If you bought a house during the era of high rates you actually made out. The high rates suppressed house prices, and you were able to refinance throughout the following period of dropping interest rates, meanwhile any bond investments you made at the same time went bonkers with the dropping rates as well.
It's actually much worse to be buying a house now. The low rates we've had over the last decade have inflated house prices, and if interest rates ever go back up, it will depress house appreciation rates, kill any bond investments you might have, and no way to benefit by refinancing. Yet another way the millennials are screwed.