It's not debt that fucks with housing it's usery.
With the average person paying 3-8% on a six figure purchase annually, and 6% on fees to purchase, it's pretty clear that over time banks end up owning most the houses/land and can then control the price to ensure they appreciate more than income. Forcing people into renting.
What's a person do when they sell their house? Either buy another and pay the fees/interest, or invest in another bank controlled market allowing banks to use that money to buy more housing.