> Those zoning decisions are anything but stupid.
> They're carefully thought out to achieve a certain
> goal. The question that's being asked in TFS is: is
> that goal forcing families and lower income people
> out of San Fransisco? A corollary to that is: is that an
> accident or on purpose?
I don't think it's actual malice, just stupidity... or maybe nostalgia turned up to 11. There's a significant, and very vocal, segment of the populace in San Francisco who have this image in their head of what San Francisco was when they moved here* and they want nothing to change... ever. (Look for invocations of Herb Caen's mythological SF of the 1950s, pining for the "Summer of Love", or baffling nostalgia for the pre-Moscone-Center SOMA filled with skeezy no-tell hotels that made the neighborhood more "real" than it is now.) So they fight tooth and nail to prevent any development, redevelopment, or new construction. Their latest and greatest triumph of NIMBY-ism was a ballot measure stopping all development on the eastern waterfront; requiring a new ballot measure to build anything on it. It was sold to prevent an unsightly "wall on the waterfront"... a waterfront that, except for the ferry building and ballpark, consists of decrepit warehouses and piers that are slowly rotting and collapsing into the bay, all the way from Pier 39 down to Candlestick. So, as new people move in and the housing stock is stagnant, supply vs. demand causes prices to rise. When those prices rise, the demographics of the neighborhood changes. And the live-in-the-past crowd re-doubles their efforts to hold the city in stasis.
(*And, make no mistake, the bulk of the "San Francisco is full. Nerds get out." crowd, including current and former ringleaders such as "Broke-Ass Stuart" and Ted Gullicksen are and were transplants themselves.)
You get things like housing developments being stopped because they include lofts, studios, and one-bedroom units targeted at singles. "Not family friendly!", people cry. Well, guess what? If singles can't find housing for singles; we're going to find a few other singles we get along with, and go in as roommates in a multi-bedroom house in the Mission or the Avenues... a house that could have held a family with kids. Bet you wish you hadn't torpedoed that SOMA high-rise full of "microapartments" and 1-bedrooms now, eh? Ostensibly good and progressive policies in place to help low-income residents afford housing work not by the city subsidizing said housing, but by forcing landlords and developers to offer it at artificially low prices. This, of course, leads to said developers and landlords raising the prices on the rest of their stock so they can still make their profits. Rent control and prop 13 keeping units off the market, preventing churn and kicking up prices on the remainder even more. With the poor covered by affordable and BMR requirements and the rich, by definition, having no difficulty affording the rest; those of in the middle class are getting squeezed the most.
Basically, a segment of the population just has to grow up, accept that you can neither stop the clock nor go back in time, and understand that on a 47 square mile peninsula, we can't build out, we have to build up. San Francisco's population density is not especially high by world city standards. Redevelopment with height restrictions lifted (Zoning currently restricts a very large percentage of the city to 4-stories.) could double, triple, or more, the housing capacity of the city without touching a single foot of our parks and green space. The same NIMBYs would then cry "But that will increase traffic.". But the beauty is that sort of density makes for better walkability provides the critical mass for a New York or Tokyo style subway system, covering the entire city as opposed to the paltry few rail lines and utterly dysfunctional busses we have now, to work. (Gods, I'd love for it to be truly feasible to give up my car entirely.)