It's pretty simple really: When the systems themselves reach a certain affordability threshold, sales of the games will increase dramatically. I don't know what that price threshold is, but I imagine it's much much lower than the Vive's current price.
Agreed, price is a major factor in adoption. However, there is evidence that VR prices could come down to much more affordable levels for people in the near future (maybe within 3-10 years?). For example, currently Google Cardboard VR is really cheap (assuming one already owns a compatible smartphone), but it's low-quality makes it little more than a novelty right now. However, it's a really cheap starting point and technology steadily improves over time. I don't think it's unreasonable to believe tech advances will allow VR systems priced similar to current Google Cardboard, but with quality more similar to current Vive, Oculus, etc.