Yea, that's a tough call. Although without any kids you don't have to worry about school districts/etc and could probably find a nice weenie spot high up, although still pricier than 8 months ago, but you lose the close to work, close to town, close to restaurants/etc.
We have been in a similar situation since we moved to VT. We looked at many places up high on the eastern slopes in the 2 times we have house shopped, but ended up in the valley because of all the reasons I listed above in addition to family nearby and school proximity. What's funny is that whenever we mentioned to realtors that we were also looking at places up in the Bromley/Stratton area, some would say "oh no, you dont want to be up there, you will never be able to get into town in the winter" like it was Sapporo or Snoqulamie pass...lol.