That contractor doesn't want your business, Dennet. He high-balled you because he isn't interested in jobs that don't make him at least a certain amount of money. My dad used to be a contractor, though in a different area, and he used to tell me all kinds of stories that sound like your story.
So, please don't give up. Not all contractors are like that contractor. I'm sure if you keep shopping, you can find a guy who is willing to do the work at prices you can more afford with materials that are closer to what you want.
Rooting for you!