The biggest climb that I have done is the Tudons from the coast near Benidorm. That is from sea level to an altitude of just over 1,000 metres and is not as steep as the Ventoux. My legs, heart and lungs were fine because I was riding up at a sensible, steady pace. What got me was backache. From about halfway up the climb my back muscles were starting to give me a lot of discomfort The last few kms were really unpleasant. I was surprised by that because I do an awful lot of climbing round here (and much of that is steep), but there is nothing that requires unrelenting effort for such a long time.
I'm not sure if back/core exercises would help with that? I find that my back is always the weakest link. As it is now, I know that I could not cope with something as hard as Ventoux without several stops to stretch my back.