Alpe d'Huez is worth doing, although I'd say that's mainly through its iconic Tour status.
In its own right, its not the most attractive climb. There are some far more attractive routes that you can take either up to Alpe d'Huez, or that branch off the main climb which turn into balcony roads down the valley. They're all doable if you're there for 3 to 4 days.
There's also the Col de Sarenne which comes up to Alpe d'Huez from the far side and was in the Tour a couple of years back. It's an absolute brute of a climb.
I tried 3x Ventoux in 2016 and failed, because I got stomach cramps the night before. Managed the first ascent from Malaucene, which is tough with about 3 to 4kms of 11%, descended to Bedoin and completely died about 7kms on the second ascent. Went back to Bedoin and returned to Malaucene round the side of the mountain, which still gets up to about 480m. I was completely done in when I got back.