The fastest I've ever been (on a straight descent with good visibility) was somewhere between 45-50mph. I can't remember exactly because it was some time ago.
My partner used to terrify me when we rode together because she'd go flying down narrow, windy roads, where you couldn't see if there was a person, a cow or a parked car round the next bend (or a car coming towards you on the wrong side of the road), and there was no way she could have stopped. But on a nice, wide major road with good visibility, she'd be hanging onto the brakes. I think she might be a bit more sensible now, but we don't ride together any more, so I'm not sure.
From that, you can probably gather that I never ride so fast that I couldn't stop if there was something in the road round the next bend, but I'm not afraid to get up to a decent speed if I can see well. I don't go all out to see how fast I can possibly go on clear roads either. Once I'm past what I feel is a comfortable spinning speed, I'm happy to freewheel the rest of the way down a hill.