You have got to be seriously nutty to want to go cycling when you've got a cold, really. I just want to lie down and complain.
In the case I mentioned above, I'd already spent a few days doing the lying down and complaining thing, but the day after the symptoms disappeared, I thought I was ready to jump back on my bike and attack the Heptonstall Road (a 10+% climb)! I hadn't factored in time for my body to actually repair itself properly once the bug was on the way out.
Nowadays, I give myself at least 3 or 4 days after the symptoms of a cold have gone, and even then I'll stick to relatively easy rides (25 miles/2,000 ft of climbing) until I am sure that I've fully recovered.