I work in clinical research and there are whole swathes of people employed to ensure exposure to radiation is minimised as there are risks associated with excessive exposure. That's because ionising radiation doesn't play nice with your DNA. The problem occurs when it knocks out the sections of your DNA which help prevent cancers, i.e. destroy or impair genes responsible for cell repair, validation genetic sequences, controls on cell division, cell growth, voluntary cell death and so on. As you slowly stack these up you have the workings of an unstable and immortal cell which may lead to invasive cancers. This is why continuous exposure to toxic air pollution, alcohol, smoking, processed foods and all things yucky eventually lead to cancers. Yet this needs to happen to dozens of genes, all within the same cell before any cancers can emerge, even then there's a chance the immune system will fight it off before it becomes a problem.
But as I said, the act of doing nothing is often worse in the long run. Lets say I wouldn't want to have a CT scan daily, but 3-5 a year would still be a negligible dose. The chart on page 1 does a good guide of summing it up.
CT scans have been used for two decades so the risks and rewards are well understood. Thalidomide at the time was not, ask Vickster rightly pointed out, lessons we learnt in the 50s and we now have tighter controls on medications, particularly for those in use prior to licensing. Exposure to radiation sounds like a scary concept, but we are being barraged by the stuff on a daily basis and baring standing in a nuclear reactor, our evolutionary past enables us to cope remarkably well with the exposures we face on a typical day.
Unless you spend your life standing inside the Fukushima reactor, this one time you have a CT scan might be the fabled straw that breaks the camels back, but most likely it is not. So, given that information I hope you can make a slightly more informed decision on whether the risk is worth it.