I had better stop before this gets too costly, having read
@Jenkins post I re-routed the cable, but nothing changed, I think I have damaged things too much before routing it properly. I thought one possible explanation was a faulty shifter. This was because its operation had become very loose when slackening tension and very tight and difficult to ratchet up when trying to move it down the gears. So I removed it from the handlebars and took the cover off. When I did so, I moved one of the shifter levers and the coiled spring flew off; unfortunately I didn't have a good look at it beforehand so I couldn't tell how it should be fitted to the mechanism - I think I worked out where the inner ends slots in, but I cannot tell where the outer end should go, hopefully anyone reading this will be able to tell what I am on about from the below photo:
After a lot of fiddling about I managed to put it back together, but I didn't know if the coil was properly fitted. I managed to reconnect the barrel adjuster, but when I came to fix the derailleur end of the cable there wasn't enough slack and I couldn't clamp it - that might have something to do with the cable being internally routed through the frame but I'm not sure about that.
Interestingly (or maybe not?) the barrel adjuster at the shifter end of the cable does have a spring, unlike the derailleur end, as can be seen from yet another photo:
Anyway, sometimes you have to admit you are beaten, and I am well and truly beaten on this one. It will probably cost me a new cable and a new shifter, but at least I have learned something about the workings of gear cables and shifters, hopefully provided some interest for others, and having used the word so often in the past couple of days, I can now spell "derailleur" without having to look it up in a dikshonry.
P.S. Thanks for all the help and interest.