I lived at Timberscombe near Minehead for a while, always seemed to be raining and together with the rather grim looking [grey] sea it wouldnt be my choice. Tivertons OK Tauntons too big n busy, My choice would be where there's fewest people! usually means less hassle
As an incomer (16 years ago from Warwickshire) I live in West Somerset, 4 miles along the coast from Minehead. Each to their own and all that, but personally, I do not recognise the above comments at all re this area. Fantastic countryside, Exmoor, superb coast, and whilst it's not a "blue" sea, that's simply because it's the second biggest tidal reach in the world. ... and as for fewer people, it's one of the the least populated areas in England, certainly lot quieter than the Tiverton "metropolis". Great for cycling, walking, fishing, paddle boarding, whatever floats your particular boat, plus, of course, the best cider in the world. Also, whilst we "suffer" holiday makers in the season, this is at a lot lower level than the Devon and Cornish hotspots, plus London / rest of the "civilised" (you know, all the shops you may want, traffic, congestion, etc., etc) world not a million miles away with mainline staion at Taunton and the M5 etc.
Whilst I work in Taunton, I personally wouldn't want to live there, but that's mostly down to the fact that I was born in a city, then moved to a large town, then a market town, then a village and now a hamlet (theme developing here).
Fab Foodie, PM if you want to ask any questions fella, happy to try and help fella.