@SkipdiverJohn in the specific case of MTBs used for "proper" off road riding, I think technology, geometry and riding styles have changed quite substantially over the last twenty years, so regularly swapping or adding bikes is not quite so crazy as replacing a perfectly good tourer/commuter every few years.
EG a modern slack angle 29er hardcore hardtail will be a totally different experience on the same trails to an older short travel xc full susser. Pretty much any well maintained MTB will get the job done, but these aren't really supposed to be purely practical machines, they're big boys toys, and I don't think the variety of bikes on offer is a bad thing for the consumer (even if they do only exist to push more sales!).
In the specific case of 1x12 drive trains I completely agree with your post - probably cheaper to produce, parts are crazy expensive to replace and there was never any consumer demand for it that I'm aware of.
Other innovations I have to grudgingly admit are worth the silly price tags, particularly dropper posts - it's not all emperors new clothes.
I'd also have to agree that most MTBs are capable of vastly more than their rider uses them for, and that something like a rigid 90s MTB would be more appropriate for a lot of people.
If you've not tried one, I'd recommend renting a modern MTB from a trail centre and having a thrash around on one. Once you get over the silly wide handlebars I'll bet you'll enjoy it and find yourself and to ride terrain you'd never considered possible before.