You don't have to justify yourself Johnno . For me, a lot of what you're describing IMO is down to feel. It's not quantifiable and likely never will be. That's not to say it's not valid...if you don't like the feel and want to spend some cash...that's all the justification you need.
The fly in the ointment is in my personal experience as follows...
Anbrosio wheelset as standard on my old Bianchi...good solid, heavy wheels...flexed a little but generally ok. I felt like something better and brought Fulcrum 5s. Brilliant wheels IM (limited) E, they're still running now, never let me down, much lighter and I feel like they spin up much faster downhill.
Did my average times decrease on my regular old 50 mile circuit....not that I noticed, but they feel better ( not quantifiable) and I get the pleasure out of them.
Life isn't dictated by a slide rule and fact and figures...if you want and feel you'd get some benefit, go for it