I have had no insight behind the scenes in any bicycle R&D other than the usual stuff agents are exposed to. One of my agencies was an Italian company and although I in theory had free access to their engineers and R&D, the language barrier was enormous and I learn nothing other than what was recommended practice and what was in the manual. Some of their designs were pure rubbish - Fiat or Alfa Romeo level rubbish. All my insight is hindsight. But I have been privileged by enjoying long conversations and analysis with one engineer who I considered to be the world's foremost thinker in bicycle engineering even though he never worked in bicycle R&D - he was employed in Fortune 500 labs in California but cycled more than he worked. His bicycle related designs only saw the light of day on his own bike and on some of his mates'. He never patented them even though he was in the business of patenting stuff. His list of professional inventions is impressive and his employers patented dozens of them. After his death, I notice the likes of FSA are freely using at least one of his designs. He motivated me to patent one of my own designs, which I did. The altimeter algorithms in your bike computers are his, but were originally designed for mountaineers. He first described stress-relieving in wheels and plenty of other inventions and techniques out of the mainstream of bicycle design and worked hard at helping anyone who wanted to listen, to understand how and why and often used to approach bicycle companies at trade shows and fire questions at the poor sods on the stands.
I have had conversations with Shimano agents who visited Shimano in Japan and Indonesia once a year and apparently the R&D there is as big as claimed, with sometimes several engineers dedicated to just one component. I'm sure it is pretty similar at SRAM and much smaller at Campagnolo. I have no more insight in there than anyone else here. AlI have is hearsay - stuff such as sworn secrecy, access to only a few etc etc.
One can only spend a portion of turnover on engineering and Campag's turnover is tiny compared to Shimano. Suntour relied on the talent of one or two people who came up with several excellent designs, including the invention of index shifting we all use. In spite of the massive R&D going into Shimano products, the company is not immune to duffs and big ones at that. Octalink I has to be one of the industry's silliest inventions ever - even a junior engineer can calculate and predict that it is a failure. Octalink II patched it up but it is still a patch job.
Other times designers are pushed into a corner by legacy standards and have to do the best they can with what they have. Today's bicycle frame design is essentially 100 and a bit years old and had two great flaws that came all the way down the line almost from its first iteration. One of these is now sorted out but the remaining one is the BB. Legacy BB shell sizing and the move to carbon frames that cannot be threaded in the BB shell has given us the press-fit BB which is a very poor design. Attempts to fix it, even with a re-invention of the BB shell by way of BB30, was unsuccessful. The other big flaw that thankfully got fixed 27 years ago is the quill stem. It was all wrong. The clamping mechanism was at the bottom, leaving the top to move around inside the steerer. The Aheadset (threadless headset) was pure genius. This didn't come out of Shimano though but out of an independent one-man band. The patent only expired two or three years ago but enjoyed tremendous success with most companies other than Chris King licensing it and giving us all the pleasure of rigid stems that don't freeze into the frame after a season's hard riding and a bit of rain.
My impression of R&D in the bicycle business is that a few big companies do the ambitious projects such as Di2 and thankfully a few dedicated loners and independent mavericks give us the true innovation. Many f the larger companies producing components, especially hubs, just get it all wrong. I'm not saying the likes of Hope and Tune don't understand why their bearings don't last, but I do resent the way they gloss over the problem as if it doesn't exist. Hub bearings and hub seals have a long way to go.
Arguably the best hub technology (a word I hate) lies with Campagnolo who very successfully prevents Shimano from improving it's hubs with an excellent patent for replaceable cup-and-cone components. Anyone who's had to scrap a Shimano wheel because of rust in the hub has to have thought that there is a better way. Chris King and Campagnolo have a better way, yet all of Shimano's horses and engineers cannot find a better way of making a hub. Instead they blow their trumpet with glitzy stuff like new this and new that every year.Di2 is good but I would have thought they'd have sorted their hub weakness by now.
BTW, the auto anti cross chaining trick was first introduced in Shimano's MTB range on XTR, a year or two ago. Nice to hear it has come to the road.
Whenever you see a new cycling gadget or new design or new approach to an old application, stop for a minute and ask "what problem does that solve?". You'll be surprised at how many new things don't solve problems. The developer often cannot answer that question.
Another question that cuts to the chase is "can you quantify that"? If more people asked those questions and demanded rational answers we'll all be better off. For instance: "These wheels roll better." Can you quantify that? This frame is more responsive. "Can you quantify that?"
Kudos to any designer who says no, it doesn't solve a problem but damn, isn't it beautiful?
