I find golf incredibly hard to pay attention to, but at least its relatively simple to understand, get the lowest score, take the least amount of shots, job done.
I think one of the biggest issues with cycling is that its slow to get into because compared to most popular sports its much harder to understand what the actual aim of the sport is, the different Jerseys and how you would get each or what the point is, Tour winner vs TT vs Points vs winning Team vs KOM etc, with the various leaderboards for each.
If you compare to F1 which im a fan of, although im aware a lot also find it boring, the viewing figures have been on the up over the past few years with prime time BBC1 coverage. Particularly at the start of the season, the commentators do their best to explain everything as it crops up in the most simplest of terms, to the point of being annoying to the die hard fans, purely with new viewers in mind. When ive put TdF on tv the commentators have been chatting about things unrelated to the race itself, from their meal last night to discussing architecture and im left clueless to how this mass of cyclists riding through pretty scenery relates at all to whos going to win anything. I tend to leave it on the TV in the background, wait for the distance to run out and watch the final sprint.