While Newtonian physics is no longer universal truth in physics nowadays, Newton was not wrong in the present context. I think your visualisation of what the systems are doing is.
As you know Newton's 2nd law of motion says force is mass times acceleration. I trust you are also aware that work done (or a change in energy) is force times distance travelled in the direction of the force. Ok? If not please see
this.
Your problem in understanding shows up as an error in your question - the fact of the matter is those two accelerating objects have not covered the same distance.
To visualise what we are talking about, we could look at a spoke nipple where one normally finds it on the wheel, and another welded to the centre of the wheel's axle. Since the two are otherwise identical, they illustrate two masses with the former being a rotational mass with significant inertia because it is far from its centre of rotation (i.e. the centre of the wheel's axle), while the other nipple essentially only translates in line with the bike (if you don't want to ignore its rotation along its own axis at the axle you can consider this
tit nipple is sitting on the saddle of the bike instead, ok?).
I hope you can now be convinced, by visualising the very different trajectories of the two spoke nipples, that when the bike is accelerated it takes more energy to accelerate the nipple on the rim. Not only does it travels in a straight line (since after a mile the bike has travelled the nipple has also travelled a mile),
it is also spinning up and travelling in a circular trajectory with a radius of the wheel. In contrast the nipple at the axle or on the saddle only has a trajectory in a straight line only. Since the spinning up trajectory experienced by the first nipple is the only difference and is an add-on, unless you think Newtonian physics says it can be accelerated and moved about this circular trajectory by consuming no energy at all, the nipple at the rim has to consume more energy than the one on the axle/saddle to be accelerated for the same mile with the bike.
If you believe spinning that spoke nipple up consumes no energy, then you would effectively be saying it takes no more effort to spin up from still a rim made of lead than of alloy while both rims are on a bike on a stand.
What the above shows, is saving rotational mass has greater effect on performance than an equivalent non-rotational mass. In some idealised conditions (constant speed, no braking etc.), the two nipples may end up consuming the same amount of energy over distance in theory, but those are idealised conditions that are not achievable by cycling in the real world.
From what you wrote, I believe your visualisation for comparison is different to the above - you were thinking about items that essentially have parallel but otherwise identical trajectory as the bike, and which do require the same energy to accelerate if they have the same mass. The problem with that visualisation, is that a rotational mass does not have the same trajectory as a non-rotational mass.