Bear with me here. A thought experiment:
I wear a hi-viz jacket in the winter. What would happen if I took it off and didn't wear it. I'll try and be honest here.
For the first few days (maybe even the first couple of weeks) I might actually be concerned about not wearing my jacket at night. I might try cycling more conservatively, i.e. more submissively. As time progresses, however, I would get used to not wearing the hi-viz and I would probably forget what it was like to wear it. At this point I would expect that my cycling would return to normal, i.e. I would cycle the same way I did with the hi-viz on. This would be because I would no longer think about what I was or was not wearing as I would be getting on with the job of cycling and enjoying it.
I would suggest that the time I would be in greatest danger would be during the transition, where I would be riding more submissively. I probably would not take my road position as I should etc because I was worried about my visibility.
However, this effect would be temporary.
Going the other way it is possible for a short while that someone might feel safer for a couple of weeks because of wearing a jacket. But, similar to my example this effect would disappear once it became the norm.
In fact it could be the case that changing either way is just as dangerous, and it would be the change itself that would be dangerous and not the fact that you were moving to hi-viz that was the problem
So in conclusion changing may cause transitory reduction in safety, but in the long term no effect.
So if you conclude (as I do) that hi-viz improves your visibility under certain circumstances then I suggest that long term safety is improved at the expense of a short term reduction in safety. Of course if you are just starting cycling the transitory period would not matter as the major risks of learning a new skill would result in far greater risks (so long as you wore hi-viz at the start).
Discuss.....