You don't lose the upper body muscle as such because you're constantly contracting the muscles for your front and back, it does not cause too much stress to the upper body and will always engage the muscle but for some people are wanting to target that area this will not be enough.
For you I recommend to keep your upper body training which you can also do without expensive weights, I'm a gym instructor and I use affordable weights at home to keep my training when I feel my gym is not an environment I want and I enjoy achieving the goals that some people decide to go overly expensive on their gear when it is not always the best thing to do.