We went the opposite in our house. We had a 40" TV a few years back but when my sister was having trouble with her TV we gave it to her and went back to a 32". Our living room isn't that big at 15' by 13' and the 40" just seemed to dominate the corner of the room. We did try to get something in between but couldn't find anything between 32" and 40".
My son OTOH has just bought a 50" for his flat and it makes a huge difference to him because he is partially sighted.
I must admit, my 50" tv still looks big when its not on..... Its a big feck off black rectangle on the wall in my smallish lounge.
But it all makes sense when i turn it on and watch something on it.