This could be a hearing issue - but not so likely. It is most likely a Room acoustic problem. Try headphones.

It cannot really be an issue as far as I understand it.

Sense of dimension is primarily a combination of both ears and not one of them ( except if one of your ears has a completely different shape) acoustic shadow, the perception should be good..

Try different equipment and setups-
Open you daw and just pan a sound here and left and right , maybe have it move with automation to see whats going on

However it sounds like the game had made you become aware of an existing problem in your setup- that i would guess happens outside of the game as well.