All it means when people say “you’re speaking from a place of privilege” is that you’re likely to underestimate how bad the problem is by default because you are never personally exposed to that problem. It’s not a moral judgement of how difficult your life is.
Lemme put it this way. As a right handed person how often have y’all noticed the lack of left handed desks? Right handed accommodated controllers? Where a computer mouse is naturally placed? Not very often I assume. You won’t notice the everyday inconveniences in a world that is MADE FOR YOU.