This totally works. You don't have to tell them, and they can't deny you. Just don't say something along the lines of "I won't wear a mask because it sucks"……. They aren't your doctor or your lawyer. Just say I can't wear a mask for medical issues.
That, in fact, is my medical reason for not wearing a mask, but I won't disclose that to a shop keeper. All they need to know is I have a medical condition. The medical condition is between my doctor and I……if I had a doctor. And that is none of their bees wax either.