Yeah I imagine there is a big difference between being on vacation and around people you don't know and pretty much hearing people you do know insult you in your face because they think you don't understand what they are saying. It's like hearing colleagues gossip about you because they didn't know you are in the same room but even worse because 1. they know you are there and 2. they underestimate how well you speak there language. Part of me would be angry and hurt but the introvert and overthinking part of me would prevent me from confronting them.
The most I said „the foreigner is doing what? Sometimes I can understand things.“ I got a nervous smile and a pink face.