I think it's a very relatable experience/fear! Not to confirm your worst fear, but I 100% told stories from class to my parents back in the day.
For me personally, I tend to tell my most horrific lived stories very nonchalantly and casually in VERY public settings. Much to the dismay of the people who happen to be in earshot of my storytelling. You have to think of it as a form of celebrity.
Whether we're conscious of it or not, we've probably ALL been the subject of someone's dinnertime conversation or a "tea-time" text chain between best friends. It's inevitable. But if they are talking about you, at least they remember you!