A good friend of ours is a retired athlete and member of his sport's HOF. We met late in his career but I saw enough to know - in spite of the financial rewards - I would never want to go through what he did and still does, albeit to a lesser extent these days.
Knowing him as I do, I also know what he endured was exacerbated by his personality. He's a great guy, fantastic family man, but he's a very private person and the notoriety that comes with being a sports superstar was a great drudgery for him and by extension his family and even friends when we were with him in public.
Try to imagine what's it's like when simple things like going to the grocery store, a mall, a movie or restaurant or wherever always turns into a circus of people who want to see you, touch you, want an autograph, a picture taken, etc., etc. And not just once in a while but every time you stepped out of your house in those days.
I'm nowhere near as private a person, but still there is no way in the world I'd want to put up with the nonstop attention/adulation. Probably sounds weird to some but having experienced it many times with our friend I know I'd be sick and tired of it in a very short time.