This is just my POV, but from my personal experience with trying to get people into GG, the biggest reason most people won't play it is it's high learning curve. When starting out at the game, you'd pretty much have to be playing a gold character to have any chance of winning against someone who has but the slightest amount of experience over you. This problem is compounded practically 100 times over for me, having been playing XX for like 2 years now (I'm too lazy to order AC online despite my intense desire to play competitively, we were damn lucky to find XX and/or even Isuka used as it was) Granted my ability with most of the cast compared to my mains is exponentially less, but I still know all of the characters' basic tactics, and understand the physics and game mechanics so well that it is damn near impossible for me to not just utterly destroy anyone who wants to try to play it with but the slightest effort.
It just takes far more practice to get even somewhat good at GG than the average person cares to invest time in. Also, there's no fighter like GG (well, now there's Blazblue, but nobody in my area has played it yet) it just doesn't function anything like mainstream games that most are familiar with. Hell, gameplay wise I'd almost go so far as to say the only similarity GG has to other fighters is it's 2d status. Toss the "Anime factor" in on this, and it's not hard to see why the game that has a loyal, but admittedly quite small following. (when compared to other fighters, anyway)