As usual, the problem is not what's claimed (classism, elitism, bullyism as claimed bases for a would-be grammarchy). It is the absolutism and oversimplification used to make and support the claim.
Where a given rule aids clarity? GRAMMAR ELITE GOOD. They win not because "It's a rule"; they win not because the power of rules can "shut down" and "bully" people into silence! And what sheep, if so; what cowards to be shut down, shamed by ignorance of some silly rule!
Where the rule truly is silly, is useless, the grammarchists are easily dethroned and ridiculed for insisting on it. Where the rule powerfully aids clarity, though, the grammarchists rule. By divine right: the will of the people to a language capable of wielding meaning with power.
It's case by case, as almost always it is. Absolutists can eat a dick!
But even there, my generalization is overbroad. In a given instance, many of them will choose not to.