It is the Nemesis of pedantry to be always wrong. Your true prig of a pedant goes immensely out of his way to be vastly more correct than other people, and succeeds in the end in being vastly more ungrammatical, or vastly more illogical, or both at once. The common pronunciation, the common idiom, the common meaning attached to a word, are not nearly good enough or fine enough for him; he must try to get at the original sound, at the strict construction, at the true sense—and he always manages to blunder upon something far worse than the slight error, if error it be, whioh he attempts to avoid in his superfine correctness. There are people so fastidious that instead of saying ‘camelia,’ the form practically sanctified by usage and by Dumas Fils (for even Dumas Fils can sanctify), they must needs say ‘camella,’ a monstrous hybrid, the true but now somewhat pedantic ‘Latin’ name being really ‘camellia.’ There are people so learned that instead of talking about Alfred the Great, like all the rest of us, they must needs talk about Ælfred, and then pronounce the word as though the first half of it had something or other to do with eels, whereas the true Anglo-Saxon sound thus clumsily expressed is simply and solely the common Alfred. There are people so grammatical that they must needs dispute ‘against’ their opponent instead of disputing with him, in complete ignorance of the fact that the word ‘with’ itself means ‘against’ in the early forms of the English language, and still retains that meaning even now in ‘withstand,’ ‘withhold,’ ‘withdraw,’ and half-a-dozen other familiar expressions. To such good people one is tempted to answer, in the immortal words of Dr. Parr to the inquirer who asked that great scholar whether the right pronunciation was Samaria or Samareia, ‘You may thay Thamareia if you like, but Thamaria ith quite good enough for me.’
There’s a great passage about the etymological fallacy (the connection in which it was adduced at Wordorigins):
And this leads us on to a second habit of the microscopic critic, which I venture to describe as the Etymological Fallacy. Your critic happens to know well some one particular language, let us say Greek or Latin; and so far as the words derived from that language are concerned (and so far only) he insists upon every word being rigidly applied in its strict original etymological meaning. He makes no allowance for the natural and beautiful growth of metaphor, and the transference of signification, which must necessarily affect the usage of all words in the course of time; he is aware that the root of ‘mutual’ in Latin implies reciprocal action, and so he objects to the harmless English colloquial expression ‘Our Mutual Friend,’ which the genius of Dickens has stamped so indelibly upon the English language that all the ink of all the pedants will never suffice to wash out the hall-mark. I use the mixed metaphor quite intentionally, because it exactly expresses the utter hopelessness of the efforts of banded pedantry.
I wonder if that’s the origin of the phrase (in its modern sense)? The whole thing is worth a read; [I just wish I knew who had written it -- Punch suggests it was the editor of the magazine, who would have been James Payn at this time].
Update. Apparently the author is Grant Allen; see Jan Freeman’s comment below.