Victor Mair has a post at the Log full of the kind of detailed historical/philological investigation I love. A correspondent wrote that “in Chinese the word for beard (胡子) has an archaic root meaning ‘foreign,'” and Mair, who had long “wondered if all of these expressions [húshuō(bādào) 胡说(八道) ‘nonsense; ridiculous; bullshit,’ húrén 胡人 ‘barbarian,’ húzi 胡子 ‘beard,’ húnào 胡闹 ‘act wild; be mischievous,’ etc.] … had something to do with wild, bearded barbarians from the west,” decided to look into it. He says:
… it begins to get really tricky, because it is possible that certain non-Sinitic peoples to the north and northwest were thought of as hú 胡 because they had hú 胡 (“beards”) and that these hú 胡 folk behaved in a very hú 胡 (“wild; uncontrolled; unruly”) fashion. But this is a semantic and etymological minefield upon which we must tread cautiously.
Some points to consider:
1. The earliest meaning of hú 胡 is generally considered to be “tissue drooping down under the chin of an animal (e.g., dewlap)” — note that the character has a “flesh” radical.
2. By extension, it came to mean “part of a weapon that hangs down”, and this is probably also how the meaning “beard” arose (“the pendulous mass of hair under a man’s chin”).
3. Hú 胡 also developed the meaning of “neck” (the part of an animal behind the thing hanging down) and “broad; large”, which I’ve written about extensively in Victor H. Mair, “Was There a Xià Dynasty?“, Sino-Platonic Papers, 238 (May, 2013), 1-39. See esp. p. 9 where the Old Sinitic reconstruction of hú 胡/鬍 (“beard; bearded person”) is given as *’ga (in Jerry Norman’s spelling system according to David Branner), together with cognates in Tibetan.
There are a bunch more points, some speculation, and an image of “a band of musicians with a dancer on top of a camel’s back.” Check it out.