Emoji as a Substitute for Gesture.

Lauren Gawne of La Trobe University writes for The Conversation about some research she’s been doing:

Over the last three decades, psychologists, linguists, and anthropologists, along with researchers from other traditions, have come together to understand how people gesture, and the relationship between gesture and speech.

The field of gesture studies has demonstrated that there are several different categories of gesture, and each of them has a different relationship to the words that we say them with. In a paper I co-authored with my colleague Gretchen McCulloch, we demonstrate that the same is true of emoji. The way we use emoji in our digital messages is similar to the way we use gestures when we talk.

She goes on to write about what gestures and emoji have in common, illustrative and metaphoric emoji, “beat” gestures, and the limitations of emoji:

Gestures and speech are closely synchronised in a way emoji and text can’t be. Also, the scope of possibilities with gesture are limited only to what the hands and body can do, while emoji use is limited to the (currently) 2,823 symbols encoded by Unicode.

Thanks, Trevor!

For those who don’t know, emoji has nothing to do with emotion; AHD:

Japanese: e, picture (from Old Japanese we, from Early Middle Chinese γwəjh) + moji, writing (from Old Japanese monji, moji, from Early Middle Chinese mun dzɨh (also the source of Mandarin wénzì) mun, mark, writing (from Old Chinese , soot + –n, n. suffix, since Chinese ink is traditionally made from soot) + dzɨh, symbol, character; see KANJI).

And totally unrelated but worth noting: Erik McDonald of XIX век has made his translation of “It Didn’t Come Off” (Не сошлись), an 1867 novella by Sof’ia Engel’gardt (see this LH post), available for free download as a dual-language e-book — links in .epub, .mobi/Kindle, and .pdf format here.

Comments

  1. melissa boiko says

    Wouldn’t “emoji” (1997 in English) be influenced by “emoticon” (1988)? I wonder how many speakers reanalyse it as emo- from emotion. (Though “emoji” in Japanese is much older (≤1928; all dates from OED; Google books has 1986 hits for “emoticon”).

    I find it neat that “moji” is equivalent to “-ji”, which makes an analysis emo-ji, parallel to emot-icon, cross-linguistically plausible, though not etymological.

    Incidentally the OED also theorizes that “e-moji” may have been based on “picto-graph”.

  2. Wouldn’t “emoji” (1997 in English) be influenced by “emoticon” (1988)?

    There’s a discussion of the pair at the Log.

    I wonder how many speakers reanalyse it as emo- from emotion.

    Quite a few, I’m sure, which is why I provided the etymology.

    I find it neat that “moji” is equivalent to “-ji”, which makes an analysis emo-ji, parallel to emot-icon, cross-linguistically plausible, though not etymological.

    Nice!

  3. See now Keith Houston’s The Emoji Tongue (“If 😂 was a word, would that make emoji a language?”):

    At first glance, it is tempting to file emoji alongside writing systems such as cuneiform, hieroglyphics, or Chinese characters, all of which started life as pictures of objects. After all, that is how emoji got started. But merely existing as a collection of images is not enough for emoji to be called a script, since all of those other examples served as visual representations of spoken languages (Akkadian, Old Egyptian, and Old Chinese respectively), while emoji are still unmoored from any spoken equivalents. You may choose to read aloud the emoji in a message you receive (“Love you lots, kissing face emoji, heart emoji, hugging face emoji”), but all you are doing is describing a set of pictures. They no more form part of the written grammar of that text than would a picture of the Eiffel Tower.

    That said, some have successfully expanded beyond their native tongues. The Latin alphabet, for example, is now used for hundreds of languages worldwide. Some of those, such as the Romance languages, are derived from spoken Latin and brought the alphabet along with them; others, such as Turkish, have had the Latin alphabet retroactively applied. Likewise, and not without controversy, Chinese characters have since branched out into Japan and Korea. As such, it is just about possible to imagine that emoji, despite their wordless origins, could achieve scripthood by being retroactively applied to Japanese, English, or any other spoken language, in the same way as those other scripts in the past. It is not difficult to imagine that 😊 could translate to “smile,” or that 🚗 might mean “car.”

    Emoji resembles the Latin alphabet in another way too. Just as the same sequence of letters can mean different things in different languages, some emoji mean different things in different cultural contexts. In Japan, for instance, 🙏 means “thank you” and not “praying” or “high five” as it does in other places. The Japanese words for “poo” and “luck” sound similar, and so 💩 has connotations of serendipity that do not travel well. And ♨️ is not a symbol for a plate of hot food but instead a cartographic symbol for onsen, or hot springs. Mismatched meanings are not confined to Japan: In some countries, the insouciant 💅 has distinctly sexual connotations. Equally, the thumbs-up emoji (👍) is seen as a rude gesture in some places, with the same going for the “OK” hand gesture (👌), which can mean “nothing” or “zero.” In other contexts, 👌 can be a white-supremacist symbol, and in yet others it represents an orifice into which a 👍 might be inserted.

    Even where an emoji has broadly the same meaning across different cultures, seemingly inconsequential details can have a disproportionate effect on its perceived meaning. In China, for instance, the lack of expression around the eyes of many common smileys, such as 🙂, gives them a dismissive or mocking air, whereas the more expressive eyes of 😁 and 😄 are less ambiguous.

    In these respects, emoji are as chameleonic as any other common script. Yet there is one crucial characteristic shared by many scripts (and alphabets in particular) that emoji do not yet possess. In linguistic terms, emoji are not symbolic. […]

    Gretchen McCulloch and Lauren Gawne, a pair of linguists who specialize in online language and gestures respectively, offered an answer. In a 2019 paper entitled “Emoji as Digital Gestures,” McCulloch and Gawne make a convincing case that emoji act as the body language of the web. Citing earlier studies that demonstrated that a majority of emoji are used to augment the meaning of the words they accompany, McCulloch and Gawne show how emoji in all of the roles above—metaphors and other figures of speech; punctuation; “formatting”; and plain old pictures—give us an extra channel of information that allow us to emphasize, reinforce, and sometimes even subvert our written meanings. If our skeptic needs a reason to care about emoji, this is it: emoji are valuable partners for our writing, adding literal and metaphorical color to staid old letters and numbers and punctuation and occasionally, in creative and delightful ways, replacing them entirely.

  4. Speaking of emoji tongues, I’m one of those that are annoyed at how the expressive :p became a picture of a person grinning with their tongue simply stuck out. I take it the asymmetric gesture of :p doesn’t exist in Japan?

  5. David Marjanović says

    …what asymmetric gesture? I’ve been taking for granted that :-p is just an ASCIIfication of :-þ .

  6. That’s what I thought as well.

  7. huh! i’ve always, like Rodger, understood it as specifically a tongue out the side of the mouth, which to me is a rather different gesture. i wonder whether that’s influenced by early encounters with it as [edit: goddamit the robots replaced my smiley with an emoji and now i have to learn to comment-tag in here – but for the lolz* i’ll leave it uncorrected here; it’s :- followed by capital P] 😛 , or whether the thorn interpretation simply never occurred to me.

    it seems worth pointing out – by which i mean making an argument – that emoji aren’t gestures as some act of innovation, but because they’re direct descendents not just of smilies but of a whole field of emotive-gesture-descriptors: smilies; nominal initialisms like rotfl and brb (which fills a semantic space that would in physical space be as likely to be filled by gestures as words); tone tags like /jk or /srs**; adaptations of comment-markers for descriptions like /wink/ or /smirks at your discomfiture/ or /screaming crying throwing up/. all of which have strategies for approximating gesture-speech synchronization (and expanding the possible relationships in time and space between gesture and speech) and have tended to start from but expand beyond the possibilities of gesture allowed by the human body. i hope there’s a linguistics grad student out there working on the semantics of “lol” vs [laughing to tears emoji] vs “rofl” vs [tilted laughing to tears emoji], across different age cohorts, platforms, and communities.

    and because i cannot escape literature, jeanne thornton’s intense new novel A/S/L*** has a number of scenes that take place in IRC channels, and i don’t think anyone has thought to ask her about that aspect of writing it well (which i think she does) – i’ll try to remember to, next time i see her read.

    .
    * not really a part of my idolect, but the only appropriate phrasing, i think.
    ** sarcastic uses of tone markers being, of course, another tone/gesture move.
    ** the IRC cruising protocol, not the manual/visual language.

  8. That emoticon was :P with off-center tongue because there was no centered tongue. I doubt whoever made it up was thinking: If only ASCII included a thorn! How effective it is in evoking a particular facial expression also depends on the font, and this is presumably why there are both majescule and minuscule variants.

    Another early emoticon that didn’t catch on like :) ;) :( :/ :O was :Q and probably for two different reasons. One, how much it looks like a smoking face again depends a lot on the font. Two, it doesn’t convey a clear emotional message the way the others do.

  9. i’m not sure you’re wrong, but i’m not sure i am either – and not really answerable, since the adoption of :- P / :- p was a collective effort not a consensus one.

    to me, the bodily gestures/expressions of [centered tongue out] and [sidelong tongue out] are very distinct in a way that means it wouldn’t occur to me (it violates my gesturegefül) to represent what to me is the semantic center of :- P with :- Þ or vice versa, or to see the two as representable by a single sign, just as it wouldn’t feel cromulent to interchange :- ) and :- } or :- o and :- O (to give some minimal pairs within the ASCII-based gestural space).

    i wonder whether this is a phonemic/phonetic situation, either at the level of our different can-i-say-lects (not really, i think: reading systems, maybe?) of the smiley writing system, or at the level of the layers of our gestural repertoires/systems that we experience the writing system as connected to.

    (now, of course i’m thinking about other levels: off the cuff, my gestural ideolect’s interaction with the smiley glyphs means :- P , :- Þ , ;- P , but generally * ; – Þ (the ⟪hairbow/flower⟫ makes it cromulent again, though, which is a pitfall of just carrying the notation over).

  10. David Marjanović says

    Yes, font matters a lot in smileys (serifs in particular can completely change an expression!), and I’ve seen :- P and :-Þ (the latter mostly from myself).

    So what does it mean to stick one’s tongue out on a side?

  11. So what does it mean to stick one’s tongue out on a side?

    Yes, one problem is that I simply don’t have that as part of my gesturelect and so it wouldn’t occur to me to interpret an emoticon in that way.

  12. So what does it mean to stick one’s tongue out on a side?

    One is in elementary school doing long division?

  13. @rozele: Of course, there are undoubtedly differences of opinion about the precise meaning of emoticons, both between individuals and larger usage communities. The default meanings can also shift over time.

    I do, for what it’s worth, understand sticking the tongue straight out and sticking it out to the side as different facial expressions. That’s actually why I am pretty confident that, at least relatively early on,* :P was generally meant to indicate a straight tongue. I found that as a visual representation, :P seemed somewhat inapt, as it might imply side tongue, when it was clear in the majority of cases that the people using it meant straight tongue.

    * You will note that my emoticons look different from most others in this discussion. The forms without the nose hyphen are older, and having been using them all the way back in the golden age of Usenet, I have never updated the forms I use to include noses, or widened my repertoire as larger character sets became available. Although I appreciate more elaborate emoticons as a form of ASCII art, I don’t use them myself.

  14. Stu Clayton says

    @rozele: jeanne thornton’s intense new novel A/S/L

    Is that a good one of hers to start with ? I’ve fallen into the black hole of AO3 and Heartstopper fanfic. Currently I spend a lot of time reading and being devastated by it all. I Had No Idea.

Speak Your Mind

*