A Stylistic Analysis of Graphic Emoticons: Can they be Candidates for a Universal Visual Language of the Future?
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Original printed in: Proceeding of World Conference on Educational Media, Hypermedia and Telecommunications
(ED-Media), 2008, p. 972-977
A Stylistic Analysis of Graphic Emoticons:
Can they be Candidates for a Universal Visual Language of the Future?
Junichi Azuma
University of Marketing and Distribution Sciences
Kobe, Japan
junichi.azuma@nifty.com
Martin Ebner
Graz University of Technology
Graz, Austria
martin.ebner@tugraz.at
This paper, describing the current popularity of graphic emoticon usage in
Japanese social network services, blogs and mobile phone communication,
illustrates that these graphic emoticons are now evolving into ideographic images
as substitutes for words or phrases from accessory markers showing the emotional
state of the writer. It will be argued that the behavior of these graphic emoticons
will give us valuable insights for implementing a universal auxiliary visual
language that will overcome the barrier of language differences. Possible
educational applications of the future universal visual language will be also
proposed.
Introduction
Emoticons are everywhere. Graphic emoticons are really ubiquitous in Japanese free blog sites, social network
services and mobile phone email communications. Even Westerners know that when we input typographic
emoticons such as :-), some software, such as Microsoft Word or email client programs, automatically displays the
result in graphic emoticons; (for example, ). All the major Japanese mobile phone providers are moving toward
the standardization of the codes for graphic emoticons, and the use of graphic emoticons is now extremely common
in mobile phone communications among young Japanese people. If the graphic emoticons continue to increase in
popularity, and if the emoticon-like images or simple animation becomes more and more powerful and capable of
conveying richer meaning, they may develop into a universal constructed visual language in the near future.
But, why have emoticons become so popular in cyberspace? Before considering the impact of the graphic
emoticons in our future communication, let us examine how emoticons began to be used in cyberspace.
As is well known, ASCII-based emoticons are thought to have been used in cyberspace for the first time by a
researcher called Scott Fahlman. That was in 1982 and he used the mark :) to show that the sentence was written as
a joke, as opposed to the use of the mark :( to show that the sentence was written in a serious manner. These marks,
to be read sideways, were named “emoticons,” since all these marks were used to show the emotion of the writer.
In fact, when the Internet first began to be used, cyberspace was still a place mainly for male engineering researchers.
It was chiefly a place for scientific communication, and because the bandwidth was not so large and the network did
not physically allow communication of multimedia objects, it was in principle a “text only” world. Of course no
gestures or facial expressions were visible there. As the network could not yet deal with sounds or speech, prosodic
features such as intonation, rhythm or loudness were not perceivable. There were no ways to convey the subtle
changes in our voice quality in cyberspace at that time. It was the place where logic preceded the emotion of the
communicators. In addition, communication in cyberspace by means of the written language had another important
feature. Once transmitted, it was impossible or nearly impossible to recall the message. These features naturally ledOriginal printed in: Proceeding of World Conference on Educational Media, Hypermedia and Telecommunications
(ED-Media), 2008, p. 972-977
to the difficulty of comfortable and smooth communication in cyberspace, as it is known that the contribution of the
purely linguistic elements in our actual face-to-face communication is much smaller than non-verbal information
and prosodic features of our real face-to-face communication (Mehrabian, 1971). In fact it is rather difficult to have
a language-only communication online, especially if both parties have just encountered each other for the first time
and share little information about one another. In real life, the first encounter will of course put some pressure on
both parties but with the help of paralinguistic and prosodic information accompanying the purely linguistic kernel,
they will understand one another quickly and the communication between them will become rather smooth in a short
time.
In cyberspace we often encounter a very violent argument or in a sense a furious verbal fight. Such a fight might not
arise if both parties encountered each other in real life and began the discussion face-to-face. In such a severe verbal
fight in a BBS (Bulletin Board System) or in a social network service, we very often see a posting such as “I did not
mean that in my previous posting.” Then the clarification or meta-clarification of the past postings adds more oil to
the flames and the situation will often become uncontrollable.
Softeners and Emoticons as Lubricants in Cyberspace
Considering such situations, it was quite natural that some people started to use “softeners,” intentional indicators of
paralinguistic or prosodic features, by just using normal characters or letters. In the western world, you can see
examples of this phenomenon in writing such as “Nooooooo pleeeeeese Tom. I am sorry, don't hurt meeeee.” or
“dooooooooooon't goooooooooooo awayyyyyyyyyyyyyyyyyyy plllllllllllease!!!!!!!!!!!!!!!” The irregular spelling
imitates the actual phonetic features, mainly representing prosodic features. Very often, these irregular spellings
show the lengthening of a certain syllable or a segment but, interestingly enough, our “inner ears” tend to perceive
some kind of prominence with a significant pitch movement at these points. On the other hand, if we write
in Japanese (literal translation: “This paper is quite nutty (smile).”), then we
understand that the writer posted this phrase in a casual and humorous way. It is natural that emoticons or smilies
have gradually entered cyberspace to provide the language-only and seemingly logic-only cyberspace
communication with an emotional and human touch ;-). In a sense, an emoticon, very often placed at the end of a
phrase or a sentence, is a typographic version of a paralinguistic or prosodic feature. In East Asia, especially in
Japan, people developed their own style of emoticons, or in the Japanese language, “kaomoji” (face marks or face
characters). Normally, these East Asian emoticons are to be read vertically, such as in the sentence “Well, this paper
is not so bad as you might think (^_^).” Although their style is different from the Western style, these are also quite
intelligible to people all over the world.
East Asian languages have a double byte character code system, and this allows more variety of emoticons in
cyberspace. The next example of Figure 1 shows some Japanese double byte code emoticons showing the
movement of greetings or “hurrah!,” which appeared in the Chinese character input menu.
Figure 1: Some examples of the Japanese double byte code emoticons.
As was pointed out at the beginning of this paper, when we access the major Web-based free Email services, we
notice that graphic emoticons are also usable there. Graphic emoticons are becoming quite popular in today’s
cyberspace. Many Japanese blog services as well as social network services now support the HTML-based postingsOriginal printed in: Proceeding of World Conference on Educational Media, Hypermedia and Telecommunications
(ED-Media), 2008, p. 972-977
with graphic emoticons, and though the input system is still primitive and the repertoire of graphic emoticons is still
limited, use of graphic emoticons is really expanding these days.
Stylistic Analysis of Japanese Graphic Emoticons
As the name itself shows, an emoticon is used to represent a certain emotional state of a writer, and as has been
previously argued, this behavior is considered to be a writer’s intention to somehow demonstrate prosodic or
paralinguistic features clearly, even in the text-centered cyberspace. This “paralinguistic use” of emoticons has been
popular since the time of character-based or ASCII-based emoticons. However, corresponding to the emergence of
graphic emoticons, we can now encounter new styles of emoticon usage. One of the new waves of graphic emoticon
usage is using an emoticon just before or after a word or phrase whose meaning the emoticon denotes. For example,
means:
“I found a letter and felt very glad. ”
Here, the words “found” and “letter” are accompanied by graphic emoticons, and the meaning of the sentence
becomes clearer. Another example,
means :
“In a sense, our home is quiet and peaceful (smile)”
As is clear, the phrase “our home” is accompanied by a house-shaped graphic emotion. This new type of emoticon
usage can be interpreted as reinforcing the meaning of an actual word or phrase. Of course, during the days of
character-based emoticons it was not so easy to create a variety of emoticons denoting the real things around us. Let
us call this new method of using emoticons “emphatic use.”
The most recent trend in using graphic emoticons is really extreme. Some young people in Japan omit a word or a
phrase in a sentence but use only a graphic emoticon illustrating the meaning of the lexical item. Some examples
are shown below:
Meaning in English: “…afterwards, suddenly broke down. ” ( = mobile phone)
Meaning in English: “I was busy because of the part-time job and the training. ” ( =automobile)
Meaning in English: “We had a short rest at my and went to the second hand shop in a …
These examples are really extreme, and of course there are not so many people in Japan who normally use this style
of writing for blog postings or email communication. However, this “lexical use” of graphic emoticons is
becoming more and more popular these days, especially among young Japanese females.
Descriptive Study of Graphic Emoticons
To investigate the extent to which the new styles of graphic emoticons, i.e., emphatic use and lexical use, are
popular, actual blog articles were analyzed. All the emoticons that appeared in blog postings of “AS” and “AM”
(both are female collage seniors) from August till October 2007 in Rakuten Blog, one of the largest free blog hostingOriginal printed in: Proceeding of World Conference on Educational Media, Hypermedia and Telecommunications
(ED-Media), 2008, p. 972-977
sites in Japan, were analyzed and categorized into four types; paralinguistic use (graphic emoticons), paralinguistic
use (character-based), emphatic use and lexical use. Figure 2 shows part of a posting by AS on her blog, and the
results of the analysis are shown in Figure 3.
As Figure 3 shows, paralinguistic use of emoticons is dominant even today, and we can also see a substantial
amount of character-based or ASCII-based emoticons of paralinguistic use (11% in AM’s postings and 20% in AS’s
postings).
Figure 2: Example of blog posting by AS
Figure 3: Results of Analysis of Emoticon UseOriginal printed in: Proceeding of World Conference on Educational Media, Hypermedia and Telecommunications
(ED-Media), 2008, p. 972-977
The proportion of the new usage of graphic emoticons (emphatic and lexical) is about 10% in both cases, though
there is an individual difference concerning which type of usage, emphatic or lexical, one prefers. For example, AM
strongly prefers emphatic use of graphic emoticons to lexical use (11% and 2%, respectively). As was expected, no
character-based emoticons were found that were used in an emphatic or lexical context. This is perhaps due to the
fact that we need a lot of concrete objects (home, hospital, pen, mountains, etc.) in order for them to be used as
emphatic or lexical emoticons, and one can easily imagine how troublesome it is to create these objects only using
the existing characters.
Power of Graphic Emoticon in International Communication Settings
As one can see, if a graphic emoticon works as a lexical item, it will be functioning as an ideographic word, like a
Chinese character. And if the meaning of the graphic emoticon is very easy to intuitively recognize among people
who speak different languages, they can in a sense behave as lexical items of a primitive universal visually
constructed language. An international experiment was conducted in order to test this hypothesis using a small
number of Japanese university students at the University of Marketing and Distribution Sciences in Kobe (14 in
total) and Austrian university students and researchers at Graz University of Technology (9 in all).
A special community blog site compatible with graphic emoticons was opened at TU Graz LearnLand
(https://tugll.tugraz.at/emotions/) and everybody was instructed to use as many graphic emoticons as possible in the
postings, where the common language to be used was set to English. In a sense, participants were told basically to
use graphic emoticons and write in English when they cannot properly express their meaning in graphic emoticons.
The assumptions of this project were as follows:
1) Even in an international context where the native language of users is not English, simple communication will be
feasible with the help of graphic emoticons.
2) Participants in the experiment will have no difficulties in inventing ways to use graphic emoticons as lexical
items. This will cause a dramatic increase of the lexical use of graphic emoticons in the total blog community.
As the actual screen shot of the TU Graz LearnLand community page shows (see Figure 4), participants took part in
the communication project in a positive manner. Communication of the period from October 11 till November 26,
i.e., all postings including comments on main postings was analyzed. The communication seemed quite active,
though the content of the communication was not so serious and perhaps even shallow in nature, partly because the
variety of emoticons available was limited. Anyway, it can be concluded that the first assumption of the project was
verified. However, it should be also noted that communication problems sometimes occurred. A good example is
“ ” In this context the Austrian member thought it would be appropriate to use a
graphic emoticon of a sunrise with a mountain (like Mt. Fuji) to show the meaning of “Japan,” but Japanese
participants did not understand its meaning.
One more example of communication difficulty is “ ”. Japanese participants did not
understand the intended meaning of this posting from an Austrian member at all, but as the writer here is a young
Austrian researcher who has to write a lot of academic papers and proposals, it was later considered to mean “A
weekend full of paper writing, with the deadline drawing near and a lot of business telephone calls.” It is considered
that these misunderstandings were caused because the writer imposed a certain favorite and arbitrary meaning on a
graphic emoticon. As other members did not perceive the meaning of the emoticon as intended by the writer, a
communication problem was caused. This of course shows the limits of graphic emoticons in the context of
international communication, but at the same time we must note that the writers made extensive efforts to use
graphic emoticons as lexicons.
The authors also analyzed all the writings in these community blog pages and categorized all the emoticons used
here into four types again, 1) paralinguistic use (graphic emoticons), 2) character-based paralinguistic use, 3)
emphatic use and 4) lexical use. The results of the analysis (see Figure 5) show that the lexical use of graphic
emoticons prevailed (477 cases, 56%) throughout all communications. Since participants were only instructed to
use as many graphic emoticons as possible, and not told how to use the graphic emoticons, it seems they did indeedOriginal printed in: Proceeding of World Conference on Educational Media, Hypermedia and Telecommunications
(ED-Media), 2008, p. 972-977
learn how to use graphic emoticon as an ideographic lexical item. On this basis, we believe that the second
assumption of the research project was also verified.
Figure 4: Screenshot of TU Graz LearnLand
Figure 5: Emoticon Use at TU Graz LearnLand Community
Idea of Universal Visual Language
If the trend of the replacement of linguistic units by graphic emoticons goes further, and if this tendency becomes
global, we will be able to devise a universal visual language that will work as a special kind of auxiliary language
used for communication using computers, mobile phones and other computer-based instruments. Since visual
language will make use of simple, easy-to-understand pictures, it will really make international communication easy,
overcoming the barrier of language differences.
Of course it is not clear if today’s graphic emoticons will evolve into the universal visual language of the future.
However, if we recognize that graphic emoticons, or in other words, simple digital pictures and gif animations, are
widely used in the network environment, especially in Japan, our ICT environment is getting closer to the next-
generation cyberspace which is compatible with the universal auxiliary visual language. As this visual language is
“digital” in nature, it will be easy to edit and reuse the “utterance” once produced. Because of this, it will gain muchOriginal printed in: Proceeding of World Conference on Educational Media, Hypermedia and Telecommunications
(ED-Media), 2008, p. 972-977
more popularity than traditionally constructed visual languages, such as Blissymbolics, LoCos or Elephant’s
Memory. Traditionally constructed visual languages often use combinations of symbols to represent a variety of
related vocabulary words or differences in “parts of speech.” For example, Blissymbolics uses the marker “^” to
represent movement. Thus the symbol “ ” denotes a “person’s legs,” while “ ” denotes “to walk.” If we use
today’s technology, we can just use gif animation to show the movement. Thus, a single “man-like picture” (like
“ ”) will represent “a man” and a gif animation of the same “man-like picture” with the legs moving slowly will
represent a verb “to walk” or a phrase “a man is walking.” In addition, if the movement of the legs of the man-like
gif animation picture is quick, it will then represent “the man is walking very quickly” or “the man is running.”
As Maurer et al. (2003) suggests, we can add some temporal change of color to an object to show the change of a
certain state. For example, let us suppose if the circle-like object has white color, that person is healthy. Similarly,
let us say if the color is dark, the person is unhealthy or in the case of black, he or she is dead! The situation like “a
man in a small boat got sea-sick” will be illustrated with a picture shown in Figure 6, with the color of the person’s
face changing gradually darker. It is true that today’s graphic emoticons still lack systematic morphology and
sophisticated syntactic rules, but slightly more advanced types of graphic emoticons, still or animated, will greatly
help to produce barrier-free communication tools for the hearing-impaired or orally-challenged, innovative
instruments for education, new communication media for international tourism, etc.
Figure 6: Simple Animation Showing the Situation “A man in a small boat got sea-sick.”
(Actually a single frame: the face of the man gradually turns darker.)
Educational Application
Let us consider that we have a set of digital pictures together with simple animations to teach the core vocabulary of
a language. If several different languages have a similar set of core vocabulary items, then we will be able to create
a hyperlink between a picture and the corresponding word or the phrase of each language. If this is implemented
electronically, it will be rather easy for us to create an electronic multi-lingual dictionary.
This looks just like a multi-lingual version of the famous “Duden Bildwörterbuch” but as pointed out in Azuma
(2007), the big difference is that the picture or the simple animation and the lexical explanation in the language do
not exist in the same object (for example, on the same page of a dictionary made of paper). The lexical explanation
in each language exists as metadata outside the picture, as shown in figure 7, and this means that the same picture or
animation can be referenced and used as many times as required. If the phrase “read a book carefully” is included in
the electronic multi-lingual dictionary, we can again use the same picture object for the “book” to illustrate the
meaning of the phrase.
book
lang=”en”
hon
lang=”ja”
lang=”de” Buch
Figure 7: Picture/Animation Object and Multi-lingual Lexical Explanation as MetadataOriginal printed in: Proceeding of World Conference on Educational Media, Hypermedia and Telecommunications
(ED-Media), 2008, p. 972-977
By expanding the idea of this multi-lingual dictionary, we will also be able to implement a simple universal visual
language editor. It can be built in a small gadget such as a mobile phone or some kind of PDA. One can first
choose the language mode (English or French or Chinese, etc.) according to the native tongue of the user, and then
he or she will write some sentence in their own language. The editor will then translate it into a sequence of well-
formed universal visual language. As the picture/animation-like visual language is quite easy for all the people in
the world to understand, we may be able to get rid of language barriers if we communicate with the universal visual
language. The receiver of the universal visual language can easily convert it into their own language if necessary
and all the translation tasks will be done by the editor. If you want to know the meaning or the pronunciation of a
certain visual symbol in a foreign language, you can switch to the mode of the desired language and then just touch
the symbol or the animation with your finger. The editor will tell you the corresponding word or phrase and, if
necessary, it will read it with a TTS-generated speech sound. There have been great developments in the technology
of language-to-language translation systems up to this point, but this type of universal visual language editor will be
another option in next-generation communication, or for use as a learning tool.
Conclusion
In this paper, possibility of implementing a universal auxiliary visual language was discussed, based on the current
popularity of graphic emoticon usage in Japanese social network services, blogs and mobile phone communication
and increased use of graphic emoticons as ideographic images as substitutes for lexical items. Results of an
international project verifying the international communicability of graphical emoticons have also supported the
assumption that international communication using graphic emoticons is on some level possible.
However, as a matter of fact, the idea of the computer-mediated universal auxiliary visual language is not new.
Lennon and Maurer (2001) already proposed the prototype of the universal auxiliary visual language called MUSLI
(A MUlti-Sensory Language Interface) and MIRACLE (Multimedia Information Repository, A Computer-supported
Language Effort) was the expansion project of MUSLI (See Maurer et. al., 2003). The important thing is, as pointed
out through the stylistic analysis of graphical emoticons used in cyberspace, we have more and more improvement
of technological infrastructure where the universal auxiliary visual language is supposed to be implemented.
In the near future, public signs, instructions for vending machines, automated teller machines or other interactive
machines will be using the universal visual language. The hearing-impaired or orally-challenged people will also
enjoy the benefit of this new visual language. Even the people who speak different languages will be able to
communicate through the email, blogs or social network services just by using the symbolic signs and a meeting in
the multi-lingual situation will be much easier because of the new type of communication tool compatible with
universal visual language. In fact, as Maurer et. al. (2003) argues, within just 10 years, we will have achieved a
paperless society and will be carrying a small but powerful computer capable of presenting high resolution movie
images that is compatible with the new type of visual language. Though we are not sure if today's graphic
emoticons will evolve into the next-generation universal visual language, it will be at least worth pondering the
possibility of such an innovative communication system of the future.
References
Azuma, J. (2007). Application of Pictures and Animation to Language Teaching in the Digital Age, Phraseologie
Kontrastiv und Didaktisch, Filozofske fakultete Maribor, 71-82.
Lennon, J. A. and Maurer, H. (2001). MUSLI: A hypermedia interface for dynamic, interactive, and symbolic
communication, Journal of Network and Computer Applications, 24-4, 273-291.
Maurer, H., Stubenrauch, R. and Camhy, D. (2003). Foundations of MIRACLE - Multimedia Information
Repository: A Computer-based Language Effort, Journal of Universal Computer Science, 9-4, 309-348.
Mehrabian, A (1971). Silent Messages, Wadsworth.You can also read
