Home
News
You are here

Dummy guide to Emoji: History, Nature and Usage

By Damian M.

Updated: Dec 09, 2016, 1:25 AM

9comments

Dummy guide to Emoji: History, Nature and Usage

Emoji have become a standard shortcut in communication across the globe. You can see them in just about any social media post, text message or online comment. And while we all use them, not many of us ask themselves how they work and where they came from.

Well, brace yourselves, because you're about to get these questions answered and then some.

Emoji and Emoticons – no difference there, right?

iOS, Google, Microsoft and Samsung (top to bottom)

Wrong, actually. There's a significant difference between the two, despite the fact that a lot of people call Emoji Emoticons and vice versa.

Emoticons are pictures, made up of standard characters. Like the most popular smiley face, which consists of a colon, a hyphen and a bracket, and looks like this: :-). It's a crude and completely improvised way of expressing emotions in text. But it works.

Emoji, on the other hand, usually consist of a single Unicode character. More on Unicode (and why I say “usually”), later. The smiley face emoticon from above can be expressed with the “Slightly Smiling Face” Emoji, which is depicted on the right. Do note that this Emoji will look different on different devices, unlike the smiley face Emoticon above, which will look exactly the same.

And there are also these things called Stickers, introduced in modern IM platforms. They are a whole different subject in general, but some IM developers insist on calling them Emoji for reasons, completely unknown to mankind.

Now, let's forget about Emoticons and Stickers for the time being and focus on the task at hand.

Where did these Emoji come from?

Okay, I promise I will get to the origin of Emoji soon. Bear with me for a minute, because there will be a short prelude first.

Once upon a time, the Tower of Babel was a thing, then it suddenly wasn't and people started speaking different languages. No matter do you believe the previous sentence or not, it's a fact that with the dawn of the digital age, different languages gave birth do different keyboards and different character encoding standards. These encoding standards had the sole purpose of transforming letters, numbers and other characters into binary code (ones and zeroes) for computers to understand, then reverse the process on the other end.

But, if you lived in Russia, and you wanted to send a message to your friend in Afghanistan, you had a problem. And a serious one at that. The difference of encoding standards meant that your Afghan friend might see complete gibberish appear on their screen, because the bits generated from your Russian encoder would mean something different for your friend's Afghan encoder.

Enter, Unicode Consortium. The Consortium is a non-profit organization with the sole purpose of making one unified encoder that will understand all characters of any language the same way on both ends. This way, when you send a message from the US to Israel, it will look exactly the same on both ends.

Now, that's wonderful in theory, but it turns out to be quite a difficult task to achieve in practice, because the Consortium wanted to have backwards compatibility, too. The idea was to convert text to Unicode, then convert it back into your country's old standard, and suffer no losses of characters or meaning at all.

Shigetaka Kurita, Father of Emoji

Now, finally, we'll talk about Emoji. In the late 1990s, Emoji were a Japan-specific phenomenon. No one outside of Japan really knew about them. They were introduced by Japanese cell phone manufacturers, because the local character encoders had some space to spare (it's actually a bit more complicated than it sounds, but it doesn't really matter). So, with no new glyphs to add to the set of characters, an engineer by the name of Shigetaka Kurita thought to himself, “hey, I know. Let's allow people to send tiny images to each other”, and created the first Emoji.

The first set consisted of barely 172 Emoji, each with a size of 12x12 pixels. A modest collection, compared to the thousands and thousands of glyphs we have available right now. But despite the limited catalog, Emoji took off in Japan and swept across the nation, because they allowed to send images that required the same bandwith as text. Tiny, ugly and basic images, but images nonetheless.

So far, so good. But how does this tie in with the boring Unicode thingy we talked about? Well, the Unicode Consortium went from one country to the other, adding each alphabet to their list of characters, carefully assigning each letter a number (or code point) and everything. Then it reached Japan and started adding its characters to its ever-expanding catalog.

At some point someone must have stopped and said “Wait, that's a picture of a rice bowl”. And this is when the Unicode Consortium had to choose. Either leave the Emoji out of the Unicode standard and risk sending incomplete messages every time someone converts from the Japan-specific standard to Unicode, or add the Emoji on top of what was already developed by the Consortium. Needless to say, the folk behind Unicode chose the latter.

Okay, everything is falling into place now. The Emoji exist, the Unicode standard gave them to the world and they are suddenly viral, right? Well, no.

iOS 5 was the first iOS with Emoji Keyboard

No one outside Japan used Emoji, despite them being available to everyone. It's just that no one knew about them, because they were hidden behind so many menus and under so many characters that most users never actually saw them. Not to mention that they were supported by most mobile devices, but you had no actual way of typing them.

Then Apple came along. The company decided that it wants to sell its iPhone in Japan, so it had to include the (fairly well-hidden) option of typing Emoji, so Japanese users can enjoy the same smiley face support they've had up until then. And this is when it took off.

Someone, somewhere around the globe, must have noticed the new option on their iPhone and sent a pile of poo to half of their contacts. Then the recipients of said pile of poo must have asked how the sender did it. And it became viral.

Okay, I get it, but how do Emoji work?

Emoji work exactly the same way as regular text – a Unicode code point corresponds to every character in the Unicode catalog, Emoji included. When a device sends a message, it sends a series of Unicode code points. When another device receives said message, it interprets the code points and displays letters, numbers and Emoji.

It's actually a bit more complicated than that, but this is the basic principle. However, if what I said is true, Emoji should look exactly the same on all platforms, right? But we all know that's not the case. Then how can I be correct?

The answer is rather simple. Every platform has its own Emoji font of sorts. Just like you can set your IM client to display incoming messages in Comic Sans MS, while the user on the other end is typing them in Arial. The same code points are sent and received, but they are displayed in a different style. It just so happens that, when it comes to text, the standard fonts to use are all the same. While Emoji, on the other hand, have no standard font that every platform relies on. Instead, different fonts are created by different companies.

And just like the letter "A" differs in style between Lucida and Times New Roman, so do Emoji look different on iOS and Android.

This is also the reason for some platforms not to support Emoji altogether, because they have no dedicated font or “pictures” for it and they display blank squares, question marks or other such symbols.

If every Emoji is a single character, why does Twitter count the Police Officer emoji as two?

Ah, a very good question. See, the “Police Officer” Emoji is actually one character. But you probably want to type in a police officer with dark skin, right? Well, there's this thing called “Skin Tone Modifier”, which is a whole separate Emoji.

Basically, instead of typing just “Police Officer”, you'd type “Police Officer” + “Dark Skin”. And since the skin tone modifier is a separate Emoji, you actually type in two characters.

What about the Female Police Officer with Dark Skin, though? This takes a whooping total of five characters! Here we have something else at play, called Zero-width Joiner (ZWJ), which is a Unicode character by itself.

These female officers cost us five Twitter characters each.

ZWJ was first used in languages that had to join characters in a different way, depending on other characters before and after them. Arabic would be a great example, as a lot of characters there look different in combination with other characters.

Same goes for Emoji. The ZWJ puts the Police Officer, Female and Dark Skin Modifier Emoji into one picture. So, what is actually typed in Twitter is “Police Officer”, ZWJ, “Female”, ZWJ, “Dark Skin Modifier”. You could think of ZWJ as the plus sign of Emoji.

If two Emoji can exist on their own (excluding Skin Tone Modifiers), they need a ZWJ to display as one picture, instead of two. This is called a ZWJ Sequence. And since the Skin Tone Modifier in the example is also a part of the sequence, it needs to follow the format and requires a ZWJ too, despite not needing it when it's not in such a sequence. This makes for a total of five Unicode characters and makes you limit your thoughts about that female cop to 135 symbols, instead of the 139 you expected.

We couldn't find a ZWJ image, because it's invisible. So, here are some cute hamster Emoji.

Confused yet? Then don't even get me started on the different family variations that take up to 7 characters, because they are a ZWJ sequence of four different family members. They have no skin tone modifiers support just yet, but you can do the maths of what they'd take from your Tweet, when they inevitably do. Spoiler alert – 14 characters, if separate skin tones are applied to each family member.

And if that wasn't enough to make your mind explode, there are also Emoji that can be displayed as text. For example the "Heavy Black Heart" Emoji. It does look red on all platforms, but this doesn't mean that it got the wrong name. In fact, it's based on the solid heart text symbol, which is black by default (︎). So, platforms need to know that we want the Emoji and not its text representation.

In order to do that, the "Heavy Black Heart" Emoji consists of two Unicode characters. One is the heart we already covered, and the other is an invisible symbol, called "Variation Selector-16". Said invisible character basically tells the system that the preceding characters should be displayed as Emoji and not text. Therefore, ︎ + "Variation Selector-16" = ️.

And this is still not the end of it. Emoji country flags are a whole different subject. In fact, they're not even flags, but a combination of two Unicode characters, called "Regional Indicator Symbols". These are, basically, the letters from A to Z, each surrounded by a rectangle of dots. And when you want to type in the flag of the United Sates of America, you're basically typing the letters U and S, each surrounded by dots, which is two symbols. And the Unicode Consortium chose this approach, so it won't have to deal with new flags popping up left, right and center with new countries forming each year and old countries changing flags, when they change regimes. The Consortium has covered all possible country code combinations and lets Unicode-based platforms deal with the rest.

Crazy, huh? That's the price of Unicode being universally applicable to anyone, though. And if you ask me, it's well worth it.

The original Emoji set.

Why did I need to know all this?

Because knowledge is power, I guess. And next time someone gets angry that their shooting range invite was displayed with a water pistol, you will know how it happened. Not to mention that this explains how the Love Hotel Emoji got there in the first place. Hint, such establishments are pretty popular in Japan.