Digitally Mediated Communication of Emojis By Visually Impaired Users: The Case of Sarcastic WhatsApp Messages

1. Introduction

With the advancement of technology, digitally mediated communication (DMC), such as e-mail or instant messaging, has become part of the fabric of daily life. Since DMC lacks non-verbal cues such as facial expressions, intonation, and gestures, communicators have had to develop new non-verbal cues. Symbols for facial expressions are represented by capitalization and exclamations for shouting (Harris and Paradice, 2007; Riordan and Kreuz, 2010). Emojis are another convention included in this issue of DMC, which is well-suited to social media communication (Barbieri et al., 2016).

Due to the prevalence of emojis in social media communications, visually impaired social media users are encountered emojis often. It was reported that 93.1% of blind or low-vision users encounter emojis each month (Tigwell et al., 2020). In Saudi Arabia, there are around 375,000 visually impaired people with different types and degrees of impairments (General Authority for Statistics, 2017). With assistive technology, many blind and visually impaired individuals use screen readers, also known as text-to-speech, such as VoiceOver (for iOS users) or TalkBack (for Android users), to interpret information on the screen. When reading a message that includes an emoji, a description of the emoji is read out loud. For emojis output, there are emoji keyboards that can narrate to them, so they are able to read all the emoji lists on the display screen and select the one they prefer to use. This accessibility support for screen readers and entry methods has made it possible for VIU to use emojis easily.

Since many emojis are a direct replacement for facial expressions, gestures, and other non-verbal cues to the user’s emotions and intent, it may be assumed that VIU would not find it difficult to communicate through, understand, and produce emojis. However, the functions of emojis have developed beyond providing a mere representation of a user’s face, gestures, or body language (Scott, 2022). Emojis do not always correspond to a specific meaning, so they are often employed creatively. Emojis are capable of performing a variety of roles, and they are often considered to help fill communication gaps in written forms. It is possible to interpret messages more accurately with the use of an emoji, for example, to indicate a joke, a positive attitude, irony, or mark humor or sarcasm. This prompts the receiver of the message to infer what that intended meaning is, rejecting the literal meaning and considering alternative meanings or intents. Such limitations hinder VIU from using emojis easily, causing social exclusion for visually impaired users and reducing their communication efficacy.

2. Related Work

The first appearance of emojis was in Japan in the late 1990s, and the roots of the present-day emoji can be traced back over thirty years (Scott, 2022). Emojis, which is a Japanese word for

a “picture word”, are defined by Pavalanathan and Eisenstein (2015), as a “picture character”. In 2015, emojis were publicly recognized by Oxford Dictionaries. The much attention received for studying emojis in numerous fields makes it a hot topic for scholars. Several studies have attempted to explore the use of emojis as a mode of communication from different angels, for example, from a socio-semiotic view (Alshenqeeti, 2016) or from a global (Arafah & Hasyim, 2019), cultural stand (Kimura-Thollander & Kumar, 2019). Other empirical research studies explored the pragmatic functions of emojis (e.g., Li & Yang, 2018 & Maíz-Arévalo, 2015…etc.). Some other scholars focused their investigation on the emotional meanings of emojis. Cheng (2017) affirms that emojis tended to be employed in positive messages and far less in sad or angry messages. User’s/recipient’s attention and responses can be affected by emojis (Hjärtström et al., 2019).

It has been reported that the inclusion of emojis in a text message would assist Internet users to perceive the correct attitude, emotion, and intent of the message. Further, emojis were also found to have pragmatic functions, and different types of emoji could serve different pragmatic functions in different communication settings, for example, politeness strategy (Kavanagh, 2016), face-saving strategies (Maíz-Arévalo, 2015), and of rapport among interlocutors (Maíz-Arévalo, 2015) as well. In short, emojis can clarify message intention, compensating for the absence of nonverbal cues in written communication. (Thompson & Filik, 2016).

Communication through using emojis is just like any kind of communication. To be successful, it requires sharing the same understanding, knowledge, and assumptions and adhering to similar rules of cooperative interactions. In the discourse of emojis, when a sender communicates something, the receiver interprets what she/he believes the sender’s intent is. If the receiver fails to interpret the message as what the sender intended, miscommunication may occur. Thus, the realization of a message that contains an emoji can take a locutionary meaning or illocutionary force. It carries the locutionary meaning based on the clear meaning that the statement, along with the emoji, takes. If the meaning has its force on the sender, then it takes the illocutionary force.

The use of emojis can enhance the clarity of a message or clarify its intent (Cramer et al., 2016). Still, emojis can change the interpretation of a message. The face with tears of joy emoji might be used to indicate a joke or sarcasm, to mitigate the force of a complaint, or to simply indicate that a humorous message has been received and appreciated. Al-Rashdi’s (2018), study examined the functions of emojis in naturally occurring WhatsApp conversations between Omani men and women, friends and relatives. The results demonstrated that emojis do not only serve as indicators of users’ emotions but also serve other communicative functions, such as an indication of celebration; an indication of approval of others’ messages; responses to expressions of thanks and compliments; conversational openings and closings; linking devices; and an indication of the fulfillment of a requested task. Since the investigation in this research is on the use of emojis to clarify sarcastic versus literal intent, the next discussion is merely on this function.

Emojis are strongly associated with jokes and informal discourse. A recent study by Thompson and Filik (2016), clearly demonstrated that tongue and wink faces are the most reliable indicators of sarcasm. The concept of irony is usually expressed in a way to convey the opposite of what was intended. Suppose Julie suggests fixing a broken cup with sticky tape, and Chris says great idea! By being indirect, the speaker increases the likelihood that his or her intentions will be misunderstood. The ironic intent of Chris might have been expressed through appropriate voice tones or gestures if this scenario had been spoken. In a written message, however, the absence of these nonverbal options may lead Julie to mistakenly believe that Chris's idea is great. Previous studies have confused the terms sarcasm and irony; however, sarcasm is generally accepted as an ironic method of expressing negative feelings toward someone (Kreuz & Glucksberg, 1989). This form of irony is the focus of the current research.

Focusing on visually impaired users, the available literature is still scarce in focus on this population in regard to the use of emojis. Among these, Tigwell et al. (2020) studied how VIU use and encounter emoji online and investigated the challenges they experience. The authors used online questionnaires and sequential interviews with screen reader users to understand more about the challenges that emerged from the survey findings. Issues that ultimately resulted in social exclusion were raised.

3. Significance Of The Study

It has been emphasized in the literature that emojis introduce several specific challenges for VIU in social communication (Tigwell et al., 2020). Despite the ample research conducted on emojis, no single research study, to the best of our knowledge, has attempted to explore the cultural and pragmatic functions of emojis by VIU. This study intends to fill in this gap in the literature. Previous studies have primarily used rating tasks, multiple-choice questions, or corpus analysis. By utilizing free production tasks, the study provides clear empirical data on the use and function of emojis in written communication that contribute to the existing literature. Participants in this study were directed to produce messages guaranteed to be correctly interpreted by the receiver as either literal or sarcastic rather than being receivers or observers of messages.

4. Aim Of The Study

Filling the gaps in the literature, this study aims at examining the use of emojis by VIU in digital contexts. Specifically, focusing on Saudi WhatsApp users, the aim is to uncover how VIU uses emojis in WhatsApp messages to clarify sarcastic versus literal intent. It is from a linguistic perspective that this research focuses the investigation.

5. Study Participants

Convenience sampling was followed to recruit the participants in the present study. Meeting specific criteria of eligibility, a total of forty-seven visually-impaired WhatsApp users participated in the study (23 males 49% and 24 females 51%). They all were Saudi native Arabic speakers from different age groups, with the majority aged between 18–40). Twenty-five of the participants were partially visually impaired, while twenty-two were blind fully. Table 1 shows the subjects’ demographic information.

6. Ethical Considerations

All ethical requirements of the data collection methods were applied. Participants were familiarized with what they were taking part in, and they were given informed consent to their participation prior to taking part and were free to withdraw at any time. The ethical considerations covered anonymity, in which no identifying information was collected. The online data collection method used was set to anonymize responses in which participants’ personal information, such as IP address and location data, were not collected.

7. Study Methodology Instrument

Rigorous use of appropriate research methods is required for developing an understanding of sarcasm in digitally mediated communication and online pragmatics. Scot (2022: 150), stated that “one way to investigate the ways in which the discourse context affects language use is to control the context and ask users to produce utterances.” The researchers in this study used two related task experiments to compare how users behave and perceive different contextual conditions.

8. Procedures

To take part an invitation to take part in the research was sent out to the public through the researchers’ Facebook and Twitter accounts. The invitation included a description of the target population and the inclusion criteria so as to include responses only from the intended participants. The participation consisted of two parts. Firstly, the participants were directed to fill in their demographic information through an enclosed Google Forms link. The first part of the form contained questions that were intended to ensure the eligibility of the participation. Further, it contained a consent form and information about the experimental tasks. Proceeding to the next part was for those who reported that they use emojis and WhatsApp. The second part was for the two experimental tasks in which the participants were instructed to imagine conversations as being between themselves and a friend via WhatsApp.

9. Study Design

This study follows the instrument design in Thompson and Filik’s (2016) study. Again, the experiment consisted of two tasks. These are detailed next. The items of the tasks were written in simple Saudi Arabic that can be read and understood by the target population. However, a pilot study was conducted to ensure the clarity and readability of the items in the two tasks and the ease of the system used. The pilot study was administered to twelve people with similar characteristics. They were asked to answer the questions and report any items they found unclear. Modifications were made in accordance with their responses and suggestions.

9.1 Task 1

It was the objective of Task 1 to examine the effects of sarcastic as opposed to literal messages, criticism as opposed to praise, as well as the combination of both. Following the methodology of Thompson and Filik (2016), task 1 consisted of six sets of scenarios, each with four variants (Table 1 is an example). Participants were asked to imagine the scenario as a sequence of WhatsApp messages between them and one of their friends. The task's main objective was to manipulate the factor of intention by modifying the last line of each scenario so that it is made clear that the intent behind the comment is either literal or sarcastic. The last line of each conversation was either superficially positive (e.g., Brilliant idea!) or superficially negative (e.g., Horrible idea!). The students were instructed not to change or add any words but rather to consider alternative ways of expressing their intentions. To avoid priming the use of emojis and to make the experiment's aim clear, these instructions were written without direct mention of emojis.

As shown in Table 2, there are two instances representing the manipulation of the Polarity factor (praise vs. criticism). An intention to praise remains positive in a superficially positive comment when it is meant seriously. Second, sarcastic comments convey a negative message, thus turning superficially positive comments into criticism. A superficially negative comment has the opposite effect.

9.2 Task 2

The first task examined the frequency with which emojis were used in marking sarcastic and literal intent, as well as praise and criticism, relative to one another. In task 2, the aim was to examine the perception of a WhatsApp message that includes an emoji within a conversation. The scenarios were similar to those used in Task 1, but the materials had two variants (as in example 1). In each conversation, the final comment would include emojis intended either as literal or sarcastic comments. As part of the task, participants were required to judge perceived intentions as either serious or sarcastic. Thus, the factor of intention (literal or sarcastic) was manipulated. Task 2 consisted of 12 items, each with two conditions/options (i.e., literal vs. sarcastic). To complete this task, the participants used their screen readers.

Example (1): A scenario used in Task 2 (with two conditional variants)

Friend: So, how was the exam? بشر كيف كان اختبارك

You: I really can’t tell... والله ما ادري وش أقول لك

Friend: Well, you didn’t look confident 🤣😁شكلك جبت ام العيد

10. Data Analysis

Two native speakers of Saudi Arabic were recruited as raters for this study to rate the responses. The two raters were given a rating scale. They were instructed to give a 0 point for each item left with no use of emoji or, in the case of containing an emoji but inappropriate one(s), and 1 point for an acceptable answer. The data gathered from the experiments were analyzed quantitatively, descriptively, and qualitatively. IBM Statistical Program for Social Science (SPSS® version 26) served the quantitative analysis items in terms of production probability, frequency, percentage, mean, and standard deviation (SD). T-test and correlation analyses were methods used in the analysis of the data. Further, another layer of analysis was needed. A"coding and counting" paradigm of classical content analysis (Herring, 2004, p.2) was used for emoji occurrences and frequencies.

11. The Finding Of The Study

The probability/likelihood of producing an emoji revealed that there is an effect of the intended meaning. As Table 1 illustrates, emojis were significantly more likely to be included in literal messages than in sarcastic ones. the message polarity also has a main effect on emoji production. Emojis were more likely to occur in praise cases than in criticism.

Task 1 investigated how emojis are employed for the purposes of clarifying meaning in literal versus sarcastic messages, as well as in praising and criticizing versions of each. First, although there were no explicit directions to use emojis in this experimental task, the number of participants who employed emojis to alter the messages and indicate intention was 42. The total number of emojis produced by the participants in Task 1 was around 540, and about 6 unique emoji forms or types were used. This included positive emotion emojis such as happiness, joy, and excitement (😃🤣😁), neutral/moderate emotions such as (😶😑🫤), and negative emotion forms such as being sad, angry, and upset (😼😠🤬😡). Table 4 shows the categories of the most used emojis and their definitions.

As shown in Table 5, participants tended to use emojis for the aim of expressing the intended interpretations. The literality of messages seemed to be linked more with emojis than sarcastic intended ones. In fact, many of the emoji instances were appropriately used to indicate the right intention. The mean number of emojis used appropriately to indicate the intent behind the comment is literal praise was 4.19 (out of 6), and 4.36 (out of 6) for literal criticizing comments. The situation was different for sarcastic messages. The participants used emojis in both praise and criticism sarcastic scenarios with less appropriacy (mean = 2.4 for praise and mean = 2.31 for criticism messages).

For further meaningful analysis, the qualitative analysis of the emojis used showed that the winking face with the tongue out (😜) and a squinting face with the tongue out (😝) were the most frequent emojis signifying sarcastic tone messages. And the slightly smiling face emoji (🙂) was prevalent in literal remarks and literal praise.

One additional key finding that emerged in the analysis was the participants’ impairment type and its association with the use of emojis. As shown in Table 6, there is a link between the user’s type of impairment and their production of emojis. Relying solely on screen-readers, fully blind users tended to produce fewer appropriate emojis in general, and their use of emojis in sarcastic messages was barely appropriate.

Shifting the focus, Task 2 examined the interpretation of WhatsApp messages that are attached with emojis within a conversation in similar scenarios to those used in Task 1. As shown in Table 7, literal messages enclosed by emojis were interpreted more accurately by VIU (mean= 7.99; SD= 0.84) compared with sarcastic situations (mean =5.15; SD =1.05).

Additionally, there found a clear association between the impairment type and users’ interpretations of the perceived meaning behind messages with emojis. Similar to the findings of Task 1, participants with partial impairment outperformed their fully blind counterparts (see Table 8).

The analysis also revealed a gender difference, although this was not the primary focus of the research. Emoticons and other communication tools have been discussed in the literature in relation to gender effects. Gender in this study was further examined to check if it deems an additional predictor of emoji production and perception. Data in Table 9 and Table 10 illustrate further details.

Researchers did not measure whether or not individual emoji categories are more likely to be produced in relation to gender. As Table 9 shows, gender seemed to not interact with literality. Both males and females acted similarly in both literal and sarcastic situations, as statistically, there are no significant differences at the level of significance 0.001 between male and female emoji users. And the same is true for the task of interpreting the messages (as in Table 10). No statistically significant differences were found between male and female VIU.

12. Conclusion

The aim of this study was to provide empirical evidence regarding VIU's use of emojis to clarify and interpret their intentions in written online communications. Based on the findings of this study, several major conclusions can be drawn. First, emphasizing the essentiality of emojis in online communication, the high frequency of emoji use by VIU can be attributed to its high functionality and efficiency in compensating for the absence of nonverbal cues in written communication and clarifying instant message intention. This finding is supported by Li and Yang (2018). Second, emojis tended to be likely linked more with literal WhatsApp messages than sarcastic ones. There was a greater prevalence of emoji use in praise situations than in criticism, which shows how message polarity influences emoji use. Emotions and interactions are primarily signified by emojis. Furthermore, tongue and wink emojis are often associated with sarcasm, whereas smiling faces are often associated with literal comments and praise. This finding comes in agreement with what is evident by Thompson and Filik (2016). Additionally, distinguishing sarcasm from the sincerity tone of a WhatsApp message was more challenging to VIU, and it becomes even more problematic to fully blind ones.

Other observations from the data analysis include the following. Impairment degree is linked with both how emojis are produced and how they are perceived. Participants with partial impairment are better users of emojis than their fully blind counterparts. Furthermore, confirming the findings of Thompson and Filik (2016), in that gender seemed to not interact with literality. In sum, emojis are undoubtedly a popular and important way of communicating in everyday social interactions on social media networks.

13. Limitations And Suggestions For Future Research

The current investigation was subject to some limitations. The primary limitation lies in the nature of the scenarios used. This research used mocked scenarios in which the participants were requested to imagine real WhatsApp conversations. Acknowledging the limitations of such an emoji-completion task, collecting naturally occurring social media conversations to examine VIU’s use of emojis would be ideal for future research. However, the purpose of the current study has been served. Including additional methods, such as interviews or focus groups, to provide a more comprehensive understanding of how VIU use emojis in WhatsApp communication could also yield additional interesting findings. Furthermore, to enhance the generalizability of the findings, it may have been necessary to consider expanding the study to include a larger and more diverse sample of VIU across different regions and cultures. Despite this, the sample size in this study is representative.

For future research to explore the use of emojis by VIU, additional suggestions can be made. Future research could explore how different types of emojis and their combinations affect the interpretation of sarcasm and literal intent in WhatsApp messages by VIU. Additionally, scholars could investigate how VIU perceive and use other forms of digitally mediated communication, such as voice messages or audio descriptions of emojis. Lastly, this study focused merely on the pragmatic functions of emojis in e-discourse. The pragmatics of other online communication/interaction devices, such as tagging or other image-based forms, could be targeted for the investigation of VIU communication. Importantly, focusing the investigation on reaction GIFs and memes would yield interesting findings considering the inaccessibility of such graphics to VIU as screen readers are limited in this.

Conflict Of Interests

The authors declare that they have no conflict of interest.