- Open Access
An analysis of emotion-exchange motifs in multiplex networks during emergency events
Applied Network Science volume 4, Article number: 8 (2019)
In this paper, we present an analysis of the emotion-exchange patterns that arise from Twitter messages sent during emergency events. To this end, we performed a systematic structural analysis of the multiplex communication network that we derived from a data-set including more than 1.9 million tweets that have been sent during five recent shootings and terror events. In order to study the local communication structures that emerge as Twitter users directly exchange emotional messages, we propose the concept of emotion-exchange motifs. Our findings suggest that emotion-exchange motifs which contain reciprocal edges (indicating online conversations) only emerge when users exchange messages that convey anger or fear, either in isolation or in any combination with another emotion. In contrast, the expression of sadness, disgust, surprise, as well as any positive emotion are rather characteristic for emotion-exchange motifs representing one-way communication patterns (instead of online conversations). Among other things, we also found that a higher structural similarity exists between pairs of network layers consisting of one high-arousal emotion and one low-arousal emotion, rather than pairs of network layers belonging to the same arousal dimension.
A crisis event is a sudden event that creates threats to people, property, and /or the environment (Shaluf et al. 2003). Though crisis events come in many forms, including human-made (terrorism, riots, shootings), natural disasters, organizational crises, technological crises (e.g., software failure, industrial accidents) (Farazmand 2016; Lerbinger 1997; Seeger et al. 1998) the common feature of all such events is the sense of urgency (Farazmand 2016) as well as the accompanying feelings of panic, fear, danger, and shock (Darling 1994).
During crisis events, human behavior and the attitude towards the crisis is influenced by the information people have access to (Bakker et al. 2018). For example, Panagiotopoulos et al. (Panagiotopoulos et al. 2014), Starbird et al. (Starbird et al. 2013), and Sutton & Shklovski (Sutton and Shklovski 2008) reported on the negative effects of rumors and misinformation during various crisis events, highlighting the importance of information that is issued by official sources (Crump 2011; Heverin and Zach 2010; Huang et al. 2017; Waters and Williams 2011).
Sutton & Shklovski (Sutton and Shklovski 2008) suggested that crises not only create a need for information, but also for human conversation. In this sense, online social networks (OSNs) are seen as a supporting medium for information sharing in such unexpected events. In fact, recent studies have found that OSNs play an important role during crisis events as a channel where people seek and share information to make sense of the situation, as well as organize themselves and look for help. For example, the use of online social media in crisis events has been investigated during natural disasters such as the 2009 Oklahoma grass-fires (Vieweg et al. 2010), the 2009 Marseille fire (De Longueville et al. 2009), or the 2011 and 2013 floods in Brisbane (Hung et al. 2016), as well as in terror attacks such as the 2013 Boston Marathon Bombing (Guo 2017) and the 2015 terror attacks in Paris (Cvetojevic and Hochmair 2018), or the riots during the 2017 G20 summit in Hamburg (Kušen and Strembeck 2018a).
Today, Twitter monthly counts over 300 million active users (Valenzuela and Binns 2018). Due to its active community and growing popularity, Twitter has become an important platform for message dissemination. As a result, the large volume of data sent via Twitter provides a valuable source for gaining insights into various aspects of user behavior, such as information dissemination, the flow of emotions during an event, as well as the structural patterns that emerge via user interactions.
In this paper, we focus on the analysis of crisis events resulting from shootings or terror attacks. Such events are characteristic for a chain of shorter threatening episodes (Farazmand 2016) and, unlike natural disasters, often happen unexpectedly without a prior warning.
A common feature of shootings and terror attacks is that they evoke high emotional reactions. In this respect, OSN content is a valuable source of data for analyzing such reactions. As pointed out by Balon and Rime (2016), people consistently tend to use more emotional wording when writing messages as compared to the direct (face-to-face) sharing of emotional experiences. As a possible explanation to this finding, Balon & Rime (Balon and Rime 2016) refer to non-verbal emotional expressions such as one’s body language, vocal tone, and facial expressions that a person elicits when communicating face-to-face, while in textual messages written cues (textual expressions, smileys) are primary emotion carriers.
With respect to emotional expressions resulting from crisis events, related studies have shown that one’s expression of negative emotions serves as a coping mechanism and improves a person’s mental well-being (Neubaum et al. 2014). However, not only negative emotions (such as anger and fear) are shared on social media but also a number of positive emotions, such as joy and relief. In fact, prior studies have shown that in negative events positive emotions may even prevail over negative emotions because they help reduce stress, lower the intensity of high arousal emotions, and increase the feelings of hope, gratitude, and compassion, as well as empathy (Guo 2017; Folkman and Moskowitz 2000; Fredrickson et al. 2003; Kim and Niederdeppe 2013; Kušen et al. 2017b). This human tendency to use positive emotions as an antidote against the effects of negative emotions is referred to as the undoing hypothesis (Fredrickson et al. 2000). In the context of online social media interactions, such a use of positive emotions may serve as a public coping mechanism (Guo 2017; Kušen et al. 2019).
Although the existing body of literature on the use of online social media during shootings and terror attacks provides valuable insights into human behavior during highly emotional events, there is a lack of studies on the structural patterns that emerge from the communication behavior of OSN users. In this context, triads (subgraphs consisting of three vertices) serve as basic building blocks of more complex patterns. The study of triads already started in the 1950s, when the German sociologist Georg Simmel examined the role of triads in (offline) social networks (Simmel 1950). In particular, Simmel distinguished between dyads and triads, arguing that an addition of a third member to a dyad significantly changes group dynamics.
When analyzing online social networks, such triads take the form of 3-node-subgraphs that differ with respect to the direction of the edges in these subgraphs. In this context, a network motif represents a subgraph pattern which occurs significantly more often in a real-world network as compared to the subgraph patterns identified in a corresponding null model (Milo et al. 2002). In 2002, Milo et al. were first to propose network motifs as basic building blocks of complex networks (Milo et al. 2002). Since then, network motifs have been extensively applied to various types of biological networks, such as gene networks (Alon 2007), metabolic networks (Beber et al. 2012), and protein-to-protein interaction networks (Yeger-Lotem et al. 2004). In recent years, the concept of network motifs has also found its application in other domains. For example, Tran et al. (2015) found that networks across different domains, such as protein structure networks, animal (social) networks, and co-authorships of scientific papers share a common set of motifs. However, up to date, there is a lack of studies which apply network motifs to the study of interaction patterns over online social media, and in specific patterns that emerge as people communicate emotions.
In this paper, we provide an empirical investigation of the interaction patterns that emerged on Twitter during five recent shootings and terror attacks. In particular, we focus on the temporal emergence of motifs that appear when users communicate via emotionally-charged messages. We call such statistically significant subgraph patterns emotion-exchange motifs. For our analysis, we derive a communication network where each node represents an OSN user and each directed edge represents a message that has been sent from one node to another. In order to perform an in-depth analysis, we identify different types of edges, each corresponding to messages that convey one of the eight basic emotions found in Plutchik’s wheel of emotions (Plutchik 2001) (anger, disgust, fear, sadness, joy, trust, surprise, anticipation). In particular, we propose a multiplex network as a framework to study the emergence of motifs during highly emotional events.
Our paper focuses on two aspects: First, we examine whether common motifs exist that are shared across multiple topic-wise related events (in our case: shootings and terror attacks). Second, we examine temporal aspects of the functional roles that different motifs play in each of the events we analyze.
Our findings suggest that a variety of emotion-exchange motifs emerge across multiple data-sets as people exchange emotional messages during shootings and terror events. The quantity of such motifs decreases during the post-event period, with a smaller number of motifs still prevailing even two weeks after the event initially occurred. We also found that while a majority of motifs exhibits a high edge density as users communicate negative emotions, there was a single motif that was characteristic for the frequent communication of anticipation and surprise.
The remainder of this paper is organized as follows. In “Related work” section we summarize related work, followed by a brief description of the crisis events studied in this paper (“Events of study” section). We outline our research method in “Method” section and provide the results in “Results” section. A discussion on the results is given in “Discussion” section. “Conclusion” section concludes the paper.
An extensive body of literature exists on human behavior during crisis events. We thus focus on the use of online social media during terror and shooting events (“Use of social media in crisis communication” section). Furthermore, in “Analyzing online communication patterns via network motifs” section we summarize the application of network motifs as an approach to studying network patterns in general.
Use of social media in crisis communication
Previous studies have shown that the use of social media during crisis events differs from their general use especially with respect to an increased information sharing behavior (Hui et al. 2012). While studying the communication during three crisis events that occurred in 2005 (July 7 London attacks, the New Orleans hurricane, and the Pakistan-Kashmir earthquake), Thelwall & Stuart (2007) have shown that social media users generally exhibit three types of communication needs: 1) a need for general information about the event, 2) a need for personal information about specific individuals (e.g. an inquiry if a friend is safe), and 3) a need for information usage as a human need to communicate about the crisis by either warning others or making sense of the situation.
In addition, a number of other studies have also pointed to the increased sharing of informational content during crisis events. In fact, social media users often post messages about a crisis event before professional news media arrive on the scene (Murthy 2011; Simon et al. 2014). Such ad hoc citizen journalists provide instantaneous access to information and thus contribute to the coverage of breaking news. For example, citizens instantaneously shared eyewitness information during the 2008 Mumbai bomb blasts and the 2009 downed US Airways flight (Murthy 2011) and during the 2015 Paris terror attack (Cvetojevic and Hochmair 2018). However, though providing a convenient messaging channel, OSNs were also shown to be prone to the spread of misinformation. People tend to make sense of the situation by rumoring which, again, may lead to negative consequences if a rumor turns out the be untrue. For example, Starbird et al. (2013) reported on different types of misinformation that spread over Twitter during the 2013 Boston marathon bombing, one of which miss-identified a college student of being a bomber. Within minutes, many Twitter users and some traditional media outlets passed on (retweeted) this misinformation, contributing to its attention. The corrections, however, by far did not reach the amount of attention that the misinformation received.
The need for personal information was discussed in Nilsen et al. (2018) which reported on social media messages for identifying survivors of the 2011 terror attacks in Oslo and Utøya, Norway. Another report on the need for personal information sharing (Mazer et al. 2015) studied the 2014 school shootings in Fern Creek High School in Louisville, Kentucky and Albemarle High School near Charlotte, North Carolina.
In addition to information sharing by eyewitnesses, social media messages are also used for threat assessment and as a communication tool between citizens and the authorities. One such example is given in Simon et al. (2014) which reports on the role of the officials during the 2013 Westgate mall siege in Kenya. While communicating with the siege victims, the officials were shown to be more positive in their messaging behavior to increase the emotion of trust and credibility. As noted in Guy et al. (2013), officials are often trained to control the level of their emotional arousal during crisis events and communicate compassion and empathy to the citizens affected by the event. However, though providing a good source of information to the officials, Oh et al. (2011) also found that the terrorists involved with the 2008 Mumbai attacks actively monitored online media to enhance their own situational awareness.
Though the role of social media during crisis events has predominantly been studied in terms of situational awareness (informational purpose), related studies also point to the emotional support, emotional healing, and emotional release that social media can provide during crisis events (Fraustino et al. 2012). In Norris et al. (2006), a crisis event is defined as a traumatic event that is experienced collectively. Supporting such a characterization of a crisis event, empirical studies have shown that fear generally comes forth as a dominant emotion during events which can be described as unpredictable and out of the public control (such as shootings and terror attacks) (Jin 2009). Given the high emotionality experienced collectively in such events, users tend to form emotional support communities (Brummette and Sisco 2015). In fact, Guo (2017) found that social media fostered collective hope and instrumental support within the online community after the 2013 Boston marathon bombing.
Nilsen et al. (2018) indicated that the survivors of the 2011 terror attacks in Oslo and Utøya, Norway used social media as a therapeutic channel to emotionally cope with the experience and find out how other survivors were coping after the incident or mourn their friends or family members who did not survive the attack. While studying the 2015 attacks in Paris, Dewan et al. (2017) found that the role of images posted on OSNs is as important as the textual messages for judging the public perception of a crisis event. They found that images posted about the 2015 attacks in Paris on average carried a more positive sentiment in contrast to corresponding textual messages. Furthermore, they found that though the text of an OSN message may be negative (e.g., “Horrible news!”), the image attached to the post may actually convey a positive message of support (e.g., a picture of the Eiffel tower lit up in the colors of the French flag).
A recent study (El Ali et al. 2018) examined the connection between cultural factors and sympathy bias in the OSN coverage of global terror attacks. In particular, El Ali et al. examined tweets authored by Western news media outlets and Arab news media outlets about two terror attacks – the 2015 Beirut attack and the 2015 Paris attack. Their findings suggest that there are distinct cultural differences in journalism practice between the two cultures. In particular, tweets sent by Western news media were found to be less sympathetic than those sent by Arab news media (for both attacks).
Analyzing online communication patterns via network motifs
In recent years, a number of studies have taken various approaches to examine the formation of communication patterns in different online social networks. For example, Borondo et al. (Borondo et al. 2014) focused on community detection to study the communication patterns that emerge in a political conversations about the 2012 Catalan elections on Twitter. Moreover, few studies considered network motifs for investigating structural aspects of online communication. For example, Adamic et al. (2008) applied motif detection to a network reconstructed from Yahoo’s question-answer community. The motifs identified in the study indicated a functional role of certain interaction patterns. For example, a triad motif A→B;C→B;C→A can be interpreted as a help-seeking behavior in a Q&A community, where an expert (person C) helps users A and B, while a less skilled user A and the expert C provide help to a help-seeker B.
Topirceanu et al. (2016) detected motifs in undirected monoplex friendship networks reconstructed from Google Plus, Facebook, and Twitter. Coletto et al. (2017) applied network motifs to characterize structural patterns that emerge during discussions on controversial and non-controversial topics. They examined the presence of dyadic motifs in which the edges carry one of the two possible semantic meanings – “replies to” and “follows”. The findings suggested that non-controversial topics are characteristic for A replies to and follows B while controversial topics exhibit a high presence of A replies to B (whom A does not follow), thus showing that there are statistically significant patterns that are characteristic and discriminatory for both types of interaction networks.
Rotabi et al. (2017) applied motif detection on a Twitter data-set to study the strength of strong ties by analyzing an undirected communication network. Barash et al. (2013) further utilized motifs to study how network structure supports the occurrence of a rumor on Twitter. They found that rumors may spread outside the follower network of a user and that it is fairly uncommon that a single Twitter user is exposed to rumors from disconnected sources.
Zhao et al. (2010) discovered motifs in a communication network derived from the Facebook’s wall postings and a city’s phone call records. They focused on the concept of communication motifs that serve as indicators of the information propagation processes within the network. In particular, Zhao et al. searched for occurrences of long chain motifs (a chain of at least three unique nodes), motifs with multiple reciprocal edges, loops (a chain followed by an edge pointing to the initial node in the chain), and out-star motifs (a broadcasting behavior). The authors found that star motifs are more representative for the Facebook data compared to the phone call record which contains predominantly chain motifs.
Paranjape et al. (2017) proposed the concept of δ-temporal motifs to study the temporal flow of messages. The study identified the presence of the so-called blocking motifs that represent a form of communication in which a node has to wait for a response before continuing with the communication. Paranjape et al. showed that the posting behavior on Facebook walls exhibits a higher count of blocking motifs when compared to an email network.
Juszczyszyn et al. (2008) utilized network motifs to study how the structure of an email communication network changed over time. They found that the email communication network is characteristic for its high presence of out-star motifs, i.e. there is a significant number of broadcasting nodes which never receive a reply. Moreover, the authors have shown that the presence of specific motifs highly depends on the season, e.g. during holiday months the authors found a high number of fully connected triads rather than the out-star motifs that are characteristic for the work months.
Thus far, there has been a limited number of studies on motif detection in multiplex networks. In 2017, Battiston et al. provided a mathematical framework for motif detection in multiplex brain networks (see Battiston et al. (2017)). In particular, they classify motifs in three levels based on their number of nodes, subgraphs that are generated by a particular motif in the aggregated network, and the exact connection patterns across different network layers.
With respect to the study of communication motifs and emotional communication via a multiplex model, the related work remains rather limited.
Events of study
Fur the purposes of this paper, we analyzed a data-set consisting of 1.9 million Twitter messages related to five recent shootings and terror attacks that happened in three different countries.
Event 1 - School shooting in Tehama County, California, USA. On November 15, 2017, an attacker shot four people dead in a remote Northern California community before heading to the elementary school Rancho Tehama. Unable to open the school door, the shooter rammed through the school’s locked gate with a stolen van and fired his rifle through windows, injuring several children.
Event 2 - Supermarket siege in Trèbes, France. On March 23, 2018, a member of the Islamic State went on a shooting spree in Carcassonne, France, killing one person and seriously injuring one on his way to the military barracks, where he fired shots at four police officers. Following the attack at the military barracks, the killing spree then culminated in a three-hour hostage-taking at a supermarket in Trebès, a small French city of 5000 people. Around fifty people were taken hostage, with three persons being shot, including a French gendarme who exchanged himself for a hostage. The attack left 4 victims dead and about 16 injured.
Event 3 - Shooting at the YouTube headquarters, San Bruno, California, USA The shooting at the YouTube headquarters in San Bruno happened on April 3, 2018, when a female shooter injured 3 victims before shooting herself dead with a handgun. After the shooting, it was reported that the shooter was upset with YouTube for stopping paying her for views of the videos she uploaded to the platform.
Event 4 - Van attack in Münster, Germany. On April 7, 2018, an attacker drove a van into the tourist square of the city center of Münster, Germany. The attack left two victims killed and about 20 injured, before the attacker shot himself.
Event 5 - School shooting in Santa Fe, Texas, USA. On May 18, 2018, a 17-year-old student of the Santa Fe High School shot 8 fellow students and two teachers dead and additionally injured 13 victims. The shooter intended to kill himself, but instead surrendered himself to law enforcement. Later on, it was debated whether the attack could have been prevented since the attacker posted images on his Facebook profile which suggested his intentions.
Our research method includes seven steps, as shown in Fig. 1. We first extracted and pre-processed data from Twitter as described in “Data extraction and pre-processing” section. We then performed an emotion detection procedure and labeled each tweet with respect to Plutchik’s eight basic emotions, as explained in “Emotion detection” section. Next, we reconstructed the communication network (“Construction of the communication network” section) and defined a multiplex network that captures different types of emotion-related edges (“Construction of a multiplex network” section). Subsequently, we generated synthetic random networks (null models), as described in “Null model construction” section and identified motifs in the real-world networks (“Motif detection” section). Finally, we analyzed the detected motifs.
Data extraction and pre-processing
For data collection, we used Twitter’s Search API to extract publicly available tweets in English language sent during the five events of study (see “Events of study” section). For each of the events, our data extraction started with the day the respective event occurred and concluded about a week after the event. After extracting the tweets, we removed duplicates and tweets that only contain elements which are uninformative with respect to emotion detection (such as URLs). After the pre-processing step, our data-set comprised 1,944,972 tweets.
To detect emotions, we applied our emotion detection procedure (implemented in R) which annotates emotions with respect to Plutchik’s wheel of emotions (see Kušen et al. (2017a)). In particular, our procedure uses the NRC emotion-word lexicon (Mohammad and Turney 2013) and the AFINN lexicon for the intensity of an affect (Hansen et al. 2011), as well as a set of heuristics that people naturally use to detect emotions in written texts (Taboada et al. 2011) (e.g. amplifiers, maximizers, downtoners, negation). Moreover, the procedure also considers features that are characteristic for online social media texts such as smileys and common abbreviations. We parallelized the emotion detection procedure to speed up the process. On a server computer with two Intel Xeon E5-2630v3 CPUs @ 2.40GHz (8 cores/16 threads respectively) and 288 GB RAM, the emotion extraction procedure took 5 days.
Construction of the communication network
In our analysis, we focus on messages sent between pairs of users. In particular, Twitter users can directly send tweets to another user via @screenname. Based on this information, we reconstructed a directed communication network for each event that may contain multiple edges. Each edge in these networks is labelled according to the dominant emotion (anger, fear, sadness, disgust, joy, trust, anticipation, or surprise (Plutchik 2001)) that appeared in the respective message. The number of vertices and edges in the different communication networks are shown in Table 1.
Construction of a multiplex network
Since OSN users can participate in different types of interactions on Twitter, a multiplex model can be regarded as an appropriate framework that captures different types of interactions without a loss of information. For example, Omodei et al. constructed a multiplex network to distinguish between different types of user interactions on Twitter (replying, retweeting, mentioning) (Omodei et al. 2015). Similar to Omodei et al. (2015), we constructed a multiplex network for each day of data extraction, in order to study the structural patterns that emerge when users communicate specific emotions. The respective multiplex model consists of eight layers, where each layer corresponds to one of the eight basic emotions (anger, fear, sadness, disgust, joy, trust, anticipation, surprise).
To gain more insights into interlayer dependencies, we do not only consider individual emotion layers but in addition aggregated valence layers (positive, negative). To this end, we aggregated the edges found on the negative emotion layers (anger, fear, disgust, and sadness) into an aggregated layer called negative layer, and the edges found on the positive emotion layers (joy, anticipation, and trust) into an aggregated positive layer. In addition, we derived a valence interlayer which captures the active vertices found on each valence-specific layer and their adjacent vertices that are active on the two aggregated valence layers.
Finally, we aggregated all positive- and negative-emotion layers and layer surprise to derive the overall aggregated network. Note that this procedure has also been applied in other related studies to capture multilayer patterns (see, e.g., (Battiston et al. 2017)). The 8 different emotion-annotated layers and the 4 derived layers of our multiplex model are sketched in Fig. 2.
Table 2 summarizes information (mean number and standard deviation of vertices, edges, network density, maximum degree, and mean degree of the vertices) about each layer of our real-world networks averaged over the time-period of data extraction.
Null model construction
In order to detect emotion-exchange motifs, we generated synthetic random networks that correspond to each of the real-world network. In particular, we used the stub-matching algorithm over each of the 8 emotion layers and each of the 4 derived (aggregated) layers of our multiplex network (as shown in Fig. 2). In order to speed up the algorithm’s performance, we used parallel processing (multi-threading) and assigned a pool of worker threads that process each iteration of the null model generation independently.
The stub-matching algorithm, as described by Newman et al. (2001), uses the concept of stubs which can be interpreted as “sawn-off arrow heads" of the incoming and outgoing edges. The stub-matching algorithm randomly chooses a pair of stubs which are joined with an edge. If in the process a multiple edge or a self-loop is generated, the resulting network is considered invalid and the algorithm restarts the procedure.
Although some authors argue that the generation of 300 null models is sufficient (see, e.g., Schlauch and Zweig (2015)), we followed the approach that is frequently taken by related studies and generated 1000 null models (see, e.g., Dormann et al. (2009); Shi and Shi (2014)) for each motif detection procedure. In particular, we generated 1000 synthetic networks for each of the 8 multiplex layers and the 4 derived layers for each day of each event of study. This procedure resulted in a total number of 720,000 null models.
For motif detection, we first applied the ESU subgraph enumeration algorithm (Wernicke 2006) to enumerate all subgraphs of size k (for the purposes of this paper, we set k=3) and the VF2 isomorphism testing algorithm (Cordella et al. 2004) to check whether a pair of subgraph candidates is isomorphic to each other. This helped us group isomorphic subgraphs into categories. Algorithm 1 describes our procedure for the detection of emotion-exchange motifs in detail.
We ran all 720,000 null models (see “Null model construction” section) through our motif detection procedure. For motif detection (see Algorithm 1) we used two machines, a desktop computer with Intel Xeon CPU E3-1240 v5 @ 3.5GHz (4 cores/8 threads) and 32 GB RAM as well as server computer with two Intel Xeon E5-2630v3 CPUs @ 2.40GHz (16 cores/32 threads) and 288 GB RAM. On those machines, the motif detection procedure took approximately 6 weeks to complete and in total (i.e. for all 720,000 null models) produced an output of over 1.2 terabyte of data to analyze.
In recent years, a couple of extensions of traditional network motifs have been proposed. In this section, we introduce some of those extensions and discuss how our emotion-exchange motifs differ from them.
In Fox et al. (2017), Fox et al. propose contextual motifs as an extension to network motifs that considers the context under which a particular motif occurs to better predict hypo- and hyperglycemic events in waveform patient data. Their findings suggested that the discovery of such contextual motifs are clinically more meaningful than the context-less counterparts and lead to a more fine-grained interpretation of their meaning. Our proposal of emotion-exchange motifs can also be regarded as a type of a context under which a particular motif emerges. However, due to the different nature of social communication networks and (medical) patient data the type of context differs significantly. Moreover, Fox et al. infer the context of their contextual motifs whereas we do not infer the emotion but use a multiplex emotion network to identify emotional communication patterns.
Another extension to network motifs has been proposed by Zhao et al. (2010). They proposed the concept of communication motifs which take over specific forms – long chains, motifs with multiple reciprocal edges, out-stars, and loops (a chain followed by an edge pointing back to the initial node). Aside from considering different emotions, our approach in particular differs in the motif detection procedure. For this paper, we focused our motif detection on 3-subgraphs found in the respective real-world networks but without restricting it to a specific subgraph form (shape).
To the best of our knowledge, our emotion-exchange motifs are the first motif extension that targets the exchange of emotions.
Emotions expressed during shootings and terror events
Since the expression of emotions is volatile over time (Li et al. 2017), we subsequently report on the temporal evolution of emotions in our data-sets. As noted in previous studies, shootings and terror attacks are typically unexpected events that cause an initial shock and fear in people affected by those attacks. A supporting theoretical model, the Integrated Crisis Mapping (ICM) model (Jin et al. 2012), postulates that emotions reveal the public’s interpretation of an event and on the one hand distinguishes between emotions that are perceptual and experienced as one’s primary coping strategy to make sense of the situation, and on the other emotions expressed after the stakeholders’ engagement (varying from the image perceived by public – responsibility or helplessness). According to the ICM model, terror and shooting attacks are characteristic for the initial expression of fear and sadness. Further studies have shown that in the post-event period an initial shock may evolve into anger and a search for a culprit to blame (Guo 2017; Rosas 2015; Tapia et al. 2014), as well as the expression of positive emotions and empathy to emotionally bond and support the ones affected by the event (Guo 2017; Kušen et al. 2017b; Lin and Margolin 2014).
Such an emotional response is also evident in our data-sets. Figure 3 shows the temporal flow of emotions on Twitter as well as their intensities during the five crisis events. All figures show a timeline on which the first date is the first (initial) day of the crisis event followed by the post-event dates and their corresponding emotional intensities. In our data-sets, fear is a common dominant emotion in all data-sets, with anger and sadness consistently following after the expression of fear in the post-event period.
The boxplots shown in Fig. 4 present the aggregated emotion intensity over time. Overall, Twitter users tend to express comparable emotions during all five crisis events (averaged Kendall’s τ is a strong positive 0.86). However, considering the temporal aspect of the emotional expressions over Twitter, the correlation coefficient varies between a moderate positive and a strong positive correlation (see Fig. 5).
In particular, our analysis reveals that Twitter users express highly correlated emotions when first experiencing and /or hearing about a shooting or terror attack (τ=0.76, notice the consistent high intensity of fear, anger, and sadness expressed on the first day of each crisis event in Fig. 3).
Consistent with the related work on crisis events, in our data-sets fear serves as an expression of an initial feeling of shock and /or uncertainty:
“Multiple shots fired @YouTube HQ, unknown number of injuries or victims. Still an active shooter?” (YouTube shooting),
“TERRIBLE! Killer Goes on Shooting Spree in Rural Northern California, killing 4!” (Tehama county shooting),
In comparison, anger and sadness consistently express two different types of reactions to the event, either fury and hate towards the one to blame, e.g. “I absolutely HATE the people that run YouTube, but whoever did this is a even more pathetic son of a bitch.”, or the expression of empathy for the ones directly affected by the shooting or terror attack, e.g. “Horrible deadly attack in Muenster in Germany. My thoughts are with the victims and their loved one. Hell to the killer”.
After the initial reaction to the respective event, the correlation drops to a moderate positive (0.46<τ<0.58). As shown in Fig. 3, the intensities of specific emotions vary in the post-event period among the data-sets, depending highly on the information revealed about the event. For example, the Trebes (France) data-set exhibits a high intensity of trust (notice the light blue line in Fig. 3-b) in addition to fear, sadness, and anger. This specific emotion celebrates the act of the selfless sacrifice of the French gendarme (“France honors a hero, police officer, Arnaud Beltrame who volunteered to trade places with hostages during the Trebes attack”). After their initial reaction, Twitter users also increase the expression of disgust as a reaction to the actions of the person responsible for the event, to comments of the local politicians, or to the reports by the local news agencies:
“I think it is utterly disgusting that the government hasn’t done shit about all these assholes shooting up schools” (after the Santa Fe school shooting),
“Hah. Biiiitch. But good direction! Bad execution. Pardon my pun you pathetic halitosis puke of a woman.” (upon revealing the name of the female shooter at the YouTube headquarters).
Consistent across all data-sets, Twitter users collectively and continuously express their empathy in the post-event period (annotated as sadness by our emotion detection algorithm):
“Such senseless violence :’(My heart goes out to all affected by the #YouTube HQ shooting. So terribly sad to hear. Please stay safe"
“Prayers to all the victims and victim’s families of the #SantaFeShooting. This is beyond horrific”
“Very very sad, heart hurts for these victims.” (Tehama County (North California) school shooting),
as well as thankfulness (joy) to the ones helping the victims recover or saving them:
“Thankful that Dr. Andre Campbell is taking great care of the victims of the YouTube HQ shooting.”,
“God bless the wonderful memory of Lt. Col. Arnaud Beltrame! Hero of all heroes! Viva La France!”.
Emotions during direct messaging behavior
So far, we have reported on the emotions generally expressed during the five crisis events. Next, we examine the direct messaging behavior between the Twitter users during the five events and study 1) whether the emotions communicated directly between two or more users are comparable to the ones expressed in general, 2) which common emotion-exchange motifs are formed as users directly communicate during the five events, and 3) the functional role of the different emotion-exchange motifs.
In this part of the analysis, we first excluded all retweets from our data-set (retweets contain the string RT @screenname text which signals that a tweet originally published by @screenname has been retweeted). Upon cleaning the data-set, we then analyzed the remaining @-traces (in total 3.77% of the overall data-set, i.e. 73404 out of 1,944,972 tweets).
As shown in Fig. 6, emotions communicated directly between two users are generally consistent with the overall emotional intensity for each of the shooting and terror attacks that we analyzed. In particular, when disregarding the temporal aspect, emotions communicated between a pair of users exhibit a strong (0.6≤τ<0.9) to a very strong (τ≥0.9) positive correlation (for the Tehama County shooting τ=0.93, YouTube shooting τ=1, Santa Fe school shooting τ=0.93, Trebes siege τ=1, and Münster van attack τ=0.86). However, when considering the temporal exchange of emotions, we observe that initially the emotions communicated directly between a pair of users and the ones broadcasted (i.e. tweets excluding those with direct @-mentioning of another Twitter user) are strongly (0.6≤τ<0.9) to very strongly (0.9≤τ≤1.0) correlated (see Fig. 7).
Thus, upon experiencing or hearing about the shooting or terror incident, Twitter users tend to initially exchange directed messages that convey predominantly fear, anger, and sadness. While positive messages are expressed as well, they initially remain low in their intensity.
We found that Twitter users initially express shock and disbeliefFootnote 1:
“@screenname I don’t want to talk #Politics, I’m in shock. This last shooting at YouTube hit closer to home than most.”
“@screenname I am also feeling this deeply! I work at a school and it’s absolute hitting me hard!”,
as well as information seeking behavior (expressing sadness in a combination with trust or anticipation):
“@screenname Have you heard about the shooting? Its sad what’s happening at YouTube HQ. I hope they are all ok.”
“@screenname Prayers for Northern California. Glad the shooter is no longer a threat. What is known about them?”
In the post-event period, the correlation between the daily emotion intensity expressed via broadcasting and retweeting on the one hand and direct messaging between two users on the other, however, diverges to a set of values ranging from weak (0.0≤τ<0.3) to very strong (0.9≤τ<1.0) correlation with one value even exhibiting a negative correlation (Münster van attack, day 8: τ=−0.2519).
We also found that the emotion intensities sent during the two school shooting incidents and the YouTube shooting remain strongly correlated with the general emotions expressed on Twitter about the respective events. As discussed above, the post event period is characteristic for the expressions of anger and fear:
“@screenname I live in Tehama County, and there is NOTHING anywhere–not even the local channels... #RanchoTehama",
“@POTUS On your watch. Again, please do nothing. Works great. #RanchoTehama",
“@screenname (politician) still no real sympathy for the #RanchoTehama victims. You are pathetic. #GunControlNow",
@screenname perhaps the NRA should prove their heroism and form groups of volunteers to trade places with hostages (Trebes siege),
but also sadness, joy, and anticipation (associated with empathy):
“I feel sorry for the victims of the @YouTube HQ shooting. Such a terrible tragedy.",
“@screenname Prayers for Northern California. Glad the shooter is no longer a threat. What is known about them?",
“@screenname When will this stop? Condolences to the families of the victims and all the injured and the witnesses" (Münster van attack)
Next, we show the emotion-exchange motifs that emerged on Twitter during the five events in our study (see also “Method” section). All motifs were identified for a p-value of p<0.05. We first generalize the identified motifs by disregarding the message exchange frequency (represented as the edge weight) and use the MAN-labelling scheme (Davis and Leinhardt 1972). As shown in Table 3, the generalized emotion-exchange motifs take over multiple shapes, with a predominant occurrence of 021D (frequency = 0.453). Of these generalized motifs, five are considered commonFootnote 2 emotion-exchange motifs.
Our analysis has revealed that although all of the nine motifs from Table 3 appear at some point during the data extraction period, each day is characteristic for a single dominant motif (as shown in Fig. 8). In particular, we observe a high dominance of the one-way message-sending motif 021D over the first nine days of the data extraction period, followed by a one-way message-receiving motif 021U during the following days. We also observe that motif 021U has the longest duration for the data extraction period. Interestingly, it is only on the first day of the data extraction (i.e. the day when the event initially happened) that we observe the presence of all nine motifs presented in Table 3.
Thus, upon learning about a shooting or a terror attack when we observe a high intensity of fear, anger, and sadness (see also Fig. 6), users form not only a one-way message-sending or a message-receiving motif, but also motifs that involve a two-way messaging behavior (reciprocal edges) as found in motifs 120U, 111D, 120D, and 201, message chains (motif 021C), and closed triads without reciprocal edges (030T). Below, we also show that motif 030T, which is in its frequency the third most occurring motif throughout the data-extraction period, has a specific role as users communicate non-negative emotions (such as anticipation and surprise).
Though the motifs in our data-sets can be generalized into MAN-labelled motifs where we disregard edge weights (see Table 3), it is still interesting to consider the weights as they reveal the message-exchange frequency in a given motif. As shown in Fig. 8, all nine motifs only appear together on the first day of the event, with the variabilityFootnote 3 dropping gradually over time. Overall, the motifs of the highest relative variability in our data-set are 111U and 120D (both containing reciprocal edges) (see Table 3), whereas the two most frequently occurring one-way motifs (021D and 021U) are overall low in their variability (0.09 and 0.17, respectively).
We also notice a steady trend in the decrease of the edge weights in the identified motifs. As shown in Fig. 8 the average number of edges per motif (i.e. the number of messages exchanged among three users) peaks within the first three days since the corresponding event initially happened. In particular, the number of messages received by a single user (motif 021U), messages exchanged in a chain (021C), and messages exchanged within a reciprocal closed triad (120U) are the most dense on the first day of each event. This behavioral pattern shows a human tendency to seek and exchange information in an initial moment of uncertainty. We further observe that the message sending motif (021D), message exchange within a closed triad (030T), and a message exchange with a pair of users engaging in a two-way communication (111U) peak on the day after the event when more information about the respective shooter and the number of victims have been publicly announced.
Next, we show that the distribution of the motifs differs across the emotion layers (see “Construction of a multiplex network” section). For instance, three one-way message-exchange motifs (030T, 021D, and 021U) are formed as users exchange sadness, while the communication of anger exhibits a wider variety of motifs (in particular, all nine motifs are found as users exchange anger). Our findings clearly suggest that emotion-exchange motifs that contain reciprocal edges emerge only when users exchange anger and fear or if at least one of these two emotions appears in a combination with any other emotion found in the Plutchik’s wheel. Furthermore, we found that the expression of sadness, disgust, surprise, and positive emotions are characteristic for a one-way communication.
As depicted in Fig. 9, certain emotion-exchange motifs are more common in specific days of the respective data extraction period, depending on the emotion layer the motifs appear on. Motifs 021D and 021U are the most dominant motifs in our data-sets. These are also the only two motifs that occur across all layers of our multiplex network. We notice that the reciprocal motifs 120D, 120U, 201, and 111D, on the other hand, appear predominantly on the negative-valence layers or layers that involve a combination of positive and negative emotions. These are the layers representing fear, anger, and negative emotions, as well as the interlayer and the overall aggregated layer. In contrast, the message-sending motif 021D, the message-receiving motif 021U, and the closed triad 030T appear commonly on the positive-valence layers (joy, trust, anticipation).
Some emotion-exchange motifs appear more often on certain layers indicating that specific emotions or specific combinations of emotions inspire Twitter users to send a higher frequency of messages. For example, as shown in Table 4, and although the message frequency is comparable across the layers of the multiplex network, the highest number of messages in motifs is exchanged when users communicate a combination of positive and negative emotions (μ=5.16,var=1.23) (referred to as interlayer, see “Construction of the communication network” section), followed by surprise (μ=4.82,var=0.97) and anticipation (μ=4.72,var=1.35).
As depicted in Fig. 10, each motif can be characterized with respect to the layer on which it exhibits the highest density (i.e. message exchange frequency). In particular:
motif 021D consists of on average ne=4.935 edges (max=11) on layer surprise,
motif 021U contains ne=5.220 edges (max=12) on layer interlayer,
motif 030T contains ne=6.268 edges (max=16) on layer surprise,
motif 021C contains ne=6.354 edges (max=18) on layer surprise,
motif 120U contains ne=6.096 edges (max=14) on layer fear,
motif 111D contains ne=6.006 edges (max=14) on layer fear,
motif 111U contains ne=6.279 edges (max=16) on layer aggregated,
motif 120D contains ne=6.501 edges (max=18) on the negative layer, and
motif 201 contains ne=6.163 edges (max=14) on the negative layer.
In general, the most dense daily emotion-exchange motifs in our data-sets have on average more than 6 edges (6.0≤ne≤6.50) with an upper bound between 14-18 edges. These motifs occur predominantly as Twitter users exchange messages that convey emotions of a negative valence (fear, anger, sadness, or a combination of negative emotions). The corresponding motifs are 120D (layer anger, fear, interlayer, negative, aggregated), 021C (layer sadness, fear, anticipation), 111D (layer fear), 120U (layer fear, aggregated), 111U (layer interlayer, negative, aggregated), and the rarely-occurring motif 201 (layer negative, aggregated). In contrast, we only found one emotion-exchange motif (030T) which exhibits a high density (ne≥6.0) on a single positive layer (anticipation) and on the surprise layer.
With respect to the significance profiles (SP) (Milo et al. 2004) of the motifs, we further studied whether there are similarities in the local structure of the emotion-annotated networks and systematically compared them based on two emotion theories.
In general, two approaches exist for studying the role of emotions in human communication. While the first approach studies the similarities of emotions belonging to the same valence polarity (positive or negative emotions) (see, e.g. Ferrara and Yang (2015); Kim et al. (2013)), other authors argue that examining polarity alone is not enough (see, e.g. Russell (1980)). Even though some emotions might be of the same valence polarity, they still cause dissimilar reactions in people in terms of message sending. For example, (Berger 2011) argues that two negative emotions, such as anger and sadness, will cause different message sending patterns due to the level of arousal they cause in people.
For our analysis, we therefore first examine whether the pairs of networks representing the exchange of emotions of the same valence (e.g. anger and sadness) are better correlated than the pairs including emotions of different valence polarities (e.g. anger and joy). Next, we include the dimension of arousal and examine whether the emotions belonging to the same arousal category (e.g. high arousal emotions: fear, anger) will exhibit a higher structural similarity compared to the networks of emotions of the same valence but of a low arousal (such as sadness).
To this end, we correlated the daily significance profile (SP) values of the sequence of motifs appearing on a specific emotion-annotated layer.
Networks representing the exchange of emotions of the same valence. We found that in general (i.e. when disregarding the temporal aspect) negative emotion-exchange networks (i.e. the layers/networks in our multiplex network that represent negative emotions) correlate slightly better within their own valence category (average Pearson’s r=0.79±0.13)Footnote 4 than with the networks of the positive emotion valence (average Pearson’s r=0.73±0.11), while the positive emotion-exchange networks exhibit the strongest similarity (average Pearson’s r=0.90±0.08). However, when considering the temporal aspect (as shown in Figs. 11 and 12) the correlation of the emotion-exchange networks is highly volatile over time. While the SP-values of negative emotion-exchange networks correlate positively with one of the positive emotion-exchange networks in 40.62% of all cases, a positive correlation of negative-valence networks among each other occurs in only 28.75% of all possible cases. Moreover, the SP-values of positive emotion-exchange networks positively correlate in 50% of the possible cases. Thus, given the comparatively low number of cases where negative emotion-exchange networks structurally exhibit a high correlation among each other implies that the temporal aspect plays a significant role while examining structural similarities of such networks.
Networks representing the exchange of emotions of the same arousal. Finally, we consider the dimension of arousal. Following Russell’s circumplex model (see Russell (1980)), we consider fear, surprise, joy, anticipation, disgust, and anger as high arousal emotions, while sadness and trust are considered to be low arousal emotions. We found that the networks that are formed as Twitter users communicate high arousal emotions are structurally less similar among each other (average Pearson’s r is 0.74±0.17) compared to the correlations between pairs of networks where one network represents a high and the other a low arousal emotion (average Pearson’s r=0.90±0.10).
We also found that when considering the temporal aspect, emotion-exchange networks of the same arousal category correlate positively, but less frequently over time compared to the correlation between pairs of high and low arousal emotion-exchange networks (high arousal emotion-exchange networks are positively correlated in 37% of the cases and low arousal emotion-exchange networks are positively correlated in 37% of the cases. In contrast, pairs where one network represents a high and the other a low arousal emotion positively correlate in 41% of all cases).
In the analysis we performed for this paper, we observed a high dominance of the negative emotions fear, anger, and sadness that are not only broadcast over Twitter, but also exchanged directly among Twitter users, indicating the expression of shock, uncertainty, empathy, or even rage. As pointed out by Matsumoto et al., culture influences one’s emotion regulation (Matsumoto et al. 2008) and, according to Markus & Kitayama, shapes emotional experiences (Markus and Kitayama 1991). Such cultural differences were also demonstrated in user studies. For example, Matsumoto (Matsumoto 1989) showed that the individuals from interdependent cultures (such as many Asian, African, and Latin-American cultures) assess the expression of negative emotions (e.g., anger and sadness) with a lower intensity compared to those of American or Western European cultures. Since our motif detection procedure relies on emotion-labeled networks, cultural perception and expression of emotions play a significant role in the resulting emotion-exchange motifs. In this paper, our focus lies on the events that emerged in American and Western European cultures.
When analyzing direct message exchanges between Twitter users, we identified nine emotion-exchange motifs that frequently occur in our data-sets, with 021D (B→A,B→C) and 021U (A→B,C→B) being the two most common ones over the course of time. These motifs in particular, are formed as users communicate all emotions considered in this study (i.e. these motifs alone are not good indicators of a communication pattern formed as a particular type of emotion is exchanged among three users). This finding, however, does not come as a surprise as it can be attributed to the purpose of Twitter’s communication service.
In particular, Twitter is predominantly used as a (one-way) broadcasting platform for large(r) scale information-sharing purposes (Hughes and Palen 2009). Thereby Twitter differs from other services, such as Facebook for example, where one often receives a high(er) number of direct responses to a post, resulting in a multi-way communication pattern. However, a Twitter user may directly mention any number of other Twitter users via @screenname-mentioning. In our analysis, we found that a single user mentions on average μ=1.49±1.11, max=13 other users via @screenname-mentioning. Moreover, politicians (such as presidents, chancellors, or governors) and well-known news channels (such as BBC or CNN) that reported on a terror or shooting event are frequently mentioned on Twitter. For instance, 7.13% (5235 times) of all directed messages mention Donald Trump (@realdonaldtrump), while CNN (@cnn) is mentioned in 3.37% of the messages (2474 times) in our data-set.
Thus, the two motifs (021D and 021U) can be considered typical motifs that emerge on Twitter. In fact, of all possible motifs shown in Table 5, we found that those which contain predominantly reciprocal edges are, in general, not representative for the direct communication patterns on Twitter. Thus, motifs such as 120C, 210, and 300 could not be identified as emotion-exchange motifs.
Nonetheless, we also found that a small number of motifs with reciprocal edges (in particular, 111D, 111U, 201, and 120U, see Table 5) are actually characteristic for the exchange of two specific emotions, which are anger and fear. Both of which are attributed as negative emotions of a high arousal (see Russell’s circumplex model in Russell (1980)). According to Berger (2011), high arousal emotions are associated with message-sending behavior. This finding might serve as a possible explanation why the two high arousal emotions in our data-sets inspire reciprocity. For example communication of uncertainty (fear) frequently results in the emotion-exchange motif 111D:
user A: “@screenname (user B) the killer was clearly a male! Nothing about him was female"
user B: “@screenname (user A) They’re saying that the shooter was a trans woman"
user C: “@screenname (user A) She was a biological female :O A female shooter is a rarity: an FBI study LINK"
Compared to anger and fear, sadness (which is also a negative emotion and therefore belongs to the same valence dimension as anger and fear) is an emotion of low arousal (see Russell (1980)). We found that emotion-exchange motifs that occur as people communicate sadness contain predominantly non-reciprocal (one-way) edges. Sadness in our data-sets often occurred as Twitter users communicated empathy by mentioning a fellow Twitter user via @screenname. For example, forming the chain motif 021C:
user A posted: “In pain & grief over yet another #TexasShooting @screenname (user B)"
user B posted: “#sad sad sad #SantaFeHighSchool @screenname (user C) cries with you. This should not be happening. #NoMore #MakeItStop".
Another example of the communication of sadness forming the 021U motif:
user A posted: “:(:(@screenname (user B) offers to pay for the funerals of all the #SantaFeHighShooting victims#TexasShooting"",
user C posted: “@screenname (user B) Hearing that you are going to pay for all the victims funerals from the #SantaFeShooting got me teary eyed".
Even though emotion-exchange motifs with reciprocal edges are not characteristic for the exchange of sadness, we still found that the expression of one’s empathy and grief frequently receives attention in terms of likesFootnote 5. In particular, tweets expressing sadness attract substantially more likes (μ=57.18,sd=1059.01) compared to anger (μ=8.09,sd=241.93) or fear (μ=4.58,sd=77.44) (see Table 6).
Such structural differences in the communication of negative emotions in our data-sets reveal that not all negative emotions are exchanged in the same manner. While comparing the local structures formed as users communicate individual negative emotions or a combination of negative emotions, we found that the similarities among such negative-emotion network layers are almost as strong as when paired with the individual positive-emotion network layers (Pearson’s r ranging between 0.73 and 0.79). Yet, positive-emotion network layers are found to be structurally highly similar (r=0.90).
When considering the temporal aspect, we found that networks formed over the first four days after a shooting event occurred are in general structurally more similar compared to days 5-8 after an event occurred. In particular, over the first four days, the negative emotion-exchange networks correlate strongly among each other (r=0.94±0.08), while the correlation drops to r=0.84±0.17 over days 5-8. The same pattern holds for the positive emotion-exchange networks (r=0.83±0.0 over the first four days and r=0.53±0.56 over days 5-8). However, the most substantial decrease is visible in the similarities between pairs of networks where one represents positive and the other negative emotion-exchanges (for the first four days: r=0.85±0.13, for days 5-8: r=0.45±0.59). Thus, as the five events in our study become less newsworthy over time, we also observe a drop in the structural similarities of the corresponding emotion-exchange network layers.
For our study the main restrictions result from the tools we used for data extraction, pre-processing, preparation, and analysis. In particular, we used Twitter’s API to extract publicly available tweets. One significant limitation is an API restriction which only allows for the extraction of tweets that are at most seven days old. Thus, if not planned properly, the data extraction cannot be repeated because the API restricts access to tweets older than a week. Moreover, Twitter explicitly says that not all tweets are indexed or made available by the Twitter API. Thus, even though we performed a systematic procedure where we extracted the new tweets on a daily basis, we cannot rule out the possibility that we missed relevant tweets due to this API restriction. In order to get a good sample for our data analysis, we carefully selected event-specific hashtags and key words while extracting Twitter messages sent in the aftermath of the five events.
Another limitation results from our emotion detection algorithm which utilizes existing emotion-word lexicons and relies on their completeness. To mitigate errors and incompleteness in our study, we therefore used the most accurate word-emotion lexicon that was available to us and combined it with enhanced detection capabilities for the detection of negation, amplifiers, downtoners, and specific types of emojis (see Kušen et al. (2017a)). Berrios et al., found that humans can experience and express mixed emotions (see Berrios et al. (2015)). Though in our motif detection procedure we consider one dominant emotion per edge only, it would be interesting to extend the work and allow for multiple emotion labels on every single edge (e.g., a user may express anger and sadness in the same message).
One of the major bottlenecks of our motif detection procedure lies in the computational demands of the motif detection algorithm that we applied. Topirceanu et al. also pointed to the computational challenges one faces when detecting motifs in complex networks, such as those based on real-world OSNs (see Topirceanu et al. (2016)). In addition, De Domenico et al. (2015) pointed to the increase in computational demands for computations on multilayer networks. As shown by De Domenico et al., one can reduce a multiplex network to a minimal set of layers that are necessary to accurately represent the structure of this multiplex network without loss of information about the studied phenomenon (see De Domenico et al. (2015)). Thus, in our future work we plan to further enhance our procedure to appropriately reduce the multiplex model to the layers that are deemed non-redundant and informative with respect to their corresponding real-world network.
Finally, one cannot exclude the potential influence that bot accounts might have had on the overall emotional intensity of the events we analyzed. As discussed in our previous study (see Kušen and Strembeck (2018b)), bots may be responsible for a higher presence of the so-called shifted-emotions, i.e. emotions that are not initially expected for the event of interest such as negative emotions during positive events. Thus, by acknowledging the potential for bots to impact the emotionality of a Twitter discourse, it would be interesting to detect bots in our data-set and study whether their presence affects the emergence of emotion-exchange motifs (see, e.g., Kušen and Strembeck (2018c)).
In this paper, we provide an analysis of the emotions exchanged on Twitter during five shootings and terror attacks. Consistent with the integrated crisis mapping (ICM) model, we found that fear, anger, and sadness are predominant emotions on the day a respective event happened, as well as during the post event period. In particular, this type of messaging behavior is an expression of human coping mechanisms applied in the aftermath of crisis events.
In our analysis, fear predominantly revealed the feelings of shock and uncertainty, anger exposed fury and hate towards the one to blame, and sadness revealed the feeling of empathy. We found that the initial emotional reactions about the five events strongly correlate on the first day of the event, with the correlation still remaining positive but only moderately strong in the days that follow. Such a drop in the correlation is due to the exposure of further event-specific information (e.g. the number of victims or information about the shooter). Thus, depending on the way each of the respective events resolved over time, we also observed the presence of disgust, anticipation, or even joy.
Though Twitter is predominantly used for message broadcasting (one-way communication), we also examined the direct messaging behavior (via Twitter @-mentioning) related to the five events in our study. We found that Twitter users generally exchange emotions that are initially strongly consistent with the overall emotions about the respective event. In particular, we found that the subgraphs resulting from direct messaging scenarios include a number statistically significant patterns that we call emotion-exchange motifs. We showed that nine different emotion-exchange motifs emerged during the five crisis events, each revealing more information about the local structural patterns that are formed as users communicate specific types of emotions. Moreover, we also found that the communication of anger and fear, or a combination of anger and fear with any other emotion, inspire a wider range of structural patterns including one-way and reciprocal-edged motifs. In contrast, the communication of sadness, disgust, surprise, as well as any positive emotion is characteristic for a one-way communication.
We also showed that two-way emotion-exchange motifs occur predominantly one day after the event, when more information about the shooter and the number of victims has become publicly available. In general, we found that the communication of negative emotions is characteristic for a higher density of edges within a motif, compared to the communication of positive emotions.
Finally, we systematically compared the local structure of the different emotion layers in the multiplex network by using the significance profiles of the identified motifs. When observing the two valence dimensions only, we found that the local structure that emerges as users communicate specific emotions is highly volatile over time. In particular, we found that when considering the temporal dimension pairs of positive and negative emotion-exchange networks (such as anger and joy) are more often positively correlated than any pair of two negative message-exchange networks. Furthermore, by considering the dimension of affective arousal, our findings revealed that a higher structural similarity emerges between pairs of networks of high- and low-arousal emotions, rather than pairs of networks belonging to the same arousal dimension.
In our future work, we will explore the presence and the functional role of emotion-exchange motifs in other case studies and other platforms. Moreover, we will continue to study the emergence and the evolution of temporal emotion-exchange motifs.
Note that we replaced the Twitter users’ screennames with @screenname to preserve their anonymity, aside from that all tweets are in their original form.
For a motif to be considered common, it has to appear in at least three data-sets
We define a variability of a motif with respect to its edge weight, where the edge weight marks the number of messages exchanged between a pair of nodes.
All values are reported for the significance level of p<0.05.
On Twitter, a user can like someone’s tweet to show appreciation or agreement with the content of the tweet.
Adamic, LA, Zhang J, Bakshy E, Ackerman MS (2008) Knowledge Sharing and Yahoo Answers: Everyone Knows Something In: Proceedings of the 17th International Conference on World Wide Web, 665–674.. ACM, New York. https://doi.org/10.1145/1367497.1367587.
Alon, U (2007) Network motifs: theory and experimental approaches. Nat Rev Genet 8:450–461. https://doi.org/10.1038/nrg2102.
Bakker, MH, van Bommel M, Kerstholt JH, Giebels E (2018) The influence of accountability for the crisis and type of crisis communication on people’s behavior, feelings and relationship with the government. Public Relat Rev 44(2):277. https://doi.org/10.1016/j.pubrev.2018.02.004.
Balon, S, Rime B (2016) Lexical Profile of Emotional Disclosure in Socially Shared Versus Written Narratives. J Lang Soc Psychol 35(4):345–373. https://doi.org/10.1177/0261927X15603425.
Barash, V, Milic-Frayling N, Smith MA (2013) A Method for Discovering Dynamic Network Motifs by Encoding Topic Propagation In: Proc. of the IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT) - Volume 01, 451–458.. IEEE Computer Society, Washington, DC.
Battiston, F, Nicosia V, Chavez M, Latora V (2017) Multilayer motif analysis of brain networks. Chaos Interdiscip J Nonlinear Sci 27(4):047404. https://doi.org/10.1063/1.4979282.
Beber, ME, Fretter C, Jain S, Sonnenschein N, Müller-Hannemann M, Hütt MT (2012) Artefacts in statistical analyses of network motifs: general framework and application to metabolic networks. J R Soc Interface 9(77):3426–3435. https://doi.org/10.1098/rsif.2012.0490.
Berger, J (2011) Arousal Increases Social Transmission of Information. Psychol Sci 22(7):891–893.
Berrios, R, Totterdell P, Kellett S (2015) Eliciting mixed emotions: a meta-analysis comparing models, types, and measures. Front Psychol 6:428. https://doi.org/10.3389/fpsyg.2015.00428.
Borondo, J, Morales A, Benito R, Losada J (2014) Mapping the online communication patterns of political conversations. Physica A Stat Mech Appl 414:403–413. https://doi.org/10.1016/j.physa.2014.06.089.
Brummette, J, Sisco HF (2015) Using Twitter as a means of coping with emotions and uncontrollable crises. Public Relat Rev 41(1):89–96. https://doi.org/10.1016/j.pubrev.2014.10.009.
Coletto, M, Garimella K, Gionis A, Lucchese C (2017) Automatic controversy detection in social media: A content-independent motif-based approach. Online Soc Networks Media 3-4:22–31. https://doi.org/10.1016/j.osnem.2017.10.001.
Cordella, LP, Foggia P, Sansone C, Vento M (2004) A (sub)graph isomorphism algorithm for matching large graphs. IEEE Trans Pattern Anal Mach Intell 26(10):1367–1372. https://doi.org/10.1109/TPAMI.2004.75.
Crump, J (2011) What are the police doing on Twitter? Social media, the police and the public. Policy Internet 3(4):1–27. https://doi.org/10.2202/1944-2866.1130.
Cvetojevic, S, Hochmair HH (2018) Analyzing the spread of tweets in response to Paris attacks. Comput Environ Urban Syst. https://doi.org/10.1016/j.compenvurbsys.2018.03.010.
Darling, JR (1994) Crisis Management in International Business: Keys to Effective Decision Making. Leadersh Org Dev J 15(8):3–8. https://doi.org/10.1108/01437739410073047.
Davis, JA, Leinhardt S (1972) The Structure of Positive Interpersonal Relations in Small Groups, 1–55.. Houghton Mifflin, Boston.
De Domenico, M, Nicosia V, Arenas A, Latora V (2015) Structural reducibility of multilayer networks. Nat Commun. https://doi.org/10.1038/ncomms7864.
De Longueville, B, Smith RS, Luraschi G (2009) “OMG, from Here, I Can See the Flames!”: A Use Case of Mining Location Based Social Networks to Acquire Spatio-temporal Data on Forest Fires In: Proc. of the 2009 International Workshop on Location Based Social Networks, 73–80.. ACM, New York. LBSN, https://doi.org/10.1145/1629890.1629907.
Dewan, P, Suri A, Bharadhwaj V, Mithal A, Kumaraguru P (2017) Towards Understanding Crisis Events On Online Social Networks Through Pictures In: Proc. of the IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, 439–446.. ACM, New York, NY, USA. ASONAM, https://doi.org/10.1145/3110025.3110062.
Dormann, C, Frund J, Bluthgen N, Gruber B (2009) Indices, Graphs and null models: Analyzing bipartite ecological networks. Open Ecol J 2:7–24.
El Ali, A, Stratmann TC, Park S, Schöning J, Heuten W, Boll SC (2018) Measuring, Understanding, and Classifying News Media Sympathy on Twitter After Crisis Events In: Proc. of the CHI Conference on Human Factors in Computing Systems, 556:1–556:13.. ACM, New York. CHI, https://doi.org/10.1145/3173574.3174130.
Farazmand, A (2016) Global cascades in best and worst practice in crisis and emergency management. CRC Press, Taylor and Francis Group, Boca Ratom.
Ferrara, E, Yang Z (2015) Quantifying the effect of sentiment on information diffusion in social media. PeerJ Comput Sci 1:1.
Folkman, S, Moskowitz JT (2000) Stress, Positive Emotion, and Coping. Curr Dir Psychol Sci 9(4):1–115. https://doi.org/101111/1467-872100073.
Fox, I, Ang L, Jaiswal M, Pop-Busui R, Wiens J (2017) Contextual Motifs: Increasing the Utility of Motifs Using Contextual Data In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 155–164.. ACM, New York. KDD, https://doi.org/10.1145/3097983.3098068.
Fraustino, JD, Brooke L, Yan J (2012) Social Media Use during Disasters: A Review of the Knowledge Base and Gaps. Final Report to Human Factors/Behavioral Sciences Division, Science and Technology Directorate, U.S. Department of Homeland Security, College Park, MD: START, 1–39.
Fredrickson, B, Mancuso R, Branigan C, Tugade M (2000) The Undoing Effect of Positive Emotions. Motiv Emot 24(4):237–258.
Fredrickson, BL, Tugade MM, Waugh CE, Larkin GR (2003) What good are positive emotions in crisis? A prospective study of resilience and emotions following the terrorist attacks on the United States on September 11th, 2001. J Pers Soc Psychol 84(2):36–376. http://dx.doi.org/10.1037/0022-3522.214.171.1245.
Guo, SJ (2017) The 2013 Boston marathon bombing: Publics’ emotions, coping, and organizational engagement. Public Relat Rev 43(4):755–767. https://doi.org/10.1016/j.pubrev.2017.07.003.
Guy, ME, Newman MA, Ganapati NE (2013) Managing emotions while managing crises. Int J Emerg Serv 2(1):6–20. https://doi.org/10.1108/IJES-07-2012-0033.
Hansen, LK, Arvidsson A, Nielsen FA, Colleoni E, Etter M (2011) Good Friends, Bad News - Affect and Virality in Twitter. In: Park JJ, Yang LT, Lee C (eds)Future Information Technology, 34–43.. Springer Berlin Heidelberg, Berlin.
Heverin, T, Zach L (2010) Twitter for city police department information sharing. Proc Am Soc Inf Sci Technol 47(1):1–7. https://doi.org/10.1002/meet.14504701277.
Hughes, A, Palen L (2009) Twitter adoption and use in mass convergence and emergency events. Int J Emerg Manag 6(3):248–260.
Hui, C, Tyshchuk Y, Wallace WA, Magdon-Ismail M, Goldberg M (2012) Information Cascades in Social Media in Response to a Crisis: A Preliminary Model and a Case Study In: Proc. of the 21st International Conference on World Wide Web, 653–656.. ACM, New York. https://doi.org/10.1145/2187980.2188173.
Hung, KC, Kalantari M, Rajabifard A (2016) Methods for assessing the credibility of volunteered geographic information in flood response: A case study in Brisbane, Australia. Appl Geogr 68:37–47. https://doi.org/10.1016/j.apgeog.2016.01.005.
Huang, Y, Wu Q, Hou Y (2017) Examining Twitter Mentions Between Police Agencies and Public Users Through the Lens of Stakeholder Theory In: Proc. of the 18th Annual International Conference on Digital Government Research, 30–38.. ACM, New York. https://doi.org/10.1145/3085228.3085316.
Jin, Y (2009) The effects of public’s cognitive appraisal of emotions in crises on crisis coping and strategy assessment. Public Relat Rev 35(3):310–313. https://doi.org/10.1016/j.pubrev.2009.02.003. Special Section on China Public Relations.
Jin, Y, Pang A, Cameron G (2012) Toward a Publics-Driven, Emotion-Based Conceptualization in Crisis Communication: Unearthing Dominant Emotions in Multi-Staged Testing of the Integrated Crisis Mapping (ICM) Model. J Public Relat Res 24(3):266–298. https://doi.org/10.1080/1062726X.2012.676747.
Juszczyszyn, K, Kazienko P, Musial K, Gabrys B (2008) Temporal Changes in Connection Patterns of an Email-Based Social Network In: Proceedings of the 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology - Volume 03, 9–12.. IEEE Computer Society, Washington, DC. WI-IAT, https://doi.org/10.1109/WIIAT.2008.245.
Kim, HK, Niederdeppe J (2013) The Role of Emotional Response during an H1N1 Influenza Pandemic on a College Campus. J Public Relat Res 25(1):30–50. https://doi.org/10.1080/1062726X.2013.739100.
Kim, HS, Lee S, Cappella JN, Vera L, Emery S (2013) Content Characteristics Driving the Diffusion of Antismoking Messages: Implications for Cancer Prevention in the Emerging Public Communication Environment. J Natl Cancer Inst Monogr 47:182–187.
Kušen, E, Strembeck M, Conti M (2019) Emotional Valence Shifts and User Behavior on Twitter, Facebook, and YouTube, 63–83.. Springer International Publishing, Cham. https://doi.org/10.1007/978-3-030-02592-2_4.
Kušen, E, Cascavilla G, Figl K, Conti M, Strembeck M (2017a) Identifying Emotions in Social Media: Comparison of Word-emotion Lexicons In: Proc. of the 4th International Symposium on Social Networks Analysis, Management and Security (SNAMS).. IEEE, Los Alamitos.
Kušen, E, Strembeck M (2018) On the Public Perception of Police Forces in Riot Events – The Role of Emotions in Three Major Social Networks During the 2017 G20 Riots In: Proc. of the 3rd International Conference on Complexity, Future Information Systems and Risk (COMPLEXIS).. SciTePress, Setubal.
Kušen, E, Strembeck M (2018) Why so Emotional? An Analysis of Emotional Bot-generated Content on Twitter In: Proc. of the 3rd International Conference on Complexity, Future Information Systems and Risk (COMPLEXIS).. SciTePress, Setubal.
Kušen, E, Strembeck M (2018) Investigation of Emotion Exchange Motifs in Bot/Human Interactions During Riot Events In: 2018 Fifth International Conference on Social Networks Analysis, Management and Security (SNAMS), 19–26.. IEEE, Red Hook. https://doi.org/10.1109/SNAMS.2018.8554860.
Kušen, E, Strembeck M, Cascavilla G, Conti M (2017b) On the Influence of Emotional Valence Shifts on the Spread of Information in Social Networks In: Proc. of the 9th IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 321–324.. ACM, New York. https://doi.org/10.1145/3110025.3110031.
Lerbinger, O (1997) The crisis manager: facing risk and responsibility. Erlbaum, Mahwah.
Li, J, Conathan D, Hughes C (2017) Rethinking Emotional Desensitization to Violence: Methodological and Theoretical Insights From Social Media Data In: Proceedings of the 8th International Conference on Social Media & Society, 47:1–47:5.. ACM, New York. https://doi.org/10.1145/3097286.3097333.
Lin, YR, Margolin D (2014) The ripple of fear, sympathy and solidarity during the Boston bombings. EPJ Data Science 3(1):31. https://doi.org/10.1140/epjds/s13688-014-0031-z.
Markus, H, Kitayama S (1991) Culture and the self: Implications for cognition, emotion, and motivation. Psychol Rev 98(2):224–253. https://doi.org/10.1037/0033-295X.98.2.224.
Matsumoto, D (1989) Cultural influences on the perception of emotion. J Cross-Cult Psychol 20:92–105.
Matsumoto, D, Seung Hee Y, Sanae N (2008) Culture, emotion regulation, and adjustment. J Pers Soc Psychol 94(6):925. https://doi.org/10.1037/0022-35126.96.36.1995.
Mazer, JP, Thompson B, Cherry J, Russell M, Payne HJ, Kirby EG, Pfohl W (2015) Communication in the face of a school crisis: Examining the volume and content of social media mentions during active shooter incidents. Comput Hum Behav 53:238–248. https://doi.org/10.1016/j.chb.2015.06.040.
Milo, R, Itzkovitz S, Kashtan N, Levitt R, Shen-Orr S, Ayzenshtat I, Sheffer M, Alon U (2004) Superfamilies of Evolved and Designed Networks. Science 303(5663):1538–1542. https://doi.org/10.1126/science.1089167.
Milo, R, Shen-Orr S, Itzkovitz S, Kashtan N, Chklovskii D, Alon U (2002) Network Motifs: Simple Building Blocks of Complex Networks. Science 298(5594):824–827. https://doi.org/10.1126/science.298.5594.824.
Mohammad, SM, Turney PD (2013) Crowdsourcing a Word-Emotion Association Lexicon. Comput Intell 29(3):436–465.
Murthy, D (2011) Twitter: Microphone for the masses?Media Cult Soc 33(5):779. https://doi.org/10.1177/0163443711404744.
Neubaum, G, Rosner L, von der Putten AMR, Kramer NC (2014) Psychosocial functions of social media usage in a disaster situation: A multi-methodological approach. Comput Hum Behav 34:28–38. https://doi.org/10.1016/j.chb.2014.01.021.
Newman, MEJ, Strogatz SH, Watts DJ (2001) Random graphs with arbitrary degree distributions and their applications. Phys Rev E 64(2):026118. https://doi.org/10.1103/PhysRevE.64.026118.
Nilsen, LG, Hafstad GS, Staksrud E, Dyb G (2018) Five reasons for using social media among young terror survivors: Results from the Utoya study. Comput Hum Behav 84:285–294. https://doi.org/10.1016/j.chb.2018.03.006.
Norris, F, Galea S, Friedman M, Watson P (2006) Methods for disaster mental health research. Guilford Publications, New York City.
Oh, O, Agrawal M, Rao HR (2011) Information control and terrorism: Tracking the Mumbai terrorist attack through Twitter. Inf Syst Front 13(1):33–43. https://doi.org/10.1007/s10796-010-9275-8.
Omodei, E, De Domenico M, Arenas A (2015) Characterizing interactions in online social networks during exceptional events. Front Phys 3(59). https://doi.org/10.3389/fphy.2015.00059.
Panagiotopoulos, P, Bigdeli AZ, Sams S (2014) Citizen-government collaboration on social media: The case of Twitter in the 2011 riots in England. Gov Inf Q 31(3):349–357. http://dx.doi.org/10.1016/j.giq.2013.10.014.
Paranjape, A, Benson AR, Leskovec J (2017) Motifs in Temporal Networks In: Proc. of the 10th ACM International Conference on Web Search and Data Mining, 601–610.. ACM, New York.
Plutchik, R (2001) The Nature of Emotions. Am Sci 89(4):344–350.
Rosas, OV (2015) The Emotional Framing of Terrorism in Online Media: The Case of Charlie Hebdo, 134–152.. Palgrave Macmillan, London. https://doi.org/10.1057/9781137546821_9.
Rotabi, R, Kamath K, Kleinberg J, Sharma A (2017) Detecting Strong Ties Using Network Motifs In: Proc. of the 26th International Conference on World Wide Web Companion, 983–992.. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva.
Russell, JA (1980) A circumplex model of affect. J Pers Soc Psychol 2:1161–1178. http://dx.doi.org/10.1037/h0077714.
Schlauch, WE, Zweig KA (2015) Influence of the Null-Model on Motif Detection In: Proc. of the IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, 514–519.. ACM, New York. https://doi.org/10.1145/2808797.2809400.
Seeger, MW, Sellnow TL, Ulmer RR (1998) Communication, Organization, and Crisis. Ann Int Commun Assoc 21(1):231–276. https://doi.org/10.1080/23808985.1998.11678952.
Shaluf, IM, Ahmadun F, Said AM (2003) A review of disaster and crisis. Disaster Prev Manag Int J 12(1):24–32. https://doi.org/10.1108/09653560310463829.
Shi, H, Shi L (2014) Identifying Emerging Motif in Growing Networks. PLoS ONE 9(6):1–12. https://doi.org/10.1371/journal.pone.0099634.
Simmel, G (1950) The sociology of Georg Simmel. Free Press, New York.
Simon, T, Goldberg A, Aharonson-Daniel L, Leykin D, Adini B (2014) Twitter in the Cross Fire–The Use of Social Media in the Westgate Mall Terror Attack in Kenya. PLoS ONE 9(8):1–11. https://doi.org/10.1371/journal.pone.0104136.
Starbird, K, Maddock J, Orand M, Achterman P, Mason RM (2013) Rumors, False Flags, and Digital Vigilantes: Misinformation on Twitter after the 2013 Boston Marathon Bombing In: Proceedings of the iConference, 654–662.. iSchools, Grandville.
Sutton, J, Shklovski I (2008) Backchannels on the front lines: Emergent uses of social media in the 2007 southern California wildfires In: Proc. of the 5th International ISCRAM Conference.. ISCRAM, State College. ISCRAM’08.
Taboada, M, Brooke J, Tofiloski M, Voll K, Stede M (2011) Lexicon-based Methods for Sentiment Analysis. Comput Linguist 37(2):267–307.
Tapia, AH, LaLone NJ, Kim HW (2014) Run amok: Group crowd participation in identifying the bomb and bomber from the Boston marathon bombing. In: Hiltz SR, Plotnick L, Pfaf M, Shih PC (eds)ISCRAM.. ISCRAM Association, State College.
Thelwall, M, Stuart D (2007) RUOK? Blogging Communication Technologies During Crises. J Comput Mediated Commun 12(2):523–548. https://doi.org/10.1111/j.1083-6101.2007.00336.x.
Topirceanu, A, Duma A, Udrescu M (2016) Uncovering the fingerprint of online social networks using a network motif based approach. Comput Commun 73:167–175.
Tran, NTL, DeLuccia L, McDonald AF, Huang CH (2015) Cross-Disciplinary Detection and Analysis of Network Motifs. Bioinforma Biol Insights 9:BBI.S23619. https://doi.org/10.4137/BBI.S23619.
Valenzuela, C, Binns K (2018) Twitter Announces Fourth Quarter and Fiscal Year 2017 Results. https://about.twitter.com. [Online; Accessed 24 May 2018].
Vieweg, S, Hughes AL, Starbird K, Palen L (2010) Microblogging During Two Natural Hazards Events: What Twitter May Contribute to Situational Awareness In: Proc. of the SIGCHI Conference on Human Factors in Computing Systems, 1079–1088.. ACM, New York. CHI, https://doi.org/10.1145/1753326.1753486.
Waters, RD, Williams JM (2011) Squawking, tweeting, cooing, and hooting: Analyzing the communication patterns of government agencies on Twitter. J Public Aff 11(4):353–363. https://doi.org/10.1002/pa.385.
Wernicke, S (2006) Efficient Detection of Network Motifs. IEEE/ACM Trans Comput Biol Bioinformatics 3(4):347–359. https://doi.org/10.1109/TCBB.2006.51.
Yeger-Lotem, E, Sattath S, Kashtan N, Itzkovitz S, Milo R, Pinter RY, Alon U, Margalit H (2004) Network motifs in integrated cellular networks of transcription-regulation and protein-protein interaction. Proc Natl Acad Sci 101(16):5934–5939. https://doi.org/10.1073/pnas.0306752101.
Zhao, Q, Tian Y, He Q, Oliver N, Jin R, Lee WC (2010) Communication Motifs: A Tool to Characterize Social Communications In: Proc. of the 19th ACM International Conference on Information and Knowledge Management, 1645–1648.. ACM, New York.
Availability of data and materials
The data-sets used and /or analyzed during the current study are available from the corresponding author on reasonable request.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.