Characterizing networks of propaganda on Twitter: a case study

The daily exposure of social media users to propaganda and disinformation campaigns has reinvigorated the need to investigate the local and global patterns of diffusion of different (mis)information content on social media. Echo chambers and influencers are often deemed responsible of both the polarization of users in online social networks and the success of propaganda and disinformation campaigns. This article adopts a data-driven approach to investigate the structuration of communities and propaganda networks on Twitter in order to assess the correctness of these imputations. In particular, the work aims at characterizing networks of propaganda extracted from a Twitter dataset by combining the information gained by three different classification approaches, focused respectively on (i) using Tweets content to infer the"polarization"of users around a specific topic, (ii) identifying users having an active role in the diffusion of different propaganda and disinformation items, and (iii) analyzing social ties to identify topological clusters and users playing a"central"role in the network. The work identifies highly partisan community structures along political alignments; furthermore, centrality metrics proved to be very informative to detect the most active users in the network and to distinguish users playing different roles; finally, polarization and clustering structure of the retweet graphs provided useful insights about relevant properties of users exposure, interactions, and participation to different propaganda items.


Introduction
The 2016 US presidential election veritably marked the transition from an age of 'post-trust' [1], to an era of 'post-truth' [2], with contemporary advanced democracies experiencing a rise of anti-scientific thinking and reactionary obscurantism, ranging from online conspiracy theories to the much-discussed "death of expertise" [3]. The long-standing debate about the relationship between media and public good has been reinvigorated: the initial euphoria about the "openness" of the Internet [4] has been taken over by a widespread concern that social media may instead be undermining the quality of democracy [5]. Media outlets, public officials and activists are supplying citizens with different, often contradictory "alternative facts" [6]. In this context, social media platforms would be fostering "selective exposure to information", with widespread diffusion of "echo chambers" and "filter bubbles" [7,8]. Propaganda actions may be now more effective than ever, representing a major global risk, possibly able to influence public opinion enough to alter election outcomes [9,10,11].
As a first step towards the disruption of these networks of propaganda, researchers have been trying to model the social mechanisms that make users fall prey of partisan and low-quality information. From a psychological point of view, news consumption is mainly governed by so-called "informational influence", "social credibility", "confirmation bias" and "heuristic frequency" [12,13]. This means that social media users tend to shape their attitude, belief or behavior based on arguments provided in online group discussions, using popularity as a measure of credibility, privileging information that confirms their own prior beliefs and/or that they hear regularly. These phenomena are exacerbated by the general incapability of making good use of the great amount of available information, a problem which can be modeled relying on the dualism of information overload vs. limited attention [14], or on the principles of information theory and (adversarial) noise decoding [15]. However, there is still a lack of evidence in the literature regarding the processes that lead to the structuration of digital ecosystems where polarized and unverified claims are especially likely to propagate virally. Are these a natural consequence of the existence of communities with homogeneous beliefs -i.e., echo chambers -and of the organized actions of "propaganda agents", or are we missing a piece?
To provide a first answer to this and other related questions, the present paper takes a data-driven approach. Specifically, we aim at demonstrating the importance of characterizing networks of propaganda on Twitter by combining the information gained by three different classification approaches: (i) using the content of tweets to determine users' "polarization" with respect to a main theme of interest; (ii) telling apart users having an active role in the diffusion of different propaganda and disinformation items related to that theme; (iii) analyzing social ties to identify topological clusters and users playing a "central" role in the network. Our main goal is addressing the following research questions: • Is modularity-based network clustering "stable" or are the patterns of cohesion among users dependent of the topics of discussion? In other terms, is the exposure/participation to propaganda of a given user a direct consequence of his/her own global interactions with other users?
• Can we use centrality metrics for detecting users playing specific roles in the production-diffusion chain of propaganda? If yes, what metrics should we mostly rely on? And are these users "consistently" involved in the diffusion of related yet different propaganda items?
• What is the role of polarization in the analysis? Can we reliably study networks of propaganda without taking into consideration the political/social "goal" of a propaganda item?
Our methodology will be applied to a case study concerning the constitutional referendum held on December 4, 2016 in Italy, by means of a dataset composed of over 1.3 millions tweets. As a side result, we will provide insights regarding the reasons of the success of specific propaganda items and the existence of "propaganda hubs" and "authorities", i.e., accounts that are critical in fostering propaganda and spreading disinformation campaigns.

Related work
As reported by a recent Science Policy Forum article [16], stemming the viral diffusion of fake news largely remains an open problem. The body of research work on fake news detection is vast and heterogeneous: linguistics-based techniques [17,18,19] coexist with network-based techniques [20,21,22] as well as machine-learning-based approaches [23,24]. Yet, (semi-)automatic debunking seems not an adequate response if considered alone [25,26]. Experimental evidence confirms the general perception that, on average, fake news get diffused farther, faster, deeper and more broadly than true news [27]. Users are more likely to share false and polarized information and to share it rapidly, especially when related to politics [28], while the sharing of fact-checking content typically lags that of fake news by at least 10 hours [29]. Furthermore, debunking is often associated to counter-propaganda and disseminated online through politically-oriented outlets, thus reinforcing selective exposure and reducing consumption of counter-attitudinal fact-checks [26]. Besides the technical setbacks, the existence of the so-called "continued influence effect of misinformation" is widely acknowledged among socio-political scholars [30], thus questioning the intrinsic potential of debunking in contrasting the proliferation of fake news. In this regard, the efforts deployed by major social media platforms seem insufficient. As of 2017, Twitter -the most widely studied of such platforms -expressed an alarmingly shallow stance towards disinformation, stating that bots are a "positive and vital tool" and that Twitter is by nature "a powerful antidote to the spreading of false information" where "journalists, experts and engaged citizens can correct and challenge public discourse in seconds" [31]. In the meanwhile, based on two millions retweets produced by hundreds thousands accounts in the six months preceding the 2016 US presidential election, researchers were coming to the conclusion that the core of Twitter's interaction network was nearly fact-checking-free while densely populated of social bots and fake news [32].
Characterizing misinformation and propaganda networks on social media thus recently emerged as a primary research trend [33,10,34]. Data collected on social media are paramount for understanding disinformation disorders [34]: they are instrumental to analyze the global and local patterns of diffusion of unreliable news stories [6] and, to a broader level, to understand the relevance of propaganda on public opinion, possibly incorporating thematic, polarity or sentiment classification [28], thus unveiling the structure of social ties and their impact on(dis)information flows [35]. Investigating the relation between polarization and information spreading has also been shown to be instrumental for both uncovering the role of disinformation in a country's political life [34] and predicting potential targets for hoaxes and fake news [36]. Finally, recent work used network-based features as instruments to describe, classify and compare the diffusion networks of different disinformation stories as opposed to "main-stream" news, making a promising step towards text-independent fake news detection [37].

Background
After the crucial 2013 election, that had imposed an unprecedented tri-polar equilibrium in the Italian political scenario, the 2016 referendum determined the collapse of the entire political scene, with the defeat of the center-left "Democratic Party" and the successive resignation of its leader and head of government, Matteo Renzi, architect of the consultation. The government reform was in fact strongly defeated, with "NO" percentages at 59.12% and "YES" at 30.88%. Offline trends showed how political polarisation and divisions among party leaders fostered the grassroots activism of the YES and NO front committees, reinforcing opposite views regarding the reform. The NO faction was a composite formation supported by both left-wing and right-wing parties, with alternative yet sometimes overlapping political justifications. Subsequently, the 2018 elections sanctioned the major rise of two euro-skeptic and populist formations, "5 Stars Movement" and "The Northern League", who were the main actors of opposition to the 2016 referendum.
The constitutional referendum offered to these rising parties an extraordinary window of opportunity in propaganda building, by imposing carefully selected instrumental news-frames and narratives and using social media as strategic resources for community-building and alternative agenda setting. Social media -and Twitter in particular -have in fact constituted a strategic tool for newly born political parties, that through the activation of the two-way street mediatization could incorporate their proposals into conventional media, still maintaining a critical, even conspiratorial attitude towards traditional media [38,39]. More generally, the dichotomous structuration of referendum offered to both political alignments the chance to align the various issues along a pro-anti/status quo spectrum. The cleavage was strategically used by both coalitions, which adopted opposite frames to stress their position: • on the one hand, the referendum was framed as a tool of "rottamazione", the process of political renovation at the center of Renzi's political agenda; • on the other one, on the NO front, it was inserted in the broader cleavage between anti-parties and traditional parties, pointed as an expression of old interests and privileges.

Propaganda items
Following the literature, in order to identify the main topics and themes of disinformation of the political campaigning we relied on the activity of fact-checking and news agencies who reported lists of (dis)information news stories that went viral during the referendum campaign. Mostly based on the work by fact-checking web portal Bufale.net [40], online newspaper Il Post [41], and political fact-checking agency Pagella Politica 1 [42], we were able to identify twelve main stories, that include both general theories and very specific news pieces, allowing us to study disinformation at different levels of granularity.
To widen the scope of the analysis, we considered news, theories and topics of discussion that could be associated to information disorders in its broader sense. This includes factual (i.e., verifiably true/false) claims as well as stories (e.g., hearsays, rumors and conspiracy theories) that cannot be deemed true/false with certainty, with no distinction between deliberate and organised disinformation/propaganda and unintentionally propagated misinformation. Differently from related work [37] that used the presence of a specific url for collecting tweets associated to a news story of interest, we set up a set of text-based queries in order to search our dataset for tweets that discuss a given topic in a broader sense. We manually identified, for each propaganda item, a wide range of keywords and associated hashtags, considering several variants of each query by taking into account synonyms and the possible presence of typos. We then proceeded to verify the accuracy of the query results by manually cross checking a random sample of tweets for each topic. In a previous work we classified these stories into four categories [43], by distinguishing entirely fabricated content from manipulated items and broader propaganda pieces. Here, we decided to focus upon the four most shared Propaganda Items (PI), and namely: PI1 A newspiece about alleged vote rigging organized by government forces; PI2 A second item framing the referendum as the political product of an illegitimately elected parliament and/or government; PI3 A third news, claiming that victory of the YES would make Italy yield national sovereignty to EU institutions (especially referring to an hidden clause in art.117); PI4 A fourth -more general -piece supporting the claim that a victory of the YES would have caused a shift towards authoritarianism.
All the most diffused news items can be broadly located along the spectrum of different arguments of conspiracy theories, traditionally driven by a belief that a powerful group of people is manipulating the public, while concealing their activities. As some scholars have demonstrated [44], conspirationism is associated with different sub-dimensions of populist attitudes -people-centrism, anti-elitism, and a good-versus-evil view of politics -, with coup d'état attempts and secret plots organised by politicalélites to gain further power or consolidate their privilege or the explicitly plot to notch the integrity of the electoral process by gaining unauthorized access to voting machines and altering voting results.

Classification of tweets and users
After having identified the most relevant news-pieces in our dataset, we aimed at gaining a better understanding of users in our dataset and the relation between polarization and disinformation. To classify the stance of each tweet with respect to the referendum question, we adopted a semi-automatic self-training process, described more in detail in [43]. The underlying idea is that political exchanges in social-media platforms exhibiting "a highly partisan community structure" with "homogeneous clusters of users who tend to share the same political identity" [45]. This is reflected on Twitter by the usage of different patterns of hashtags by supporters of opposite factions [46]. We therefore built a hashtag graph, pruned it of a few irrelevant or uninformative yet popular hashtags and used Louvain's algorithm to cluster such hashtags based on their mutual co-occurrence patterns. We found the two greatest clusters to clearly identify the YES and NO fronts, thus we used hashtags in these clusters to extract a training set composed of tweets labelled as follows: −1 (NO) if the tweet only contains hashtags from the NO cluster; +1 (YES) if the tweet only contains hashtags from the YES cluster; 0 (UNK) if the tweet contains a mix of hashtags from the two clusters. After some tuning, we selected doc2vec [47] feature vectors and a Gradient Boosting Classifier to define a text-based classifier to be used to extend the labeling to all tweets in the dataset.
On the whole, UNK tweets were substantially negligible -although this may be due to limitations of the classifier [43] -while NO tweets were almost 1.5x more frequent than YES tweets, supporting the diffused belief that the NO front was significantly more active than its counterpart in the social debate. Significantly, we also obtained a continuous score in [-1,1] for users, since a user can be classified with the average score of his/her tweets. These user-level scores are used in the following sections for correlating polarization with other network properties of our corpus of users.
For the sake of clarity and completeness, the hashtag graph and its cluster-graph -wherein each cluster is contracted into a single node -are shown in Figure 1. We see that: (i) hashtags used by the NO and YES supporters are strongly clustered; (ii) "neutral" hashtags (such as those used by international reporters) also cluster together; (iii) a few hashtags are surprisingly high-ranked, such as "#ottoemezzo", a popular political talk-show being central in the NO cluster -thus confirming regular patterns of behavior in the "second-screen" use of social network sites to comment television programs [48]. In particular, the two largest clusters of hashtags clearly characterize the two sides: the YES cluster is dominated by the hashtags "#bastaunsì" ("a yes is enough") and "#iovotosi" ("I vote yes"), whereas the NO cluster by "#iovotono" ("I vote no"), "#iodicono" ("I say no") and "#renziacasa" ("Renzi go home"). In this perspective, the jargon of both communities show clear segregation and high levels of clustering by political alignments, as expected.

Polarized retweet graphs
The main objects of analysis of this paper are a set of interaction networks extracted from a Twitter dataset of more than 1.3 million tweets. Each of these networks is formally represented as a graph G = (V, E), whose vertex set V models a corpus of social media users. As often done in the literature [49,34],  we consider a directed and weighted retweet graph, wherein the existence of an edge e = (u, v) means that user u retweeted user v at least once in the dataset. With respect to other possible network representations, retweet graphs have been found to present many of the characteristics of social graphs (e.g., being "small-world" and "scale-free") and to better describe the trust relationship between social media users [49]. In our graphs, edges are weighted by a parameter w e equal to the number of retweets between a given pair of users. Nodes are instead endowed with a "polarization" attribute p u ∈ [−1, 1] -defined in the previous section -equal to the average polarization of the tweets and retweets of that user. Specifically, in this paper we consider the following six graphs: • The whole retweet graph, obtained from the entire dataset 2 .
• The P/D (Propaganda/Disinformation) retweet graph, obtained from the set of all tweets that matched any of the queries defined in the Background Section, i.e., tweets related to any of the 12 news stories.
• The PI1, PI2, PI3 and PI4 retweet graphs, induced by the set of tweets that satisfied each of the four selected propaganda items, taken individually.
The subgraph of the whole graph composed of the 1000 vertices having greatest pagerank is shown in Figure 2. We can clearly see a few features of the graph that will be better discussed in the following: a general prevalence of NO edges (i.e., tweets), multiple NO-leaning clusters and a single main YES-leaning cluster.
A first relevant perspective on our dataset is obtained by considering how the vertex set of the P/D graph may be decomposed based on the belonging of its users to the individual PI graphs. Figure 3 shows that, altogether, the four items of propaganda we selected involve approximately 92% of all users of the P/D graph, with users only involved in other propaganda/fake news stories summing up to just 7.99%. We can thus safely focus on these four items without a significant loss in the generality of our results. Somewhat surprisingly, almost 68% of the users taking part to any of the four selected PIs are actually only involved in a single one of them, and only a negligible 1.5% is involved in all four items. This immediately warns us of the pitfalls of considering disinformation as a whole.
A second aspect to consider is the distribution of the polarization attribute p u across the six graphs. Figure 4 shows that, overall, users appear to be strongly polarized, with two huge spikes at -1 and +1 for the whole graph. When we switch to networks of propaganda, however, users seem to be generally less polarized. This apparently counter-intuitive phenomenon is a consequence of our scoring method and of the much higher average activity of users involved in these networks. Indeed, a user's polarization is well-definite when that user has a single tweet and gets blurrier as the number of tweets increases, because of the contribution of many tweets not all of which are necessarily equally polarized. The average number of tweets per user in our propaganda networks is 8 to 14 times greater than the average computed over the whole graph. At the same time, users with a single tweet are 37% of the whole graph, but just 1% to 5% of the P/D and PIs graphs.
The distribution of PI1, PI2 and PI3 follows the overall trend of the P/D graph, that is, a general prevalence of NO users over YES users. Since all 4 selected items, as well as most of the 12 items, are pro-NO, this may be interpreted as a prevalence of propaganda over counter-propaganda. In that sense, PI4 is the exception: a clear example of a topic mostly used by one side (the YES coalition) to accuse the other of using deceptive propaganda. This is a first element in favour of the importance of accounting for polarization when characterizing these propaganda networks and their users.

Clustering structure
The clustering structure of a retweet graph highlights relevant properties of how users and groups of users interact with each other, and of how easily information flows through the graph. Along this line, recent work provided clear evidence that modularity-based clustering applied to retweet graphs brings to light communities of users with strong homophily/affiliation within which propaganda and polarized information spreads especially well [50,46]. By characterizing and comparing the clustering structure obtained for our six graphs through the well-known Louvain algorithm we expect to better understand the emergence of networks and sub-networks of propaganda and measure their persistence. To start, in Figure 5 we plot the size distribution of communities for the P/D and PIs retweet graphs. At a high level, we see that the distributions of all PIs graphs are somewhat similar -especially for PI1 and PI3and that in all cases only a few clusters have a relevant size. Figure 6 shows the polarization of the 10 largest clusters for each of the six graphs. To obtain a single polarization score for a given cluster c, we computed the number of YES users in c, denoted Y c , the number of NO users in c, N c , and defined p c = Yc−Nc Yc+Nc . This definition guarantees that p c = +1 if N c = 0, p c = −1 if Y c = 0 and p c = 0 if Y c = N c . Yet, if compared with just taking the average polarization of the users in c, this measure is more robust with respect to classification accuracy -under the assumption that telling apart YES and NO users is easier than measuring the exact polarization of each user. We can clearly see that the clusters of the networks of propaganda are generally and significantly more polarized than the clusters of the whole graph. We also see that the overall prevalence of NO users in the P/D, PI1, PI2 and PI3 graphs already emerged in Figure 4 is reflected in a greater number of NO clustersthe same happening in PI4 for the YES front.
The main clusters of the whole graph deserve special attention. As already observed in the literature [46], in fact, they quite clearly reflect political affiliation: • Cluster 262 (16710 members) appears to group together members and supporters of the "Democratic Party", including Government members (such as PM Matteo Renzi and the Minister of Reforms Maria Elena Boschi), the official YES Committee and Renzi's foundation 'Leopolda', among the others.
• Custer 3 (11111 members) is expressive of the "5 Star Movement" community. Only two of the most active users (Minister Danilo Toninelli and Senator Elio Lannutti) are official party members, however, whereas the most influential actors belong to the militant base.
• Cluster 2 (7671 members) groups the members of the souverainist right, including the two politicians Matteo Salvini and Giorgia Meloni, their political parties, and a number of supporters.
In this context, two large and barely-polarized clusters come to light. On the one hand, cluster 19 (13085 members) seems to validate the claim that "structure segregation and opinion polarization share no apparent causal relationship" [51]. It includes left-wing opponents to the referendum as well as media accounts and has very low polarization (-0.04), a probable evidence of the willingness of the leftwing members of the NO alignment to maintain a cross-partisan interaction with the democrats. On the other hand, cluster 181 (6493 members) completely escapes the party affiliation logic. Apart from @europeelects, which produces poll aggregation and election analysis in the European Union, all the other accounts belong to international militants of the souverainist and anti-globalization movement: they are Brexit supporters, Italian pro-Trump advocates, or journalists covering such topics in their reporting activities. Now, we aim at assessing to which extent the obtained clusters are influenced by the choice of a specific PI, that is, whether the patterns of cohesion among different users seem to be coherent across different topics of discussion. In Figure 7 we use the Adjusted Mutual Information (AMI) to compare the clusters emerged in different graphs. It is worth recalling that the AMI of two partitions is 1 if the two partitions are identical, it is 0 if the mutual information of the two partitions is the expected mutual information of two random partitions 3 , and it is negative if the mutual information of the two partitions is worse than the expected one. Of course, when comparing the partitions obtained for any two graphs, we just consider the users that are common to both graphs. In addition, in Figure 8, we provide a more pointwise analysis of the 10 greatest communities of each PI graph, showing how users of these clusters distribute over the greatest 30 communities of the whole and P/D graphs. Precisely, in each heatmap the cell at the intersection of row i and column j measures the proportion of users of cluster i in the considered PI graph that lie in cluster j of the compared graph.
The two figures together provide clear evidence that users are clustered in a rather unstable way, especially when we compare networks generated by individual PIs with the whole retweet graph and with each other. The topological organization of the NO front is adequately expressive of different ideological affiliations of NO sponsors, but these differences are not clearly visible in the participation to clusters of the PI1, PI2 and PI3 networks. Assuming that selective exposure and social validation are core driving polarization mechanisms [51], two main interpretations are possible: either (i) being generally NO-leaning is enough to trigger the exposure to these three PIs, with the actual political community a user belongs to playing a marginal role; or (ii) the interactions occurring globally on Twitter -and, as such, global information flows -are only partially responsible of the tendency of users to diffuse propaganda and disinformation items. In PI4, on the other hand, most clusters are de facto sub-clusters of a macrocommunity of the whole retweet graph. This is easily explained by the different polarization emerged in Figure 6: the macro-community is cluster 262, which we already identified as the "YES community". The YES cluster seems to be driven by both an effort of community building and the attempt to de-legitimate the NO front by debunking its news-claims and propaganda items. As a consequence, YES users stay attached in the P/D and in other PI graphs, while they splits into sub-communities in PI4, showing a stronger degree of internal homogeneity and highlighting a polarized conversational archetype, with partisan actors and segregated community structure and discussion.

Users' Centrality
In this section we study the role played by the users in the four propaganda items selected and discussed in the previous sections. To assess the activity of each user we compute the following centrality measures on the retweets graph: PageRank, In-Degree, Out-Degree, Authority Score and Hub Score [53]. The centrality measures we chose are often used for networks analysis and their interpretation depends on the phenomenon modeled by the network. In our graph, the In-Degree tells us which are the users that are more often retweeted, i.e., the users creating contents that are spread on the network. The PageRank tells us which are the users that are most likely "visited", i.e., the users whose contents are most probably read if the retweets graph is used to surf the network. The Out-Degree tells us which are the users that more often retweet, i.e., the users playing a main role in the information diffusion. Finally the Hub Score and Authority Score are interconnected: the former tells us which are the users that more often retweet contents created by an authority, we call these users hubs; the latter tells us which are the users creating the main contents about a discussion topic, we call these users authorities. The main difference between the Hub Score and the Out-Degree is about the content of the retweets done by a user, in the former case the user retweets authoritative contents, in the latter case the user does not show any preference about the tweet's origin. Similarly, the main difference between the Authority Score and the In-Degree resided in the type of users that usually retweet contents produced by a given user, in the former case the users retweeting these contents are hubs, in the latter case there is no distinction among users. Thus, a user has a high Authority Score if she has a high degree and the users retweeting her contents are hubs. Tables  1, 2, 3, 4 and 5 show for each propaganda item the top 10 users of each metric. The color of each cell denotes the polarization of the user, blue is used for NO supporters and red is used for YES supporters.
The color's intensity shows how strong is the polarization, i.e., the darker is the color the more polarized is the user.
Our results show a few relevant aspects. First, if we look at the whole retweet network the most active users are almost all NO supporters, as showed in Table 1, albeit the number of NO and YES users is quite balanced in the network -as showed in Figure 4. Indeed, we find only few accounts belonging to the YES supporters in the top position of the five metrics. Additionally, by analyzing individual propaganda items we observe that the set of most active users, their ranking and their polarization change depending on the considered PI and metrics. In PI1, PI2 and PI3 the most active accounts are NO supporters, while in PI4 the YES supporters are the most active and numerous, in accordance with Figure 4 and despite PI4 also being a pro-NO item. The analysis of users polarization is thus essential to understand the role played by the main actors inside each network: if we only considered the centrality of the users, without looking at their polarization, we would not be able to distinguish between accounts that are contributing to the diffusion of a fake news and accounts that are working against, i.e., the debunkers. We also see, again in accordance with the analysis presented in the Clustering structure Section, that different PIs see different users take on different roles. Well known public figures -such as "mattosalvinimi", "giorgiameloni" and "matteorenzi" -are a minority with respect to grassroots activists, and users playing a central role in a specific network of propaganda and/or with respect to a specific metrics are absent or not as relevant in other cases -such as "cinmir89" or "proudman811".
To further investigate the persistence of these rankings across different networks of propaganda, in Figure 9 we present a set of correlation matrices that broadly corroborate the previous findings. Specifically, we computed the correlation between each propaganda item for each centrality measure in order to better understand the role of the users that were active in more than one propaganda item. We rely on Spearman's rank correlation coefficient, rather than the widely used Pearson's, because we are neither especially interested in verifying linear dependence, nor we do expect to find it. We are more interested in the possible monotonic relationship between centrality measures as determined by Spearman's correlation.
As already observed, combining the centrality and polarization data, we notice that in the PI4 network there is a different community of users that is active and that is spreading information with respect to the other PI networks. This behaviour is clearly visible from the Hub Score matrix (Figure 9d) and partially from the Out-Degree matrix (Figure 9b). In the former the anti-correlation in row PI4 shows the existence of a different community of spreaders in PI4 with respect to other PI. A community composed of users that are absent, less active or play a different role in other networks. In the Out-Degree matrix we have almost no correlation in row PI4 except for PI2. This difference is due to the presence of a small, not negligible, community of YES supporters in PI2 as showed in Figure 6. Whereas, the PageRank and In-Degree matrices show that the relevance of the accounts creating contents is more stable than those diffusing the information. Finally, the Authority Score matrix shows that, although the contents creator accounts are stable, their role change in the network. The same account is considered more authoritative in one network and less in the other.
What is happening in the other networks can be better understood by looking at figures 9f-9i where we computed separately the correlation for NO and YES supporters for the Authority and Hub Scores, that overall appear to be the most informative. To better focus our analysis we excluded the PI4 row. Our results show that among YES supporters the content creators accounts are not stable and their role change depending on the propaganda item selected. On the other hand, the role of the accounts spreading information is more stable, meaning that for different networks there are different authorities, but the hubs are the same. For what concerns the NO supporters we have that both authorities and hubs relevance changes depending on the propaganda network. Thus there is probably a more efficient synergy among NO supporters between authority and hub accounts.

Conclusions
The paper aimed at providing new insights into the dynamics of propaganda networks on Twitter. The results of our study are partly in line with existing research. Modularity-based clustering, applied to retweet graphs, pictured a wide panorama of communities of users with strong homophily/affiliation and polarized position. As expected, the clusters of propaganda networks were generally and significantly more polarized than the clusters of the whole graph and the topological organization proved to be highly representative of the ideological affiliation of users. The comparison between clusters in different graphs reveals that users' clusters are rather dynamic, particularly when comparing networks generated by individual propaganda items with the whole retweet graph and with each other. It seems that global clusters, often associated with information exposure, are only partially responsible of the tendency of users to diffuse propaganda and disinformation items. When it comes to taking a position on a controversial topic, users tend to group with different people with respect to those they usually connect to in the whole graph, and the "high-level" polarization of a user -such as the NO vs. YES leaning in our case -may have a more prominent role than his/her political Table 3: PI2 retweet graph: top 10 accounts by centrality.
In-Degree Out-Degree Authority Score Hub Score PageRank   claudiodeglinn2  marino29b  claudiodeglinn2  nativiitaliani  claudiodeglinn2  matteosalvinimi  gincarbone  matteosalvinimi  giorgiomorresi  angelosica1965  luisaloffredo28  nativiitaliani  patriotail  mania48mania53  matteosalvinimi  patriotail  luisaloffredo28  luisaloffredo28  gincarbone  sevenseasmarina  sevenseasmarina  giorgiomorresi  deglclaudio  caspanistefania  deglclaudio  giorgiameloni  malaspinadavide  civico21  pietrof70  patriotail  liberatilinda  piras zia  angelosica1965  ilpellicano88  xmeridio78  civico21  ori254  sevenseasmarina  celestinoceles7  civico21  deglclaudio  celestinoceles7  carmentpf  archidevivaio  liberatilinda  5bc32772e3fb467 claudiodeglinn2 valy s gidal randagio toscaross affiliation. This is especially visible for users involved in propaganda -as opposed to counter-propaganda. The combined analysis of cluster-to-cluster intersections and centrality metrics additionally indicates how different propaganda items are associated to different users with authoritative roles. The correlation of centrality metrics across different networks provides further insights: (i) the Authority and Hub Score seem the most informative metrics for studying networks of propaganda, thanks to their ability to tell apart content creators and spreaders; (ii) the role of content creators is taken by different users for different propaganda items, independently of clusters polarization; (iii) spreaders are instead generally more "consistent".
Overall, the propaganda community depicted in this study, far from being monolithic, has a considerable degree of internal variability, in terms of central actors, topics and opinion polarization. Polarization with respect to a main theme, transversal to the considered propaganda items, emerged as a fundamental parameter in governing users behavior. Further directions of research could involve other clustering algorithms as well as dynamic influence metrics, in order to gain deeper knowledge on the relationship between exposure to propaganda and the general structure of users interaction.

Availability of data and materials
Part of the code used in this paper will be included in the network analysis toolbox DisInfoNet, currently under development by the partners of the Project "SOMA" at https://gitlab.com/s.guarino/disinfonet. DisInfoNet is presented in a previous conference paper [43] and will be released by the end of the SOMA Project. The entire dataset used during the current study is not publicly available due to Twitter's policies. The ids of the tweets are available from the corresponding author on reasonable request.

Funding
This work was supported in part by the Project "SOMA", funded by the European Union's Horizon 2020 research and innovation programme under grant agreement No 825469. The European Commission had no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript. Any opinion, finding, and conclusions expressed in this paper only reflect the views of the authors.