Skip to main content

A network perspective on intermedia agenda-setting


In Communication Theory, intermedia agenda-setting refers to the influence that different news sources may have on each other, and how this subsequently affects the breadth of information that is presented to the public. Several studies have attempted to quantify the impact of intermedia agenda-setting in specific countries or contexts, but a large-scale, data-driven investigation is still lacking. Here, we operationalise intermedia agenda-setting by putting forward a methodology to infer networks of influence between different news sources on a given topic, and apply it on a large dataset of news articles published by globally and locally prominent news organisations in 2016. We find influence to be significantly topic-dependent, with the same news sources acting as agenda-setters (i.e., central nodes) with respect to certain topics and as followers (i.e., peripheral nodes) with respect to others. At the same time, we find that the influence networks associated to most topics exhibit small world properties, which we find to play a significant role towards the overall diversity of sentiment expressed about the topic by the news sources in the network. In particular, we find clustering and density of influence networks to act as competing forces in this respect, with the former increasing and the latter reducing diversity.


The news media is an important information source for much of the world’s population, as it feeds into opinions and choices (McCombs and Shaw 1972; McCombs et al. 1997; Wilczek 2016). Frequently referred to as the ‘gatekeepers of information’, journalists have been shown to play a powerful role in influencing public opinion through determining what stories (or elements thereof) are presented to the public, how content is framed, which elements are emphasized, and how the public forms associations between topics covered by the news (Guo et al. 2012; Wilczek 2016). While much research has been done on identifying bias in the mainstream media (Wilczek 2016), and has more recently focused on ‘fake news’ and misinformation (e.g., Vargo et al. (2018); Shu et al. (2019); Sikder et al. (2020)), little research has thus far looked at studying the dynamics underpinning the formation of opinions within the media. News organisations do not construct narratives in isolation, but are instead influenced by each other. This phenomenon is the subject of intermedia agenda-setting theory (Harder et al. 2017), which explains the ‘co-orientation’ of the narratives in the media as the result of exposure to both economic and local social factors (Guo et al. 2012). The topics that the media covers and the context in which headlines are presented have been shown to impact observable phenomena ranging from voter behaviour (McCombs et al. 1997) to macroeconomic indicators (Nyman 2016). Affective attributes of the news content, such as sentiment and tone, have been shown to affect voters’ judgments on political issues (Coleman and Wu 2010; Sheafer 2007) and people’s perception of corporations (Kim and Kiousis 2012). Because of its far-reaching impact, it is valuable to understand the opinion dynamics of the media at the level of individual news organisations as it ultimately determines what content is made available to the public as well as how it is presented.

The media continues to play a central role in directing what information the public attends to, how it is presented, and with what they associate that information. It is therefore important to have a stable, balanced and diverse set of opinions being presented to the public. Research on agenda-setting theories of the media (McCombs and Shaw 1972) generally either aggregate the media into a single representative agent or treat the media as a collection of independent actors. Here, we demonstrate an alternative representation in which news sources can be understood as nodes in a social network whose actions are influenced by the other nodes in the network. By framing the system as a social network, we are able to exploit the tools provided by network science to aid in our understanding of how such a system evolves. Numerous studies have demonstrated how network structure impacts signal propagation (Delre et al. 2007; Watts and Dodds 2007). However, we are not aware of any research to date that has employed a network approach to explore the propagation of information within the media.

In many real-world social networks, connections and interactions between agents can be explicitly gathered from the data, such as through citation in academia (Li et al. 2019), or liking and following in social media (Anger and Kittl 2011; Livan et al. 2017). In the domain of the news media, whilst there are naturally interactions between agents, these are latent and network structures must first be inferred from the available data before one can study the dissemination and propagation of news and sentiment through them. Previous work on studying the dissemination of news data from a social network modelling approach has explored what content the media presents (Vargo and Guo 2017) and how breaking news stories are picked up and propagated through the media (Liu et al. 2016), but not how the network of interactions of the news sources impacts how content is framed.

In this work, we tackle the above by presenting a method to operationalise intermedia agenda-setting theory in which each news source is a potential node in a directed network and edges exist between news sources when there is empirical evidence that the sentiment expressed by one source on a topic is Granger-caused by that of another. We implement the proposed method and employ it to answer the following research questions (RQs):

  • RQ1: Can ‘bellwether’ news sources be identified? That is to say, sources who have influence over the way in which the rest of the media portray a story. Moreover, are such bellwethers more central in networks that capture intermedia influence?

  • RQ2: Is there emergent structure in the networks that can potentially impact how a news story is depicted and propagated across the media?

  • RQ3: How does the structure of these networks impact the diversity of sentiment that exists across the media on a given topic?

We propose and implement the aforementioned method, which constitutes our first contribution. We perform topic modelling on a collection of news articles to identify 200 topics discussed in the news media between May and December 2016. Sentiment analysis is then used to model the views adopted by each news source and how these vary over time. Lead-lag relationships between the sentiment expressed across news sources are identified to construct networks representing how sentiment is transmitted and adopted by the media. Our second contribution is then in applying this method to explore the diffusion of sentiment in the media. Our final contribution is towards the discussion of the ability of the media to self-regulate and provide a varied perspective for public opinion formation.

We find:

  1. 1.

    There exist significant lead-lag relationships in which specific news sources can influence the sentiment with which a story is presented.

  2. 2.

    Intermedia influence networks are able to capture characteristics that are consistent with intermedia agenda-setting theory, such that certain elite news sources sit more centrally.

  3. 3.

    Influence is distributed across media sources and bellwether behaviour varies depending on the topic.

  4. 4.

    A significant proportion of topics exhibit ‘gatekeeping’ tendencies in which a small subset of news sources have disproportionately large influence.

  5. 5.

    Influence networks exhibit small-world characteristics with communities of densely connected subgroups.

  6. 6.

    The overall diversity in sentiment is impacted by the structural features of intermedia networks. In particular, the density of such intermedia networks has a negative impact on the sentiment diversity on a topic whereas the strength of the network clustering has a positive impact on it.

The remainder of this paper is organised as follows: in the “Related work” section we provide a background and overview of related work on influence analysis in social networks. In the “Methodology” section we outline our methodology, and in the “Results” section we provide a detailed analysis of our results and discuss their implications. Finally, the “Conclusion” section provides some conclusive remarks and proposes potential further work.

Related work

Since the beginning of the 20th century, numerous theories have emerged that try to explain the role of the media in society and the impact that it has on public opinion (and vice versa). Arguably the most broadly accepted theory today is based on agenda-setting theory (McCombs and Shaw 1972) and extensions thereof. It argues that the media does not regulate what audiences think, but rather it tells them what to think about by controlling what content is accessible. It paints the media as ‘gatekeepers’ of information about reality by selecting, omitting, and framing what issues are reported on, and to what extent. This then prompts the public to perceive selected issues as being more/less important than others, or drawing links between events, which ultimately changes how the audience engages with the issue and thereby influences the public’s narrative (McCombs 2005).

Beyond looking at the relationship between the media and the public, intermedia agenda-setting research concerns itself with measuring the extent to which news content transfers between media. The highly regarded media have been shown to exert influence over their peers (Vargo and Guo 2017; Harder et al. 2017). This “co-orientation” within the media can be attributed to several factors. One explanation is that it is the result of ‘churnalism’ whereby media sources re-hash the content of their peers without having to expend the resources that are required to construct the narrative (Harder et al. 2017; Lim 2011). Another theory is that co-orientation is sociopsychological in origin, whereby the major news sources are regarded as better judges of what is (subjectively) newsworthy (Harder et al. 2017; Harcup and O’Neill 2017). Recently intermedia agenda-setting has also considered the transfer of content between different types of media, such as whether, and when, the mainstream media influences social media, and vice versa. It has been shown, for instance, that the online news media influences the content of politically aligned Twitter accounts (Harder et al. 2017).

So far, intermedia agenda-setting research has focused on testing a priori hypotheses about the expected influence exerted by certain news sources. For example Vargo and Guo (2017) find that The New York Times and The Washington Post have the ability to set the agenda of others. In Welbers et al. (2018), the authors find a similar effect for Algemeen Nederlands Persbureau (ANP) in the Dutch press, while Lim (2011) identifies intermedia agenda-setting effects of and on other South Korean online media websites.

Here, by operationalising intermedia agenda-setting in network terms, we circumvent the need for a priori assumptions about which sources may act as agenda-setters. A core component in this is to automatically capture and interpret how the agents in the news media influence one another. As will be discussed in the next section, the concept of ‘influence’, and how it is measured, has emerged as a popular field of research in applied mathematics and computer science.

Social network influence

There is no single mathematical definition of social network influence. This is largely because it varies between contexts, and even within the same domain, it can be unclear how influence is defined. Peng et al. (2018) describe four basic components that define influence, regardless of the domain: (1) social influence exists between two agents, the influencer and the influencee (aka. the influenced); (2) influence is a function of uncertainty, with zero uncertainty if the influencer has no doubt that the influencee will perform an action and the maximum amount of uncertainty when the influencer has no influence over the influencee’s actions; (3) the level of influence can be represented by a real number which describes the uncertainty of the influencer’s ability to dictate the influencee’s action; (4) influence does not need to be symmetric.

Consistent with the above criteria, numerous evaluation metrics have been proposed to assign influence values to agents in social networks, some based purely on centrality and network topology, and others on entropy measures. Degree, closeness, PageRank, HITS and Katz centralities have all been used in past studies (Kiss and Bichler 2008; Peng et al. 2018; Romero et al. 2011; Weng et al. 2010).

While influence measures based on network topology are convenient because they may be calculated for an arbitrary network, they ignore potentially valuable information about the content of the signals being passed through networks. Entropy measures of influence are one method for including signal content. Ver Steeg and Galstyan (2013), for example, propose the use of transfer entropy, a measure for determining how much better the sequence of signals emitted by one agent can be predicted by including the signal history from another agent. Other methods have focused on whether the signals transmitted by one agent Granger-cause those of the other agents within the same network (Vargo and Guo 2017; Stern et al. 2018). We will build networks of influence based on the latter approach.

Opinion and emotion influence

Prior research has approached the question of how actors influence one another through emotion from two differing disciplines. The idea of how emotions and opinions are influenced has been studied extensively in the fields of psychology, cognitive science and sociology (Hatfield et al. 1993; Moors 2009; Van Kleef et al. 2011). More recently though, computational and mathematical models have explored how sentiment and opinions spread through social networks. The addition of emotion in the context of influence within social networks has been examined in Twitter (Stieglitz and Dang-Xuan 2013; Ferrara and Yang 2015; Xiong et al. 2018), Facebook (Coviello et al. 2014), and blog posts (Hui and Gregory 2010; Song et al. 2007). Semantic information expressed by actors in social media has been proposed as a method to augment influencer detection beyond what is possible using only network topology (Song et al. 2007; Zhou et al. 2009). Inversely, incorporating information about a user’s influence based on network topology has been used to augment sentiment analysis (Eliacik and Erdogan 2018; Hui and Gregory 2010). Furthermore, contagion models have been developed to study how emotion and emotive content propagates and diffuses throughout social networks; sentiment in social media-based content has been shown to correlate with both the speed and quantity of information sharing (Brady et al. 2017; Hill et al. 2010; Stieglitz and Dang-Xuan 2013; Xiong et al. 2018).


Identifying sentiment-Loaded topics

Whereas prior related studies have looked at how news stories are picked up on, we are concerned not only with what is discussed but also with capturing how it is presented. In other words, understanding from what angle the story or ‘theme’ is depicted as well as how this varies across different news sources and different topics. Therefore, our goal is to construct time-series that encapsulate two factors: (1) the prominence that a source assigns to a recurring topic, and (2) the sentiment with which it covers it. We do this by performing topic modelling on news articles to identify what the stories are about; we then scale the topic assignments according to the sentiment of the articles. This gives us an indicator of the affinity between the source of the articles and the topics the article covers.

Our method produces a |T|×|A|×|K| tensor, where |K| is the number of topics, |A| is the number of news sources, and |T| is the length of our time-series. In other words, each news source of an article aA has |K| distinct time-series that describe how their sentiment towards each of the topics evolves through time. Topic extraction is performed by training a Latent Dirichlet Allocation (LDA) topic model (Blei et al. 2003) with |K| topics on a corpus containing news articles from a range of different sources. LDA is an unsupervised learning technique that models documents as a multinomial distribution over |K| topics, and topics as a multinomial distribution over |V| words. It has been shown to successfully capture semantically coherent clusters of topics in medium-length texts such as news articles, blogs and journals (Blei et al. 2003). Using the LDA model, each article in the corpus is mapped to a multinomial topic distribution that encodes how ‘much’ of each of the |K| topics is entailed within the article. Topics are modelled as being static over time (i.e., the per-topic word distributions are independent of time) because long-term media coverage has been shown to have a greater effect on policy-making than topics that are covered over the short- or medium-term through repetition and cumulative exposure (Joly 2016).

The sentiment for each article is subsequently measured following Tuckett et al. (2014), i.e., counting the relative frequency of ’excitement’ and ‘anxiety’ keywords that have been shown to capture the affective impact that articles may have on a reader. That is, the sentiment value of an article is the number of excitement keywords minus the number of anxiety keywords, all as a fraction of the length of the article. Despite there being more sophisticated methods of capturing the sentiment of a document, using pre-defined keywords allows us to explicitly remove those words from the vocabulary that constitutes the topic distribution of words in the LDA model. This ensures that when the sentiment dynamics of each of the topics are later analysed, they are not biased by intrinsically ‘positive’ or ‘negative’ topics.

To capture how a particular news source ‘feels’ about a topic on any given day, each of that source’s documents, from that day, are assigned a topic distribution using the LDA model. The topic probabilities for each of that source’s articles are scaled by the sentiment score of that article and then summed together across articles. Repeating this process daily for each news source allows us to populate a tensor G where Gt,a,k is the sentiment of a news source a on topic k on a day t. The steps for assigning topic-sentiment time series are presented in Algorithm 1.

The sentiment score assigned to a given news source on a given topic on a given day is influenced by three factors: the strength of the emotion in the content they published, the topic entropy of their articles (i.e., the probability mass of the articles that are attributed to a topic), and amount that they invest in a topic (i.e., the number of articles that have non-zero probability mass assigned to the topic). In other words, a source’s score will be high if there is substantial positive emotion expressed in their content and/or their writing is about a specific topic and/or there is a proportionately large number of their articles about said topic. Concerning the final point, we note that news sources who publish more frequently may be assigned greater absolute sentiment scores on a topic. Our analyses throughout the rest of the paper, therefore, will only hinge on the relative fluctuations of such scores, and will be agnostic to absolute size.

Topic-Level influence networks

There have been many proposed definitions of influence, and metrics vary widely depending on the discipline, application area and data. The definition used here follows that of Vargo and Guo (2017) and Stern et al. (2018) in which, for two agents A and B, A is considered to influence the opinion of B on a given topic if the prior opinion expressed by A on that topic increases our ability to predict that of B. More formally, if the conditional probability of observing B(t) is dependent on A(tδ(t)) for some time t and lag δ(t), then A influences B. This lead-lag relationship is measured using the Granger causality test (Granger 1969). Source A’s sentiment leads that of B on a particular topic if A’s sentiment time series Granger-causes that of B on that topic with a p-value lower than 0.05. We account for family wise error rate by applying the false discovery rate correction, which we implement with the Benjamini-Hochberg procedure (Benjamini and Hochberg 1995).

For each pair of news sources, once lead-lag relationships between agents are identified, they can be used to populate topic-level influence networks. Graphs depicting the relationships between pairs of news sources are then constructed separately for each topic. Where a lead-lag relationship has been determined, both agents are added to the relevant graphs as nodes. An edge is drawn between them, with an arrow leading from the influencer to the influencee, resulting in an unweighted directed graph. This procedure is sketched in Fig. 1.

Fig. 1

The network-construction process: A topic-specific sentiment time-series is inferred for each news source based on the content that they publish. Networks are inferred for each topic based on the empirically measured conditional dependence between the sentiment time-series of the news sources

Network sizes vary across topics because a news source is only added to a network if it has at least one edge. This results in sources that are not active in the discussion of a given topic, or which have no impact on their peers, to be excluded. Figure 2 shows an example of such a network.

Fig. 2

An example of the intermedia influence networks for a topic about the 2016 US presidential election. Each node is an agent within the news media and an edge exists going from an influencer to an influencee

Opinion diversity

One of the questions we wish to address is whether we can deduce how diverse the sentiment surrounding a topic is, given what we are able to capture of the influence that news sources have on one another. We, therefore, desire a measurable statistic that represents how varied news sources are in their sentiment towards a given topic. For this purpose, we define a measure of ‘opinion diversity’ on a topic k as the median of the daily cross-sectional variance of sentiment across news sources:

$$\begin{array}{*{20}l} y_{k} &= \text{median}\left\{\text{Var}[G_{t,\bullet,k}]| t \in \{0,\dots,T\} \right \} \end{array} $$
$$\begin{array}{*{20}l} &= \text{median} \left\{\frac{1}{|A|}\sum_{a}^{|A|}(G_{t,a,k}-\overline{G_{t,\bullet,k}})^{2}| t \in \{0,\dots,T\} \right \} \ . \end{array} $$

Low values of y indicate a general consensus in sentiment, whereas higher values reflect greater heterogeneity. We choose the median as a measure of central tendency as it is robust to skewed distributions, which in our domain arise when a topic receives a burst of news activity, such as a breaking news event.


The source of the news articles used in this research is the LexisNexis Newsdesk dataset. The dataset consists of 113,000,000 articles from over 34,000 news sources dating from 6 May 2016 to 31 Dec 2016. In addition to the content of the news articles, the dataset contains further metadata, including the source’s country and broad topic labels such as ‘Top Stories’, ‘Sport’, and ‘Fashion’. To only focus on relevant articles, only those with between 100 and 1700 words in length and tagged under ‘Top Stories’ are kept. Furthermore, to eliminate sources that only sporadically publish relevant content, all sources with an average publication rate lower than once per day are ignored. We also remove news aggregators, such as Yahoo! News, and local subsidiaries of large national sources. This leaves 313,276 articles from 97 news sources usable in this study.

Standard pre-processing is performed on each of the texts including tokenisation, lemmatisation and the removal of stopwords, before performing topic modelling and sentiment extraction on the articles.

Article topics are assigned using an LDA model of k=200 topics trained using the Gensim Python package (Rahman and Wang 2016). The number of topics was chosen to maximise the model’s coherence (Mimno et al. 2011) and a subset of the topics are presented in Appendix. Qualitative inspection of the topics learned by the LDA model suggests that the model successfully learns to cluster words based on identifiable news topics. These topics include Britain’s exit from the European Union, which assigns high probability to words including [‘europe’, ‘brexit’, ‘farage’], the scandal involving the state-sponsored doping of Russian athletes [‘olympic’, ‘doping’, ‘russia’] and changes to central banks’ interest rates [‘bank’, ‘rate’, ‘bond’].

Bellwether behaviour

Here we present the results and discussion concerning RQ1, which asks whether the sentiment that news sources adopt is influenced by those of their peers.

There has been extensive research into herd behaviour and interdependence within the fields of behavioural economics and social psychology, in which studies have shown that an individual’s bias towards a particular group is a significant contributor towards their social relations (Hogg 2013). This behaviour has also been proposed as a factor in determining how stories are framed in the media (Wilczek 2016).

We distinguish between two indications of potential peer influence. The first ‘one-versus-all’ approach considers whether the dynamics of a news source’s sentiment on a particular topic influences, or is influenced, by the average sentiment expressed by the remainder of their peers and competitors, which we will refer to as the consensus. The second is whether there is a measurable relationship between distinct pairs of agents that cannot be explained away by the consensus.

To test whether an individual news source acts as a leading indicator on that topic it is first removed from the consensus and then tested to see whether the time-series that represents the consensus on a topic is Granger-caused by the news source in question. The inverse is also performed to determine if the sentiment of the source are Granger-caused by the consensus. This test is performed for each topic and each news source that contributes to that topic. We refer to sources that act as leading indicators as ‘bellwethers’ and an example of this is presented in Fig. 3.

Fig. 3

An example of how topic-specific sentiment changes over time. The example shown here is the ‘Brexit’ topic. The BBC (orange) is a bellwether on this topic as it leads the consensus (blue) whereas CNN (green) lags the consensus

Figure 4 presents an overview of the relative frequencies at which a news source leads versus lags the consensus. Of the 15,800 possible lead-lag relationships, there are 379 (2.4%) cases in which a source Granger-causes the consensus and 256 (1.6%) instances where a news source is Granger-caused by the consensus.

Fig. 4

Histogram of the number of topics that each news source lead and lag

Each of the 97 news sources acts as bellwether for at least one topic with the average news source being credited as a bellwether for 5 different topics. A manual inspection of the set of agents suggests that regional and national-level outlets are more likely to lead on topics within their own geopolitical environments than they are on non-local topics. This is conceivably because they are attuned to issues that are of concern to their readership base. Examples of this are the Miami Herald being a leading source on topics surrounding the ‘Orlando Nightclub Shooting’, or the London Evening Standard being a leading indicator on the topic concerning ‘London’, ‘Londoners’ and ‘Sadiq Kahn’. Conversely, a minority of media outlets are leaders of a proportionately large number of topics. These are found to generally be larger outlets, such as the New York Times and The Telegraph, both of whom act as bellwethers on 13 topics, The Los Angeles Times with 10 topics, and The Washington Post with 9. Considering the instances where a source lags the consensus, the results also indicate that no single news outlet is wholly independent of other sources, but rather all have at least one topic for which they are trend followers.

This first result supports the notion that there is not only some level of interdependence in the topics which the media chooses to report on but also in the manner in which they do so. Next, we consider influence at a pairwise level and whether this phenomenon is mirrored in the influence networks, described in “Topic-Level influence networks” section. In particular, we test whether the bellwether news sources are more central in intermedia influence networks than those who are not bellwethers. For this, we assign each news source to one of four groups depending on whether it leads the consensus of a topic (i.e., a bellwether), whether it lags the consensus, whether it neither leads nor lags the consensus or whether it both leads and lags the consensus (mutual Granger causality). The PageRank centrality of each news source is calculated for each of the topic-influence networks that it sits inFootnote 1. The distributions of the resulting centrality measures of the groups, shown in Fig. 5, are then compared by performing a Kruskal-Wallis H-test (KW test) for independent samplesFootnote 2, which is a non-parametric test of whether two or more samples originate from the same distribution. More specifically, the null hypothesis of the KW test is that the medians of the samples are the same. We use the KW test to test whether the median centrality of a news source on a topic differs depending on whether that source was earlier identified as leading the topic consensus, lagging it, neither, or both.

Fig. 5

The distribution of PageRank centrality values grouped by whether a node is a bellwether, whether it lags the discussion, whether it neither leads nor lags the discussion or whether it both leads and lags the discussion

The KW test is performed jointly on the four groups and the results reject the null hypothesis that the groups are generated from the same distribution (p=1.7×10−4). While from this result we can assume that there is at least one group whose distribution stochastically dominates the others, it does not state which one. To determine where the stochastic dominance occurs we furthermore perform the test between pairs of groups, the results of which are shown in Table 1. For any pair of groups, if the null hypothesis is rejected, then we can simply check which of the two groups has the larger median. The results table shows us that when a news source is a bellwether on a topic, it is also more likely to have higher centrality in the network associated with that topic than when it is not, but there is no statistical difference between those who lag the consensus and those who are neither leaders nor laggers.

Table 1 The results of the Kruskal-Wallis H-test performed for each pair of groups. Adjusting for multiple comparison, the 10%, 5% and 1% significance thresholds are 0.016, 0.008 and 0.0016 respectively

The structural properties of intermedia influence networks

Having established that there is evidence of intermedia agenda-setting in which news sources are responsive to the content published by one another, we proceed by considering the structural properties of the networks that capture the intermedia influence.

Intermedia gatekeeping

The ‘gatekeeping’ function of the media is well documented in which, even in the age of social media, the editors act as information filters and barriers to the content that is disseminated to the public (Shoemaker and Vos 2014). What is less well understood is whether there is intermedia gatekeeping in which a small subset of actors have influence that spans disproportionately far. We employ our inferred networks to address this question by testing whether individual news sources have disproportionately large influence that could enable them to govern the agenda of the discussion on particular issues. The scope here is not to definitively state whether or not certain players dominate the sentiment on a topic, but rather to ask whether the network structure facilitates this behaviour. We reason that if the media does have highly influential gatekeepers who can directly impact others, then we would expect to see a considerable proportion of topics where the associated network has at least one source that is significantly more central than would be expected to find if influence were assigned randomly. First we consider descriptive statistics about the aggregate influence. As seen in Fig. 6, we find that the larger news sources, such as The Los Angeles Times, The Telegraph, CNN and BBC, in general have greater total out-degree across networks. This qualitatively shows that influence is not uniformly distributed. However, from a quantitative standpoint it does not suggest whether this indicates disproportionately influential actors. We, therefore, run the following binomial test: for each network, gG, we begin by considering the out-degree of the most central node, kmax(g). We then ask what the probability would be of observing a node with out-degree kkmax(g) under an Erdős-Rényi network generation process that has the same average degree as g. For a network g with n nodes and np edges, if the probability, P(kkmax(g)), as given in equation 3, is smaller than a threshold α (where α includes Bonferroni correction), then we can argue that g has at least one disproportionately central actor. We repeat this procedure for the second most central node, (kmax−1(g)), third most central (kmax−2(g)) etc.

$$ P(k\ge k_{\text{max}}(g)) = 1 - \sum_{i=1}^{k_{\text{max}}(g)} \binom{i}{n} p^{i} (1-p)^{n-k} $$
Fig. 6

The cumulative out degree of the top 20 topics

The results, seen in Fig. 7 show that, after performing this test on each of the 200 topics, we observe 63 cases (31.5%) where we can reject the null hypothesis that the centrality of the most central node in the network could be generated from an Erdős-Rényi process. From this result it is not possible to definitively conclude that there is systematic gatekeeping in the media. However, it does provide strong evidence that there are certain topics for which a small subset of the media hold a disproportionate amount of influence. We sought to corroborate this by performing a rich-club analysis (Colizza et al. 2006) (the rich-club effect is a widely known measure of the excess of interaction between nodes with a large degree in a network). However, the results were inconsistent and only a small minority of topics showed any evidence of the rich-club effect. This is to be expected given that connectivity in the domain of intermedia influence reasonably tends to be acyclic. From this we can conclude that for the topics that do exhibit gatekeeping tendencies, the influence appears to be enjoyed by only one or two sources who, however, are largely insulated from other influential sources.

Fig. 7

The percentage of topics who reject the null hypothesis of P(kkmaxx) for varying x

Small-World properties

The small-world property of many empirical social networks has been shown to have a profound impact on how efficiently information spreads through them (Latora and Marchiori 2001). Due to their high clustering and small characteristic path length (Watts and Strogatz 1998), the nodes of networks that exhibit the small-world property locally sit within a close-knit community, whilst simultaneously still allowing for information to pass efficiently through the network as a whole. In networks that exhibit the small-world phenomenon, innovations (i.e., novel perspectives) by an individual are likely to spread through the network and be picked up by others. In the context of intermedia influence networks, this phenomenon translates to most sources appearing to be independent of one another, while in fact requiring only few steps for a new spin on a story to propagate through the network as a whole.

There have been several proposed measures of small-worldness, each of which relies on a slightly varying interpretation of how a small-world network is defined (Humphries and Gurney 2008; Telesford et al. 2011; Neal 2015). One popular method, proposed by Humphries and Davies (2008), measures small-worldness using \(\sigma = \frac {\frac {C}{C_{r}}}{\frac {L}{L_{r}}}\), where C and L are the clustering coefficient and average path length of the network respectively, and Cr and Lr are the clustering coefficient and average path length of an equivalent random network. Though having been criticised as overestimating small-worldness on large networks (Telesford et al. 2011), σ has the benefit of having a simple heuristic for classifying a network as small-world: if σ>1 then the network is said to be small-world. The networks we consider are not particularly large (ranging from between 20 to 50 nodes per network), making the σ value an appropriate choice. We use the definition proposed in Fagiolo et al. (2007) that extends the definition of clustering from the undirected to directed graph case:

$$C = \frac{1}{N}\sum_{i \in G} \frac{\frac{1}{2}\sum\limits_{j,h \in G}(e_{ij}+e_{ji})(e_{ih}+e_{hi})(e_{jh}+e_{hj})}{(K^{out}_{i} + K^{in}_{i})(K^{out}_{i} + K^{in}_{i}-1)-2\sum\limits_{j \in G}e_{ij}e_{ji}} \ , $$

where N is the number of nodes in the network, enm is an indicator function of whether there is an edge going from node n to node m, and \(K^{in}_{n}\) and \(K^{out}_{n}\) are the in-degree and out-degree of node n respectively. The average shortest path between directed nodes is defined as (Rubinov and Sporns 2010):

$$L = \frac{1}{N(N-1)}\sum\limits_{j\in G, i \neq j} d_{ij} \ , $$

where dnm is the shortest directed path length from node n to node m.

To generate equivalent random networks that maintain equivalent degree sequences, we follow previous studies (Liao et al. 2011) by randomly reshuffling edges, whilst preserving the in-degree and out-degree for each node.

Figure 8 shows a histogram of the σ values of the topic networks. 87.5% of the networks have a value of σ>1, a strong indication that the media exhibits small-world tendencies in the manner in which they influence one another to adopt a particular angle or spin on a story.

Fig. 8

Histogram showing the distribution of small-worldness, σ. A value of σ<1 is considered not to be small-world

Opinion diversity

Now that we have a comprehensive view of the topology of intermedia influence networks, we proceed to explore the impact that the topology has on the level of opinion diversity observed across the media.

Having a variety of opinions, emotions and viewpoints expressed by differing sources in the media is imperative to giving the public an opportunity to gain exposure to alternative perspectives on a particular issue (Mutz and Martin 2001). The above analysis of the structure of the networks indicates that there are both instances of identifiable community structures throughout the networks (which may isolate news sources from one another), as well as relatively short average distances between nodes (which conversely, may result in new information or perspectives spreading and being adopted through the network efficiently). In this final section, we seek to quantify the net impact that these two competing forces have on the diversity of sentiment expressed in the news media. To test this we propose a series of linear models that attempt to capture these effects and compare them based on the significance of their coefficients, while also controlling for the size and density of the networks.

We compare three linear regression models and examine their coefficients. The dependent variable, log(y), is the log of the opinion diversity, defined in Eq. (1). Each of the models contains as independent variables the average shortest path length L as well as the size of the network N (measured as the number of nodes) as a control variable. The first model, M1 also includes the average clustering coefficient, C, as an independent variable while the second model, M2 includes the network density, d, instead. The third model, M3 considers the combined effect that both d and C have on log(y).

Some topics covered in the news media may be intrinsically highly emotive, though not necessarily be particularly divisive or material to overall public opinion. Topics such as sport or celebrity gossip fall into this category, compared to, for example, media coverage about election candidates or international affairs. Therefore, the topics are inspected manually by the authors, and all of the topics that are clearly about sports, celebrity gossip or lifestyle/entertainment are ignored. This results in a sample size of 161 networks.

The results provided in Table 2 show the regression coefficients of the three models. In all three of the models, sentiment diversity increases as a function of the size of a network however the average shortest path length is not a significant covariate. In M1 and M2, neither the average clustering coefficient nor the density adds explanatory power beyond that of just the network size. In the third model, however, we see compounding effects in which increasing network density reduces diversity while higher clustering increases it. A possible interpretation of this result is that greater interconnectedness, in general, captures the known effects of intermedia agenda-setting in which news sources tend to mimic one another and mutually reinforce each other. However, the formation of local subgroups in the media contrasts this effect and allows different groups to express different sentiment on the same issue and therefore reduce the global consensus.

Table 2 The regression coefficients of each of the three linear models. Model M1 is log(y)=ω0+ω1L+ω2N+ω3C, model M2 is log(y)=ω0+ω1L+ω2N+ω4d, model M3 is log(y)=ω0+ω1L+ω2N+ω3C+ω4d, where y is the opinion diversity, L is the average shortest path length, N is network size, C is the average clustering coefficient, d is the network density. The values shown below pertain to the coefficients ωi (i=0,…,4) with p-values reported in brackets. (p<0.1;p<0.05;p<0.01.)


In this paper, we presented a method to operationalise intermedia agenda-setting by identifying networks of relationships in the media that capture how news sources influence one another to adopt particular sentiment on issues.

We used network analysis to explore the propagation of sentiment about a topic in the media. In doing so, we revealed characteristics of online news dissemination and the sentiment adoption process of the media. We found that, consistent with the expectations of intermedia agenda-setting theory, there is identifiable influence among the news media, both where sources lead/lag the herd, as well as on a peer-to-peer level. While those who lead the consensus sentiment vary from topic to topic, we found that elite news sources, such as the New York Times, Washington Post etc., are more likely to be identified as a bellwether than those who are not elite. Moreover, we found that the larger, well established news sources also tend to have higher centrality in the networks that directly measure peer-to-peer influence. This observation contrasts the findings of other recent studies that explore the level of influence maintained by different actors in the media. A study by Vargo and Guo (2017) on the influence between types of media concluded that, in response to a shifting audience, the large elite media have become attentive to the agendas of smaller online news sources. Our results suggest that, while the elite media sources can be attentive to others, they do still have comparably more power.

When compared to a null model, we identified that some topics covered by the media form network structures in which influence is highly centralised to a small subset of news sources. However this effect was only observed on 31.5% of the available topics, so we are unable to definitively state whether or not such effects are systematic. We did, however, find pervasive evidence of small-world characteristics between interacting news sources, and that this phenomenon is pervasive across topics. Previous work on the network structure of news dissemination has also found small-world properties when investigating the propagation of breaking news (Liu et al. 2016). While such work demonstrated that breaking news has the ability to propagate quickly, by focusing on the associated sentiment of prolonged topics, the observed small-worldness in our results indicates that changes in the framing of a story are also subject to being rapidly disseminated. Finally, by comparing descriptive networks statistics to the dynamics of how the media discusses topics, we found evidence that intermedia influence can be a positive factor in presenting diverse perspectives. In particular, we saw that while greater degrees of interconnectedness (in the form of more densely connected networks) were associated with reduced diversity in the overall sentiment that the media presents on an issue, this effect was diminished for topics in which the media forms clusters of influence.

As the first study of its kind to look at the impact that the structure of an influence network has on how perspectives are adopted in the media, there are several questions that we hope to answer in future work. Notably, we would like to include entity-specific information to our model, primarily whether the results change when explicitly controlling for the size of a news source, whether there are partisan effects or differences between news agencies and publishers, and how the opinion dynamics vary differently within communities versus between them. We also believe that there is more to be understood of the nature of the influence through further studying the impact of the magnitudes and duration of the lead-lag relationships.


Fig. 9

The inferred topics with the 20 highest coherence values

Availability of data and material

The data that support the findings of this study are available from LexisNexis but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of LexisNexis.


  1. 1.

    The direction of the edges of each graph are reversed to reflect that PageRank assigns centrality to nodes whose incoming edges have high centrality

  2. 2.

    The assumptions for an ANOVA are not satisfied



Latent Dirichlet Allocation




  1. Anger, I, Kittl C (2011) Measuring Influence on Twitter In: Proceedings of the 11th International Conference on Knowledge Management and Knowledge Technologies, i-KNOW ’11, 31–1314.. ACM, New York.

    Google Scholar 

  2. Brady, WJ, Wills JA, Jost JT, Tucker JA, Van Bavel JJ (2017) Emotion shapes the diffusion of moralized content in social networks. Proc Nat Acad Sci U S A 114(28):7313–7318. WOS:000405177100063.

    Google Scholar 

  3. Benjamini, Y, Hochberg Y (1995) Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. J R Stat Soc Ser B Methodol 57(1):289–300.

    MathSciNet  MATH  Google Scholar 

  4. Blei, DM, Ng AY, Jordan MI (2003) Latent Dirichlet Allocation. J Mach Learn Res 3:993–1022.

    MATH  Google Scholar 

  5. Coleman, R, Wu HD (2010) Proposing Emotion as a Dimension of Affective Agenda Setting: Separating Affect into Two Components and Comparing Their Second-Level Effects. Journal Mass Commun Q 87(2):315–327.

    Google Scholar 

  6. Colizza, V, Flammini A, Serrano MA, Vespignani A (2006) Detecting rich-club ordering in complex networks. Nat Phys 2(2):110–115.

    Google Scholar 

  7. Coviello, L, Sohn Y, Kramer ADI, Marlow C, Franceschetti M, Christakis NA, Fowler JH (2014) Detecting Emotional Contagion in Massive Social Networks. PLOS ONE 9(3):90315.

    Google Scholar 

  8. Delre, SA, Jager W, Janssen MA (2007) Diffusion dynamics in small-world networks with heterogeneous consumers. Comput Math Organ Theory 13(2):185–202.

    MATH  Google Scholar 

  9. Eliacik, AB, Erdogan N (2018) Influential user weighted sentiment analysis on topic based microblogging community. Expert Syst Appl 92:403–418.

    Google Scholar 

  10. Fagiolo, G, Moneta A, Windrum P (2007) A Critical Guide to Empirical Validation of Agent-Based Models in Economics: Methodologies, Procedures, and Open Problems. Comput Econ 30(3):195–226.

    Google Scholar 

  11. Ferrara, E, Yang Z (2015) Measuring Emotional Contagion in Social Media. PLOS ONE 10(11):0142390.

    Google Scholar 

  12. Granger, CWJ (1969) Investigating Causal Relations by Econometric Models and Cross-spectral Methods. Econometrica 37(3):424–438.

    MATH  Google Scholar 

  13. Guo, L, Vu HT, McCombs M (2012) An Expanded Perspective on Agenda- Setting Effects, Exploring the third level of agenda setting. Rev Comun 11:51–68.

    Google Scholar 

  14. Harcup, T, O’Neill D (2017) What is News?. Journal Stud 18(12):1470–1488.

    Google Scholar 

  15. Harder, RA, Sevenans J, Van Aelst P (2017) Intermedia Agenda Setting in the Social Media Age: How Traditional Players Dominate the News Agenda in Election Times. Int J Press Polit 22(3):275–293.

    Google Scholar 

  16. Hatfield, E, Cacioppo JT, Rapson RL (1993) Emotional Contagion. Curr Dir Psychol Sci 2(3):96–100.

    Google Scholar 

  17. Hill, AL, Rand DG, Nowak MA, Christakis NA (2010) Emotions as infectious diseases in a large social network: the SISa model. Proc R Soc Lond B Biol Sci 277(1701):3827–3835.

    Google Scholar 

  18. Hogg, MA (2013) Intergroup Relations. In: DeLamater J Ward A (eds)Handbook of Social Psychology, Handbooks of Sociology and Social Research, 533–561.. Springer, Dordrecht.

    Google Scholar 

  19. Hui, P, Gregory M (2010) Quantifying Sentiment and Influence in Blogspaces In: Proceedings of the First Workshop on Social Media Analytics, SOMA ’10, 53–61.. ACM, New York.

    Google Scholar 

  20. Humphries, MD, Gurney K (2008) Network ’Small-World-Ness’: A Quantitative Method for Determining Canonical Network Equivalence. PLoS ONE 3(4).

  21. Joly, J (2016) Disentangling media effects: The impact of short-term and long-term news coverage on Belgian emergency assistance. Cooperation Confl 51(4):428–446.

    Google Scholar 

  22. Kim, JY, Kiousis S (2012) The Role of Affect in Agenda Building for Public Relations: Implications for Public Relations Outcomes. J Mass Commun Q 89(4):657–676.

    Google Scholar 

  23. Kiss, C, Bichler M (2008) Identification of influencers — Measuring influence in customer networks. Decis Support Syst 46(1):233–253.

    Google Scholar 

  24. Latora, V, Marchiori M (2001) Efficient Behavior of Small-World Networks. Phys Rev Lett 87(19):198701.

    Google Scholar 

  25. Li, W, Aste T, Caccioli F, Livan G (2019) Reciprocity and impact in academic careers. EPJ Data Sci 8(1):20.

    Google Scholar 

  26. Liao, W, Ding J, Marinazzo D, Xu Q, Wang Z, Yuan C, Zhang Z, Lu G, Chen H (2011) Small-world directed networks in the human brain: Multivariate Granger causality analysis of resting-state fMRI. NeuroImage 54(4):2683–2694.

    Google Scholar 

  27. Lim, J (2011) First-level and second-level intermedia agenda-setting among major news websites. Asian J Commun 21(2):167–185.

    Google Scholar 

  28. Liu, N, An H, Gao X, Li H, Hao X (2016) Breaking news dissemination in the media via propagation behavior based on complex network theory. Physica A Stat Mech Appl 453:44–54.

    Google Scholar 

  29. Livan, G, Caccioli F, Aste T (2017) Excess reciprocity distorts reputation in online social networks. Sci Rep 7(1):1–11.

    Google Scholar 

  30. McCombs, M, Shaw DL (1972) The Agenda-Setting Function of Mass Media. Public Opin Q 36(2):176–187.

    Google Scholar 

  31. McCombs, M, Llamas JP, Lopez-Escobar E, Rey F (1997) Candidate Images in Spanish Elections: Second-Level Agenda-Setting Effects. Journal Mass Commun Q 74(4):703–717.

    Google Scholar 

  32. McCombs, M (2005) A look at agenda-setting: Past, present and future. Journal Stud 6(4):543–557.

    Google Scholar 

  33. Mimno, D, Wallach HM, Talley E, Leenders M, McCallum A (2011) Optimizing Semantic Coherence in Topic Models In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP ’11, 262–272.. Association for Computational Linguistics, Stroudsburg. event-place: Edinburgh, United Kingdom.

    Google Scholar 

  34. Moors, A (2009) Theories of emotion causation: A review. Cognit Emot 23(4):625–662.

    Google Scholar 

  35. Mutz, DC, Martin PS (2001) Facilitating Communication across Lines of Political Difference: The Role of Mass Media. Am Polit Sci Rev 95(1):97–114.

    Google Scholar 

  36. Neal, Z. (2015) Making Big Communities Small: Using Network Science to Understand the Ecological and Behavioral Requirements for Community Social Capital. Am J Community Psychol 55(3):369–380.

    Google Scholar 

  37. Nyman, RBE (2016) An Algorithmic Investigation of Conviction Narrative Theory: Applications in Business, Finance and Economics. Doctoral. UCL (University College London). Accessed 17 July 2017.

  38. Peng, S, Zhou Y, Cao L, Yu S, Niu J, Jia W (2018) Influence analysis in social networks: A survey. J Netw Comput Appl 106:17–32.

    Google Scholar 

  39. Rahman, MM, Wang H (2016) Hidden Topic Sentiment Model In: Proceedings of the 25th International Conference on World Wide Web, WWW ’16, 155–165.. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva.

    Google Scholar 

  40. Romero, DM, Galuba W, Asur S, Huberman BA (2011) Influence and Passivity in Social Media In: Proceedings of the 20th International Conference Companion on World Wide Web, WWW ’11, 113–114.. ACM, New York.

    Google Scholar 

  41. Rubinov, M, Sporns O (2010) Complex network measures of brain connectivity: Uses and interpretations. NeuroImage 52(3):1059–1069.

    Google Scholar 

  42. Sheafer, T (2007) How to evaluate it: The role of story-evaluative tone in agenda setting and priming. J Commun 57(1):21–39.

    Google Scholar 

  43. Shoemaker, PJ, Vos TP (2014) Media gatekeeping In: An Integrated Approach to Communication Theory and Research, 89–103.. Routledge, Abingdon.

    Google Scholar 

  44. Shu, K, Wang S, Liu H (2019) Beyond News Contents: The Role of Social Context for Fake News Detection In: Proceedings of the Twelfth ACM International Conference on Web Search And Data Mining, WSDM ’19, 312–320.. ACM, New York.

    Google Scholar 

  45. Sikder, O, Smith RE, Vivo P, Livan G (2020) A minimalistic model of bias, polarization and misinformation in social networks. Sci Rep 10(1):1–11.

    Google Scholar 

  46. Song, X, Chi Y, Hino K, Tseng B (2007) Identifying Opinion Leaders in the Blogosphere In: Proceedings of the Sixteenth ACM Conference on Conference on Information and Knowledge Management, CIKM ’07, 971–974.. ACM, New York.

    Google Scholar 

  47. Stern, SO, Tuckett D, Smith RE, Nyman R (2018) Measuring the Influencers in the News Media’s Narratives In: 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis And Mining (ASONAM), 698–701.

  48. Stieglitz, S, Dang-Xuan L (2013) Emotions and Information Diffusion in Social Media—Sentiment of Microblogs and Sharing Behavior. J Manag Inf Syst 29(4):217–248.

    Google Scholar 

  49. Telesford, QK, Joyce KE, Hayasaka S, Burdette JH, Laurienti PJ (2011) The Ubiquity of Small-World Networks. Brain Connectivity 1(5):367–375.

    Google Scholar 

  50. Tuckett, D, Smith RE, Nyman R (2014) Tracking phantastic objects: A computer algorithmic investigation of narrative evolution in unstructured data sources. Social Networks 38:121–133.

    Google Scholar 

  51. Van Kleef, GA, Van Doorn EA, Heerdink MW, Koning LF (2011) Emotion is for influence. Eur Rev Soc Psychol 22(1):114–163.

    Google Scholar 

  52. Vargo, CJ, Guo L (2017) Networks, Big Data, and Intermedia Agenda Setting: An Analysis of Traditional, Partisan, and Emerging Online US. News. Journal Mass Commun Q 94(4):1031–1055.

    Google Scholar 

  53. Vargo, CJ, Guo L, Amazeen MA (2018) The agenda-setting power of fake news: A big data analysis of the online media landscape from 2014 to 2016. New Media Soc 20(5):2028–2049.

    Google Scholar 

  54. Ver Steeg, G, Galstyan A (2013) Information-theoretic Measures of Influence Based on Content Dynamics In: Proceedings of the Sixth ACM International Conference on Web Search And Data Mining, WSDM ’13, 3–12.. ACM, New York.

    Google Scholar 

  55. Watts, DJ, Strogatz SH (1998) Collective dynamics of ’small-world’ networks. Nature 393(6684):440–442.

    MATH  Google Scholar 

  56. Watts, DJ, Dodds PS (2007) Influentials, Networks, and Public Opinion Formation. J Consum Res 34(4):441–458.

    Google Scholar 

  57. Welbers, K, Atteveldt Wv, Kleinnijenhuis J, Ruigrok N (2018) A Gatekeeper among Gatekeepers. Journal Stud 19(3):315–333.

    Google Scholar 

  58. Weng, J, Lim E-P, Jiang J, He Q (2010) TwitterRank: Finding Topic-sensitive Influential Twitterers In: Proceedings of the Third ACM International Conference on Web Search And Data Mining, WSDM ’10, 261–270.. ACM, New York.

    Google Scholar 

  59. Wilczek, B (2016) Herd Behaviour and Path Dependence in News Markets: Towards an Economic Theory of Scandal Formation. J Interdiscip Econ 28(2):137–167.

    Google Scholar 

  60. Xiong, X, Li Y, Qiao S, Han N, Wu Y, Peng J, Li B (2018) An emotional contagion model for heterogeneous social media with multiple behaviors. Physica A Stat Mech Appl 490:185–202.

    MathSciNet  Google Scholar 

  61. Zhou, H, Zeng D, Zhang C (2009) Finding leaders from opinion networks In: 2009 IEEE International Conference on Intelligence and Security Informatics, 266–268.

Download references


We thank Nora Ptakauskaite, Simone Righi, and David Tuckett for helpful comments on preliminary versions of this manuscript.


GL acknowledges support from an EPSRC Early Career Fellowship (Grant No. EP/N006062/1).

Author information




SS designed the research and conducted the experiments. GL and RES supervised the research. All authors wrote the manuscript. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Giacomo Livan or Robert E. Smith.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Stern, S., Livan, G. & Smith, R.E. A network perspective on intermedia agenda-setting. Appl Netw Sci 5, 31 (2020).

Download citation


  • Intermedia agenda-setting
  • Network influence
  • Opinion dynamics