 Research
 Open Access
 Published:
Effective and scalable methods for graph protection strategies against epidemics on dynamic networks
Applied Network Science volume 4, Article number: 18 (2019)
Abstract
Dynamic networks are networks with temporal relationship features which evolve over time by the inclusion and deletion of nodes and edges. Suppressing the epidemic spreading in such networks is quite challenging. The problem of protecting a limited number of nodes to restrain the spreading of malicious attacks or dangerous rumor in the networks is called graph protection problem. However, most of existing strategies only consider to protect at once regardless the evolving network structure and incoming attacks over time, i.e., these strategies either preprotect important nodes before the epidemic starts or postallocate the protection while the attacks have already spread over the network. In this paper, we introduce multipleturns protection strategies, which divide the size of protection budget into several turns and protect nodes according to the currently observed temporal snapshot of dynamic networks. We construct a minimum vertex cover of the input network efficiently using reinforcement learning approach. To capture the state of the input network, a featurebased representation of each node is constructed using a graph embedding technique. Experimental evaluations show that our proposed methods, namely ReProtect and ReProtectp effectively restrain epidemic propagation in synthetic and realworld network datasets. By protecting about 15% of nodes, our methods can obtain up to 84% of surviving nodes and outperform other baseline methods on two popular epidemic models: SIS and SIR.
Introduction
With the rising popularity of massivescale online social networks such as Facebook, Instagram, Twitter, etc., people are more connected and can share information with each other (Zhuang et al. 2013). These platforms play a vital role in the dissemination of positive information such as new ideas, innovations, and hot topics. However, they may also become channels for the spreading of malicious rumors, misinformation, or even dangerous virus and malware. The rumor spreading can severely threaten public safety and financial stability. For instance, some people may post on social networks a rumor about an upcoming big earthquake. It will cause chaos among society and hence may hinder the normal public order. In this situation, it is necessary to find individuals, if their account deactivated or removed from the network, would block further rumor spreading. The problem is known as graph protection problem where the goal is to protect a number of nodes to restrain the epidemic propagation by maximizing the ratio of surviving nodes in a network (Wijayanto and Murata 2017; 2018b). In this problem, the protection budget constraints the number of nodes we are allowed to protect.
Realworld social networks and collaboration networks have highly dynamic structures, and they evolve rapidly over time (Zhan et al. 2017; Wang et al. 2016). The inherently dynamic nature of the network leads to dynamic network representations. Dynamic networks are defined as networks which evolve over time by the addition and removal of nodes and edges (Bakker et al. 2018; Moore et al. 2006; Zhuang et al. 2013). Dynamic networks have temporal relationship features which specify the number of connection among nodes that are active at a certain time.
Restraining the epidemic spreading in dynamic networks is obviously more challenging than in static networks because of the temporal changing of the network structure. However, most of the existing work failed to address the incoming rumor or virus attacks during the temporal transition in dynamic networks. The existing strategies either preemptively protect critical nodes prior to epidemic attacks, behaving as prevention efforts (Prakash et al. 2010; Chen et al. 2016; Wijayanto and Murata 2017), or postemptively allocate the protection while the epidemics have already propagated over the network, simulating as delayed reactions (Zhan et al. 2017; Zhang et al. 2017; Zhang and Prakash 2015; Song et al. 2015). In this paper, we introduce a multipleturns graph protection strategy by dividing the protection budget into several turns and protecting nodes based on the currently observed temporal structure of networks.
On the other hand, most of the current works of graph protection strategy mostly fall into one of the following drawbacks: (1) protecting only particular areas of the networks, as demonstrated by centralitybased methods (Prakash et al. 2010; Buono and Braunstein 2015; Zhao et al. 2014) (2) scalability issue, as demonstrated by dominator treebased methods (Zhang and Prakash 2014; Zhan et al. 2017; Zhang and Prakash 2015) (3) lack of convergence guarantee in large size networks, as shown by eigendecompositionbased methods (Tong et al. 2010; Chen et al. 2016; Wijayanto and Murata 2017; Prakash et al. 2010). We propose the construction of minimum vertex cover to determine the protected nodes in an efficient and scalable method. The minimum vertex cover (MVC) is the set of nodes which cover all edges of networks in a minimum possible size of nodes. As we explain later, MVC serves as the protection threshold of the network (see “Proposed methods” section for our detailed explanation).
In recent years, reinforcement learning (RL) approaches have obtained many stateoftheart results in solving various complex problems (Mnih et al. 2015; Riedmiller 2005). RL allows autonomous agents to learn to improve their performance with experience. In this work, we utilize RL approach to construct MVC from the currently observed network snapshot. Specifically, we propose nstep fitted QLearning to obtain the MVC solution of input network by leveraging the neural network as a function approximator. Neural network architecture allows us to efficiently accelerate the training and execution of our proposed methods in minibatch processing and multiple graphics processing units to deal with large size networks. In order to handle the different size and structure of each temporal snapshot of dynamic networks, each node is represented in a fixedlength feature vector using a graph embedding technique.
Extensive evaluations in both synthetic and realworld network datasets show that our proposal effectively restrains epidemic spreading. In Email network dataset, by protecting about 15% of nodes, our methods can achieve up to 84% of surviving nodes and outperform other baseline methods. Comprehensive evaluations under two most popular epidemic models, i.e., SIS and SIR, confirms the effectiveness and scalability of our methods.
The novelty of our methods arises primarily from the application of more stochasticity and learning ability to graph protection problem, specifically on dynamic networks. In largescale social networks, the changing of relationship structure and rumor spreading patterns may come and arise on a regular basis. Therefore, there is an opportunity to learn the current condition into a model using reinforcement learning. By learning the given temporal structure of observed networks and existing epidemics, this will provide a new incentive to predict future protection from previously learned actions in the same dynamic networks.
This paper extends our preliminary idea in (Wijayanto and Murata 2018a). In addition to the contents in (Wijayanto and Murata 2018a), this paper includes the following: detailed explanation of the proposed methods; evaluation on synthetic networks, as well as more realworld network datasets; review of the relevant related work; discussion of scalability and computational complexity; evaluation of parameter sensitivity; addition of stronger baselines methods such as Betweenness, GraphShield and NetShield+; and evaluation on SIR epidemic model.
The remainder of this paper is organized in the following manner. We formalized the problem and definition in “Problem formulation” section. The review of recent most related studies is presented in “Related work” section. Our proposed methods, namely ReProtect and ReProtectp are described in “Proposed methods” section. The result of experimental simulations are provided in “Evaluation” section. Finally, concluding remarks of our work is provided in “Conclusion” section.
Problem formulation
In this section, we formalize the definitions and problems used throughout this paper. We summarize the symbols and notations in Table 1.
Definition 1.Protecting a node means removing all of its corresponding edges. The number of nodes we are allowed to protect is constrained by the protection budget (\(k \in \mathbb {Z}_{> 0}\)). At time t, a node in a network can belong to any of the following states: susceptible and infected. Attacking a node means initially infect the node in a network. Figure 1 shows the example of protection and attack in a static network.
Definition 2. Graph Protection Problem
Let G=(V,E) be an undirected connected graph with set of nodes V and set of edges E. Let θ be the surviving ratio of nodes that remain uninfected at the end of epidemics.
Given an input graph G, SIS or SIR epidemic model, and a protection budget k, the goal is to find a set of nodes S∈V such that θ is maximized, subject to the size of S is equal to constraint budget k. The protection is performed by removing all edges connected to the set of nodes S in graph G to get a new graph G^{(S)}.
Definition 3. Dynamic Network
Let {1,⋯,T} be a finite set of discrete time steps. Let V^{D}={1,⋯,n} be a set of nodes which appear within time {1,⋯,T}. Let G_{t}=(V_{t},E_{t}) be a graph representing the snapshot of the network at time t. V_{t}⊆V^{D} is a subset of nodes V^{D} observed at time t. (t,u,v) represents an edge from vertex u∈V_{t} to v∈V_{t} at time t.
A dynamic network G^{D}=(V^{D},E^{D}) is a series 〈G_{1},⋯,G_{T}〉 of static networks where each G_{t}=(V_{t},E_{t}) is a snapshot of nodes and their edges at time t such that \(V^{D} = \bigcup _{t} V_{t}\). For the sake of consistency, the time during which the nodes are observed is assumed as finite. Following the definition by (Habiba et al. 2010) and (Bakker et al. 2018), the temporal length of G is assumed to be divided into discrete steps {1,⋯,T}. The nontrivial problem of appropriate time discretization is beyond the scope of our work.
Definition 4. SIS Epidemic Model
SusceptibleInfectedSusceptible (SIS) is an epidemic epidemic model which define that each node in graph G with N number of nodes would be in one of the following two states: susceptible and infected. Let (t) be the number of susceptible nodes, and let (t) be the number of infected nodes at time t. At each timestamp t, susceptible nodes can be infected by their infected neighbors with infection rate β. Also, each infected node can get recovered to susceptible state with recovery rate δ. In the homogeneous case of wellmixed populations, this model can be formalized as nonlinear differential equations:
being s(t)=(t)/N and i(t)=(t)/N the respective proportions of states at time t. A continuoustime epidemic process under constant infection rate β and recovery rate δ on any network can be described by Markov theory. Following the definition of SIS epidemic in network by PastorSatorras (2015), the individualbased meanfield (IBMF) and degreebased meanfield (DBMF) approach can be used to analytically simulate the SIS model.
Definition 5. SIR Epidemic Model
In SusceptibleInfectedRecovered (SIR) model, each node in graph G belong to any of the susceptible, infected, or recovered state. Each of recovered node is resistant of any infection. Let (t) be the number of recovered nodes. Following the definition by Kermack and McKendrick (1927), for the homogeneous case of wellmixed populations, this model is formalized as:
being s(t)=(t)/N, i(t)=(t)/N, and r(t)=(t)/N the respective proportions of states at time t. In addition to IBMF and DBMF approach, following the definition of SIR epidemic in network by PastorSatorras (2015), we can analytically describe the SIR model using generating function approach, where the probability that a link exists is related to the probability of transmission of the disease from an infected node to a connected susceptible one.
Definition 5. Multipleturns Graph Protection Problem on Dynamic Networks
Let G^{D}=(V^{D},E^{D}) be an undirected dynamic graph as an input, with a series of a known sample 〈G_{1},⋯,G_{T}〉 of snapshots where each G_{t}=(V_{t},E_{t}) represent a static network at time t. Let k be the protection budget, which k<V^{D} and allocated into several turns according to the number of observed snapshots of G^{D}. Protection budget for snapshot G_{t} at time t is denoted by k_{t} such that \(k = \sum _{t=1}^{T} k_{t}\).
Let us denote S, a set of k protected nodes from graph G^{D} and \(S = \sum _{t=1}^{T} S_{t}\) where S_{t} denote a subset of k_{t} protected nodes of snapshot graph G_{t} at time t. Protection means removing corresponding edges of the set of nodes S_{t} in graph G_{t} to get a new graph \(G_{t}^{(S)}\). Under random attack strategies, l nodes are randomly attacked (i.e., initialized as infected nodes) from graph G^{D} such that \(l = \sum _{t=1}^{T} l_{t}\) at each turn in time t. We define θ as the ratio of surviving nodes of graph G^{D}.
Given an input graph G^{D}, SIS or SIR epidemic model, and a protection budget k, the goal is to find S such that θ is maximized, subject to the size of S is equal to constraint budget k, i.e., calculating the following combinatorial optimization:
Related work
In this section, we review the relevant existing studies related to our work. We first review the fundamental work of epidemic modeling on dynamic networks, then we discuss some related work on graph protection strategy and its application in dynamic networks. Finally, some problems related to graph protection on dynamic networks are presented.
Fundamental work of epidemic modeling on dynamic networks
The properties of dynamic networks are essentially different from those in static networks. (Braha and BarYam 2006; 2009) found that the overlap of the centrality in dynamic networks and that in the aggregated (static) network is quite low. They also demonstrated that the static topology is unable to capture the dynamic properties of social networks. Hill and Braha (2010) propose a reinforced random walk approach to explain dynamic centrality phenomena and qualitatively reproduce the characteristic features of realworld networks. Those studies (Braha and BarYam 2006; 2009; Hill and Braha 2010) provide an important foundation of dynamic network properties.
Holme presents a systematic review of dynamic networks and discusses methods for topological and temporal structure analysis (Holme and Saramäki 2012; Holme 2015). More specifically, PastorSatorras et al. (2015) discuss a fundamental review of epidemic model on dynamic networks, which also recently emphasized by Enright and Kao (2018).
Graph protection strategy and its application in dynamic networks
The study of graph protection strategies has mostly been introduced by assuming the static topologies of network structure. PastorSatorras and Vespignani investigated the effect of random uniform and targeted highdegree immunization of individuals on homogeneous complex networks and scalefree networks (PastorSatorras and Vespignani 2002). Chen et al. proposed NetShield (Tong et al. 2010) and NetShield+ (Chen et al. 2016) which use the properties of matrix perturbation to find a set of nodes in static networks to be preemptively protected (Tong et al. 2010). Zhang and Prakash (2014; 2015) developed DAVA and DAVAfast, two postemptive polynomialtime heuristics methods which merge all infected nodes into a supernode by building a weighted dominator tree of input network. NIIP (Song et al. 2015) extracts a maximum directed acyclic graph from a static network then implements a Monte Carlo simulation to approximate the distribution of k over each time point t given the probability of a functional node getting infected. Wang et al. investigated a rumor blocking in static networks by considering dynamic Ising propagation model which consists of the individual tendency and global popularity of the rumor Wang et al. (2016; 2017). Under the constraint of user experience utility, they proposed DRIMUX method to protect a set of nodes in t time interval to limit the spreading of rumor.
In dynamic networks, Prakash et al. proposed greedy algorithms, called NLDS, as preemptive protection of the dynamic networks (Prakash et al. 2010). The methods are composed on different variants which select protected nodes based on the highest degree centrality, acquaintance (random neighbor) or the largest eigenvalue of the adjacency matrix. Liu & Gao investigated a different task of influence blocking in dynamic email networks (Liu and Gao 2011). They introduced an adaptive AutonomyOriented Computing which actively propagates the vaccination patches to counter a virusembedded email spreading. VAILDN is introduced by Zhan et al. (2017) as a postemptive scheme protection. By merging all infected nodes into one supernode and building a weighted dominator tree of modified input network, VAILDN determines the protected nodes based on each subtree benefit comparison.
Table 2 shows the comparison of our proposed method to the relevant existing work on graph protection strategy. To summarize, none of the existing works investigated the suppressing the epidemic spreading by multipleturns graph protection strategies on dynamic networks.
Problems related to graph protection on dynamic networks
There are some problems related to our work. Epidemic containment using link deactivation (Bishop and Shames 2011; Van Mieghem et al. 2011; Matamalas et al. 2018), aims to deactivate a set of links (instead of nodes) to contain epidemic spreading in the networks. Van Mieghem proposes a link removal approach to decrease the spectral radius of graph during epidemic spreading (Van Mieghem et al. 2011). Bishop discusses a mechanism for reducing the speed of disease propagation (Bishop and Shames 2011). Matamalas introduces an epidemic controlling approach based on the deactivation of most important links transmitting the disease (Matamalas et al. 2018). These studies are different from our focus as they are focusing on link selection instead of node selection. Additionally, in the realworld social networks, nodes represent users while links/edges represent friendship connections among users. For a network administrator, such as in Facebook or Twitter, it is more reasonable to temporarily deactivate a certain user in the case of rumor spreading than to deactivate part of the users’ friendship relations. While in human contact networks, it is more plausible to immunize an important person than to restrict a combination of several peertopeer interactions.
Network dismantling (Braunstein et al. 2016; Ren et al. 2018) is another problem related to our work. It is the problem of determining a minimum set of nodes in which removal breaks the network structure into subcritical connected components at minimum cost. Braunstein et al. (2016) provides insightful finding that the dismantling problem is an intrinsically collective problem and that optimal dismantling sets cannot be viewed as a collection of individually wellperforming nodes. Ren et al. (2018) proposed a method based on the spectral properties of a nodeweighted Laplacian operator to solve the problem.
Influence maximization problem on dynamic networks is also related to our work. While in the influence maximization we aim to maximize the influence spreading (information diffusion) (Tong et al. 2017; Murata and Koga 2018), the graph protection tries to restrain and contain any of those spreading process. Tong et al. (2017) introduced a greedy adaptive seeding strategy as an efficient heuristic for maximizing influence in dynamic social networks. Murata and Koga (2018) proposed three new methods for solving the problem which are the extensions of the methods for static networks.
Proposed methods
In this section, we propose new methods for multipleturns graph protection problem in dynamic networks, namely ReProtect and ReProtectp. To restrain the spreading of epidemic in dynamic networks, we divide the protection budget wisely into several turns. The protected nodes are selected in each turn according to the currently observed temporal snapshot of dynamic network. Using the multipleturns protection, we aim to address the changing of network structure and incoming rumor or virus attacks during the temporal transition in dynamic networks.
Figure 2 illustrates our proposed method in each turn, which takes a temporal snapshot of dynamic networks at time t as an input and determines the set of protected nodes. In each given turn, we determine the most critical set of nodes of the input network. A node is considered as a critical node if it is assumed that protecting such node contribute to block largescale epidemic spreading (Chen et al. 2016; Wang et al. 2016, 2017).
The main idea of our method can be described in the following key points:
1. Minimum vertex cover (MVC)
At first, we aim to find the set of the most critical nodes in the input network. Many previous studies suggest that a certain critical node criterion is best for a certain type of network structure. For instance, degree centrality is most suitable for dense and highly centralized network (Lawyer 2015; Chen et al. 2016), while betweenness centrality and connectivity are well fit for clustered networks with the existence of graph bridges (Italiano et al. 2012; Khan et al. 2015; Lawyer 2015).
We propose to consider a minimum vertex cover (MVC) as a criterion to determine set of critical nodes from networks. Given a graph G=(V,E), a vertex cover is a subset of the nodes V_{c}⊆V such that every edge of G is connected to V_{c}. Hence, this set of nodes V_{c} in graph G cover every edge in G. A minimum vertex cover is a vertex cover with the smallest possible number of nodes. Every graph trivially has a vertex cover where V_{c}=V. Figure 3a shows the vertex cover, and Fig. 3b shows the minimum vertex cover for the same graphs. The complexity of vertex cover problem is NPComplete, and that of the minimum vertex cover problem is NPHard.
As shown in Fig. 2, our input is a static network G_{t}, the observed snapshot of dynamic network at time t. We aim to completely cover all the connections in G_{t}, which are represented by edges, by the smallest possible size of nodes. The size definition of MVC is intuitively aligned with the limited size of the protection budget in graph protection problem. Following the definition of graph protection problem, we can show the role of MVC as the protection threshold in a network.
Theorem 1
(Protection Threshold) The protection threshold is the minimum required size of S to disconnect graph G such that no propagation may occur among nodes. Given an undirected connected graph G=(V,E), a minimum vertex cover of G is also a protection threshold of G.
Proof
A vertex cover V_{c} of G is a subset of the nodes V_{c}⊆V such that (u,v)∈E⇒u∈V_{c}∨v∈V_{c}. A minimum vertex cover \(V_{c}^{*}\) is a V_{c} with the smallest size as follows:
Since all edges in graph G is covered by \(V_{c}^{*}\):
then by removing all corresponding edges in G connected to \(V_{c}^{*}\) we get \(G^{(V_{c}^{*})} = \left (V_{c}^{*}, E^{(V_{c}^{*})}\right)\). Thus, \(G^{(V_{c}^{*})}\) has no edge, i.e., \(E^{(V_{c}^{*})} = \{\}, \left E^{(V_{c}^{*})}\right  = 0\).
According to Definition 1 and 2, protecting the set S of nodes in G is removing all edges of G connected to S. This is a minimax function of minimizing the size S to get the maximum edges in G covered as follows:
Consequently, by protecting minimum vertex cover \(V_{c}^{*}\), i.e., \(S = V_{c}^{*}\), then G^{(S)} has no edge. Hence, a minimum vertex cover \(V_{c}^{*}\) of G is also a protection threshold of G. □
2. Topk highest degree MVC
Let us recall that MVC is a set of nodes without any requirement of ordering. Intuitively, given k budget, selecting any k nodes from \(V_{c}^{*}\) may result in a different set of nodes. Additionally, not all of the node in MVC should have the same priority to be protected within a limited budget. We consider that the more connected a node v to its neighbors in G, the more critical node v to be protected. Hence, after obtaining MVC nodes from the input network, we reorder MVC nodes using their degree value within the input network.
Suppose that at time t we are given an input temporal snapshot graph G_{t} and protection budget k_{t}. Under the constraint of limited protection budget (k_{t}), we select top k_{t} MVC nodes based on their degree value within graph G_{t}.
3. Reinforcement learning as solution approximation
Despite the protection threshold guarantee of MVC, finding the MVC nodes of graphs is NPHard (Hartman and Weigt 2006). We consider a reinforcement learning (RL) approach to approximate the solution. RL approach aims to obtain an optimal solution by maximizing the cumulative rewards without given any predefined deterministic policies (Mnih et al. 2015; Khalil et al. 2017). Such advantage enables us to exploit the known best policy while also consider exploring unknown policies to obtain an optimal solution.
More specifically, we leverage the nstep fitted QLearning (Khalil et al. 2017) to obtain MVC approximation with an efficient training process and scalable implementation. Hence, our proposed methods take the advantage of nstep QLearning (Sutton and Barto 1998) and fitted Qiteration (Riedmiller 2005).
We let the nstep fitted QLearning iteratively learn to construct a vertex cover (V_{c}) solution of the input network. We define the RL environment as follows:

State (\(\mathbb {S}\)): set of currently selected V_{c} nodes from input graph

Action (\(\mathbb {A}\)): add new node v to vertex cover set \(\mathbb {S}\)

Reward (\(\mathbb {R}\)): 1, as our goal is to get the minimum size of vertex cover, we set a penalty for adding a new node into V_{c} set.

Termination criteria: all edges are covered
To quantify how good is taking an action \(a \in \mathbb {A}\) given a state \(s \in \mathbb {S}\), in QLearning, we have the QFunction (Watkins 1989). QFunction evaluates the pair of state and action and maps it into a single value, called QValue, using the following Bellman optimality equation:
where \(s \in \mathbb {S}\) is a given state, \(a \in \mathbb {A}\) is the current action, r is the current reward, λ is the discount factor of the future rewards, \(s' \in \mathbb {S}\) is the next state, and \(a' \in \mathbb {A}\) is the next action. The calculation of QFunction is performed and updated iteratively for each possible pair of state and action. The result of all QValue is stored in a table, called QTable. The best action for a given state is indicated by the highest QValue.
To obtain the maximum expected cumulative reward achievable from a given pair of state and action, we can compute the optimal QFunction, denoted as Q^{∗}, using the following equation:
where s_{0} and a_{0} are the initial state and action respectively, t indicates a step which consists of: observe a state, perform an action, retrieve a reward, and observe the next state.
As the number of all possible pair of state and action can be very large, calculating the QValue in QTable is not efficient. Especially, if we are handling a largesize input network, using QTable is computationally infeasible and resourceconsuming. A nonlinear function approximator can be used to estimate the optimal QFunction in Eq. (8) such that:
where Ψ is the function parameters (weights) of our nonlinear function approximator Q(s,a,Ψ). A neural network or a kernel function can be used as the nonlinear function approximator of QFunction (Sutton and Barto 1998).
Recent studies show that neural networks or convolutional neural networks achieve stateoftheart results as function approximators (Mnih et al. 2015; Sutton and Barto 1998). The neural network architecture also speed up learning in finite problems, due to the fact that it can generalize from earlier experiences to previously unseen states (Mnih et al. 2015). In this paper, we propose a convolutional neural network as the function approximator of optimal QFunction. Recall that in QFunction, our input is a given state and action to obtain QValue as output. The state is the given input graph with currently selected V_{c} nodes. The actions are the possible nodes to be included into current V_{c}. In convolutional neural network architecture, our input should represents both of those state and action. Hence, we need a same fixedlength feature representation of the graph and each of its node. Therefore, in our construction of minimum vertex cover, our function approximator in Eq. (9) will be denoted as:
where \(h(\mathbb {S})\) and v represent the fixedlength feature representation of the state \(\mathbb {S}\) and an action of adding node v using the neural network set of weights Ψ.
4. Graph embeddings as featurebased representations
We leverage an efficient and scalable graph embedding technique, called Structure2Vec (Dai et al. 2016; Khalil et al. 2017), to embed the input graph and each of its node. This graph embedding technique computes a ddimensional feature embedding μ_{v} for each node v∈V, given the current partial solution \(\mathbb {S}\).
Given a temporal snapshot graph G_{t}, we embed each node v by constructing a ddimensional embedding μ_{v}. All of \(\mu _{v}^{(0)}\) entries are initialized as zero, and for every v∈V we update it iteratively in T iterations as follows:
with x_{v} is node v own tag, whether being already selected or not. Selected node will be given tag = 1, otherwise 0. N(v) is the set of neighbors of node v in graph G_{t}. \(\sum _{u \in N(v)} \mu _{u}^{(t)} \) is the feature of node v neighbors. w(u,v) is the neighbors’ edge weight, to consider the weighted connection in weighted graph. ψ_{1},ψ_{2},ψ_{3}, and ψ_{4} are the function parameters (weights) which specified as \(\psi _{1} \in \mathbb {R}^{d}\), \(\psi _{2} \in \mathbb {R}^{dxd}\), \(\psi _{3} \in \mathbb {R}^{dxd}\), and \(\psi _{4} \in \mathbb {R}^{d}\). ReLU is the rectifier linear unit activation function applied elementwise to input where ReLU(x)=x if x>0 and 0 otherwise.
Here we will explain how to get the function \(\hat {Q}(h(\mathbb {S}_{t}),v; \Psi)\) in Eq. (10). Once the embedding μ_{v} for each node v∈V is calculated using Eq. (11) after T iteration, we get \(\mu _{v}^{(T)}\). The pooled embedding of the entire graph G_{t} is then given by
Then we can use it to estimate the optimal QFunction in Eq. (10) as follows:
being \(\sum _{u \in V} \mu _{u}^{(T)}\) is the pooled embedding of the entire graph. ψ_{5},ψ_{6}, and ψ_{7} are the neural network parameters (weights) which specified as \(\psi _{5} \in \mathbb {R}^{2d}\), \(\psi _{6} \in \mathbb {R}^{dxd}\), and \(\psi _{7} \in \mathbb {R}^{dxd}\).
To this end, we show that the pooled embedding of the entire graph is used as a surrogate to represent the state. And the embedding of each node is used as a surrogate to represent the action. The function \(\hat {Q}(h(\mathbb {S}),v)\) is depend on the collection of seven parameters \(\Psi = \{\psi _{i}\}_{i=1}^{7}\) which are learned during the training phase and will be evaluated during the evaluation phase. Figure 4 shows the architecture illustration of neural networks used in this paper.
a. Training Phase
Algorithm 1 illustrates our proposed training phase. In each training iteration, our method returns the neural network’s set of parameters Ψ which successfully get V_{c} from graph G. In line 5, we specify how to select a new node by balancing exploration and exploitation. In this case, the exploration means selecting a random nodes with probability ε. The exploitation means we aim to get the maximum expected cummulative rewards, i.e. by selecting a node which maximizes the function \(\hat {Q}(h(\mathbb {S}_{t}),v; \Psi)\). \(h(\mathbb {S}_{t})\) is the embedding of state \(\mathbb {S}\) at step t. The exploration probability ε is set to decrease from 1.0 to 0.05 linear to the iteration step. To efficiently train the neural network, we perform batch processing as described in line 9.
The loss function which learned to minimize is as follows:
being \(y = \sum _{i=0}^{n1} r(S_{t+i},v_{t+i}) + \lambda \max _{v}' \hat {Q}\left (h(\mathbb {S}_{t+n}), v'; \Psi \right)\). n is the number of step updates.
b. Evaluation Phase
Algorithm 2 illustrates the evaluation phase of our proposed method. To get the besttrained neural network’s set of parameters (weights) Ψ^{∗}, we evaluate the training result against a set of given graph G^{D} available snapshots. We use this neural network set of parameters in the testing simulation of the graph protection.
c. Testing Phase
Algorithm 3 shows the testing phase of multipleturns graph protection strategy on dynamic networks. We are given an input snapshot of graph G_{t} and budget k_{t}. Each node in G_{t} is embedded into ddimensional feature vector. The size of d is equal to the embedding size during training in Algorithm 1. The minimum vertex cover of G_{t} is then constructed using the besttrained neural network’s set of parameters Ψ^{∗} resulted from Algorithm 2. Finally, we get a set S of top k_{t} degreeordered MVC nodes to be protected from the current temporal snapshot of graph G_{t}.
We also propose ReProtectp method, a variant of ReProtect, which trained on the perturbed version of each available snapshot of dynamic networks. The perturbation is performed by removing edges probabilistically from the snapshot graph. Specifically, for each edge, we generate a random number. If the edge weight is smaller than the generated random number, the edge will be removed. We introduce this variant to provide more variety to the training data and avoid possible overfitting issue.
Computational complexity analysis
Based on Algorithm 3, we present the analysis of computational complexity of our proposed ReProtect method. The cost of step 1 to initialize empty set S is constant. The step 2 and 3 are to construct an approximated MVC set of graph G_{t} which has the complexity of O(p·M) based on the analysis by Dai et al. (2016); Khalil et al. (2017). p is the constant number of node testing steps, equals to the number of nodes divided by the number step updates in QLearning. M is the number of edges. In nstep QLearning, we update the value of each action based on the rewards of taking the sequence of n actions consecutively. n is called as the number of step updates. Suppose that the number of nodes in graph G_{t} is 500 and the number of step updates is 5, then p is a constant number equals to 100. One can see that p ranges from 1 to the number of nodes in graph G_{t}.
Getting the ordered MVC nodes in step 4 has an average O(N·logN) using QuickSort, where N is the number of nodes in G_{t}. Therefore, the total computational complexity of our ReProtect method is O(p·M+N·logN). The difference of ReProtect and ReProtectp is only on training process. Similarly, we can infer that the computational complexity of ReProtectp method is also O(p·M+N·logN).
Evaluation
In this section, we provide experimental evaluations of our proposed methods. The goal of this evaluation is to answer the following questions:
1. (Effectiveness) How effective are the proposed methods in restraining epidemic spreading in both synthetic and realworld dynamic networks? We define the measurement of effectiveness using the surviving ratio (θ) of nodes in dynamic network G^{D} at the end of epidemics.
2. (Scalability) Are the proposed methods scalable with respect to the changing of graph size (in terms of the number of nodes) and different protection budget size (k)?
3. (Sensitivity Analysis) How is the effectiveness of our proposed methods in the different values of epidemic parameters, such as the infection rate (β) and recovery rate (δ)?
Dataset
We evaluate our proposed methods on various realworld dynamic network datasets, which summarized in Table 3.

Dutch College dataset is a directed network of friendship ratings among 32 university freshmen (Van de Bunt et al. 1999). Each student was asked to rate the others at seven different time points.

Hospital dataset contains the temporal network of human contacts between patients and healthcare workers in a hospital ward in Lyon, France (Vanhems et al. 2013). Data was collected in December 2010.

Hypertext 2009 dataset is the network of contacts of the attendees of the ACM Hypertext 2009 conference (Stehlé et al. 2011). In the network, a node represents a conference visitor, and an edge represents a facetoface contact.

PrimarySchool dataset contains the temporal network of contacts between teachers and children used in the study of BMC Infectious Diseases 2014 (Gemmetto et al. 2014; Stehlé et al. 2011).

Highschool 2013 dataset contains the temporal network of contacts between students in a high school in Marseilles, France (Mastrandrea et al. 2015). The data was collected in December 2011 and November 2012.

Infectious dataset is the network of facetoface people behavior during the Dublin Science Gallery 2009 exhibition (Isella et al. 2011).

Email dataset was obtained from the email communication between institution members (the core) from a large European research institution (Paranjape et al. 2017). A directed edge (u,v,t) means that person u sent an email to person v at time t in the network.
Comparison methods
Recall that to the best of our knowledge, there is no previous work has been proposed to handle the multipleturns graph protection problem on dynamic networks. Here, we investigate the performance comparison of the following methods:

None: simulates the condition without any protection.

GreedyMVC: approximates the set of MVC nodes of the input graph by greedily selects the uncovered edge with the maximum sum of degrees of its endpoints (Khalil et al. 2017). Then protects k nodes from this unordered MVC set.

Degree (Prakash et al. 2010): protects k highest degree nodes of the current snapshot of the dynamic network. This method represents the concept of NLDSDegree by Prakash et al. (2010).

Betweenness: protects k nodes with the highest betweenness centrality of the current snapshot of the dynamic network.

NetShield+ (Tong et al. 2010; Chen et al. 2016; Prakash et al. 2010): aims to protect a set of k nodes considering the largest eigenvalue of adjacency matrix. This methods represents the stronger variant of eigendecompositionbased methods by Chen et al. (2016) and NLDSEigenValue by Prakash et al. (2010).

GraphShield (Wijayanto and Murata 2017): protects k nodes by taking into account the role of graph connectivity and degree centrality.

ApproxDegree: simulates the 2approximation algorithm to get the MVC nodes (Chakrabarti; Hartman and Weigt 2006). We add the degreeordering nodes to this method for protecting the topk highest degree of MVC nodes.

ReProtect and ReProtectp: are our proposed methods.
Experimental setting
In the training phase, we use the embedding dimension size 64, batch size 64, embedding iteration 5 as suggested in Structure2Vec^{Footnote 1} (Dai et al. 2016). The setting of nstep is set to 5 and learning rate as 0.0001 and number of training iteration as 100,000. These three settings are commonly applied in nstep QLearning (Sutton and Barto 1998). In the evaluation phase, we consider the number of evaluation iteration as 100.
For a fair comparison, unless specified otherwise, all of the methods are simulated under the same setting as follows: infection rate β=0.8, recovery rate δ=0.2, and the initial number of attacked nodes (l) equals to the protection budget (k). We simulate l=k,l_{t}=k_{t}, and k_{1}=k_{2}=⋯=k_{t}. Random attack evaluation is employed in all experiments. The setting applies for evaluation in both SIS and SIR epidemic model.
All results presented in this section are the average of multiple simulations. Unless specified otherwise, we take the average from 100 simulations for each result. The initial condition is all nodes susceptible except the attacked ones, which are infected.
We let the epidemic spreading arrive at the stationary state before changing to the next snapshot of the network for SIS model. While for SIR model, we count the ratio of surviving nodes at the highest outbreak point, right before the final regime of epidemics as suggested by PastorSatorras et al. (2015). For continuity, in SIR model, we restart the epidemic spreading in the new snapshot after the final regime of epidemic spreading in the previous snapshot. Gillespie algorithm (Kiss et al. 2017) is used to simulate the epidemic spreading on networks. Additionally, we follow the time discretization method of dynamic network by Zhuang et al. (2013).
Finally, all of the experiments are performed on the same machine, Ubuntu 16.06 LTS PC with an Intel(R) Core(TM) i97900X CPU @ 3.30GHz CPU and NVIDIA GTX 1080 Ti SLI GPU.
Evaluation of effectiveness on synthetic network
We evaluate the performance of all comparison methods on a synthetic network generated using Dynamic Attributed Network with Community Structure Generator^{Footnote 2} (DANCER) (Largeron et al. 2017). Due to the simplicity setting of graph protection problem, we only consider the temporal network structure of the generated network and ignore their attribute and community assignment provided by DANCER. We generate a dynamic network with 100 nodes and ten temporal snapshots^{Footnote 3}.
Table 4 shows the average result of 100 simulations under the constraint of protection budget k=0.25N, being N is the number of nodes in the input graph. Both of our proposed methods obtain a higher ratio of surviving nodes than other competitors. ReProtectp achieves the highest protection effectiveness.
If we vary the number of given budget k, both of our proposed methods outperform the other baseline methods as shown in Fig. 5. When the given protection budget k is too small, ReProtect and ReProtectp exhibit competitive performances with other methods, but with an increasing k, they easily outperform other baseline methods, such as Degree, GraphShield, and Betweenness. On the other hand, ReProtect can also outperform other competing methods, even though it needs a bigger protection budget to obtain the similar performance of ReProtectp. An introduction of more data variety using graph perturbation into training process helps our proposed method to get a better result, as in ReProtectp.
Evaluation of effectiveness on realworld networks
On realworld networks, we compare the performance of all comparison methods on seven different datasets and two different epidemic models, i.e., SIS and SIR model. Table 5 and 6 show the result of surviving nodes ratio on SIS and SIR epidemic model respectively. The results are averaged from 100 simulations under the constraint of protection budget k=0.15N, with N is the number of nodes in the input graph. Both of our proposed methods consistently reach the highest ratio of surviving nodes. Additionally, in most cases, the proposed method with more training data variety using the perturbed graph, namely ReProtectp achieves a better result than the regular training as in ReProtect. Tables 7 and 8 present the standard deviation of the surviving nodes ratio.
To evaluate the performance comparison in different protection budget k, we vary the given k as shown in Figs. 6 and 7. Both of our proposed methods are able to outperform other competitors align with the increase given budget in all datasets, while constantly maintain competitive performance in a very small size of k. The consistency of better performance shown by our methods in many different numbers of available protection budget indicates the reliability as protection strategies.
We consider that the reinforcement learning is more suitable for graph protection on dynamic networks due to at least two major reasons. First, reinforcement learning approach using convolutional neural network as function approximator gives us a potential benefit to learn from previously solved MVC of network snapshot. By learning the given temporal structure of observed networks, this provides an incentive to predict future protection from previously learned actions in the same dynamic networks. The benefit of learning could not be obtained by traditional MVC approximation algorithms. Second, the nature of convolutional neural networks (CNN) provide us not only scalability in handling the large size networks which may contain up to billion nodes, but also easily parallelizable in multiple CPUs and GPUs. Here, we leverage our approach on top of recent advances in deep learning technology. Traditional MVC approximation algorithms are not specifically designed for this computationally expensive task.
Evaluation of effectiveness on the aggregate networks
According to the observations of Braha and BarYam (2006; 2009), the snapshots static networks are quite different from the aggregate network itself. The aggregate network is the network obtained by ignoring time and aggregating all of the temporal edges in the dynamic network (Braha and BarYam 2006; 2009). An interesting question arises, how does the multipleturns timebased protection strategies analyzed in our proposed methods compare with the protection strategies when implemented on the aggregate network? In this subsection, we report the effectiveness evaluation on the aggregate networks of the same synthetic and realworld datasets.
Tables 9 and 10 show the result of surviving nodes ratio (θ) on SIS and SIR epidemic model respectively. The results are averaged from 100 simulations under the constraint of protection budget k=0.15N, with N is the number of nodes in the input graph. Tables 11 and 12 present the standard deviation of the surviving nodes ratio. Compared with the results in Tables 5 and 6, we found that the multipleturns timebased strategies are beneficial and more effective than the aggregatebased strategies. The aggregatebased strategies are the protection strategies applied on the aggregate networks under the assumption that the timeaggregated networks are accessible and known a priori. We observe that the timeaggregation of all edges make the network denser thus require more nodes to be protected. The average degree of nodes in each snapshot of the network compared with that of in the aggregated network is shown in Table 3. In the aggregated network, the average degree of nodes is higher than in each network snapshot.
Evaluation of scalability
Let us recall our second evaluation goal, which aims to measure how scalable is the proposed method with respect to the changing of graph size and different k budget size. In this subsection, we report the result of scalability evaluation by investigating the computational running time of our proposed methods. Different values of k were used to evaluate the scalability in different scale of protection set.
To perform simulation by changing the number of nodes, we generate synthetic dynamic networks using Dynamic Attributed Network with Community Structure Generator (DANCER) (Largeron et al. 2017). We only consider the temporal network structure of the generated network and ignore their attribute and community assignment provided by DANCER. We generate dynamic networks with 10 temporal snapshots and the number of nodes is changed from N={100;200;300;500;1000;1500;2000}. The budget size is changed from {10;20;30;40;50}.
From Fig. 8, it can be inferred that our methods scale almost linearly with respect to the number of nodes. Hence, the proposed methods are scalable with respect to the changing of graph size, which means they are applicable for large size networks. Running our methods on graph with 2000 nodes takes less than 9 seconds. Further paralelization of neural network design can also be applied to speed up the running time.
Evaluation of sensitivity to epidemic parameters
In SIS and SIR model, epidemic parameters consist of the infection rate (β) and the recovery rate (δ). To analyze the sensitivity of our proposed methods, the effectiveness comparison with different epidemic parameters are shown in this subsection.
Prakash et al. (2011) demonstrated using empirical simulations that the ratio of infection rate over recovery rate \(\left (\frac {\beta }{\delta }\right)\) takes the role as constant dependent of epidemic threshold in various epidemic model including SIS and SIR. Epidemic threshold is an intrinsic property of a network. When the strength of the virus is greater than the epidemic threshold, then the epidemic would breakout (Prakash et al. 2011). The ratio of \(\frac {\beta }{\delta }\) is commonly called as the epidemic propagation rate (Wijayanto and Murata 2018b; Prakash et al. 2011).
We perfom simulations to confirm the effectiveness of our proposed methods using the same network dataset in Effectiveness on Synthetic Network subsection under three scenarios:
(1) Comparison of survival ratio θ when the epidemic propagation rate \(\left (\frac {\beta }{\delta }\right)\) changes
(2) Comparison of survival ratio θ when the infection rate (β) changes
(3) Comparison of survival ratio θ when the recovery rate (δ) changes
For a fair analysis and comparison, simulations are performed under a fixed protection budget(k).
Comparison of survival ratio θ when the epidemic propagation rate \(\left (\frac {\beta }{\delta }\right)\) changes
We change the ratio of \(\frac {\beta }{\delta }\) from \(\left \{\frac {0.9}{0.1}; \frac {0.8}{0.2}; \frac {0.7}{0.3}; \frac {0.6}{0.4}; \frac {0.5}{0.5}; \frac {0.4}{0.6}; \frac {0.3}{0.7}; \frac {0.2}{0.8}; \frac {0.1}{0.9}\right \}\). Figure 9 shows the comparison of survival ratio θ of all methods in SIS and SIR epidemic model. The results are averaged from 100 simulations under the fixed protection budget k=0.25N, with N is the number of nodes of the input network. In all of these conditions, both of our proposed methods obtain higher survival ratio θ than other competitors.
Comparison of survival ratio θ when the infection rate ( β ) changes
We change the infection rate (β) from {0.9;0.8;0.7;0.6;0.5;0.4;0.3;0.2;0.1} with fixed recovery rate (δ). Figure 10 shows the comparison of survival ratio θ of all methods. The results are presented from the average of 100 simulations with a fixed protection budget k=0.25N, where N is the number of nodes of the input network. Both of our proposed methods could achieve highest survival ratio θ regardless the value of infection rate and epidemic models.
Comparison of survival ratio θ when the recovery rate (δ) changes
We investigate the comparison of survival ratio θ by changing the recovery rate (δ) from {0.9;0.8;0.7;0.6;0.5;0.4;0.3;0.2;0.1} with fixed infection rate (β) and the protection budget k=0.25N. N is the number of nodes of the input network. As shown in Fig. 11, in SIS and SIR epidemic model, both of ReProtect and ReProtectp methods obtain higher survival ratio θ than other competitors. The results are averaged from 100 simulations.
Conclusion
In this paper, we addressed the multipleturns graph protection problem to restrain epidemic spreading on dynamic networks. The protection budget is divided into several turns and selects protected nodes based on the presently observed temporal snapshot of dynamic networks. By proving the role of minimum vertex cover (MVC) as the protection threshold of the network, we choose to protect the highest degree of MVC nodes at the size of each allocated protection budget. We introduce methods utilizing the nstep fitted QLearning to efficiently learn the MVC construction from the input graph under reinforcement learning approach. Graph embedding technique is incorporated as a featurebased representation of the input network states. We demonstrate the effectiveness and scalability of our methods, namely ReProtect and ReProtectp. Extensive evaluations on synthetic and realworld network datasets show that our proposed methods outperform other baseline methods while maintaining the scalability. Further investigation of two different epidemic model simulation, i.e., SIS and SIR model, also confirm the effectiveness and scalability of our methods.
The strategy of handling graph protection problem against nontrivial targeted attacks in dynamic networks is left for our future work. Extending our methods into a multiagent policy gradient reinforcement learning to achieve better training efficiency will also be our next consideration.
Notes
the generated network is available at http://bit.ly/syntheticnetwork
Abbreviations
 CNN:

Convolutional neural network
 DANCER:

Dynamic attributed network with community structure generator
 DAVA:

Dataaware vaccine allocation
 DBMF:

Degreebased meanfield
 DRIMUX:

Dynamic rumor influence minimization with user experience
 IBMF:

Individualbased meanfield
 MVC:

Minimum vertex cover
 NIIP:

Node immunization over infectious period
 NLDS:

NonLinear dynamical system
 ReLU:

Rectifier linear unit
 RL:

Reinforcement learning
 SIS:

SusceptibleInfectedSusceptible epidemic model
 SIR:

SusceptibleInfectedRecovered epidemic model
 TIM:

Targeted immunization method
 VAILDN:

Vaccination allocation in large dynamic networks
References
Bakker, C, Halappanavar M, Visweswara Sathanur A (2018) Dynamic graphs, community detection, and riemannian geometry. Appl Netw Sci 3(1):3. https://doi.org/10.1007/s4110901800592.
Bishop, AN, Shames I (2011) Link operations for slowing the spread of disease in complex networks. EPL (Europhys Lett) 95(1):18005.
Braha, D, BarYam Y (2009) TimeDependent Complex Networks: Dynamic Centrality, Dynamic Motifs, and Cycles of Social Interactions(Gross T, Sayama H, eds.). Springer, Berlin. https://doi.org/10.1007/9783642012846_3.
Braha, D, BarYam Y (2006) From centrality to temporary fame: Dynamic centrality in complex networks. Complexity 12:59–63.
Braunstein, A, Dall’Asta L, Semerjian G, Zdeborová L (2016) Network dismantling. Proc Natl Acad Sci 113(44):12368–12373. https://doi.org/10.1073/pnas.1605083113.
Buono, C, Braunstein LA (2015) Immunization strategy for epidemic spreading on multilayer networks. EPL (Europhys Lett) 109(2):26001.
Chakrabarti, AApproximation Algorithms: Vertex Cover (Computer Science 105  Winter 2005). http://tandy.cs.illinois.edu/dartmouthcsapprox.pdf Accessed 27 Dec 2017.
Chen, C, Tong H, Prakash BA, Tsourakakis CE, EliassiRad T, Faloutsos C, Chau DH (2016) Node immunization on large graphs: Theory and algorithms. IEEE Trans Knowl Data Eng 28(1):113–126.
Dai, H, Dai B, Song L (2016) Discriminative embeddings of latent variable models for structured data. In: Balcan MF Weinberger KQ (eds)Proceedings of The 33rd International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 48, 2702–2711.. PMLR, New York.
Enright, J, Kao RR (2018) Epidemics on dynamic networks. Epidemics 24:88–97. https://doi.org/10.1016/j.epidem.2018.04.003.
Gemmetto, V, Barrat A, Cattuto C (2014) Mitigation of infectious disease at school: targeted class closure vs school closure. BMC Infect Dis 14(1):695. https://doi.org/10.1186/PREACCEPT6851518521414365.
Habiba, Yu Y, BergerWolf TY, Saia J (2010) Finding spread blockers in dynamic networks. In: Giles L, Smith M, Yen J, Zhang H (eds)Advances in Social Network Mining and Analysis, 55–76.. Springer, Berlin.
Hartman, AK, Weigt M (2006) Phase Transitions in Combinatorial Optimization Problems: Basics, Algorithms and Statistical Mechanics. WileyVCH Verlag, Weinheim.
Hill, SA, Braha D (2010) Dynamic model of timedependent complex networks. Phys Rev E 82:046105.
Holme, P (2015) Modern temporal network theory: a colloquium. Eur Phys J B 88(9). https://doi.org/10.1140/epjb/e2015606574.
Holme, P, Saramäki J (2012) Temporal networks. Phys Rep 519(3):97–125. https://doi.org/10.1016/j.physrep.2012.03.001. Temporal Networks.
Isella, L, Stehlé J, Barrat A, Cattuto C, Pinton J, Van den Broeck W (2011) What’s in a crowd? analysis of facetoface behavioral networks. J Theor Biol 271(1):166–180. https://doi.org/10.1016/j.jtbi.2010.11.033.
Italiano, GF, Laura L, Santaroni F (2012) Finding strong bridges and strong articulation points in linear time. Theor Comput Sci 447:74–84. https://doi.org/10.1016/j.tcs.2011.11.011. Combinational Algorithms and Applications (COCOA 2010).
Kermack, WO, McKendrick AG (1927) A contribution to the mathematical theory of epidemics. Proc R Soc Lond A Math Phys Eng Sci 115(772):700–721. https://doi.org/10.1098/rspa.1927.0118.
Khalil, E, Dai H, Zhang Y, Dilkina B, Song L (2017) Learning combinatorial optimization algorithms over graphs. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds)Advances in Neural Information Processing Systems 30, 6339–6349.. Curran Associates, Inc., New York.
Khan, SA, Bölöni L, Turgut D (2015) Bridge protection algorithms – a technique for faulttolerance in sensor networks. Ad Hoc Netw 24:186–199. https://doi.org/10.1016/j.adhoc.2014.08.016.
Kiss, IZ, Miller JC, Simon PL (2017) Mathematics of Epidemics on Networks: From Exact to Approximate Models. Springer, New York.
Largeron, C, Mougel PN, Benyahia O, Zaïane OR (2017) Dancer: dynamic attributed networks with community structure generation. Knowl Inf Syst 53(1):109–151. https://doi.org/10.1007/s1011501710282.
Lawyer, G (2015) Understanding the influence of all nodes in a network. Sci Rep 5(8665):1–9. http://dx.doi.org/10.1038/srep08665.
Liu, J, Gao C (2011) Adaptive immunization in dynamic networks. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 6804 LNAI:673–683. https://doi.org/10.1007/9783642219160_71.
Mastrandrea, R, Fournet J, Barrat A (2015) Contact patterns in a high school: A comparison between data collected using wearable sensors, contact diaries and friendship surveys. PLoS ONE 10:1–26. https://doi.org/10.1371/journal.pone.0136497.
Matamalas, JT, Arenas A, Gómez S (2018) Effective approach to epidemic containment using link equations in complex networks. Sci Adv 4(12). https://doi.org/10.1126/sciadv.aau4212.
Mnih, V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, Petersen S, Beattie C, Sadik A, Antonoglou I, King H, Kumaran D, Wierstra D, Legg S, Hassabis D (2015) Humanlevel control through deep reinforcement learning. Nature 518:529–533. https://doi.org/10.1038/nature14236.
Moore, C, Ghoshal G, Newman MEJ (2006) Exact solutions for models of evolving networks with addition and deletion of nodes. Phys Rev E 74:036121.
Murata, T, Koga H (2018) Extended methods for influence maximization in dynamic networks. Comput Soc Netw 5(8):1–21. https://doi.org/10.1186/s4064901800568.
Paranjape, A, Benson AR, Leskovec J (2017) Motifs in temporal networks In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining. WSDM ’17, 601–610.. ACM, New York. https://doi.org/10.1145/3018661.3018731.
PastorSatorras, R, Castellano C, Van Mieghem P, Vespignani A (2015) Epidemic processes in complex networks. Rev Mod Phys 87:925–979. https://doi.org/10.1103/RevModPhys.87.925.
PastorSatorras, R, Vespignani A (2002) Immunization of complex networks. Phys Rev E 65:036104. https://doi.org/10.1103/PhysRevE.65.036104.
Prakash, BA, Chakrabarti D, Faloutsos M, Valler N, Faloutsos C (2011) Threshold conditions for arbitrary cascade models on arbitrary networks In: 2011 IEEE 11th International Conference on Data Mining, 537–546.. IEEE, New York. https://doi.org/10.1109/ICDM.2011.145.
Prakash, BA, Tong H, Valler N, Faloutsos M, Faloutsos C (2010) Virus propagation on timevarying networks: Theory and immunization algorithms. In: Balcázar JL, Bonchi F, Gionis A, Sebag M (eds)Machine Learning and Knowledge Discovery in Databases, 99–114.. Springer, Berlin.
Ren, X, Gleinig N, Helbing D, AntulovFantulin N (2018) Generalized network dismantling. CoRR abs/1801.01357. http://arxiv.org/abs/1801.01357.
Riedmiller, M (2005) Neural fitted q iteration – first experiences with a data efficient neural reinforcement learning method In: Proceedings of the 16th European Conference on Machine Learning. ECML’05, 317–328.. Springer, Berlin.
Song, C, Hsu W, Lee ML (2015) Node immunization over infectious period In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. CIKM ’15, 831–840.. ACM, New York.
Stehlé, J, Voirin N, Barrat A, Cattuto C, Isella L, Pinton J, Quaggiotto M, Van den Broeck W, Régis C, Lina B, Vanhems P (2011) Highresolution measurements of facetoface contact patterns in a primary school. PLoS ONE 6(8):23176. https://doi.org/10.1371/journal.pone.0023176.
Sutton, RS, Barto AG (1998) Reinforcement Learning: An Introduction. MIT Press, Cambridge.
Tong, H, Prakash BA, Tsourakakis C, EliassiRad T, Faloutsos C, Chau DH (2010) On the vulnerability of large graphs In: 2010 IEEE International Conference on Data Mining, 1091–1096.. IEEE, New York. https://doi.org/10.1109/ICDM.2010.54.
Tong, G, Wu W, Tang S, Du D (2017) Adaptive influence maximization in dynamic social networks. IEEE/ACM Trans Netw 25(1):112–125. https://doi.org/10.1109/TNET.2016.2563397.
Van de Bunt, GG, Van Duijn MAJ, Snijders TAB (1999) Friendship networks through time: An actororiented dynamic statistical network model. Comput Math Org Theory 5(2):167–192.
Van Mieghem, P, Stevanović D, Kuipers F, Li C, van de Bovenkamp R, Liu D, Wang H (2011) Decreasing the spectral radius of a graph by link removals. Phys Rev E 84:016101. http://dx.doi.org/10.1103/PhysRevE.84.016101.
Vanhems, P, Barrat A, Cattuto C, Pinton JF, Khanafer N, Regis C, Kim Ba, Comte B, Voirin N (2013) Estimating potential infection transmission routes in hospital wards using wearable proximity sensors. PLoS ONE 8(9):73970. https://doi.org/10.1371/journal.pone.0073970.
Wang, B, Chen G, Fu L, Song L, Wang X, Liu X (2016) Drimux: Dynamic rumor influence minimization with user experience in social networks In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI16), 791–797.
Wang, B, Chen G, Fu L, Song L, Wang X (2017) Drimux : Dynamic rumor influence minimization with user experience in social networks. IEEE Trans Knowl Data Eng 29(10):2168–2181.
Watkins, CJCH (1989) Learning from Delayed Rewards. Cambridge University, Cambridge.
Wijayanto, AW, Murata T (2017) Flowaware vertex protection strategy on large social networks In: Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017. ASONAM ’17, 58–63.. ACM, New York. https://doi.org/10.1145/3110025.3110033.
Wijayanto, AW, Murata T (2018) Learning adaptive graph protection strategy on dynamic networks via reinforcement learning In: 2018 IEEE/WIC/ACM International Conference on Web Intelligence (WI). WI 2018, 534–539.. IEEE, New York. https://doi.org/10.1109/WI.2018.0041.
Wijayanto, AW, Murata T (2018) Preemptive spectral graph protection strategies on multiplex social networks. Appl Netw Sci 3(1):5. https://doi.org/10.1007/s4110901800618.
Zhan, J, Rafalski T, Stashkevich G, Verenich E (2017) Vaccination allocation in large dynamic networks. J Big Data 4(1):2. https://doi.org/10.1186/s4053701600614.
Zhang, Y, Prakash BA (2014) Dava: Distributing vaccines over networks under prior information In: Proceedings of the 2014 SIAM International Conference on Data Mining, 46–54.. SIAM, Philadelphia. https://doi.org/10.1137/1.9781611973440.6. https://epubs.siam.org/doi/abs/10.1137/1.9781611973440.6.
Zhang, Y, Prakash BA (2015) Dataaware vaccine allocation over large networks. ACM Trans Knowl Discov Data 10(2):20–12032.
Zhang, Y, Ramanathan A, Vullikanti A, Pullum L, Prakash BA (2017) Datadriven immunization In: 2017 IEEE International Conference on Data Mining (ICDM), 615–624.. IEEE, New York. https://doi.org/10.1109/ICDM.2017.71.
Zhao, D, Wang L, Li S, Wang Z, Wang L, Gao B (2014) Immunization of epidemics in multiplex networks. PLoS ONE 9(11):1–5. https://doi.org/10.1371/journal.pone.0112018.
Zhuang, H, Sun Y, Tang J, Zhang J, Sun X (2013) Influence maximization in dynamic social networks In: 2013 IEEE 13th International Conference on Data Mining, 1313–1318.. IEEE, New York. https://doi.org/10.1109/ICDM.2013.145.
Acknowledgements
We thank the anonymous reviewers for their valuable suggestions and comments.
Funding
This work was supported by JSPS GrantinAid for Scientific Research(B) (Grant Number 17H01785) and JST CREST (Grant Number JPMJCR1687). A.W.W. is suported by Indonesia Endowment Fund for Education (LPDP) for the educational scholarship.
Availability of data and materials
All of the datasets used in this paper are publicly accessible online in their respective references.
Hospital, Hypertext 2009, PrimarySchool, Highschool 2013, and Infectious dataset can be obtained from SocioPatterns (http://www.sociopatterns.org/datasets).
Dutch College dataset is available at Konect (http://konect.unikoblenz.de/networks).
Email dataset can be found at SNAP (http://snap.stanford.edu/data).
The source code of our proposed methods will be accessible by request upon acceptance.
Author information
Authors and Affiliations
Contributions
All authors contribute to the writing of the paper. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Wijayanto, A.W., Murata, T. Effective and scalable methods for graph protection strategies against epidemics on dynamic networks. Appl Netw Sci 4, 18 (2019). https://doi.org/10.1007/s4110901901227
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s4110901901227
Keywords
 Graph protection
 Node immunization
 Dynamic networks
 Reinforcement learning