Effective and scalable methods for graph protection strategies against epidemics on dynamic networks

Fundamental work of epidemic modeling on dynamic networks

The properties of dynamic networks are essentially different from those in static networks. (Braha and Bar-Yam 2006; 2009) found that the overlap of the centrality in dynamic networks and that in the aggregated (static) network is quite low. They also demonstrated that the static topology is unable to capture the dynamic properties of social networks. Hill and Braha (2010) propose a reinforced random walk approach to explain dynamic centrality phenomena and qualitatively reproduce the characteristic features of real-world networks. Those studies (Braha and Bar-Yam 2006; 2009; Hill and Braha 2010) provide an important foundation of dynamic network properties.

Holme presents a systematic review of dynamic networks and discusses methods for topological and temporal structure analysis (Holme and Saramäki 2012; Holme 2015). More specifically, Pastor-Satorras et al. (2015) discuss a fundamental review of epidemic model on dynamic networks, which also recently emphasized by Enright and Kao (2018).

Graph protection strategy and its application in dynamic networks

The study of graph protection strategies has mostly been introduced by assuming the static topologies of network structure. Pastor-Satorras and Vespignani investigated the effect of random uniform and targeted high-degree immunization of individuals on homogeneous complex networks and scale-free networks (Pastor-Satorras and Vespignani 2002). Chen et al. proposed NetShield (Tong et al. 2010) and NetShield+ (Chen et al. 2016) which use the properties of matrix perturbation to find a set of nodes in static networks to be pre-emptively protected (Tong et al. 2010). Zhang and Prakash (2014; 2015) developed DAVA and DAVA-fast, two post-emptive polynomial-time heuristics methods which merge all infected nodes into a supernode by building a weighted dominator tree of input network. NIIP (Song et al. 2015) extracts a maximum directed acyclic graph from a static network then implements a Monte Carlo simulation to approximate the distribution of k over each time point t given the probability of a functional node getting infected. Wang et al. investigated a rumor blocking in static networks by considering dynamic Ising propagation model which consists of the individual tendency and global popularity of the rumor Wang et al. (2016; 2017). Under the constraint of user experience utility, they proposed DRIMUX method to protect a set of nodes in t time interval to limit the spreading of rumor.

In dynamic networks, Prakash et al. proposed greedy algorithms, called NLDS, as pre-emptive protection of the dynamic networks (Prakash et al. 2010). The methods are composed on different variants which select protected nodes based on the highest degree centrality, acquaintance (random neighbor) or the largest eigenvalue of the adjacency matrix. Liu & Gao investigated a different task of influence blocking in dynamic email networks (Liu and Gao 2011). They introduced an adaptive Autonomy-Oriented Computing which actively propagates the vaccination patches to counter a virus-embedded email spreading. VAILDN is introduced by Zhan et al. (2017) as a post-emptive scheme protection. By merging all infected nodes into one supernode and building a weighted dominator tree of modified input network, VAILDN determines the protected nodes based on each sub-tree benefit comparison.

Problems related to graph protection on dynamic networks

There are some problems related to our work. Epidemic containment using link deactivation (Bishop and Shames 2011; Van Mieghem et al. 2011; Matamalas et al. 2018), aims to deactivate a set of links (instead of nodes) to contain epidemic spreading in the networks. Van Mieghem proposes a link removal approach to decrease the spectral radius of graph during epidemic spreading (Van Mieghem et al. 2011). Bishop discusses a mechanism for reducing the speed of disease propagation (Bishop and Shames 2011). Matamalas introduces an epidemic controlling approach based on the deactivation of most important links transmitting the disease (Matamalas et al. 2018). These studies are different from our focus as they are focusing on link selection instead of node selection. Additionally, in the real-world social networks, nodes represent users while links/edges represent friendship connections among users. For a network administrator, such as in Facebook or Twitter, it is more reasonable to temporarily deactivate a certain user in the case of rumor spreading than to deactivate part of the users’ friendship relations. While in human contact networks, it is more plausible to immunize an important person than to restrict a combination of several peer-to-peer interactions.

Network dismantling (Braunstein et al. 2016; Ren et al. 2018) is another problem related to our work. It is the problem of determining a minimum set of nodes in which removal breaks the network structure into subcritical connected components at minimum cost. Braunstein et al. (2016) provides insightful finding that the dismantling problem is an intrinsically collective problem and that optimal dismantling sets cannot be viewed as a collection of individually well-performing nodes. Ren et al. (2018) proposed a method based on the spectral properties of a node-weighted Laplacian operator to solve the problem.

Influence maximization problem on dynamic networks is also related to our work. While in the influence maximization we aim to maximize the influence spreading (information diffusion) (Tong et al. 2017; Murata and Koga 2018), the graph protection tries to restrain and contain any of those spreading process. Tong et al. (2017) introduced a greedy adaptive seeding strategy as an efficient heuristic for maximizing influence in dynamic social networks. Murata and Koga (2018) proposed three new methods for solving the problem which are the extensions of the methods for static networks.

1. Minimum vertex cover (MVC)

At first, we aim to find the set of the most critical nodes in the input network. Many previous studies suggest that a certain critical node criterion is best for a certain type of network structure. For instance, degree centrality is most suitable for dense and highly centralized network (Lawyer 2015; Chen et al. 2016), while betweenness centrality and connectivity are well fit for clustered networks with the existence of graph bridges (Italiano et al. 2012; Khan et al. 2015; Lawyer 2015).

We propose to consider a minimum vertex cover (MVC) as a criterion to determine set of critical nodes from networks. Given a graph G=(V,E), a vertex cover is a subset of the nodes V_c⊆V such that every edge of G is connected to V_c. Hence, this set of nodes V_c in graph G cover every edge in G. A minimum vertex cover is a vertex cover with the smallest possible number of nodes. Every graph trivially has a vertex cover where V_c=V. Figure 3a shows the vertex cover, and Fig. 3b shows the minimum vertex cover for the same graphs. The complexity of vertex cover problem is NP-Complete, and that of the minimum vertex cover problem is NP-Hard.

As shown in Fig. 2, our input is a static network G_t, the observed snapshot of dynamic network at time t. We aim to completely cover all the connections in G_t, which are represented by edges, by the smallest possible size of nodes. The size definition of MVC is intuitively aligned with the limited size of the protection budget in graph protection problem. Following the definition of graph protection problem, we can show the role of MVC as the protection threshold in a network.

2. Top-k highest degree MVC

Let us recall that MVC is a set of nodes without any requirement of ordering. Intuitively, given k budget, selecting any k nodes from \(V_{c}^{*}\) may result in a different set of nodes. Additionally, not all of the node in MVC should have the same priority to be protected within a limited budget. We consider that the more connected a node v to its neighbors in G, the more critical node v to be protected. Hence, after obtaining MVC nodes from the input network, we reorder MVC nodes using their degree value within the input network.

Suppose that at time t we are given an input temporal snapshot graph G_t and protection budget k_t. Under the constraint of limited protection budget (k_t), we select top- k_t MVC nodes based on their degree value within graph G_t.

3. Reinforcement learning as solution approximation

Despite the protection threshold guarantee of MVC, finding the MVC nodes of graphs is NP-Hard (Hartman and Weigt 2006). We consider a reinforcement learning (RL) approach to approximate the solution. RL approach aims to obtain an optimal solution by maximizing the cumulative rewards without given any pre-defined deterministic policies (Mnih et al. 2015; Khalil et al. 2017). Such advantage enables us to exploit the known best policy while also consider exploring unknown policies to obtain an optimal solution.

More specifically, we leverage the n-step fitted Q-Learning (Khalil et al. 2017) to obtain MVC approximation with an efficient training process and scalable implementation. Hence, our proposed methods take the advantage of n-step Q-Learning (Sutton and Barto 1998) and fitted Q-iteration (Riedmiller 2005).

where \(s \in \mathbb {S}\) is a given state, \(a \in \mathbb {A}\) is the current action, r is the current reward, λ is the discount factor of the future rewards, \(s' \in \mathbb {S}\) is the next state, and \(a' \in \mathbb {A}\) is the next action. The calculation of Q-Function is performed and updated iteratively for each possible pair of state and action. The result of all Q-Value is stored in a table, called Q-Table. The best action for a given state is indicated by the highest Q-Value.

where s₀ and a₀ are the initial state and action respectively, t indicates a step which consists of: observe a state, perform an action, retrieve a reward, and observe the next state.

where Ψ is the function parameters (weights) of our non-linear function approximator Q(s,a,Ψ). A neural network or a kernel function can be used as the non-linear function approximator of Q-Function (Sutton and Barto 1998).

where \(h(\mathbb {S})\) and v represent the fixed-length feature representation of the state \(\mathbb {S}\) and an action of adding node v using the neural network set of weights Ψ.

4. Graph embeddings as feature-based representations

We leverage an efficient and scalable graph embedding technique, called Structure2Vec (Dai et al. 2016; Khalil et al. 2017), to embed the input graph and each of its node. This graph embedding technique computes a d-dimensional feature embedding μ_v for each node v∈V, given the current partial solution \(\mathbb {S}\).

with x_v is node v own tag, whether being already selected or not. Selected node will be given tag = 1, otherwise 0. N(v) is the set of neighbors of node v in graph G_t. \(\sum _{u \in N(v)} \mu _{u}^{(t)} \) is the feature of node v neighbors. w(u,v) is the neighbors’ edge weight, to consider the weighted connection in weighted graph. ψ₁,ψ₂,ψ₃, and ψ₄ are the function parameters (weights) which specified as \(\psi _{1} \in \mathbb {R}^{d}\), \(\psi _{2} \in \mathbb {R}^{dxd}\), \(\psi _{3} \in \mathbb {R}^{dxd}\), and \(\psi _{4} \in \mathbb {R}^{d}\). ReLU is the rectifier linear unit activation function applied elementwise to input where ReLU(x)=x if x>0 and 0 otherwise.

being \(\sum _{u \in V} \mu _{u}^{(T)}\) is the pooled embedding of the entire graph. ψ₅,ψ₆, and ψ₇ are the neural network parameters (weights) which specified as \(\psi _{5} \in \mathbb {R}^{2d}\), \(\psi _{6} \in \mathbb {R}^{dxd}\), and \(\psi _{7} \in \mathbb {R}^{dxd}\).

To this end, we show that the pooled embedding of the entire graph is used as a surrogate to represent the state. And the embedding of each node is used as a surrogate to represent the action. The function \(\hat {Q}(h(\mathbb {S}),v)\) is depend on the collection of seven parameters \(\Psi = \{\psi _{i}\}_{i=1}^{7}\) which are learned during the training phase and will be evaluated during the evaluation phase. Figure 4 shows the architecture illustration of neural networks used in this paper.

a. Training Phase

Algorithm 1 illustrates our proposed training phase. In each training iteration, our method returns the neural network’s set of parameters Ψ which successfully get V_c from graph G. In line 5, we specify how to select a new node by balancing exploration and exploitation. In this case, the exploration means selecting a random nodes with probability ε. The exploitation means we aim to get the maximum expected cummulative rewards, i.e. by selecting a node which maximizes the function \(\hat {Q}(h(\mathbb {S}_{t}),v; \Psi)\). \(h(\mathbb {S}_{t})\) is the embedding of state \(\mathbb {S}\) at step t. The exploration probability ε is set to decrease from 1.0 to 0.05 linear to the iteration step. To efficiently train the neural network, we perform batch processing as described in line 9.

being \(y = \sum _{i=0}^{n-1} r(S_{t+i},v_{t+i}) + \lambda \max _{v}' \hat {Q}\left (h(\mathbb {S}_{t+n}), v'; \Psi \right)\). n is the number of step updates.

b. Evaluation Phase

Algorithm 2 illustrates the evaluation phase of our proposed method. To get the best-trained neural network’s set of parameters (weights) Ψ^∗, we evaluate the training result against a set of given graph G^D available snapshots. We use this neural network set of parameters in the testing simulation of the graph protection.

c. Testing Phase

Algorithm 3 shows the testing phase of multiple-turns graph protection strategy on dynamic networks. We are given an input snapshot of graph G_t and budget k_t. Each node in G_t is embedded into d-dimensional feature vector. The size of d is equal to the embedding size during training in Algorithm 1. The minimum vertex cover of G_t is then constructed using the best-trained neural network’s set of parameters Ψ^∗ resulted from Algorithm 2. Finally, we get a set S of top- k_t degree-ordered MVC nodes to be protected from the current temporal snapshot of graph G_t.

We also propose ReProtect-p method, a variant of ReProtect, which trained on the perturbed version of each available snapshot of dynamic networks. The perturbation is performed by removing edges probabilistically from the snapshot graph. Specifically, for each edge, we generate a random number. If the edge weight is smaller than the generated random number, the edge will be removed. We introduce this variant to provide more variety to the training data and avoid possible overfitting issue.

Computational complexity analysis

Based on Algorithm 3, we present the analysis of computational complexity of our proposed ReProtect method. The cost of step 1 to initialize empty set S is constant. The step 2 and 3 are to construct an approximated MVC set of graph G_t which has the complexity of O(p·M) based on the analysis by Dai et al. (2016); Khalil et al. (2017). p is the constant number of node testing steps, equals to the number of nodes divided by the number step updates in Q-Learning. M is the number of edges. In n-step Q-Learning, we update the value of each action based on the rewards of taking the sequence of n actions consecutively. n is called as the number of step updates. Suppose that the number of nodes in graph G_t is 500 and the number of step updates is 5, then p is a constant number equals to 100. One can see that p ranges from 1 to the number of nodes in graph G_t.

Getting the ordered MVC nodes in step 4 has an average O(N·logN) using QuickSort, where N is the number of nodes in G_t. Therefore, the total computational complexity of our ReProtect method is O(p·M+N·logN). The difference of ReProtect and ReProtect-p is only on training process. Similarly, we can infer that the computational complexity of ReProtect-p method is also O(p·M+N·logN).

Dataset

Comparison methods

Experimental setting

In the training phase, we use the embedding dimension size 64, batch size 64, embedding iteration 5 as suggested in Structure2Vec¹ (Dai et al. 2016). The setting of n-step is set to 5 and learning rate as 0.0001 and number of training iteration as 100,000. These three settings are commonly applied in n-step Q-Learning (Sutton and Barto 1998). In the evaluation phase, we consider the number of evaluation iteration as 100.

For a fair comparison, unless specified otherwise, all of the methods are simulated under the same setting as follows: infection rate β=0.8, recovery rate δ=0.2, and the initial number of attacked nodes (l) equals to the protection budget (k). We simulate l=k,l_t=k_t, and k₁=k₂=⋯=k_t. Random attack evaluation is employed in all experiments. The setting applies for evaluation in both SIS and SIR epidemic model.

All results presented in this section are the average of multiple simulations. Unless specified otherwise, we take the average from 100 simulations for each result. The initial condition is all nodes susceptible except the attacked ones, which are infected.

We let the epidemic spreading arrive at the stationary state before changing to the next snapshot of the network for SIS model. While for SIR model, we count the ratio of surviving nodes at the highest outbreak point, right before the final regime of epidemics as suggested by Pastor-Satorras et al. (2015). For continuity, in SIR model, we restart the epidemic spreading in the new snapshot after the final regime of epidemic spreading in the previous snapshot. Gillespie algorithm (Kiss et al. 2017) is used to simulate the epidemic spreading on networks. Additionally, we follow the time discretization method of dynamic network by Zhuang et al. (2013).

Finally, all of the experiments are performed on the same machine, Ubuntu 16.06 LTS PC with an Intel(R) Core(TM) i9-7900X CPU @ 3.30GHz CPU and NVIDIA GTX 1080 Ti SLI GPU.

Evaluation of effectiveness on synthetic network

We evaluate the performance of all comparison methods on a synthetic network generated using Dynamic Attributed Network with Community Structure Generator² (DANCER) (Largeron et al. 2017). Due to the simplicity setting of graph protection problem, we only consider the temporal network structure of the generated network and ignore their attribute and community assignment provided by DANCER. We generate a dynamic network with 100 nodes and ten temporal snapshots³.

If we vary the number of given budget k, both of our proposed methods outperform the other baseline methods as shown in Fig. 5. When the given protection budget k is too small, ReProtect and ReProtect-p exhibit competitive performances with other methods, but with an increasing k, they easily outperform other baseline methods, such as Degree, GraphShield, and Betweenness. On the other hand, ReProtect can also outperform other competing methods, even though it needs a bigger protection budget to obtain the similar performance of ReProtect-p. An introduction of more data variety using graph perturbation into training process helps our proposed method to get a better result, as in ReProtect-p.

Evaluation of effectiveness on real-world networks

To evaluate the performance comparison in different protection budget k, we vary the given k as shown in Figs. 6 and 7. Both of our proposed methods are able to outperform other competitors align with the increase given budget in all datasets, while constantly maintain competitive performance in a very small size of k. The consistency of better performance shown by our methods in many different numbers of available protection budget indicates the reliability as protection strategies.

We consider that the reinforcement learning is more suitable for graph protection on dynamic networks due to at least two major reasons. First, reinforcement learning approach using convolutional neural network as function approximator gives us a potential benefit to learn from previously solved MVC of network snapshot. By learning the given temporal structure of observed networks, this provides an incentive to predict future protection from previously learned actions in the same dynamic networks. The benefit of learning could not be obtained by traditional MVC approximation algorithms. Second, the nature of convolutional neural networks (CNN) provide us not only scalability in handling the large size networks which may contain up to billion nodes, but also easily parallelizable in multiple CPUs and GPUs. Here, we leverage our approach on top of recent advances in deep learning technology. Traditional MVC approximation algorithms are not specifically designed for this computationally expensive task.

Evaluation of effectiveness on the aggregate networks

According to the observations of Braha and Bar-Yam (2006; 2009), the snapshots static networks are quite different from the aggregate network itself. The aggregate network is the network obtained by ignoring time and aggregating all of the temporal edges in the dynamic network (Braha and Bar-Yam 2006; 2009). An interesting question arises, how does the multiple-turns time-based protection strategies analyzed in our proposed methods compare with the protection strategies when implemented on the aggregate network? In this subsection, we report the effectiveness evaluation on the aggregate networks of the same synthetic and real-world datasets.

Evaluation of scalability

Let us recall our second evaluation goal, which aims to measure how scalable is the proposed method with respect to the changing of graph size and different k budget size. In this subsection, we report the result of scalability evaluation by investigating the computational running time of our proposed methods. Different values of k were used to evaluate the scalability in different scale of protection set.

To perform simulation by changing the number of nodes, we generate synthetic dynamic networks using Dynamic Attributed Network with Community Structure Generator (DANCER) (Largeron et al. 2017). We only consider the temporal network structure of the generated network and ignore their attribute and community assignment provided by DANCER. We generate dynamic networks with 10 temporal snapshots and the number of nodes is changed from N={100;200;300;500;1000;1500;2000}. The budget size is changed from {10;20;30;40;50}.

From Fig. 8, it can be inferred that our methods scale almost linearly with respect to the number of nodes. Hence, the proposed methods are scalable with respect to the changing of graph size, which means they are applicable for large size networks. Running our methods on graph with 2000 nodes takes less than 9 seconds. Further paralelization of neural network design can also be applied to speed up the running time.

Evaluation of sensitivity to epidemic parameters

In SIS and SIR model, epidemic parameters consist of the infection rate (β) and the recovery rate (δ). To analyze the sensitivity of our proposed methods, the effectiveness comparison with different epidemic parameters are shown in this subsection.

Prakash et al. (2011) demonstrated using empirical simulations that the ratio of infection rate over recovery rate \(\left (\frac {\beta }{\delta }\right)\) takes the role as constant dependent of epidemic threshold in various epidemic model including SIS and SIR. Epidemic threshold is an intrinsic property of a network. When the strength of the virus is greater than the epidemic threshold, then the epidemic would breakout (Prakash et al. 2011). The ratio of \(\frac {\beta }{\delta }\) is commonly called as the epidemic propagation rate (Wijayanto and Murata 2018b; Prakash et al. 2011).

We perfom simulations to confirm the effectiveness of our proposed methods using the same network dataset in Effectiveness on Synthetic Network subsection under three scenarios:

(1) Comparison of survival ratio θ when the epidemic propagation rate \(\left (\frac {\beta }{\delta }\right)\) changes

(2) Comparison of survival ratio θ when the infection rate (β) changes

(3) Comparison of survival ratio θ when the recovery rate (δ) changes

For a fair analysis and comparison, simulations are performed under a fixed protection budget(k).

Comparison of survival ratio θ when the epidemic propagation rate \(\left (\frac {\beta }{\delta }\right)\) changes

We change the ratio of \(\frac {\beta }{\delta }\) from \(\left \{\frac {0.9}{0.1}; \frac {0.8}{0.2}; \frac {0.7}{0.3}; \frac {0.6}{0.4}; \frac {0.5}{0.5}; \frac {0.4}{0.6}; \frac {0.3}{0.7}; \frac {0.2}{0.8}; \frac {0.1}{0.9}\right \}\). Figure 9 shows the comparison of survival ratio θ of all methods in SIS and SIR epidemic model. The results are averaged from 100 simulations under the fixed protection budget k=0.25N, with N is the number of nodes of the input network. In all of these conditions, both of our proposed methods obtain higher survival ratio θ than other competitors.

Comparison of survival ratio θ when the infection rate ( β ) changes

We change the infection rate (β) from {0.9;0.8;0.7;0.6;0.5;0.4;0.3;0.2;0.1} with fixed recovery rate (δ). Figure 10 shows the comparison of survival ratio θ of all methods. The results are presented from the average of 100 simulations with a fixed protection budget k=0.25N, where N is the number of nodes of the input network. Both of our proposed methods could achieve highest survival ratio θ regardless the value of infection rate and epidemic models.

Comparison of survival ratio θ when the recovery rate (δ) changes

We investigate the comparison of survival ratio θ by changing the recovery rate (δ) from {0.9;0.8;0.7;0.6;0.5;0.4;0.3;0.2;0.1} with fixed infection rate (β) and the protection budget k=0.25N. N is the number of nodes of the input network. As shown in Fig. 11, in SIS and SIR epidemic model, both of ReProtect and ReProtect-p methods obtain higher survival ratio θ than other competitors. The results are averaged from 100 simulations.