A psychological network analysis of the relationship among component importance measures

Abstract

Importance measures (IMs) in networks are indices that allow the analysis and evaluation of the network components that are most critical to the performance of the network. Such information is useful for a decision-maker as it enables taking actions to prevent or improve the performance of the network in the face of changing operational events (e.g., the identification of important links that should be hardened or made redundant). This paper presents an approach to analyze the relationships between the IMs through the use of so-called psychological networks, which estimate the characteristics of a new kind of network wherein the “nodes” correspond to IMs and the connecting links and their capacities are derived statistically using the IMs calculated. Such estimation does not use any a priori information of relationships among IMs. The approach proposed in this work defines an equivalence paradigm not described previously in the literature between the approach used in psychology and the IMs used to measure networks. As a result, the main characteristics of the relationships among IMs are derived, such as magnitude, sign, and robustness of the selected IMs. An example related to a transportation network and a set of flow-based IMs illustrates the contribution of psychological networks for understanding how the IMs interact.

Introduction

The network theory paradigm has been used with great success to model numerous real or natural systems, from electrical power systems, social relationships among people, and biological interactions. A network is defined by a set of nodes that interact with each other when they are connected through links, such that both nodes and links are known.

Network designers and analysts are often interested in knowing which network components, whether nodes or links, may be critical to the performance of the network. For example, one may wish to know which component, when taken out of service, causes the greatest disturbance in the network. To evaluate this effect, numerous indices or indicators have been defined to quantify the effect of the presence, absence, or degradation of a component in the network. Such indices, referred to as importance measures (IMs), often assess different aspects of the network topology such as distance, node neighborhood, node degree, flow, among others (Rocco and Barker 2022).

Most of the studies on the importance of components try to quantify these effects and rank components according to the extent of their effect on the system. It is typical in these studies that the use of various indicators can produce different hierarchies of components, as each indicator analyzes different facets of performance (e.g., one node may be especially important to average degree while may not be as important to the minimum shortest path). One way that researchers have used is to seek a consensus between the results derived from applying a set of indicators representing different perspectives. For example, Almoghathawi et al. (2017) used a multi-criteria decision analysis technique to aggregate different importance measures into a single ranking. In this way, it is possible to define a single hierarchy in which, depending on the selected aggregation method, the preferences of the decision-making entities could be considered. However, it is possible that such a consensus cannot be reached, and the decision-makers must assess and understand how different IMs may differ and/or “interact” with each other.

The main idea of this paper is not related to the assessment of the importance of the components of a network per se but rather to map out the possible interactions or dependency relationships among IMs: how the different importance measures are related, their magnitude and signs, the importance measures that could be considered as relevant, as well as the assessment of the stability and robustness of the estimations performed.

At first glance, one might consider assessing the interactions among the perspectives offered by different IMs using a correlation analysis, a valuable tool in many research scenarios. However, correlation analysis alone may not offer a complete understanding of relationships among variables for several reasons: (i) there could exist complex relationships, which may not be uncovered by the linear relationship between variables assumed with correlation, (ii) our approach considers multivariate relationships, where correlation analysis only captures pairwise associations, missing broader connections, and (iii) correlation analysis can suffer from omitted variable bias, as excluding key variables can result in misleading correlations.

For these reasons, we propose a different approach based on the use of psychological networks, or a network analysis of multivariate data supported by network theory that has recently been used in psychology studies (e.g., Punzi et al. 2022, Feng et al. 2022, Solmi et al. 2020, Borsboom 2017). In this approach, researchers define for a specific psychological diagnosis (e.g., an eating disorder) a set of appropriate symptoms to measure how likely a person is to possess that affliction. To this end, a questionnaire is defined with a series of questions aimed at measuring different aspects that are believed to be related to the disorder that is being evaluated. These evaluations are generally based on the use of a Likert-type ordinal scale. For example, question 25 of the Eating Disorder Examination Questionnaire (EDE-Q 6.0) (Fairburn and Beglin 1994) asks “In the past four weeks, how dissatisfied have you been with your weight?”. The possible answers range from “not at all” to “markedly” on a 7-point Likert scale (Allen and Seaman 2007).

This questionnaire is delivered to a sample, and responses are analyzed to determine which questions or symptoms define the nodes of a psychological network (PN) and, with statistical techniques, to determine any relationships between the symptoms, including their sign and magnitude. In this way, an undirected weighted network of nodes and links is created and subsequently analyzed following the guidelines derived from network theory. For example, the symptoms (nodes of the PN) that are the most important or links that have the greatest absolute magnitude are evaluated. Such a problem is different from a classical network assessment. Indeed, in the classical assessment, nodes and links (as well as their weights) are known. On the contrary, in the PN, only the set of nodes is known whereas the set of edges must be estimated using statistical procedures (Epskamp et al. 2018). In other words, the characteristics of the PN derived do not consider any a priori information on the structure of network (e.g., the relationships among IMs) as other approaches require (e.g., statistical latent variable models). Indeed, the latter assumes that the covariation between measures occurs due to an underlying unobservable common cause (e.g., Borsboom et al. 2018), while PNs assume that variables causally influence each other, not because they share a latent common cause (Epskamp et al. 2018).

The approach proposed in this work seeks to determine the relationship between the different measures of importance that apply to a given system, modeled as a network $G$, and how the measures are related to each other. For this purpose, each IM is considered as a symptom, and therefore a node of the PN that is sought to be generated: each IM evaluates a particular aspect of the components of the network $G$. Given a set of IMs, each component in network $G$ is evaluated using each measure. This is equivalent to considering each component as being an individual who answers a questionnaire, in which the questions evaluate the selected measures of importance.

As such, the similarity between the measures of importance and the symptoms is established. The statistical processing of the questionnaire will allow the generation of the PN interactions between the measures of importance, for the selected $G$ network and its subsequent analysis. To our best knowledge, this is the first time that the statistical analysis of a set of indices for assessing the importance of components is performed using the PN paradigm.

To illustrate the proposed approach, we show the assessment of the flow-importance measures proposed by Nicholson et al. (2016) to derive the ranking distribution of the importance of the links of a selected network. These measures consider the topology of the network as well as the capacity that characterizes each link. An example based on three operating conditions of the Colombian transportation network (Rocco et al. 2022) is used to illustrate the approach.

The importance and applications of the work presented are reflected in the results obtained. Indeed, they reveal the most important relationships among IMs, demonstrate the presence of negative effects, and suggest to which IMs to devote attention to improve network performance in light of varying operating conditions. In addition, the comparison of the three estimated PNs allows concluding that there are no differences, which could be interpreted as the importance of the components are not affected by the operating conditions being analyzed.

In Sect. "Proposed method" some important concepts are presented, and the proposed approach is described. In Sect. "Illustrative example: colombian transportation network", an example, based on the Colombian transportation network, illustrates the approach. Concluding remarks and areas of future work are discussed in Sect. "Conclusions and future work".

Proposed method

Consider a system under study described by a weighted graph ${G}_{W}=\left(V,E,W\right)$, where $V$ is a set of nodes, $E$ is the set of links connecting the nodes, and $W$ is a set real numbers that represent the capacity of the links in the network. Here, capacity refers to the link parameter that indicates the maximum amount of a good or commodity that can be transferred between two nodes of the network, such as the number of finished products or generated electric power. Capacity is a characteristic that is defined during the planning of the network.

Nodes are indexed with $i=1,\dots ,n;n=|V|$. Links and weights are indexed with $k=1,\dots ,m;m=|E|$. The index of the IMs with which to evaluate the importance of nodes is $l=1,\dots ,nim$, where $nim$ is the number of IMs considered. Matrix $EIM$ defines an $m\times nim$ matrix of importance measures such that ${EIM}_{k,l}$ is the evaluation of $I{M}_{l}$ for link $k$. Note that the information compiled corresponds to cross-sectional data (i.e., time is not considered).

Psychological network analysis of multivariate data

Mentioned previously, $EIM$ defines a matrix of IMs such that ${EIM}_{k,l}$ is the evaluation of $I{M}_{l}$ for link $k$. Such a matrix will be considered as basic information: each row represents an individual (i.e., a node or a link, as in the example to be discussed) and each column represents the assessment of each importance measure. The assessment could be presented as real numeric values or by the corresponding rankings. In the latter case, values in each column of $EIM$ range from 1 to $m$. To mimic a psychological questionnaire, matrix $EIM$ could be converted to a questionnaire, by defining a Likert scale of, say, five points ranging from 1 = Very Important to 5 = Not Important, though any scale could be selected. Using a Likert scale can be driven by factors like the data's format (e.g., data in physical units) or its confidential nature. In such situations, an analyst might select to reformat the data to make it more akin to a questionnaire for easier interpretation. In Sect. "Illustrative example: colombian transportation network", for example, we represent the ranking of links in the Colombian network as a Likert scale, offering a finely detailed level of granularity. After this transformation, $EIMT$ will be the “questionnaire” (to use the PN analysis terminology). If the analyst does not make any transformation, then $EIMT=EIM$.

The psychological network analysis methodology is based on four steps: (i) build the psychological network (i.e., the statistical estimation of the PN structure), (ii) describe the characteristics of the PN, (iii) evaluate the stability or robustness of the PN, and (iv) if required, compare different PNs.

In this way, the network analysis of multivariate data is a technique that mixes multivariate statistics and network assessment procedures to derive the structure of relationships in multivariate data (Borsboom et al. 2021). As a result, we build the structure of the PN as a weighted graph $P=\left({V}_{p},{E}_{p},{W}_{p}\right)$, where ${V}_{p}$ denotes a set of nodes, ${E}_{p}$ is the set of links connecting nodes, and ${W}_{p}$ is a set of weights, real numbers that represent, for example, the partial correlation between nodes. The set of links ${E}_{p}$ and the corresponding set of weights ${W}_{p}$ will be estimated from $EIMT$. Nodes are indexed with $i=1,\dots ,nim$, links and weights are indexed with $k=1,\dots ,{m}_{p};{m}_{p}=|{E}_{p}|$, and $|{E}_{p}|$ is the number of links of the PN to be determined.

Network estimation

The network estimation procedure used in the psychology literature is based on the theory of a pairwise Markov random field (PMRF) (Costantini et al. 2015, van Borkulo et al. 2014). Under this approach, a network has known nodes and the relationship between nodes (to be estimated) is quantified with “partial correlation coefficients between two nodes after conditioning on all other nodes in the dataset” (Epskamp et al. 2018). For example, in a network with three nodes $a$, $b$, and $c$, the estimation process could detect the existence of two links $\left(a,b\right)$ and $\left(b,c\right)$ with weights ${w}_{ab}$ and ${w}_{bc}$, positive and negative, respectively. This suggests that node $a$ has a positive interaction with node $b$ while controlling $c$, while nodes $b$ and $c$ have a negative interaction while controlling $a$. The interactions ${w}_{ab}$ and ${w}_{bc}$ cannot be described by the rest of nodes in the network (i.e., node $c$ and node $a$, respectively). Additionally, nodes $a$ and $c$ are “conditionally independent given node $b$” (Epskamp et al. 2018) (i.e., there exist no partial correlations between $a$ and $c$ while controlling $b$ (da Cunha Leme et al 2020)). Note that in the estimated network, based on cross-sectional data, there is no implication of directionality, thus the generated network is an undirected weighted network.

An interesting detail in this process is the use of partial correlation coefficients as weights associated with links. In fact, many authors consider that the direct use of correlations as weights “do not correct for linear relations that might be due to other variables” (e.g., Lafit et al. 2019). Partial correlations are calculated by inverting a proper matrix of correlations (e.g., Pearson, polychoric, or polyserial correlation, depending on the characteristic of the variables selected (Johal and Rhemtulla 2021)).

It is important to mention that the network estimation requires the determination of $\frac{1}{2}\left[{m}_{p}\times \left({m}_{p}-1\right)\right]$ weights and ${m}_{p}$ threshold parameters. This means that, for example, in a network with 15 nodes, there are 120 parameters to be estimated from the data.

Since the quality of the connectivity and weight estimates depends on the size of the data set, to derive non-zero weights (i.e., to mitigate the existence of false positives), a regularization technique is usually selected during the estimation process. Such a regularization technique is the graphical least absolute shrinkage and selection operator (GLASSO) approach (Friedman et al. 2008, 2014) that includes a penalty parameter λ that converts small link weights to zero. In this way, the regularization by the GLASSO approach produces a parsimonious (or sparse) and more interpretable network (Miers et al. 2020).

In general, the optimal selection of λ could be based on the minimization of the Extended Bayesian Information Criterion (EBIC) (Chen and Chen 2008). EBIC includes a tuning parameter to cope with the complexity of the model (e.g., selecting 0.5 produces a conservative approach (Solmi et al. 2020; Epskamp et al. 2018; Fried et al. 2016)).

The network estimation process ends with the visualization of the estimated network. To this aim, the Fruchterman-Reingold algorithm (Fruchterman and Reingold 1991) is selected as it tends to group nodes with high link weights. We used the software R for statistical computing (version 4.2.1, open source, available at https://www.r-project.org/). The network was estimated and visualized using the R-package qgraph (Epskamp et al. 2012).

To avoid the inclusion of two nodes with similar correlation patterns, some authors (e.g., Jones 2020) suggest using the “goldbricker function” to assess this situation. This procedure (implemented in the ‘networktools’ package, Version 1.5.0) uses different methods for deciding “if correlation pairs are significantly different” (Jones 2020), given a selected p-value threshold.

Network characteristics

An interesting property associated with a network is the ability to quantify the centrality of its nodes. Indeed, centrality measures can assess the importance of the nodes for determining the dynamics and the structure of the network (da Cunha Leme et al. 2020). For example, the betweenness of a node is a centrality index that measures how many times the node belongs to the all-pairs shortest paths in the network.

One of the measures that has recently received attention is the expected influence, defined as “the summed weight of its edges shared with the remaining nodes in the network” (Robinaugh et al. 2016, Yuan et al. 2022). The measure is especially useful in estimated networks with positive and negative partial correlations (Robinaugh et al. 2016, Yuan et al. 2022). The expected coefficient of influence indicates whether a node activates or deactivates other nodes of the network, depending on the sign of the weights. As such, nodes with higher expected influence are considered to be more important nodes.

Another common centrality measure often presented is the strength centrality, or “the sum of the absolute value of all edges linking to a given node” (Robinaugh et al. 2016). It is clear that for a node with exclusively positive edges, the expected influence centrality is equal to its strength centrality (Robinaugh et al. 2016). All of the centrality measures in this work were estimated by using R-package qgraph (Epskamp et al. 2012).

Stability or robustness of the PN

Mentioned previously, the estimation of the PN characteristics as well the link weights are based on a sample. This means that, to properly understand the link weights and the centrality measures, we must assess the accuracy and stability of the estimated network (Epskamp et al. 2017). To this aim, Epskamp et al. (2017) proposed different techniques and resampling procedures (based on the bootstrap technique (Efron 1979)) that can be applied to evaluate the precision of the estimated network. One such procedure implemented in the R-package bootnet (Epskamp 2018, Borsboom et al. 2018) is able to derive a pseudo-95% confidence interval for link weights. Since the GLASSO procedure is used as the estimation tool, the information obtained is basically used to evaluate the accuracy of the estimations and is not intended as a zero test (i.e., as a procedure to detect if a weight is zero because the confidence interval contains zero). However, the authors also propose a bootstrapped difference test to compare if pairs of link weights vary significantly from one another (Wei et al. 2021).

In addition, a procedure to quantify the stability of the estimation is also proposed. This procedure removes an increasing number of data (subsample) and determines the correlation coefficient between the original estimation and the estimation obtained with the subsample data. The correlation stability (CS) coefficient quantifies the maximum proportion of cases that can be eliminated to retain, with 95% certainty, a correlation with the original centrality greater than 0.7 (da Cunha Leme et al. 2020). To interpret the differences in centrality, Epskamp et al. (2017) indicate, as a guide, that the CS coefficients should not be less than 0.25 and preferably greater than 0.5 (Solmi et al. 2020).

Network comparison

A decision-maker may be interested in comparing the structure of two or more fitted PNs generated by two or more $EIMT$ data. Mentioned previously, no a priori information regarding the relationships among IMs is used. As such, the structure of the PN is determined only from the data set under analysis. Network comparison is then an interesting tool to evaluate the resulting PNs. For example, the analyst could: (i) assess the importance of the components for a single network under two different set of weights (e.g., different operating conditions) modeled as ${W}_{1}$ and ${W}_{2}$ (i.e., networks ${G}_{W1}=\left(V,E,{W}_{1}\right)$ and ${G}_{W2}=\left(V,E,{W}_{2}\right)$), (ii) compare the effects of modifications to the set of components (e.g., different sets of links ${G}_{Wa}=\left(V,{E}_{a},{W}_{a}\right)$ and ${G}_{Wb}=\left(V,{E}_{b},{W}_{b}\right)$, or (iii) assess different networks ${G}_{Wa}=\left({V}_{a},{E}_{a},{W}_{a}\right)$ and ${G}_{Wb}=\left({V}_{b},{E}_{b},{W}_{b}\right)$. In all of these cases, a proper set of PNs is generated. As such, comparing two or more PN structures means to statistically decide: (i) if the overall structure of the fitted PNs can be considered different or not, (ii) if the presence of links in each PN is equal, and (iii) which links (i.e., weights) can be considered to be different (Jefferies et al. 2022). In our context, the comparison among PNs could suggest, for example, that the operating conditions analyzed affects the importance of the components according to the centrality measures described in 2.2.3, or that the modifications to the set of links do not cause any difference, or that the set of IMs is heterogeneous in the sense that they do not quantify the networks in the same way.

To this aim, three tests have been proposed to evaluate the differences in networks [van Borkulo et al. 2022]: (i) the global structure test M, which quantifies the differences in the distribution of the link weights, (ii) the edge weights test, which evaluates differences for selected links and is performed if the previous test M gives a significant result, and (iii) the global network structure test S, which considers the absolute sum of all the edges between groups. In this paper, network test comparisons were performed using the R-package NetworkComparisonTest (van Borkulo et al. 2016).

PN assessment process

Figure 1 shows a typical process used to perform the required evaluations to develop a PN. Some procedures are optional and depend on the type of evaluation being performed. Here we assume (blue section) that the network to be analyzed and the set of IMs have been previously selected, and the corresponding importance assessment is already performed such that matrix $EIM$ is available. If required, matrix $EIM$ is transformed to $EIMT$ (e.g., by defining a proper Likert scale) or otherwise $EIMT=EIM$. From here, steps 2.1.2–2.1.4 (red section) are performed. If network comparisons are required, the process defined in Fig. 1 is repeated for each network and step 2.1.5 is performed.

Fig. 1

PN assessment process

Illustrative example: Colombian transportation network

In this section we present the results of the analysis of multivariate data using the approach described in 2.2. The R packages used in the evaluation were executed with the default settings, unless indicated. The assessments performed are in line with the guidelines suggested by Burger et al. (2022).

Figure 2 shows the topology of the Colombian transportation network with 51 nodes and 57 links. Table 1 shows the from-to and base link capacities. This network has been assessed by Rocco et al. (2022) to quantify the rank robustness of the importance of the links under different link capacity scenarios. To this aim, the authors defined a set of possible random scenarios of link capacities, quantified the set of importance measures defined for each link, and used a fuzzy multicriteria analysis to rank the importance of the set of links.

Fig. 2

The Colombian transportation network

Table 1 From-to and base link capacities

As a first example of the proposed procedure, we generate three networks (${G}_{1}$, ${G}_{2}$, ${G}_{3}$) by randomly varying the base capacity of the links in the interval [70, 100)%, [95, 100)%, [80, 95)%, and [70, 80)% (uniform distributions). Such variations could correspond to planned or unplanned disruptions during the normal operation of a network, such as maintenance, weather events, random failures, or even intentional actions (e.g., riots). Under these circumstances, the link capacities could be affected.

Link importance measures

While there are many importance measures in the literature (e.g., centrality measures (Newman 2010, Wu 2011, Saxena and Jadeja 2022)), here we consider a set of measures proposed by Nicholson et al. (2016) that consider how the max-flow in a network is affected by the presence of each link: All Pairs Max Flow Count (MFC), Min Cutset Count (MCC), Edge Flow Centrality (EFC), Flow Capacity Rate (FCR), Weighted Flow Capacity Rate (WFCR), and One-at-a-Time Damage Impact (DI).

The most important links derived by this evaluation could suggest to a decision maker how those links “can be reinforced, protected prior to any disruption, or expedited during the recovery stage” (Nicholson et al. 2016). For example, the IM defined as “all pairs max flow edge count (MFC)” assesses “the utilization of a given edge in all $s-t$ pairs max flow problems.” That means that if a link participates more than others in the “all $s-t$ pairs max flow” problems, then it could produce a substantial effect on network performance when such link is disrupted. Table 2 shows the set of importance measures proposed by Nicholson et al. (2016).

Table 2 Set of selected component importance measures (Nicholson et al. 2016, Almoghathawi et al. 2017)

Psychological network analysis application

For each network, we rank the links from the most important to the least important using the set of six IMs from Nicholson et al. (2016). Note that each of the three networks has the same layout of nodes and links but with randomly generated link capacities. This could be interpreted as different operating conditions for the network. Therefore, the IMs also assess different network characteristics and assist in planning for operators.

To illustrate, Table 3 shows the matrix ${EIM}_{1}$ with the ranks of the links for the first network ${G}_{1}$. For example, according to the MFC importance measure, link 28 is ranked in the first position (bolded in Table 3), followed by link 27, and so on. Note that the same link 28 is ranked in the 49th position according to MCC, thus illustrating that different importance measures could rank the components in a very different way.

Table 3 Matrix $EIM_{1}$ with the ranks for network $G_{1}$

Rather than convert these rankings into a Likert scale with, say, five or seven points, we instead opted to maintain the original granularity of the rankings and use$EIMT=EIM$. The corresponding Cronbach’s alpha indices for each data set ${EIMT}_{l},l=\text{1,2},3$, are 0.860, 0.862, 0.854 respectively, which indicate that response values are consistent.

Using the procedure described in 2.1.2, we estimated the PNs ${P}_{l}, l=\text{1,2},3$, using as input data the corresponding matrices ${EIMT}_{l},l=\text{1,2},3$. Figure 3 depicts the three estimated networks. We selected the averageLayout function of the qgraph package to organize the nodes in a unique layout for an easy visual comparison. Links in green represent positive partial correlation while red links correspond to negative partial correlations. The widths of the links are proportional to their partial correlations. At first glance, the three estimated PNs seem qualitatively equals.

Fig. 3

The three PNs estimated from different capacities. Links in green represent positive partial correlation while red links correspond to negative partial correlations with line widths proportional to their partial correlations

Note that, as consequence of the GLASSO procedure, many links are not shown since their weights are fixed to zero. Some links are shown in red, indicating the presence of negative partial correlations. This fact could be explained as if the nodes involved act as negative effects or more likely as a spurious relationship due to the sample size used. Table 4 shows the weights derived for each network. However, note that the highest absolute negative partial correlation is less than 0.145. Note that the network estimation is based in 6 $\times$ 5/2 + 6 = 21 possible parameters.

Table 4 Weights derived for each of the three PNs

A fast evaluation of the similarity among networks is based on the sum of the absolute values of the weight matrices (Van Borkulo et al. 2016). In our case, the sums are 6.01, 5.85 and 5.84, respectively, suggesting very similar values.

Using the procedure described in 2.2.5, we compare the three PNs. Since the procedure can compare only pairs of estimated networks, three comparisons are performed (the paired = TRUE option in the NetworkComparisonTest package was used since the input data refer to the same set of links). The pairwise comparison of the three PNs estimated are shown in Table 5. The results of the network invariance and global strength invariance tests suggest that the three networks are not different from each other and are “equals.” Note that the fact that the networks are statistically equal means that the analysis of the significantly different links is not required. In other situations, such analysis would reveal the presence or absence of common links as well as the magnitude differences of the relationships.

Table 5 Pairwise comparison for the three PNs, test statistics and p-values for the network invariance and global strength invariance tests

In our context, from a practical point of view, this result means that the importance of the elements is not affected when the network is exposed to these three different operating conditions. Of course, a larger sample would be necessary to make a general conclusion. But we consider that the analysis presented here is sufficient for illustrating how a network comparison in PNs could be performed.

The previous results suggest the analysis of an additional case where the information of the three networks analyzed is combined as a new data set of observations (Jefferies et al. 2022): the three matrices ${EIMT}_{l},l=\text{1,2},3$ are combined to build the global matrix ${EIMT}_{global}$. This new matrix ${EIMT}_{global}$ has a Cronbach’s alpha value of 0.858, (again indicating that response values are consistent. The results of the goldbricker procedure (package nettools, using the option method = "zou2007") suggest no further reduction of the data and that the matrix can be used as-it-is for estimating the global network.

The corresponding network is shown in Fig. 4 while the associated weights of the links are presented in Table 6. The sum of the absolute values of the weight matrix is 5.90, very similar to the values obtained for the single networks ${P}_{1},{P}_{2},{P}_{3}$.

Fig. 4

The global network estimated. Links in green represent positive partial correlation while red links correspond to negative partial correlations with line widths proportional to their partial correlations

Table 6 Weights associated to the links of the global network estimation

A notable characteristic of this network is that two-third of the links (10 out of 15) were not zero, and almost all of these links were positive (only two partial correlations are negative). The three strongest interrelationships between nodes (i.e., say weights > 0.45) in the final network are: MCC-DI (0.61), FCR-WFCR (0.53) and MFC-EFC (0.45).

It is interesting to note that the fit produces two negative weights, but their absolute values are less than 0.14. That the weights for links MFC-DI and MCC-EFC are negative could suggest that their negative contribution is due to, even if MFC, MCC, DI, and EFC are importance measures, they assess different importance aspects of the components of the network. But at this stage, it is not possible to make any strong conclusions.

The comparison of the single networks ${P}_{1}, {P}_{2},{P}_{3}$ and the global network reveals that the networks cannot be considered different. The network invariance tests M produce p-values of 0.998, 1.000, and 0.995, respectively, while the global strength invariance tests S result in p-values of 0.640, 0.831, and 0.776.

Figure 5 shows the bootstrapped 95% confidence interval of link weights as a qualitative way of measuring the accuracy of the weight estimations (using nBoots = 2000 in the bootnet package). Mentioned previously, the derived confidence intervals based on GLASSO cannot be used as a test to verify non-zero estimation. Figure 5 shows that for each estimated link there are: (i) a black dot that reflects the means derived by the bootstrap procedure, (ii) a red line that corresponds to the original estimation, and (iii) the gray area corresponding to the confidence interval estimation.

Fig. 5

Accuracy of link weights

Note that almost all of the confidence intervals are not large, suggesting, qualitatively, a good accuracy of the estimation of the link weights and a clear interpretation of the link weights ranking. In general, large confidence intervals would require a careful interpretation of the importance of the link weights. An additional important characteristic in Fig. 5 is the absence of confidence intervals in those links that are forced by the GLASSO procedure to be zero. Finally, the ranges of the negative estimations are also narrow.

To test the difference among the weight estimations, a different bootstrap procedure is used (stability difference test: alpha = 0.05). Figure 6 shows the results of this test, only for non-zero estimations. Each cell corresponds to a pair of estimated links, and there are three types of cells. The cells on the diagonal show the estimation weights. Black boxes correspond to weight estimations that are significantly different, while gray boxes suggest there are no significant differences in the estimates.

Fig. 6

Bootstrapped difference test for link weights

For example, Fig. 6 shows that the weight associated to the interaction MCC-DI differs significantly from the other values except for FCR-WFCR. Positive weights for MFC-EFC and MFC-WFCR do not indicate any significant differences as well as the negative weights for MFC-DI and MCC-EFC.

Figure 7 shows the importance of the nodes according to three well-known centrality measures: strength, betweenness, and expected influence. Since there is a tendency to consider the expected influence as the more convenient measure, we will refer only to this measure. Figure 7 shows that nodes WFCR and FCR have the highest value, meaning that these two importance measures show the strongest association with other nodes in the network. From a theoretical point of view, actions for controlling these two nodes would produce more effective actions to reduce the importance of the links in the network $G$.

Fig. 7

Importance of the IMs according to strength, betweenness, and expected influence

To clearly pinpoint the differences among the values of the expected influence of the nodes, Fig. 8 shows the bootstrapped difference test. Figure 8 should be interpreted in a similar manner as Fig. 6, relative to bootstrapped difference test for link weights. In this case, the expected influences of WFCR and FCR are different (except FCR vs MCC) while the rest of the cells in gray do not indicate any significant differences.

Fig. 8

Bootstrapped difference test for expected influence of IMs

Finally, Fig. 9 shows the stability of nodes for expected influence and strength. The red (blue) line indicates the average correlation between the node expected influences (respectively node strengths) in the full sample and subsample. The areas correspond to intervals between percentiles 97.5 and 2.5. CS-coefficients are 0.749 for expected influence and 0.673 for strength, indicating adequately stable CS-coefficients.

Fig. 9

Stability of IMs for expected influence and strength

Conclusions and future work

In this paper we study the relationships that may exist between a set of importance measures that quantify which components most affect the performance of a network. To this aim, we adapt the concept of psychological network analysis, a methodology based on the symptoms of psychological disorders. Under this construct, IMs are considered to be “symptoms,” and each component of the network is regarded as an individual whose symptoms are investigated.

To our knowledge, this novel approach has not been applied in the field of network IMs. Indeed, the equivalence between symptoms and IMs allows to statistically estimate a PN, able to show how the different importance measures are related, their magnitude and signs and the importance measures that could be considered as relevant. Additionally, the stability and robustness of the estimation is determined. The characteristics of the PN derived do not consider any a priori information on the structure of network (e.g., the relationships among IMs).

We apply the procedure to a transportation network using a selected set of flow-based IMs for links and the corresponding ranking of the links, as a Likert scale with a highly detailed level of granularity. We analyzed three network scenarios corresponding to the different operating conditions, as well as combining the three scenarios into a unique scenario. The results highlight the most important relationships, demonstrate the presence of negative effects, and suggest which IMs to pay attention to improve network performance in light of varying operating conditions. In addition, the comparison of the PNs allows concluding that there are no differences in the PNs, which could be interpreted as the importance of the components are not affected by the operating conditions being analyzed. It is important to mention that our work considers only one network with three random different operating conditions and a selected set of flow-related importance measures for links.

We believe that the additional information derived using PNs enables a better understanding of the IMs and their relationships. In fact, the network analysis relies on building a model from the numerical assessment of the IMs of the components (i.e., completely driven by the data). Since the analysis is not based on any a priori relationship, such assessment may uncover unsuspected patterns, effects, or conditions. In our practical example, we noticed that some relationships among IMs are non-existent, others have higher weights, some are inhibiting or reinforcing, and some IMs are more important than others. These numerical conclusions were unforeseen.

Future work will consider three main aspects. First, we will evaluate the effect of (i) a larger sample of operating conditions and draw stronger conclusions regarding the invariance (or lack thereof) of the importance of components of a network, and (ii) converting the information from the IMs to one or more Likert scale of different size. Second, we will assess how the relationships derived for the same set of importance measures on different networks are related, including the effects of component modifications to the topology of a previously considered network (e.g., layout of links). As evidenced in the psychological area, the determination of the PNs for different input groups (i.e., different networks) does not necessarily produce the same results. This suggests that it is possible that the procedure described in this work (applied to different networks) produces different PNs indicating that the interrelationships between the measures of importance are statistically different. This ultimately cannot be considered a disadvantage of the approach since it would highlight that the behavior of the IMs selected, when considering different networks and possibly operational aspects, is not homogenous and definitely produce different results. To this aim, a variability network analysis (Fried et al. 2018, Holtge et al. 2020) could be used to highlight the similarities or the differences. The third aspect that deserves consideration is the simultaneous assessment of different sets of importance measures (e.g., flow-based and graph theoretic-based importance measures). This analysis could highlight which importance measures act as bridge or connection among the two sets and “therefore represent potential points of effective intervention” (Burger et al. 2022).