Detecting problematic transactions in a consumer-to-consumer e-commerce network

Providers of online marketplaces are constantly combatting against problematic transactions, such as selling illegal items and posting fictive items, exercised by some of their users. A typical approach to detect fraud activity has been to analyze registered user profiles, user's behavior, and texts attached to individual transactions and the user. However, this traditional approach may be limited because malicious users can easily conceal their information. Given this background, network indices have been exploited for detecting frauds in various online transaction platforms. In the present study, we analyzed networks of users of an online consumer-to-consumer marketplace in which a seller and the corresponding buyer of a transaction are connected by a directed edge. We constructed egocentric networks of each of several hundreds of fraudulent users and those of a similar number of normal users. We calculated eight local network indices based on up to connectivity between the neighbors of the focal node. Based on the present descriptive analysis of these network indices, we fed twelve features that we constructed from the eight network indices to random forest classifiers with the aim of distinguishing between normal users and fraudulent users engaged in each one of the four types of problematic transactions. We found that the classifier accurately distinguished the fraudulent users from normal users and that the classification performance did not depend on the type of problematic transaction.


Introduction
In tandem with the rapid growth of online and electronic transactions and communications, fraud is expanding at a dramatic speed and penetrates our daily lives. Fraud including cybercrimes costs billions of dollars per year and threatens the security of our society [1,2]. In particular, in the recent era where online activity dominates, attacking a system is not too costly, whereas defending the system against fraud is costly [3]. The dimension of fraud is vast and ranges from credit card fraud, money laundering, computer intrusion, to plagiarism, to name a few.
Computational and statistical methods for detecting and preventing fraud have been developed and implemented for decades [4][5][6][7]. Standard practice for fraud detection is to employ statistical methods including the case of machine learning algorithms. In particular, when both fraudulent and non-fraudulent samples are available, one can construct a classifier via supervised learning [4][5][6][7]. Exemplar features to be fed to such a statistical classifier include the transaction amount, day of the week, item category, transactions of weapons, which are prohibited by the service because they may lead to crime. The number of sampled users of each type is shown in Table 1.

Network analysis
We examine a directed and weighted network of users in which a user corresponds to a node and an attempted transaction between two users represents a directed edge. The weight of the edge is equal to the number of attempted transactions by the seller with the buyer. We constructed egocentric networks of each of several hundreds of normal users and those of fraudulent users, i.e., those engaged in at least one problematic transaction. The egocentric network of either a normal or fraudulent user contained the nodes neighboring the focal user, edges between the focal user and these neighbors, and edges between the pairs of these neighbors.
We calculated eight indices for each focal node. They are local indices in the meaning that they require the information up to the connectivity among the neighbors of the focal node.
Five out of the eight indices use only the information about the connectivity of the focal node. The degree k i of node v i is the number of its neighbors. The node strength [40] (i.e., weighted degree) of node v i , denoted by s i , is the number of transactions in which v i is involved. Using these two indices, we also considered the mean number of transactions per neighbor, i.e., s i /k i , as a separate index. These three indices do not use information about the direction of edges.
The sell probability of node v i , denoted by SP i , uses the information about the direction of edges and defined as the proportion of the v i 's neighbors for which v i acts as seller. Precisely, the sell probability is given by where k in i is v i 's in-degree (i.e., the number of neighbors from whom v i attempted to buy at least one item) and k out i is v i 's out-degree (i.e., the number of neighbors to whom v i attempted to sell at least one item). It should be noted that, if v i acted as both seller and buyer towards v j , the contribution of v j to both in-and out-degree of v i is equal to one. Therefore, k in i + k out i is not equal to k i in general. The weighted version of the sell probability, denoted by WSP i , is defined as where s in i is node v i 's weighted in-degree (i.e., the number of attempted buys) and s out i is v i 's weighted out-degree (i.e., the number of attempted sells). The other three indices are based on triangles that involve the focal node. The local clustering coefficient C i quantifies the abundance of undirected and unweighted triangles around v i [41]. It is defined as the number of undirected and unweighted triangles including v i divided by k i (k i − 1)/2. The local clustering coefficient C i ranges between 0 and 1.
We hypothesized that triangles contributing to an increase in the local clustering coefficient are localized around particular neighbors of node v i . Such neighbors together with v i may form an overlapping set of triangles, which may be regarded as a community [42,43]. Therefore, our hypothesis implies that the extent to which the focal node is involved in communities should be different between normal and fraudulent users. To quantify this concept, we introduce the so-called triangle congregation, denoted by m i . It is defined as the extent to which two triangles involving v i share another node and is given by where Tr i = C i k i (k i − 1)/2 is the number of triangles involving v i . Note that m i ranges between 0 and 1. Frequencies of different directed three-node subnetworks, conventionally known as network motifs [44], may distinguish between normal and fraudulent users. In particular, among triangles composed of directed edges, we hypothesized that feedforward triangles (Fig. 1a) should be natural and that cyclic triangles (Fig. 1b) are not. We hypothesized so because a natural interpretation of a feedforward triangle is that a node with out-degree two tends to serve as seller while that with out-degree zero tends to serve as buyer and there are many such nodes that use the marketplace mostly as buyer or seller but not both. In contrast, an abundance of cyclic triangles may imply that relatively many users use the marketplace as both buyer and seller. We used the index called the cycle probability, denoted by CYP i , which is defined by where FF i and CY i are the numbers of feedforward triangles and cyclic triangles to which node v i belongs. The definition of FF i and CY i , and hence CYP i , is valid even when the triangles involving v i have bidirectional edges. In the case of Fig. 1c, for example, any of the three nodes contains one feedforward triangle and one cyclic triangle. The other three cases in which bidirectional edges are involved in triangles are shown in Figs. 1d-f. In the calculation of CYP i , we ignored the weights of edges.

Random forest classifier
To classify users into normal and fraudulent users based on their local network properties, we employed a random forest classifier [45][46][47] implemented in scikit-learn [48]. It uses an ensemble learning method that combines multiple classifiers, each of which is a decision tree, built from training data and classifies test data avoiding overfitting. We combined 300 decision-tree classifiers to construct a random forest classifier. Each decision tree is constructed on the basis of training samples that are randomly subsampled with replacement from the set of all the training samples. To compute the best split of each node in a tree, one randomly samples the candidate features from the set of all the features. The probability that a test sample is positive in a tree is estimated as follows. Consider the terminal node in the tree that a test sample eventually reaches. The fraction of positive training samples at the terminal node gives the probability that the test sample is classified as positive. One minus the positive probability gives the negative probability estimated for the same test sample. The positive or negative probability for the random forest classifier is obtained as the average of single-tree positive or negative probability over all the 300 trees. A sample is classified as positive by the random forest classifier if the positive probability is larger than 0.5, otherwise classified as negative. There were more normal users than any one type of fraudulent user. Therefore, to balance the number of the negative (i.e., normal) and positive (i.e., fraudulent) samples, we uniformly randomly subsampled the negative samples such that the number of the samples is the same between the normal and fraudulent types. Then, we split samples of each type into two sets such that 75% and 25% of the samples of each type are assigned to the training and test samples, respectively. Based on the training sample constructed in this manner, we built each of the 300 decision trees and hence a random forest classifier. Then, we examined the classification performance of the random forest classifier on the set of test samples.
The true positive rate, also called the recall, is defined as the proportion of the positive samples (i.e., fraudulent users) that the random forest classifier correctly classifies as positive. The false positive rate is defined as the proportion of the negative samples (i.e., normal users) that are incorrectly classified as positive. The precision is defined as the proportion of the truly positive samples among those that are classified as positive. The true positive rate, false positive rate, and precision range between 0 and 1.
We used the following two performance measures for the random forest classifier. To draw the receiver operating characteristic (ROC) curve for a random forest classifier, one first arranges the test samples in descending order of the estimated probability that they are positive. Then, one plots each test sample, with its false positive rate on the horizontal axis and the true positive rate on the vertical axis. By connecting the test samples in a piecewise linear manner, one obtains the ROC curve. The precision-recall (PR) curve is generated by plotting the samples in the same order in [0, 1] 2 , with the recall on the horizontal axis and the precision on the vertical axis. For an accurate binary classifier, both ROC and PR curves visit near (x, y) = (0, 1). Therefore, we quantify the performance of the classifier by the area under the curve (AUC) of each curve. The AUC ranges between 0 and 1, and a large value indicates a good performance of the random forest classifier.
To calculate the importance of each feature in the random forest classifier, we used the permutation importance [49,50]. With this method, the importance of a feature is given by the decrease in the performance of the trained classifier when the feature is randomly permuted among the test samples. A large value indicates that the feature considerably contributes to the performance of the classifier. To calculate the permutation importance, we used the AUC value of the ROC curve as the performance measure of a random forest classifier. We computed the permutation importance of each June 20, 2019 5/17 feature with ten different permutations and adopted the average over the ten permutations as the importance of the feature. We optimized the parameters of the random forest classifier by a grid search with 10-fold cross-validation on the training set. For the maximum depth of each tree (i.e., the max depth parameter in scikit-learn), we explored the integers between 3 and 10. For the number of candidate features for each split (i.e., max features), we explored the integers between 3 and 6. For the minimum number of samples required at terminal nodes (i.e., min samples leaf), we explored 1, 3, and 5. As mentioned above, the number of trees (i.e., n estimators) was set to 300. The seed number for the random number generator (i.e., random state) was set to 0. For the other hyperparameters, we used the default values in scikit-learn version 0.20.3. In the parameter optimization, we evaluated the performance of the random forest classifier with the AUC value of the ROC curve measured on a single set of training and test samples.
To avoid sampling bias, we built 100 random forest classifiers, trained each classifier, and tested its performance on a randomly drawn set of train and test samples, whose sampling scheme was described above.

Descriptive statistics
The survival probability of the degree (i.e., a fraction of nodes whose degree is larger than a specified value) is shown in Fig. 2a for each user type. Approximately 60% of the normal users have degree k i = 1, whereas the fraction of the users with k i = 1 is approximately equal to 2% or less for any type of fraudulent user (Table 1). Therefore, we expect that whether k i = 1 or k i ≥ 2 gives useful information for distinguishing between normal and fraudulent users. The degree distribution at k i ≥ 2 may provide further information useful for the classification. The survival probability of the degree distribution conditioned on k i ≥ 2 for the different types of users is shown in Fig. 2b. The figure suggests that the degree distribution is systematically different between the normal and fraudulent users. However, we consider that the difference is not as clear-cut as that in the fraction of users having k i = 1 ( Table 1).
The survival probability of the node strength (i.e., weighted degree) is shown in Fig. 2c for each user type. As in the case for the unweighted degree, we found that many normal users, but not fraudulent users, have s i = 1. In fact, the number of the normal users with s i = 1 is equal to those with k i = 1 ( Table 1), implying that all normal users with k i = 1 participated in just one transaction. In contrast, no user had s i = 1 for any type of fraudulent user. The survival probability of the node strength conditioned on s i ≥ 2 apparently does not show a clear distinction between the normal and fraudulent users (Fig. 2d, Table 1).
The distribution of the average number of transactions per edge, i.e., s i /k i , is shown in Fig. 3a. We found that a majority of normal users have s i /k i = 1. This result indicates that a large fraction of normal users is engaged in just one transaction per neighbor (Table 1). This result is consistent with the fact that approximately 60% of the normal users have k i = s i = 1. In contrast, many of any type of fraudulent users tend to have a larger value of s i /k i than the normal users. Therefore, multiple transactions with a neighbor seem to be a characteristic behavior for fraudulent users. These results did not qualitatively change when we discarded the users with s i /k i = 1 (Fig. 3b, Table 1).
The distribution of the unweighted sell probability for the different user types is shown in Fig. 4a. The distribution for the normal users is peaked around 0 and 1, indicating that a relatively large fraction of normal users is exclusive buyer or seller. Note that, by definition, the sell probability is at least 1/k i because our samples are sellers. Therefore, a peak around the sell probability of zero implies that the users probably have no or few sell transactions apart from the one sell transaction based on which the users have been sampled as seller. In contrast, the distribution for any fraudulent type is relatively flat. Figure 4b shows the relationships between the unweighted sell probability and the degree. On the dashed line in Fig. 4b, the sell probability is equal to 1/k i , indicating that the node has k out i = 1, which is the smallest possible out-degree. The users on this line were buyers in all but one transaction. Figure 4b indicates that a majority of such users are normal as opposed to fraudulent users, which is quantitatively confirmed in Table 1. We also found that most of the normal users were either on the horizontal line with the sell probability of one (43.2% of the normal users with k i ≥ 2; see Table 1 for the corresponding fractions of normal users with k i = 1) or on the dashed line (29.5%). This is not the case for any type of fraudulent user ( Table 1).
The distribution of the weighted sell probability for the different user types and the relationships between the weighted sell probability and the node strength are shown in Fig. 4c and Fig. 4d, respectively. The results are similar to the case of the unweighted  sell probability in two aspects. First, the normal users and the fraudulent users form distinct frequency distributions (Fig. 4c). Second, most of the normal users are either on the horizontal line with the weighted sell probability of one or on the dashed line with the smallest possible weighted sell probability, i.e., 1/s i (Fig. 4d, Table 1).
The survival probability of the local clustering coefficient is shown in Fig. 5a. It should be noted that, in this analysis, we confined ourselves to the users with k i ≥ 2 because C i is undefined when k i = 1. We found that the number of users with C i = 0 is not considerably different between the normal and fraudulent users (also see Table 1). Figure 5b shows the survival probability of C i conditioned on C i > 0. The normal users tend to have a larger value of C i than fraudulent users, whereas this tendency is not strong ( Table 1).
The survival probability of the triangle congregation is shown in Fig. 6a. Contrary to our hypothesis, there is no clear difference between the distribution of the normal and fraudulent users. The triangle congregation tends to be large when the node strength is small (Fig. 6b) and the local clustering coefficient is large (Fig. 6d). It depends little on the weighted sell probability (Fig. 6c). However, we did not find clear differences in the triangle congregation between the normal and fraudulent users (also see Table 1).
The survival probability of the cycle probability is shown in Fig. 7a. A large fraction of any type of users has CYP i = 0 (Table 1). When the users with CYP i = 0 are discarded, the normal users tend to have a smaller value of CYP i than any type of fraudulent users (Fig. 7b, Table 1).

Classification of users
Based on the eight indices whose descriptive statistics were analyzed in the previous section, we defined 12 features and fed them to the random forest classifier. The aim of the classifier is to distinguish between normal and fraudulent users. The first feature is binary and whether the degree k i = 1 or k i ≥ 2. The second feature is also binary and whether the node strength s i = 1 or s i ≥ 2. The third feature is s i /k i , which is a real number greater than or equal to 1. The fourth feature is binary and whether the unweighted sell probability SP i = 1 or SP i < 1. The fifth feature is binary and whether SP i = 1/k i or SP i > 1/k i . The sixth feature is SP i , which ranges between 0 and 1. The seventh feature is binary and whether the weighted sell probability WSP i = 1 or WSP i < 1. The eighth feature is binary and whether WSP i = 1/s i or WSP i > 1/s i . The ninth feature is WSP i , which ranges between 0 and 1. The tenth feature is the local clustering coefficient C i , which ranges between 0 and 1. When k i = 1, the local clustering coefficient is undefined. In this case, we set C i = −1. The eleventh feature is the triangle congregation m i , which ranges between 0 and 1. When there is no triangle or only one triangle involving v i , one cannot calculate m i . In this case, we set m i = −1. Finally, the twelfth feature is the cycle probability CYP i , which ranges between 0 and 1. When there is neither feedforward nor cyclic triangle involving v i , CYP i is undefined. In this case, we set CYP i = −1.
The ROC and PR curves when all the 12 features of users are used and the fraudulent type is fictive transactions are shown in Figs. 8a and b, respectively. Each thin line corresponds to one of the 100 classifiers. The thick lines correspond to the average of the 100 lines. The dashed lines correspond to the uniformly random classification. Figure 8 indicates that the classification performance seems to be high. Quantitatively, for this and the other types of fraudulent users, the AUC values always exceeded 0.98 ( Table 2).
The importance of each feature in the classifier is shown in Fig. 9a, separately for the different fraud types. The importance of each feature is similar across the different types of fraud. Figure 9a indicates that the average number of transactions per neighbor (i.e., s i /k i ), the sell probability (i.e., SP i ), and whether the weighted sell probability is equal to 1/s i (i.e., WSP i = 1/s i ) are the three features of the highest importance. In particular, s i /k i is by far the most dominant in importance. Given the results of the descriptive statistics in the previous section, a large value of s i /k i , a small sell probability, and WSP i = 1/s i strongly suggest that the user may be fraudulent.  Figure 9a also suggests that the features based on the triangles, i.e., C i , m i , and CYP i , are not strong contributors to the classifier's performance. Because these features are the only ones that require the information about the connectivity between pairs of neighbors of the focal node, it is practically beneficial if one can realize a similar classification performance without using these features; then only the information on the connectivity of the focal users is required. To explore this possibility, we constructed the random forest classifier using the nine out of the twelve features that do not require the connectivity between neighbors of the focal node. The mean AUC values for the ROC and PR curves are shown in Table 2. We find that, despite some reduction in the performance scores relative to the case of the classifier using all the 12 features, the AUC values with the nine features are still large, all exceeding 0.96. The permutation importance of the nine features is shown in Fig. 9b. The results are similar to those   when all the 12 features are used (Fig. 9a).
More than half of the normal users have k i = 1, and there are few fraudulent users with k i = 1 in each fraud category ( Table 1). The classification between the normal and fraudulent users may be an easy problem for this reason, leading to the large AUC values. To exclude this possibility, we carried out a classification test for the subdata in which the normal and fraudulent users with k i = 1 were excluded, leaving 412 normal users and a similar number of fraudulent users in each category (Table 1). We did not carry out subsampling because the number of the negative and positive samples were similar. Instead, we generated 100 different sets of train and test samples and built a classifier based on each set of train and test samples. The AUC values when either 10 or 7 features (i.e., the features excluding whether or not k i = 1 and whether or not s i = 1) are used are shown in Table 3. The table indicates that the AUC values are still competitively large while they are smaller than those when whether or not k i = 1 and whether or not s i = 1 are used as features (Table 2).

Discussion
We showed that a random forest classifier using network features of users distinguished different types of fraudulent users from normal users with approximately 0.98-0.99 in terms of the AUC. We only used the information about local transaction networks centered around focal users to synthesize their features. We did so because it is better in practice not to demand the information about global transaction networks due to the large number of users. It should be noted that AUC values of ≈ 0.96-0.99 was also realized when we only used the information about the connectivity of the focal user, not the connectivity between the neighbors of the focal user. This result has a practical  advantage when the present fraud-detection method is implemented online because it allows one to classify users with a smaller amount of data per user. The random forest classifier is an arbitrary choice. One can alternatively use a different linear or nonlinear classifier to pursue a higher classification performance. This is left as future work. Other future tasks include the generalizability of the present results to different types of fraudulent transactions, such as resale tickets, pornography, and stolen items, and to different platforms. In particular, if a classifier trained with test samples from fraudulent users of a particular type and normal users is effective at detecting different types of fraud, the classifier will also be potentially useful for detecting unknown types of fraudulent transactions. It is also a potentially relevant question to assess the classification performance when one pools different types of fraudulent as a single positive category to train a classifier.
Prior network-based fraud detection has employed either global or local network properties to characterize nodes. Global network properties refer to those that require the structure of the entire network for calculating a quantity for individual nodes, such as the connected component [14,17,29], betweenness centrality [14][15][16], user's suspiciousness determined by belief propagation [20,21,27,30,31,33,35,36], dense subgraphs including the case of communities [14,18,19,[22][23][24][25], and k-core [26,32]. Although many of these methods have accrued a high classification performance, they require the information about the entire network. Obtaining such data may be difficult  when the network is large or rapidly evolving over time, thus potentially compromising the computation speed, memory requirement, and the accuracy of the information on the nodes and edges. Alternatively, other methods employed local network properties such as the degree including the case of directed and/or weighted networks [14-16, 20, 23, 30, 33, 34, 37, 38], and the abundance of triangles and quadrangles [20,37]. The use of local network properties may be advantageous in industrial contexts, particularly to test sampled users, because local quantities can be rapidly calculated given a seed node. Another reason for which we focused on local properties was that we could not obtain the global network structure for computational reasons. It should be noted that, while the use of global network properties in addition to local ones may improve the classification accuracy [23], the present local method attained a similar classification performance to those based on global network properties, i.e., 0.880-0.986 in terms of the ROC AUC [14,17,20,21,35,36].
A prior study using data from the same marketplace, Mercari, aimed to distinguish between desirable non-professional frequent sellers and undesirable professional sellers [51]. The authors of Ref. [51] used information about user profiles, item descriptions, and other behavioral data such as the number of purchases per day. In contrast, we focused on local network features of the users (while a quantity similar to WSP i was used as a feature in Ref. [51]). In addition, we used specific types of fraudulent transactions, whereas the authors of Ref. [51] focused on problematic transactions as a single broad category. Combining network and non-network features may realize a better classification performance, which also warrants future work.