Estimation of flow trajectories in a multi-lines transportation network

Abstract

Characterizing a public transportation network, such as an urban network with multiple lines, requires the origin–destination trip counts during a given period. Yet, if automatic counting makes the embarkment (boarding) and disembarkment (alighting) counts in vehicles known, it often happens that pedestrian transfers between lines are harder to track, and require costly and invasive devices (e.g., facial recognition system) to be estimated. In this contribution, we propose a method, based on maximum entropy and involving an iterative fitting procedure, which estimates the passenger flow between origins and destinations solely based on embarkment and disembarkment counts. Moreover, this method is flexible enough to provide an adaptable framework in case additional data is known, such as attraction poles between certain nodes in the network, or percentages of transferring passengers between some lines. This method is tested on toy examples, as well as with the data of the public transportation network of the city of Lausanne provided by its Transportation Agency (tl), and gives arguably convincing estimations of the transportation flow.

Introduction

Transportation networks determine our mobility, require a considerable amount of planning and resources, and elicit much public hopes and critics. They also constitute an endless source of inspiration in formal modeling and optimization, as attested in operations research (classical optimal transportation, maximum flow problem), quantitative geography and spatial econometrics (spatial navigation, multimodality, gravity models for flows), and machine learning (recent developments in regularized optimal transportation, such as color transfer or images interpolation; see e.g. Peyré and Cuturi 2019).

This contribution addresses a straightforward, yet central question in public transportation networks: given a network made of many train, bus, subway, or tram lines, how can one estimate the real trips made by the travelers, on the sole basis of the embarkment (boarding) counts and disembarkment (alighting) counts in each vehicle? Although estimating origin–destination flows is a much addressed issue in transportation modeling (see e.g Bell and Lida 1997; Hazelton 2000; Ashok and Ben-Akiva 2002; Cui 2006 and references therein), the specific problem addressed in this contribution seems, to the best of our knowledge, original.

Pedestrian transfers of travelers between different lines here constitute the missing information, which planners traditionally attempt to estimate using census, or more recently with costly and invasive devices located at station, such as mobile phone tracking or facial recognition systems. In this article, we take a different approach, which is to produce the optimal estimation of transportation trajectories without additional data, using the principle of maximum entropy. The proposed solution consists of three consecutive steps: a maximum-entropy computation of the trip distributions, obeying marginal constraints and with a given prior; an update of the prior distribution by shrinking the components responsible for transfer overflow; and an update of marginal distributions. This iterative procedure clearly evokes the EM algorithm (see e.g. Dempster et al. 1977; Bavaud 2009), with the first step corresponding to the “expectation step” and last two steps to the “maximization step”. The first step only is required for solving the single line case, naturally much simpler but yet not trivial, and exhibiting a disembarking probability independent of the embarking stop (Markov property). This method also offers some flexibility, as it is possible to set a prior distribution taking into account attraction or repulsion poles among stops, and to fix hyperparameters limiting the number of transfers between lines.

As case studies, we test the proposed method on toy examples as well as with the data of the public transportation network¹ of the city of Lausanne, in Switzerland. Toy examples offer some kind of validation, as a transportation flow can be entirely set on toy networks and compared to the solution given by our algorithm. The real case scenario with the data from the Lausanne public transportation network demonstrates that this method is applicable on a real transportation network and can yield pertinent insights about traveler habits.

“ Notations and formalism” Section introduces the notations and the formalism, as well as the statement of the problem and the proposed solution. “Case Studies” Section  contains case studies with toy examples and the Lausanne public transportation network.  “Conclusion” Section concludes the article. Data, codes and extensive results can be found in the GitHub repository of the article.²

Notations and formalism

Lines, stops and junctions

Consider a transportation network made of lines numbered $\ell =1,\ldots , q$, of respective lengths (number of stops) $l_\ell$. Opposite lines, that is parallel lines running in the back and forth directions are considered as distinct.

The $l=\sum _{\ell =1}^ql_\ell$ stops constitute the nodes of the transportation network. Each stop $i=1,\ldots ,l$ belongs to a single line, and defines a unique next or forward stop F(i) (unless i is the line terminus) and a unique backward stop B(i) (unless i is the line start), both on the same line.

Let $S_i$ denotes the set of stops which can be reached from stop i outside lines connection (with, e.g., an acceptable walking distance), excluding i itself. A stop i is referred to as an isolated stop if $S_i=\emptyset$, and to as a junction otherwise.

Edges, trips, and the incidence matrix

Two sorts of oriented edges are involved in the transportation network:

• Intra-line edges $(i,j)=(i,F(i))$ belonging to a single line $\ell (i)=\ell (j)$

• Inter-line or transfer edges (i, j) connecting different lines $\ell (i)\ne \ell (j)$, involving walks from junction i to $j\in S_i$. The set of transfer edges is denoted by T.

A st-trip, noted [s, t], consists of entering into the network at stop s, and leaving the network at t, by following the shortest-path (i.e. achieving the minimum distance, minimum time, or minimum cost), supposed unique, leading from s to t.

The succession of edges (i, j) belonging to the st-trip, noted $(i,j)\in [s,t]$, is unique. Define the edge-trip incidence matrix as

$$\begin{aligned} \chi _{ij}^{st} = {\left\{ \begin{array}{ll} 1 &{} \text {if} (ij)\in [s,t], \\ 0 &{} \text {otherwise}. \end{array}\right. } \end{aligned}$$

(1)

Note that we can also forbid some aberrant trips across the network, for example, trips [s, t] where (s, t) is a transfer edge (travelers making this trip do not actually do not use the transportation network). The set of permitted trips across the network is denoted by P, and can be defined following some conditions (see, e.g., “Construction”).

Transportation flows

Let $x_{ij}$ count the number of travelers using edge (i, j) in a given period, such as a given hour, day, week or year. The edge flow $x_{ij}$ is denoted by $y_{ij}$ for an intra-line edge (i, j), and by $z_{ij}$ for a transfer edge (i, j). By construction, $x_{ij}=y_{ij}+z_{ij}$, where $y_{ij}\, z_{ij}=0$.

Let $a_i$, respectively $b_i$, the number of passengers embarking, respectively disembarking at stop i. By construction,

$$\begin{aligned} {\left\{ \begin{array}{ll} y_{i,F(i)}=a_i \, \text { and } b_i=0 &{} \text {if}\, i\, \text {is a line start}, \\ y_{B(i),i}=b_i \, \text { and } a_i=0 &{} \text {if}\, i\, \text {is a line terminus}, \\ y_{i,F(i)}=y_{B(i),i}+a_i-b_i &{} \text {otherwise}. \end{array}\right. } \end{aligned}$$

(2)

Also, $\textbf{a}$ and $\textbf{b}$ must be consistent, in the sense that $A_{B(i)}\ge B_i$, where $A_i$ (respectively $B_i$) is the cumulated number of embarked (resp. disembarked) passengers on the line under consideration, recursively defined as $A_{F(i)}=A_i+a_i$ (resp. $B_{F(i)}=B_i+b_i$). Moreover, $A_i=B_i$ at a terminal line stop i. This common value yields the total number of passengers transported by the line.

Let the transportation flow $n_{st}$ denotes the number of passengers following an st-trip, that is entering the network at s and leaving the network at t by using the shortest-path. One gets from (1)

$$\begin{aligned} x_{ij}=\sum _{st}\chi _{ij}^{st}\, n_{st} \end{aligned}$$

(3)

Among the passengers embarking in i, some transfer from another line, and some others enter into the network:

$$\begin{aligned} a_i=z_{\bullet i}+n_{i\bullet } \end{aligned}$$

(4)

where “$\bullet$” denotes the summation over the replaced index, as in $n_{i\bullet }=\sum _{j=1}^l n_{ij}$. Similarly, among the passengers disembarking in i, some transfer to another line, and some others leave the network:

$$\begin{aligned} b_i=z_{i\bullet }+n_{\bullet i} \end{aligned}$$

(5)

By construction

$$\begin{aligned} a_{\bullet }=b_{\bullet }=z_{\bullet \bullet }+n_{\bullet \bullet } \end{aligned}$$

where $n_{\bullet \bullet }$ counts the number of passengers, and $z_{\bullet \bullet }$ counts the number of transfers. $z_{\bullet \bullet }/n_{\bullet \bullet }$ is the average number of transfers per passenger.

As explained in “Lines, stops and junctions” Section, transfers can only occur at junctions, that is $z_{ij}>0$ implies $(i, j) \in T$. In particular, $z_{ii}=0$: no traveler is supposed to disembark and re-embark later at the same stop.

Statement of the problem and solution method

Automatic passenger counters measure the number of passengers entering and leaving lines at each stop [Boyle, 1998], that is $\textbf{a}$ and $\textbf{b}$, which provide the basic raw data of the present study, kindly provided by the Lausanne Transportation Agency (tl) for the case study in “Real Data” Section. We will suppose here that this data obeys the necessary consistency conditions for embarkment and disembarkment counts, even if, in real case studies, a rescaling must usually be performed to balance in and out-flows on each lines (given in the appendix).

Intra-line edge flows $\textbf{Y}=(y_{ij})$ can be determined by (2), but transfer edge flows $\textbf{Z}=(z_{ij})$ are, here and typically, unknown. The objective is to estimate the $l\times l$ transportation flow matrix $\textbf{N}=(n_{st})$. Many consistent solutions coexist in general, even for a single line with no transferts (“Markov property for a single line” Section). This issue of incompletely observed data can be tackled by the maximum entropy formalism Jaynes (1957), which has often been invoked in transportation modeling research Wilson (1967) Erlander and Stewart (1990).

Let $f_{st}=n_{st}/n_{\bullet \bullet }$ be the distribution of st-trips (empirical distribution) and let $g_{st}$ be some prior guess on its shape (theoretical distribution). Assuming some reasonable initial prior $g_{st}$,

• (1) We first suppose that the empirical margins $\alpha _s=f_{s\bullet }$ and $\beta _t=f_{\bullet t}$ are known. Then $f_{st}$ can be determined as the maximum entropy solution (“Maximum entropy estimate of st-trips” Section), i.e. as the distribution closest to $g_{st}$ in the Kullback–Leibler divergence sense under the margin constraints, to be calibrated by an iterative fitting inner loop

• (2) Then (“Updating the prior distribution” Section), the prior is updated to $\tilde{g}_{st}$ by shrinking, if necessary, the priors $g_{st}$, thus avoiding transfer overflow exceeding the embarking and disembarking counts at each stop. Moreover, an hyperparameter $\theta \in [0, 1]$ is used at this stage in order to control the minimum proportion of passengers entering/leaving the network at each stop.

• (3) Finally (“Updating the margin distributions” Section), the margins are updated to $\tilde{\alpha }_s$ and $\tilde{\beta }_t$.

With the new prior distribution $\widetilde{g}_{st}$ and the new margin distributions $\widetilde{\alpha }_s$, $\widetilde{\beta }_t$, we can iterate the above steps, until convergence. The only free parameter is $\theta$, whose effect is studied on toy examples in “Toy Examples” Section.

The above iterative solution method is somewhat reminiscent of the EM algorithm. As a matter of fact, the first step (maximum entropy) exactly correspond to the “expectation step” of the EM algorithm (see e.g. Dempster et al. 1977; Bavaud 2009), but steps two and three, aiming at calibrating parameters $g_{st}$, $\alpha _s$ and $\beta _t$, do not follow the maximum likelihood rationale of the “maximisation step”. Pseudocode of the algorithm is shown in Algorithm 1.

Maximum entropy estimate of st-trips

As announced, the proportion of st-trips $f_{st}=n_{st}/n_{\bullet \bullet }$ (empirical distribution) will be estimated from some prior guess $g_{st}$ (theoretical distribution) and margin constraints $\alpha _s$ and $\beta _t$ for $f_{st}$ by maximum entropy, i.e. by solving the problem

$$\begin{aligned} \min _{\textbf{f}\in \mathcal {F}}&\; \sum _{st}f_{st}\log \frac{f_{st}}{g_{st}}, \nonumber \\ s.t.&\; \sum _t f_{st} = \alpha _s, \nonumber \\&\; \sum _s f_{st} = \beta _t. \end{aligned}$$

(6)

The Lagragian is

$$\begin{aligned} L = \sum _{st}f_{st}\log \frac{f_{st}}{g_{st}} - \sum _s \lambda _s \left(\alpha _s - \sum _t f_{st}) - \sum _t \mu _t (\beta _t - \sum _s f_{st}\right), \end{aligned}$$

which gives, after deriving and setting to zero,

$$\begin{aligned} f_{st} = \phi _s \psi _t g_{st} \qquad \text {with } \phi _s := \exp (- 1 - \lambda _s) \text {, } \psi _t := \exp (- \mu _t). \end{aligned}$$

(7)

Using constraints in (6), we find

$$\begin{aligned} \phi _s = \frac{\alpha _s}{\sum _t \psi _t g_{st}}, \qquad \psi _t = \frac{\beta _t}{\sum _s \phi _s g_{st}}, \end{aligned}$$

(8)

which yields the following iterative fitting algorithm: starting with some $\psi ^{(0)}_t > 0$, one performs the iteration

$$\begin{aligned} \phi ^{(\iota )}_s = \frac{\alpha _s}{\sum _t \psi ^{(\iota )}_t g_{st}}, \qquad \psi ^{(\iota + 1)}_t = \frac{\beta _t}{\sum _s \phi ^{(\iota )}_s g_{st}}, \end{aligned}$$

(9)

until convergence to $\phi _s$ and $\psi _t$ obeying (8).³

In view of (4) and (5), the postulated margins must satisfy, for each isolated stop i

$$\begin{aligned} \alpha _i=\frac{a_i}{n_{\bullet \bullet }}\qquad \qquad \beta _i=\frac{b_i}{n_{\bullet \bullet }} \end{aligned}$$

(10)

permitting to determine the total flow as $n_{\bullet \bullet }=\frac{a_i}{\alpha _i}$, or $n_{\bullet \bullet }=\frac{b_i}{\beta _i}$ for any isolated stop i, and thus the st-flow itself as

$$\begin{aligned} n_{st} = n_{\bullet \bullet } f_{st}= n_{\bullet \bullet }\phi _s \psi _t g_{st} \end{aligned}$$

(11)

whose plugging into (3) yields the intra-line edge flows $\textbf{Y}=(y_{ij})$ and the transfer edge flows $\textbf{Z}=(z_{ij})$. Only the latter is required for subsequent algorithm steps and can be computed with

$$\begin{aligned} z_{ij} = I((i,j) \in T)\sum _{st} \chi _{ij}^{st} n_{st} \end{aligned}$$

(12)

where I(.) denotes the 0/1 indicator function.

Initialization of the prior and the margins

The geometry of the network permits to define the set of permitted st-trips across the network denoted by P. The initial prior was chosen as the uniform distribution on admissible paths, that is as

$$\begin{aligned} g_{st} = {\left\{ \begin{array}{ll} \frac{1}{\vert P \vert } &{} [s, t] \in { P}, \\ 0 &{} \text {otherwise}. \end{array}\right. } \end{aligned}$$

The initial margins were chosen to initially match $g_{st}$, namely $\alpha _s=g_{s \bullet }$ and $\beta _t=g_{\bullet t}$ for all stops. However, $g_{st}$ could be chosen more carefully in case of additional data. As a matter of fact, $g_{st}$ represent prior attractions between nodes in equation (7), before margin correction given by $\phi _s$ and $\psi _t$ and subsequent algorithm steps. An urban planner could choose to increase or decrease some of these values in order to take into account prior knowledge on commuting habits of inhabitants.

Embarkment, disembarkment constraints and hyperparameter $\theta$

After a single iterative fitting step, the resulting transportation flow $\textbf{N}=(n_{st})$ and transfer flow $\textbf{Z}=(z_{ij})$ have little chance to fulfill constraints (4) and (5) defined by $\textbf{a}$ and $\textbf{b}$, and prior distributions $g_{st}$, $\alpha _s$ and $\beta _t$ must be corrected accordingly.

When $z_{\bullet i} > a_i$, or $z_{i \bullet } > b_i$, the found solution typically predicts that there are more passengers entering, respectively exiting, at a stop i than the actual measured quantity. One could be tempted to correct prior distributions in order to have $z_{\bullet i} = a_i$, or $z_{i \bullet } = b_i$, on every problematic nodes i, but the latter solution would mean that all passengers entering (resp. exiting) the line at this stop are transiting, which seems unrealistic in most real life scenarios.

We define here the hyperparameter $\theta \in [0, 1]$ as the minimum proportion of passengers (among $a_i$ and $b_i$) entering/leaving the network at each stop (in other words, not transferring), that is

$$\begin{aligned} n_{s\bullet }\ge \theta a_s \qquad \qquad \qquad n_{\bullet t}\ge \theta b_t \end{aligned}$$

(13)

or equivalently

$$\begin{aligned} z_{\bullet s} \le (1 - \theta ) a_s\qquad \qquad \qquad z_{t \bullet } \le (1 - \theta ) b_t \end{aligned}$$

(14)

and updates of distributions will be made accordingly.

Note that, if additional data were known, we could set a particular value of $\theta$ for every node, and differing for embarkments and disembarkments. However, without additional information, we will restrain to this simpler case.

Updating the prior distribution

Overflow occurs in transfer edge (i, j) if $z_{i \bullet } > (1 - \theta )b_i$ or $z_{\bullet j} > (1 - \theta )a_j$. To avoid it, components $g_{st}$ of the prior distribution will be shrinked by a suitable ratio whenever edge flows $(i,j)\in [s,t]$ exhibit overflow. For any edge (i, j), let us compute the flow ratio $r_{ij}$ as

$$\begin{aligned} r_{ij} = \max \left( 1, \frac{z_{i \bullet }}{(1 - \theta )b_i}, \frac{z_{\bullet j}}{(1 - \theta )a_j} \right) \, \ge \, 1\hspace{5.0pt}, \end{aligned}$$

(15)

where $r_{ij} > 1$ denotes an overflow through edge (i, j). For a given origin–destination [s, t], define the orgin-destination flow ratio $\bar{r}_{st}$ as the largest $r_{ij}$ among edge flows $(i,j)\in [s,t]$, that is as

$$\begin{aligned} \bar{r}_{st} = \max _{ij} \chi _{ij}^{st} r_{ij}\, \ge \, 1\hspace{5.0pt}. \end{aligned}$$

(16)

By construction, $\bar{r}_{st} > 1$ denotes an overflow on some transfer edge between s and t. To adjust the flow, we shall divide the previous flow by this ratio

$$\begin{aligned} \widetilde{n}_{st} =\frac{n_{st}}{\bar{r}_{st}} \end{aligned}$$

(17)

and define the new prior distribution as

$$\begin{aligned} \widetilde{g}_{st} = \frac{\left( \frac{\widetilde{n}_{st}}{\phi _s \psi _t} \right) }{\sum _{s',t'} \left( \frac{\widetilde{n}_{s',t'}}{\phi _{s'} \psi _{t'}} \right) }\hspace{5.0pt}. \end{aligned}$$

(18)

where $\phi _s$ and $\psi _t$ are the values (8) obtained in the previous maximum entropy step.

Updating the margin distributions

By construction, the corrected flow $\widetilde{n}_{st}$ found in (17) now respects embarkment and disembarkment constraints. We can compute the new transfer flow on edges with

$$\begin{aligned} \widetilde{z}_{ij} = I((i,j) \in T)\sum _{st} \chi _{ij}^{st} \widetilde{n}_{st} \end{aligned}$$

(19)

and, with (4) and (5), updating margin distributions is straightforward

$$\begin{aligned} \widetilde{\alpha }_s = \frac{a_s - \widetilde{z}_{\bullet s}}{\sum _{s'} (a_{s'} - z_{\bullet {s'}})} \qquad \qquad \qquad \widetilde{\beta }_t = \frac{b_t - \widetilde{z}_{t \bullet }}{\sum _{t'} (b_{t'} - z_{{t'} \bullet })} \hspace{5.0pt}. \end{aligned}$$

(20)

Markov property for a single line

A “network” made of a single line contains no transfers, and flow estimates can be obtained at once by the maximum entropy step only.

Let $i=1,\ldots , l$ enumerate the stops in increasing order, that is $F(i)=i+1$. The initial prior is simply $g_{st}=c\; I(s<t)$ and captures solely the unidirectional nature of trips, where $c=\frac{1}{(l-1)(l-2)}$. The margins of the empirical distribution $f_{st}$, as well as the total flow, are here known

$$\begin{aligned} \alpha _s=\frac{a_s}{a_\bullet }\qquad \qquad \qquad \beta _t=\frac{b_t}{b_\bullet }\qquad \qquad \qquad n_{\bullet \bullet }=a_{\bullet }=b_\bullet \hspace{5.0pt}. \end{aligned}$$

(21)

Following (7) maximum entropy flows are of the form

$$\begin{aligned} n_{st}= n_{\bullet \bullet }\, c\, I(s<t)\, \phi _s\,\psi _t \end{aligned}$$

(22)

where (setting $\Psi _s:=\sum _{t>s}\psi _t$ and $\Phi _t:=\sum _{s<t}c\phi _s$) the constraints (8) equivalently read

$$\begin{aligned} \phi _s=\frac{\alpha _s}{c\, \sum _{t>s}\psi _t}=\frac{a_s}{n_{\bullet \bullet }\, c\, \Psi _s} \qquad \qquad \qquad \psi _t=\frac{\beta _t}{c\, \sum _{s<t}\phi _s}=\frac{b_t}{n_{\bullet \bullet }\, c\, \Phi _t} \end{aligned}$$

(23)

to be solved by iterative fitting.

Interestingly enough, the form (22) for the flows is reminiscent of the gravity flows of quantitative Geography Wilson (1967) Erlander and Stewart (1990) Bavaud (2002) Thomas-Agnan and LeSage (2021), where $\phi _s$ is the push factor, $\psi _t$ is the pull factor, and $I(s<t)$ the distance deterrence function. Yet, instead of being symmetric in s, t and decreasing with the distance $|s-t|$, the distance deterrence function is here asymmetric due to the line orientation, but otherwise constant.

This constancy entails the following Markovian behaviour for flows: let $m_{st}$ be the number of travelers embarking at stop s and still inside the line at stop $t>s$, and let $\rho _{st}$ the probability that travelers embarking at s will disembark at t. By (22),

$$\begin{aligned} m_{st}=\sum _{u\ge t}n_{su}=n_{\bullet \bullet }\, c\, \phi _s \sum _{u\ge t} I(s<u)\, \psi _u =n_{\bullet \bullet }\, c\, \phi _s (\psi _t+\Psi _t) \end{aligned}$$

The empirical estimate of $\rho _{st}$ is given by the proportion, among the travelers embarking at s and still present at $t>s$, of travelers disembarking at t, that is

$$\begin{aligned} \rho _{st}=\frac{n_{st}}{m_{st}}=\frac{n_{\bullet \bullet }\, c\, \phi _s\, \psi _t}{n_{\bullet \bullet }\, c\, \phi _s (\psi _t+\Psi _t)}=\frac{\psi _t}{\psi _t+\Psi _t}\le 1 \end{aligned}$$

which depends on t only: it appears that the disembarkment probability $\rho _t=\frac{\psi _t}{\psi _t+\Psi _t}$ at t is independent of the embarkment stop s. Said otherwise, a traveler embarking at any stop s (and thus necessarily in the line at $F(s)=s+1$) experiences the same disembarkment probability at each further stop $t>s$.

This Markov property, enjoyed by maximum-entropic flows, contrasts other possible solutions, such as the “first in, first out” (FIFO) flows (homogenizing the traveled distances among users) or the “last in, first out” (LIFO) flows (tending to generate maximally contrasted traveled distances).

Case studies

Case studies are divided in two sections. In the first section, we test the algorithm on toy examples, which are artificial networks where the transportation flow $\textbf{N}_\text {ref} = (n^\text {ref}_{st})$ is randomly drawn. These examples enable some kind of validation of the algorithm, as the ”real” transportation flow is known and can be compared to the solution $\textbf{N} = (n_{st})$ given by our method. This setup differs from the second section, which is dedicated to applying the algorithm to the real case of the public transportation network of the city of Lausanne (tl), where embankment and disembarkment flows are measured but the real transportation flow is unknown. This second case study shows that the algorithm is applicable on large, real datasets and can give insights about passengers probable routes in the network.

Error measurements

In all case studies, we obtain a estimation of the transportation flow with the algorithm, noted $\textbf{N} = (n_{st})$, starting from the real embarkment flow $\textbf{a}_\text {ref}$ and disembarkment flow $\textbf{b}_\text {ref}$. In toy examples, we also have access to the real transportation flow $\textbf{N}_\text {ref}$. There are two types of dissimilarity measures between the data and the solution proposed by the algorithm: (1) if we have access to $\textbf{N}_\text {ref}$, how much $\textbf{N}$ differs from it, and (2) how well constraints defined by $\textbf{a}_\text {ref}$ and $\textbf{b}_\text {ref}$ are respected. The first dissimilarity is measured through the mean transportation error, denoted by $\text {MTE}(\textbf{N})$, and computed as

$$\begin{aligned} \text {MTE}(\textbf{N}) = \sum _{st} \frac{n^\text {ref}_{st}}{n^\text {ref}_{\bullet \bullet }} \frac{|n_{st} - n^\text {ref}_{st}|}{n^\text {ref}_{st}} = \frac{\sum _{st} |n_{st} - n^\text {ref}_{st}|}{n^\text {ref}_{\bullet \bullet }} \end{aligned}$$

(24)

and the second one with the mean margin error, noted $\text {MME}(\textbf{N})$, defined as

$$\begin{aligned} \text {MME}(\textbf{N})&= \frac{1}{2} \sum _{i} \frac{a^\text {ref}_i}{a^\text {ref}_\bullet } \frac{|z_{\bullet i} + n_{i \bullet } - a^\text {ref}_i |}{a^\text {ref}_i} + \frac{1}{2} \sum _{i} \frac{b^\text {ref}_i}{b^\text {ref}_\bullet } \frac{|z_{i \bullet } + n_{\bullet i} - b^\text {ref}_i |}{a^\text {ref}_i} \nonumber \\&= \frac{\sum _i (|z_{\bullet i} + n_{i \bullet } - a^\text {ref}_i |+ |z_{i \bullet } + n_{\bullet i} - b^\text {ref}_i |)}{2n^\text {ref}_{\bullet \bullet }} \end{aligned}$$

(25)

where $z_{ij} = I((i,j) \in T)\sum _{st} \chi _{ij}^{st} n_{st}$ is the flow on transfer edges T. Both errors can be interpreted as a weighted mean of error percentages.

Note that, by construction, the MME should be zero when the algorithm converges. However, it can be informative to track down this error along iterations and, in some practical cases where margin constraints are impossible to fulfill, the algorithm convergence criterion is reached with MME $> 0$.

Toy examples

Construction

All constructed toy examples are built following the same approach, which aims at being simple but somewhat realistic. We fix a number of round trips $p \ge 2$, each of which is constituted of a forward line and a backward line, for a total of $q = 2p$ lines. Every line has a starting and an ending node, which are isolated nodes, and possesses $p-1$ intermediary nodes which allow transfers to the other round trips, yielding a total of $n = 2p(p + 1)$ nodes in the network. Examples of these toy networks can be found in Fig. 1.

Fig. 1

3 toy examples, with the number of round trips $p \in \{2,3,4\}$. Trips are displayed in the same color, lines possess a unique letter, and each stop correspond to a unique combination of a letter and a number. Clusters of nodes represent positions where transfers between round trips are possible

In order to be realistic, the permitted st-trips set P is constructed by considering all shortest-path between pair of nodes, excluding:

• s and t that are on the same line but with t preceding (or equal to) s in the line order.

• s and t that are on the same round trip but on opposite lines.

• s and t whose shortest-path starts with a transfer edge, ends with a transfer edge, or possesses two (or more) consecutive transfer edges.

A transportation flow $\textbf{N}_\text {ref} = (n^\text {ref}_{st})$ is drawn by setting a fixed number of passengers $n^\text {ref}_{\bullet \bullet }$, and each passenger is assigned randomly to a (s, t) pair drawn uniformly among P. From this reference transportation flow $\textbf{N}_\text {ref}$, using the edge-trip incidence matrix ${\varvec{\chi }}$ and equation (3), we can compute flow on edges $\textbf{X}_\text {ref}$ and, in turn, the number of passengers embarking $\textbf{a}_\text {ref}$ and the number of passengers disembarking $\textbf{b}_\text {ref}$ at each stop.

Algorithm iterations

First, we exhibit some algorithm iterations on a toy example with $p=2$, where 50 passengers were drawn uniformly across the $\vert P \vert = 20$ possible st-trips. Some iterations of the algorithm with $\theta =0.001$, along with MTE and MME errors, are shown in Fig. 2.

Fig. 2

Real transportation flow (green arrows) obtained by randomly drawing 50 passengers on the graph toy example with $p=2$ round trips, along with iterations 1, 2, 4, 7, 15 of the algorithm with $\theta =0.001$. MTE and MME errors are computed, and embarkment and disembarkment counts are represented respectively by the red and blue colors on nodes

On this small example, we can see that the algorithm quickly find an estimation giving a small MME error, but still give a MTE of 0.256. This result is due to the fact that only 50 passengers are drawn, giving a large deviation compared to the optimally found solution which maximizes the entropy.

MTE study

The main goal of toy examples, since we have access to the real transportation flow, is to study how the resulting MTE behaves regarding passenger st-trips distribution and hyperparameter $\theta$, on different network sizes.

The randomness of the st-trips distribution is controlled by the number of drawn passengers and we can see that the algorithm performs better if this number increases, as seen in Fig. 3. This behavior can be expected as the method is constructed to find the solution maximizing the entropy while respecting embarkment and disembarkment constraints. In fact, all MTE should converge to 0 eventually, however this rate of convergence seems to be low.

Fig. 3

Mean transportation error (MTE) vs the number of randomly drawn passengers, in toy examples with numbers of round trips p varying from 2 to 8. The hyperparameter is set to $\theta =0.001$ and 10 different draws are performed for each data point

As for the hyperparameter study, found in Fig. 4, we see that the optimal parameter seems to decrease as the network size increases, reaching value close to 0 for 8 round trips. However, this is not the only factor which helps to calibrate this parameter. As a matter of fact, it seems unrealistic, in some real life scenarios, that all passengers embarking or disembarking at junctions are all transiting, and this value will be kept at 0.1 for our real life study.

Fig. 4

Mean transportation error (MTE) vs hyperparameter $\theta$, in toy examples with numbers of round trips p varying from 2 to 8. The dot on each curve denotes the minimum. The number of drawn passengers is different for every network size and is set to $5 \cdot \vert P \vert$, in order to keep an (almost) constant variance. 10 different draws are performed for each data point

Real data

The dataset and algorithm setup

The dataset was constructed from data collected by the public transportation agency in the city of Lausanne (tl), in Switzerland. Automatic passenger counters were used to collect data on the number of passengers entering and leaving buses and subways at each stop. The dataset comprises 44 lines, 1361 stops, and over 115 million passengers per year. Each round trip forms two separate lines, which often have stops that correspond to both the inbound and outbound directions, but this is not always the case. Some stops may be unique to either the inbound or outbound direction. The initial dataset contained 13 million data rows, which were aggregated for the year 2019, and the passenger data was filtered to include only passengers who embarked and disembarked at each stop. Lines with the most errors were removed, resulting in a final dataset of 35 transportation lines and 1216 stops. The threshold for invalid lines was set when a difference between total embarkment and disembarkment counts differs more than 15% of their means. For the rest all other lines, the correction found in the appendix was applied to ensure consistency conditions.

Line edges were built from the given data, and to obtain the full network structure, we added transfer edges between different lines if the estimated walking time (using Open Trip Planner⁴ Morgan et al. (2019)) was less than 120 s. Shortest-paths between all pairs of stops were computed using the breath-first search algorithm West (2001) implemented in the igraph⁵ R library Csardi and Nepusz (2006), thus giving ${\varvec{\chi }}$. The matrix P, containing permitted st-trips, was built using the same conditions found in the toy example experiments (“Construction” Section) with an additional one. If the estimated pedestrian time between two stops s and t was shorter than the estimated time by taking the public transportation network, this st-trip was also removed from P. The graph structure, consisting in the edge-trip incidence matrix ${\varvec{\chi }}$, the permitted st-trips set P, and the transfer edges set T, was constructed beforehand testing the method.

The algorithm was run with hyperparameter $\theta = 0.1$. Tracking the mean margin error (MME) showed a rapid decrease ($<0.001$ after 13 iterations) but the (conservative) convergence criterion (chosen as when $\sum _{st} \vert f^\text {prev}_{st} - f_{st} \vert < 10^{-6}$) was reach after 316 iterations, which corresponds to approximately one hour of computing time with a single thread on a AMD Epyc2 7402 with 32GB of RAM.

Results

As the result of our algorithm consists in the $(1216 \times 1216)$ matrix $\textbf{N} = (n_{st})$, it is difficult to present them succinctly. We chose here to focus on two aspects: (1) mapping the origin and destination profiles of stops with the help of Correspondence Analysis (CA) Benzécri (1977); and (2) mapping the aggregated transfers between lines.

The first and second factorial dimensions of the destination and origin profiles of stops, obtained with CA applied on $\textbf{N}$, can be found on Fig. 5. We can see that origin and destination coordinates of each stop are quite similar when displayed on the first two factorial axes. This result can be explained if we understand that, in fact, origin and destination profiles are, to some extent, symmetrical: at the beginning of a line, stops have several choices of destinations, but very few origins “point” at them, which is exactly the reverse for stops at the end of a line. Intermediary stops are generally located near the city center, and have a similarly rich origin and destination profiles. Concerning the dimensions, we can see that the first component divides the city stops between west and east, a known dichotomy in the public transportation usage in Lausanne: the east of the city is less well served by public transport and inhabitants often use cars, while the west intensively uses public transports. The second component is harder to interpret, but seems to highlight the radial structure of the transportation network in Lausanne.

Figure 6 maps predicted transfer hubs, obtained by aggregating every transfer counts $z_{ij}$ between lines when they occurs between stops located in the same spatial cluster (same “superstop” name in the dataset). As expected, most transfers occur in the city center, with the top 5 largest hubs located at St-François (2,132,597 transfers), Lausanne-Gare (2,035,543 transfers), Bel-Air (1,908,168 transfers), Ours (1,057,282 transfers), and Chauderon (969,789 transfers). When taken individually, four of the top 5 transfer counts are predicted between line 1 and line 72 occurring at the train station (Lausanne-Gare). Line 72 is actually a subway line connecting the northern and southern regions of the city, and is by far the most used line in the network. Line 1 is also a highly frequented line and operated toward the western region of the city. It is not surprising to see most transfers occurring between these two lines, as they permit to connect the region with a high density of public transportation network users (the western part of Lausanne), to the main artery of the network (line 72).

Fig. 5

Color representations (positive values in red, negative values in blue) of the first two factorial coordinates of stops, obtained from Correspondence analysis (CA) performed on the transportation matrix $\textbf{N}$. Top row maps stops as “origin profiles” (“where passengers are going”), bottom row maps stops as “destination profile” (“where passengers are coming from”). Left column depicts the first factorial dimension, right column the second factorial dimension. Ring sizes correspond to the number of embarkments (top) and disembarkments (bottom)

Fig. 6

Localization and size of transfer hubs, obtained by aggregating estimated transfer flows between lines $z_{ij}$ in “superstops”

Conclusion

At first glance, the task of estimating the origin–destination trip counts of public transportation networks on the sole basis of embarkments and disembarkments stop counts might appear as too ambitious. The project is indeed challenging, but we hope to have convinced the reader that meaningful estimates can be obtained by applying a succession of carefully chosen, principled yet flexible steps. Some of the ingredients (maximum entropy, iterative fitting) are familiar in transportation theory, and some others (shrinkage of priors, minimal amount of entering and leaving the network) seem original. Possible theoretical and algorithmic improvements are still under investigation. Due to length constraints, the exploitation and interpretation of the estimated flow has been kept to a minimum. Yet, many issues, such as disaggregating data and regimes (week days, week-ends or holidays; time in the day), assessing the centrality of stops and junctions, or characterizing the imbalance between uphill and downhill flows (Lausanne is known for its slopes) are of primary interest for urban planning and transportation geography, and will be developed in a forthcoming work. Also, visualizing the numerous and various quantities of interest, on a spatial map or else, constitutes a challenge in itself, requiring a particular blend of creativity and rigor.

Appendix

Correction of the embarkment and disembarkment counts in a single line

It may happen that, in a line $\ell$ with stops indexed in order as $1, \ldots , l$, raw data $\textbf{a}=(a_i)$ and $\textbf{b}=(b_i)$ do not obey the following necessary consistency conditions

$$\begin{aligned} a_l = 0&, \; b_1 = 0 \\ A_{(i-1)}&\ge B_i \qquad \forall i \in \{1, \ldots , l-1\} \\ A_{l}&= B_{l} \qquad \text {for terminal stop} \,l \end{aligned}$$

where $A_i$ (respectively $B_i$) is the cumulated number of embarked (resp. disembarked) passengers on the line at stop i, i.e. $A_i = \sum _{j=1}^{i} a_j$ and $B_i = \sum _{j=1}^{i} b_j$. The first condition is easy to correct (by setting both quantities to 0), and we will assume that they are valid. The last two require an iterative procedure, explained here.

We first identify all stops i where $A_{i - 1} < B_i$ and store them, along with stop 1 and l, in a order set S. Let $\text {Prev}(i)$ be the item right before i in the set S. For all $i \in S \setminus \{1\}$, we do:

$$\begin{aligned} \hat{a}_j&=\left( 1-\frac{(A_{(i-1)} - A_{(\text {Prev}(i)-1)}) - (B_{i} - B_{\text {Prev}(i)})}{(A_{(i-1)} - A_{(\text {Prev}(i)-1)}) + (B_{i} - B_{\text {Prev}(i)})}\right) \, a_j \qquad \forall j \in \{ \text {Prev}(i), \ldots , i-1 \} \\ \hat{b}_j&=\left( 1+\frac{(A_{(i-1)} - A_{(\text {Prev}(i)-1)}) - (B_{i} - B_{\text {Prev}(i)})}{(A_{(i-1)} - A_{(\text {Prev}(i)-1)}) + (B_{i} - B_{\text {Prev}(i)})}\right) \, b_j \qquad \forall j \in \{ \text {Prev}(i) + 1, \ldots , i \} \end{aligned}$$

Before the last step, all conditions $A_{(i-1)} \ge B_{i}$ should be respected except for node l. The last step ensures that $A_{l} = B_{l}$. However, as this last step can sometimes lower the embarkment count and increase the disembarkment count on previous nodes (if $A_l > B_l$ before correction), some new nodes can now violate $A_{(i-1)} \ge B_i$. This is why this procedure must be iterated until all stops on the line verify consistency conditions.