Definition of competence and related models
There are various definitions of “competence” in literature. In (Belkadi 2006), a detailed summary of those definitions is provided. For our research needs, we choose the definition stated by Le Boterf, which defines a competent person as: “a person who knows how to act appropriately in a particular context, by choosing and mobilizing a double equipment: personal resources (knowledge, qualities, culture, emotions...) and network resources (databases, documents, expertise networks,...) ”(Le Boterf 2000).
Bonjour et al. (2002) examined the competence concept and proposed a cognitive model for defining competence. This model aims to simplify its representation and to clarify the link between knowledge and competence. Later, (Belkadi et al. 2007) proposed a method based on fuzzy logic for competence characterization related to a work situation. It consists of analyzing the characteristics of a given situation, after tracing the corresponding activity. This method can not be used in our research case, because we do not aim to model activities in order to characterize competences.
Several competence models had been proposed in the literature (Paquette 2004; Belkadi 2006; Hlaoittinun et al. 2009; Mota 2009; Coulet 2011). There is also the European e-Competence Framework (e-CF)Footnote 1 which provides a model of 40 competences related to the Information and Communication Technologies (ICT) field. E-CF framework is a common standard throughout Europe. In 2016, it became a European norm and was officially published as EN 16234-1. This framework is although specific to ICT field and does not apply to other fields. There are also business competence models such as the Compmetrica Competence ModelFootnote 2. Furthermore, many organizations have their own competence model, such as CERNFootnote 3. Other examples of competence models could be found in the APECFootnote 4 and Pôle EmploiFootnote 5 “job descriptions”.
Interaction data and social networks
In order to automatically identify user competences, one can process the data produced during working sessions such as collaborative activities or human-machine interactions. A large and growing community of researchers, mainly in Human Sciences field, has investigated data analysis including interaction data. Implementing routines to produce such data is usually a costly time-consuming task. Platforms generating interaction data sets with their context is therefore a considerable asset. For sharing traces corpus contextualized in specific formats, platforms have been developed so far, such as PSLC Datashop (Koedinger et al. 2010) or MULCE (Chanier et al. 2010). The BEATCORP platform (Courtin and Tomasena 2016) proposes tools for sharing interaction traces’ corpus of any format, and also provides analysis tools for those corpora. The platform is based on the PROXYMA (PROXY for Multiple Analyses) approach (Chebil et al. 2015). In perspective, the designed approach aspires to provide, via Web data analysis, the recommendation of resources, such as documents or skills, and the appraisal of such recommendations through evaluation methods.
Several studies have tackled the issue of using interaction data for social network analysis. In (Wu et al. 2008), a research was made on face-to-face communication network. The authors used sociometrics to measure offline interactions at the workplace in order to assess work performance and therefore improve group structuring. Aral et al. (2007) presented a research study based on email data analysis aiming to understand how social networks impact information worker productivity.
Competence recommendation system
Recommendation systems help users getting straight to the essential and benefit from “suggestions (...) which they would not have spontaneously paid attention to” (Béchet 2011). The search time can significantly be reduced by such systems. Nonetheless, these systems can achieve such a performance level only when the computed recommendations are relevant and judged useful by the user. Consequently, downstream evaluation processes must be developed. Most competence recommendation systems rely primarily on Web data. For example, the Aardvark project (Horowitz and Kamvar 2010) (now abandoned) implemented a platform for posting questions to people having the appropriate profile to respond. Those people need to belong to the same on-line social network as the question asker. The DemonD project (Delalonde and Soulier 2007) used a method similar to Aardvark project in extracting profiles, and relied on the traces of discussions within the system in order to recommend various resources: documents, discussions, people or articles. The SmallBlue project (Lin et al. 2012) extracted employees’ profiles from their personal data such as e-mails. The recommendations were proposed as a graphical representation of the user’s social connectivity with the recommended persons.
So far, among the competence recommendation systems listed above, the ones offering a graphical representation of recommended competences did not consider the role of spatial distribution of the graph nodes for providing more meaningful recommendations (the distance between the nodes, node dispersion, graph expansion, communities detection). There are spatial layout algorithms which can allow a trimmer distribution of network nodes, which would make recommendation networks more significant for the user as he/she can visualize and interpret them better. For example, the Gephi graph editor and analysis tool provides a list of these algorithms. ItFootnote 6 is a free open source interactive software for networks visualization based on graphical elements such as nodes, edges or hubs. This exploratory data analysis software is used in visual analytic research field. Gephi expects CSV files as input, where data represent network’s matrix structure (list of edges with their weight). It would therefore be propitious to use spatial layout algorithms along with recommendation algorithms and to study the incidence of this combination on the quality of the competence recommendations provided to the user.
The reviewed systems above share a common feature: endogenous data analysis. Otherwise speaking, systems taking into account relationships between users rely on their intra-system participation (except for e-mail). But if we consider that users are already active on digital systems to communicate, coordinate and produce contents (on e-mailer, Enterprise Resource Planning tools, Social Networks), we can barely assume that they would be inclined to join a new on-line social network in order to benefit from recommendations. It would then be more suitable to observe their activity in already-used systems, and offer them the most immersive recommendation service. The analysis of interaction data produced within digital systems can automatically enrich users’ profiles with operational competencies.
Summarized in (Shani and Gunawardana 2011), there are three different strategies for recommendation systems’ evaluation. The first, called off-line, is based on pre-collected data relating to behaviors of a group of users such as ranking or choosing items. Such data is used to predict their future behaviors. Called on-line, the second strategy is applied to a randomly-selected users during a real-time interaction with the system, observing and comparing their behavior to other users. User study is the third strategy, which involves some voluntary users answering qualitative or quantitative questions when using the recommendation system.
Off-line and user study are the most suitable strategies fitting to our experimental context. Off-line experiment would enable, as a first step, a number of algorithms to be executed and compared, so the most accurate algorithm can be selected thereafter. In order to collect quantitative measurements (the time taken to perform the task, recommendations’ relevance), we can observe and record users’ behavior. For this purpose, some voluntary users would perform various tasks while interacting with the system. By means of a questionnaire, qualitative data such as the ease of use of the user interface, can also be collected.
There are also metrics for recommendation system assessment in addition to strategies, such as the accuracy metric which refers to the quality of recommendations, the recall metric which calculates the rate of relevant recommended items, or Mean Absolute Error (MAE) which computes the deviation between predicted recommendations and actual users’ choices, as well as some other metrics proposed and discussed in the literature such as coverage, learning rate, novelty and serendipity or mean similarity (Miller et al. 2004; Ziegler 2005). In (Said et al. 2012), a 3D model for evaluating a recommendation system is proposed. It assesses system quality according to three axes: user requirements, technical constraints and expected business model’s objective. Said and Bellogín (2014) present, in a recent study, Rival which is a free java tool for evaluating recommendation systems that ensures comparison and reproducibility of results in any experimental context.
Our evaluation model includes some of the above metrics such as precision, recall, Mean Absolute Error, and coverage. We have completed the model with other measures presented and explained in the last section.
Most of competence recommendation systems mainly rely on text analysis (summary documents, databases), web browsing history analysis, or e-mail. Other types of activity traces can also be considered to identify experts within an organization such as activity logs (a tool/system use frequency indicator), or user interactions with professional social networks. These types of traces would allow user profile to be dynamically updated and associated competences identification to be improved.
A survey was carried out among a list of companies to identify the tools usually used by DHR or managers to find competent people for leading projects or carrying out collaborative activities. The survey also aimed to find the gaps in the used tools and determine the needs of recruiters/ managers. The results of this survey were considered while designing the proposed decision-making tool.
In this research, we describe and deploy concepts of the interaction data-based decision-making tool for competence recommendation and evaluation. The followed methodology is described and the designed model is presented. A case study was carried out to illustrate the process of identifying the appropriate competence profiles corresponding to user needs, based on interaction data analysis. Processing the big amount of available data is a major challenge, but the lack of relevant data can also complicate the competence identification process.
The competence model used in this work is a simplified model, which relates to the listed models in related works. Competences, in this model, are classified by type (hard skills, soft skills) and by level (3 levels ranging from “not competent” to “very competent”). The competences repository is a database which differs from one company to another. This database is an input to our decision-making tool proposed for the automatic competence identification and their recommendation.
When designing the platform for competences recommendation and evaluation, we chose a graphical representation for system recommendations. In fact, recommended competences are displayed as networks, which are “human-centered”. All the nodes represent people, and the connections represent the characterized relationships (trust degree, collaboration degree), with a focus on one node called “ego”. As a perspective, other representations could be considered depending on the intended utilization. For example, a “product-centric” representation could incorporate other elements into the graph which is centered on one or more products, such as the corresponding manufacturing processes, that are associated with competences profiles, themselves linked to people.