Risk Number Description of Risk Work Packages Concerned Proposed risk-mitigation measures Did the risk materialize during the 1st period? Actions taken Did the risk materialize during the 2nd period? Actions taken
1 Partner inability to execute tasks successfully or in time. This risk has low probability but could come from a partner directly leaving the consortium (e.g. SME bankruptcy) or by lack of the appropriate human resources with expertise to execute a task. WP1, WP3, WP4, WP7, WP6, WP2, WP8, WP5 To rebalance the project assigning more time or effort to the affected tasks. In the worst case, to assign the involved task to another partner (most competences can be executed by at least two partners) or to an external entity. No
2 Complex conflicts among partners. This risk has low probability and could come from partners encountering an unresolvable problem. WP1 Activation of the conflict resolution mechanism for solving the problem. In the worst case, the resolution may require one or several partners leaving the consortium and being replaced. No
3 Inability to collect relevant market requirements. This risk has low probability given that the consortium involves relevant technological companies with deep experience in software development and complex system testing. WP2 To reinforce Task 2.1 for collecting requirements from entities external to the consortium. To reinforce the AB for involving external stakeholders in the area of software testing suitable for providing relevant market requirements and feedback. No
4 Market and project feedback not aligned with planning. This has medium probability and may happen if our tests, demonstrators or market scouting show the unsuitability of some of initial ElasTest ideas for testing SiL. WP2 To follow our Agile Management methodology (see Task 2.2) and pivot to modify the project plan accordingly. In case of major modifications, a project amendment shall be requested to the EC. Yes The consortium discovered a limitation on the Web demonstrator (Task 7.3) that required manual testing, not initially considered in the DoA. Therefore, on September 2017, in its face-to-face meeting, the consortium decided to prioritize including manual testing support in the platform for Milestone 3 (January 2018).
5 Inability to integrate the ElasTest stack. This risk has low probability given the partners expertise and could come from the inherent complexity of the system and its multitude of components, which might not offer appropriate interfaces or stability. WP3, WP4, WP6, WP5 To rebalance the project assigning more time or resources to the affected tasks. To divert more effort to communication. In the worst case, split the platform into sub-platforms offering partial functionality with decreased complexity. No
6 Inability of the recommendation and cognitive Q&A engines to provide useful knowledge. This risk has high probability and comes from the inherent uncertainty of the research activities that need to be performed for creating such engines and from the need of huge amounts of training data next to the definition of usefulness. WP4 To use a strategy based on concentrating on specific types of SiL. For example, SiL for which a model can be created (e.g. SOA) shall present less uncertainty. Concentrate on SiL for which the 4 ElasTest demonstrators are created. In the worst case, these engines failing to work shall not impact on the rest of project results. Yes On December 2017, it was decided that the recommendation engine switched from a Watson-based approach, to a deep learning approach, where different techniques are exploited to achieve good test recommendation results. This risk has been mitigated, as of June 2018, and results are much promising now.
7 Inability of the ElasTest orchestration mechanism to simplify testing tasks on common SiL. This risk has low probability and could occur if specific types of SiL are identified where the orchestration concept might not be applicable for any reason. WP4 To concentrate the project on the specific types of SiL where ElasTest orchestration might be applicable and to adapt our technology to them. To focus our dissemination and exploitation strategy on the segments where such SiL types may be relevant and ElasTest might have impact No
8 Inability of some ElasTest Test Support Services to provide useful solutions to testers on all types of SiL. This risk has medium probability and could occur on some of the most innovative services such as Security Check as a Service or Sensor, actuator and device impersonation as a Service. WP5 To concentrate the activities of the involved tasks on solving problems on the four vertical domains of ElasTest demonstrators instead of on generic SiL. As a last resource, and given our modular architecture, some services might be removed from ElasTest without affecting the impact of the rest. No
9 Inability to perform the planned validation studies. This risk has low probability and could occur because of delays in platform releases or in the development of the four demonstrators, or due to complexity of empirical studies settings. WP7 To rebalance the project assigning more time or effort to the affected tasks. WP7 includes early preparation of the studies so that potential issues can be timely detected and handled. Yes In September 2017 it was detected that the web vertical (ATOS WL) could not leverage ElasTest due to a lack of manual testing support. This vertical had no automated tests, all tests over the software were performed manually through a web browser. To solve this issue, ETM and EUS components were modified to support manual testing as well. Furthermore, even with manual testing included into the platform, for this vertical it was not possible to improve any metric by using ElasTest. Therefore, as a second measure it was decided to implement an additional integration not originally foreseen in the DoA. TestLink, a tool for test management, was integrated in the platform. In addition, an initial experimentation has been performed to ensure the validation approach works, with the new features added, and all verticals were able to complete the experiment.
10 Inability to generate a critical mass for the ElasTest community before the project ends. This risk has low probability and could happen given the limited resources for disseminating and communicating the community to developers worldwide. WP8 The ElasTest community, as a whole, might fail to generate the appropriate critical mass. In that case, each of the independent component of ElasTest might be exploited independently basing on FOSS or proprietary business models so that the impact of more successful components is not constrained by the rest. No
U1 Inability to measure some of the metrics WP7 To consider external projects and/ or companies for the experimentation Yes Inability to measure Validation metric 2.4: "To increase the subjective feelings of end-users in terms of efficiency, overall satisfaction and lack of risks when using SiL-based applications in a factor of at least 1 per each in a scale of 5", because due to the time frame of the project workplan, it will not feasible to reach end-users of the four demonstrators released by ElasTest WP7 and get their feedbacks, given that they are planned for release at M36. To address this issue, the open source projects Kurento and OpenVidu (with a good number of users) will have their tests migrated to ElasTest by Fall 2018. After several releases during 2018 and beginning 2019, users of these two projects could be considered end-users of a software that is validated with ElasTest.