default A risk game to measure the impact of information quality on human threat assessment and decision making

By

A risk game to measure the impact of information quality on human threat assessment and decision making.  Jousselme, Anne-Laure; Pallotta, Giuliana; Locke, Jonathan. CMRE-FR-2015-009. June 2015.

This document describes the Risk Game, a general methodology developed at CMRE to elicit experts knowledge and know-how, including their ability to deal with information of different nature (from sensors to human witnesses), to consider the information quality (including source quality) and to reason about concurrent events. It is a "contrived" technique aimed at capturing data expressing human reasoning features while performing a specific task of maritime situation assessment. The main focus of the Risk Game is the study of the impact of information quality on the ability of human operators to assess threat and make decisions. A new methodology is presented where information is abstracted away by cards, and the quality is randomly selected by dice roll. The game has been tested during the Table Top Exercise (TTX) for harbour protection held at CMRE in November 2014. The game has been played by 32 experts, most of them OF-3 and above of the maritime domain, from 9 NATO Nations. The preliminary results obtained are promising and allowed us to identify research challenges as a basis for definition of future work. We in particular validated the elicitation method and highlighted for instance that the players? perceived relevance of information may differ from the effective relevance, that a high amount of false information increases the uncertainty of the player before decision and may lead to wrong decisions, or that the context has a high impact on the decision taken. We also draw some research avenues to be addressed in future work, such as the definitions of information quality measures which would be consistent across different domains of definition (resolution and type of scale).