Tools and guidance
Background
An important aim of CHI+MED has not just been to undertake scientific research but also to develop practical support for professionals based on that research. This support comes in a variety of forms such as guidance documents, practical techniques and resources that support them for example computer based tools.
We have developed tools and guidance for a range of purposes throughout the project including:
- to support the design and procurement of new medical devices;
- to support the case arguing that a device is safe and satisfies regulatory requirements, as well as to check such claims;
- to document and to analyse medical incidents.
PVSio-web: a Tool for Prototyping Interactive Medical Systems from Mathematical Models
Our prototyping tool, PVSio-web, helps check the safety of designs of devices and in doing so it supports designers, regulators, and hospital training and procurement staff. Working with the US regulator, the Food and Drug Administration (FDA), our tool has helped identify problems in a series of commercial medical devices. Hospitals have used it as part of training programmes highlighting safety-related design issues.
Programming safer keyed data input
We have created a tool that automatically creates data entry user interfaces that are safer to use than those created in ad hoc ways. It helps ensure that programmers cover all eventualities and the user interface responds to errors in a sensible way. User interfaces cannot be made perfectly safe. This is a fundamental problem we define and call divergence. It means that whatever is done to prevent users making mistakes there will always be some other mistake that can still be made. Programmers have at best ad hoc solutions over what to do when people enter data incorrectly, like when they include a second decimal point or put a leading 0 in a number, but often programmers just ignore these seemingly simple problems. Unfortunately, failing to manage user error well causes even worse problems for users, and this is what our tool helps prevent. One approach we developed that the tool uses is traffic lights, so the user interface and the user know when a mistake has been made. A red traffic light – perhaps augmented with a sound or vibration (and it could be any colour, or flash for colour-blind users) – alerts the user that a problem must be solved. With the tool, traffic lights are implemented flexibly, efficiently and dependably. Crucially, the programmer can then build software that does not need to handle the user errors, since the tool has done that already.
Modelling and analysis of interoperable medical systems
Different medical devices are increasingly used together and need to be ‘interoperable’ so that, for example, monitors share data with devices giving treatment. We have developed new tools for the verification and validation of interoperable medical systems. They are integrated into our PVSio-web prototyping environment for interactive (human-computer) systems. These tools have been successfully used to develop a realistic interoperable medical system prototype based on commercial products and application scenarios described by the FDA. In ongoing work they are being used to validate the FDA requirements for interoperable medical systems.
Refinement in developing and verifying user interface requirements for infusion pumps
Criteria for the acceptable safety of medical devices, safety requirements, are typically described precisely, but in natural languages like English. An important issue is reassuring regulators that the given requirements are satisfied. We have developed a refinement based approach that involves gradually transforming a requirement into a form that can be coded. It relates to the pre-market review process promoted by the FDA to provide safety assurances. Our approach helps to both design and clarify high-level safety requirements that relate to user interfaces of medical devices. It also supports the development of usage models for a range of interaction methods and their verification against safety requirements.
Using control theory to model interaction with medical devices
We have developed a new approach for evaluating the user interfaces of medical devices based on techniques from aerospace engineering and robotics. It combines manual control theory with hybrid automata. We have modeled both discrete and continuous human operator behaviour and, in particular, we have found a novel way to model a person using a medical device with up and down buttons. We can model both short button presses and holding down buttons. We have integrated the model with the CHI+MED analysis toolset, PVSio-web, and have demonstrated the model’s utility by simulating a person entering values on an infusion pump, using up and down buttons. A typical insight is the identification of user interface design flaws that would lead to a large overshoot when using up and down buttons to adjust numbers. It also helps an analyst to identify and reflect on different types of user behaviour.
Stochastic evaluation: randomly simulating users pressing buttons
Many different user interfaces for entering numbers could be made much safer without affecting their normal use. Stochastic evaluation, which involves computer simulation of users pressing buttons, is a very fast and effective way of seeing how usable and safe real or proposed designs are. We have identified ways to make common designs much safer, quickly identifying bugs or defective design choices. Our approach also allows choices to be compared by putting a value on the size of a problem’s impact on safety. The technique is automatic and best when there is an objective measure of how significant an error is. For example, ‘drug doses that are out by a factor of ten or more’ is easy to quantify and hence easy to evaluate automatically.
Standards: usability engineering standards for medical device design
We prepared a summary guidance document to signpost relevant standards for manufacturers on usability engineering. This led to raised awareness of the need for usability engineering standards during the design of medical devices with manufacturers, regulators and advisory committees. It led to our acting as expert reviewer on a white paper on Usability Engineering standards for the British Standards Institute.