Usability testing of software as a medical device (SAMD) 

The aim of the efforts and work on the usability of software that is a medical device is to ensure that it is safe for the patient and the user by reducing the risks arising from incorrect or misuse. This means that when focusing on usability, we should focus on the risks that may arise from misuse or user error and their causes. This is a particularly difficult task, as the number of potential sources of user action not in accordance with the intended use of a medical device seems infinite. These actions may result from ambiguous statements in the instructions for use or displayed messages, software errors, and a non-intuitive and misleading interface. In addition, the attitude of the user himself (the so-called human factor) can be a source of incorrect actions. He or she may be distracted and not read the messages correctly. He may also ignore them, and act in haste or according to previous habits. Trying to anticipate all of these events, reduce the likelihood of their occurrence, and limit the impact on the safety of the patient and the user is the essence of the process of working on the usability of software that is a medical device.   

Medical software plays a key role in the treatment and care of patients. To be effective, it must not only be secure but also easy to use. This is why usability tests are so important to help verify that the software as a medical device meets user expectations. Of course, this does not exclude carrying out additional usability activities focusing not on safety, but on user convenience. They are also important because they often have a significant impact on the market success of a product and its popularity among users. When working on the usability of the software we develop, we pay attention to both aspects. We prioritize safety, but we also make sure that the solutions we create are user-friendly and convenient for everyday work. 

Formative evaluation and summative evaluation 

The work we carry out on the usability of a medical device can be roughly divided into 2 stages. The first is the formative evaluation carried out during software development. During it, we work with UX (usability) specialists on the individual parts of the interface. We also periodically meet with potential users, present them with our concepts and prototype solutions and collect their comments and suggestions. After analysing these in terms of patient safety and user-friendliness, suggestions for changes to the software are made and, once approved, implemented in subsequent versions. The result of such iterative work is an increasingly clear and intuitive interface, which translates into minimising the risk to the patient associated with user error or misuse. Whether this is indeed the case and whether we have actually succeeded in minimising these risks is checked during the second stage of usability work called ‘summative evaluation’. Its purpose is to carry out an evaluation by potential users of the already developed software and to identify potential usability problems. In contrast to the first stage, this evaluation is more formal and is carried out by a larger number of users to ensure a diversity of opinions and experiences. Importantly, these are different users to those who took part in the formative eveluation and had no previous experience of the software being evaluated. This is to ensure that their evaluation is not influenced by their previous experience and knowledge of previous versions of the software. Their comments and feedback are then analysed, which may result in a decision to make further changes to the software and repeat the summative evaluation. Even if these comments do not result in the creation of a new version, they provide valuable material to be included in the user manual, communications and recommendations to the user. 

Requirements for usability testing of medical software 

The usability engineering process described above and used by Graylight is in accordance with EN 62366-1:2015, which contains quite detailed requirements for the work to be carried out in this area and the documentation to be created. It instructs manufacturers to pay particular attention to the risks posed to patient safety by user error or misuse of the device. From this perspective, the requirements to:  

  • planning and documenting activities related to usability work;
  • effectively carry out the activities in accordance with pre-established plans;
  • usability activities should focus on all parts of the software medical device life-cycle, i.e. from its design, manufacture, installation, use, to its removal from the user’s resources (computer or server); 
  • the focus of the manufacturer on eliminating the sources of potential errors already at the stage of operation of the software itself, and only if this is not possible, incorporating warnings and messages for the user;
  • readability of the interface and clarity of messages intended for the user;
  • adapting the type of usability activities to the specific software and the risk to patient and user safety associated with its misuse.

Summary 

The usability of a medical device is a very important aspect that significantly affects patient safety. This is particularly important in the case of software. In this type of medical device, the user reads and analyses results, makes modifications and generates reports via an interface. Therefore, its readability and comprehensibility for the user are crucial in order for the user to correctly interpret the information provided and to delegate the desired tasks to the software. A lot of ambiguity can occur in the human-software communication line, which can result in a risk for the patient, e.g. through misinterpretation of results or lack thereof. Work on software usability aims to minimise the likelihood and impact of such undesirable situations. 

 At GLI, the safety and usability of medical software is a priority for us. For this reason, we have implemented a process in accordance with EN 62366-1:2015, which includes, among other things, usability testing.   

To summarise our medical software usability activities are:  

  • implementation of a process in accordance with EN 62366-1:2015;
  • use of modern usability testing methods;
  • collaboration with end users.

 Our customers can be confident that the medical software we develop is safe and meets their needs. 

References and sources 

EN 62366-1:2015 Medical devices – Part 1: Application of usability engineering to medical devices