Usability testing of software as a medical device (SAMD) 

The aim of the efforts and work on the usability of software that is a medical device is to ensure that it is safe for the patient and the user by reducing the risks arising from incorrect or misuse. This means that when focusing on usability, we should focus on the risks that may arise from misuse or user error and their causes. This is a particularly difficult task, as the number of potential sources of user action not in accordance with the intended use of a medical device seems infinite.

These actions may result from ambiguous statements in the instructions for use or displayed messages, software errors, and a non-intuitive and misleading interface. In addition, the attitude of the user himself (the so-called human factor) can be a source of incorrect actions. He or she may be distracted and not read the messages correctly. He may also ignore them, and act in haste or according to previous habits. Medical device usability prioritizes foreseeing, minimizing, and safeguarding against potential software interactions.   

Medical software plays a key role in the treatment and care of patients. To be effective, it must not only be secure but also easy to use. This is why usability tests are so important to help verify that the software as a medical device meets user expectations. Of course, this does not exclude carrying out additional usability activities focusing not on safety, but on user convenience. They are also important because they often have a significant impact on the market success of a product and its popularity among users. When working on the usability of the software we develop, we pay attention to both aspects. We prioritize safety, but we also make sure that the solutions we create are user-friendly and convenient for everyday work. 

The work we carry out on the usability of a medical device can be roughly divided into 2 stages. The first is the formative evaluation carried out during software development. During it, we work with UX (usability) specialists on the individual parts of the interface. We also periodically meet with potential users, present them with our concepts and prototype solutions and collect their comments and suggestions. After analysing these in terms of patient safety and user-friendliness, suggestions for changes to the software are made and, once approved, implemented in subsequent versions. The result of such iterative work is an increasingly clear and intuitive interface, which translates into minimising the risk to the patient associated with user error or misuse.

The second stage of usability testing, called summative evaluation, actively verifies if we’ve minimized risks. Here, potential users directly evaluate the fully developed software to identify usability issues. This formal evaluation involves a larger, more diverse group than the formative stage, ensuring a wider range of perspectives and experiences. Crucially, these users haven’t used the software before, unlike those in formative evaluation.

To prevent bias from past experiences, healthcare professionals evaluate the software without prior knowledge of previous versions. Their feedback is then analysed to identify any needed changes. These revisions may lead to a new software iteration and another evaluation cycle. Even without prompting a new version, the feedback becomes valuable input for the user manual, communication materials, and user recommendations.

The usability engineering process described above and used by Graylight is in accordance with EN 62366-1:2015, which contains quite detailed requirements for the work to be carried out in this area and the documentation to be created. It instructs manufacturers to pay particular attention to the risks posed to patient safety by user error or misuse of the device. From this perspective, the requirements to:  

• planning and documenting activities related to usability work;
• effectively carry out the activities in accordance with pre-established plans;
• usability activities should focus on all parts of the software medical device life-cycle, i.e. from its design, manufacture, installation, use, to its removal from the user’s resources (computer or server); 
• the focus of the manufacturer on eliminating the sources of potential errors already at the stage of operation of the software itself, and only if this is not possible, incorporating warnings and messages for the user;
• readability of the interface and clarity of messages intended for the user;
• adapting the type of usability activities to the specific software and the risk to patient and user safety associated with its misuse.

The usability of a medical device is a very important aspect that significantly affects patient safety. This is particularly important in the case of software. In this type of medical device, the user reads and analyses results, makes modifications and generates reports via an interface. Therefore, its readability and comprehensibility for the user are crucial in order for the user to correctly interpret the information provided and to delegate the desired tasks to the software. A lot of ambiguity can occur in the human-software communication line, which can result in a risk for the patient, e.g. through misinterpretation of results or lack thereof. Work on software usability aims to minimise the likelihood and impact of such undesirable situations. 

 At GLI, the safety and usability of medical software is a priority for us. For this reason, we have implemented a process in accordance with EN 62366-1:2015, which includes, among other things, usability testing.   

To summarise our medical software usability activities are:  

• implementation of a process in accordance with EN 62366-1:2015;
• use of modern usability testing methods;
• collaboration with end users.

 Our customers can be confident that the medical software we develop is safe and meets their needs. 

References:
EN 62366-1:2015 Medical devices – Part 1: Application of usability engineering to medical devices