Quality assurance in a medical project

Understanding the needs of the industry is crucial when it comes to quality assurance in a medical project. What is crucial here is to meet the relevant norms and standards, while taking care of the attractiveness of the system. Ensuring all these conditions are met is not an easy task and validating the correctness of such a system depends on the cooperation of many people.

Quality Assurance (QA) may include the concept of a process or role where the scope of the role is not clearly defined. There are many definitions, but in this post I would like to focus on the role of the “engineer”. David Krala describes it as follows: “Quality Assurance engineers make absolutely sure that the developed software operates on the highest quality level and doesn’t contain any elements discouraging from using the service”. [1]. This is what I deal with every day in quality assurance in a medical project and I hope to explain in this article what exactly this work is about, what benefits it can bring, and where to gain its deeper understanding.

Is the product adequate?

The answer to this question results from testing the system. It enables verification and validation, which in the medical system means:

• Verification – checking if the product meets the norms, standards and specifications. It is usually carried out in the test environment.
• Validation – checking if the product meets the expectations of the stakeholders or the end users. It often involves testing the system in the circumstances in which users will use it (for example, tests in a medical device facility).

Quality assurance in medical project – testing

The level of complexity of the tests depends on the product we are dealing with. The more a product can put the health or life of a patient at risk, the more demanding the testing process is. A web application that displays the patient’s test results might have to meet fewer standards of quality assurance in medical project than the one used to control the X-ray dose. For this reason, the extent of the required actions and processes determine depending on the class of a project. The second important issue is to define the responsibility for product testing in a team. “Quality is everyone’s responsibility” (W. Edwards Deming). Therefore, although the main role for testing the system is that of a QA, the responsibility for testing a particular aspect should be defined for each individual in the team respectively.

Let’s look at the means of system verification:

• Testing – designing, managing and performing tests. Tests design on the basis of system requirements and standards that the project must meet. This process includes both the work of a QA engineer (mainly tests on an environment similar to the one in which the system will be used) and programmers (mainly unit tests).
• Code review – a review of source code performed by other team members, usually programmers. It allows to eliminate errors, implementation inaccuracies or missing areas. It can also contribute to improving the quality of the code, which makes it more transparent and more effective.
• Documentation review – review of documents such as system requirements, risk analysis or system architecture design. As medical projects require a lot of formalization, good preparation of these documents often turns out to be one of the key aspects that facilitate further work.

It is good practice to engage Quality Assurance engineers from the first phases of the medical project. This person can take part in verifying product specifications, user requirements or risk analysis. This allows us to notice inaccuracies, missing elements, shortcomings or erroneous assumptions at the very beginning of the project. Additionally, QA is able to gain the knowledge requires to design the tests much faster. Another outcome is the ability to start testing faster.

Once the product specification is complete

You can start designing tests based on it. Important elements to consider are:

• Test data – data used during system testing.
• Reference tools – emulators, alternative solutions, examples from trusted sources.
• Automation – it is worth considering the automation of system functions, which result from medical standards. These standards are rarely updated, and their verification often
  requires a lot of time and meticulousness.

Test data

Access to medical test data is a common cause for concern. One of the biggest problems is their availability and dissemination. Data presenting actual patient’s personal data must be anonymised, i.e., sensitive data must be deleted, and it is often difficult to obtain such data because the patient’s consent is required.

It is worthwhile to obtain data from systems and devices with which the created software will integrate. In this way, you can verify how well the application will handle the processing of data with which it will work daily.

You can also prepare the test data yourself, but often it is a process that requires a lot of time and attention. Basically, there are libraries that allow you to change the data in such a way as to adjust it to the needs of the tests. Although medical standards provide a fairly accurate specification of how the data is to appear, in the real world there are simplifications or certain parts of the standards are not used. There are also ambiguities and interpretation problems. New versions of the standards from quality assurance in medical project also do not take into account outdated requirements, which in some cases, however, should be provided.

Data from different devices and systems

Many medical facilities have obsolete hardware and replacing its software with newer versions is impossible. Without taking into account the old requirements of standards in the system, communication with older devices may be impossible. Creating test data on our own may expose us to omitting these issues, which in turn may bring about troublesome moments during system integration with external software used in the hospital.

Usually, the most beneficial option is to have anonymised data from different devices/systems, and a tool to view and modify it. Working on the data in circulation will allow to meet real problems occurring in the medical systems environment and editing tools will generate missing data or unusual cases.

Where to look for test data and tools to generate it?

• Open-source libraries implementing a given medical standard. For example, PyDicom and Pynetdicom python libraries have some interesting test data, and DicomLibrary provides some demonstration images.

• Medical data collections for testing, algorithm development (e.g. BraTS dataset).

• Medical document collections.

• Cooperation with a medical facility/product/customer.

Security

Medical data should be subject to special attention. Protect it in terms of preventing leakage or misrepresentation it is important thing. In the case of systems operating in a hospital network, a lot of dangers are eliminated, or a lot of responsibility is redirected to IT administration, but it should never be assumed that this completely solves the problems. If the team lacks a person with high-security qualifications, we have to think about testing by a qualified external team.

Performance and quality assurance in a medical project

The performance of an application can be measured in comparison to alternative solutions. Solutions which is determined by the experience of the software developers or end users.

Furthmore, each system has its specificity which is the focus of performance testing. For example, a system for assigning and administering drugs in a hospital facility can be based on optimizing database queries. In the case of the implementation of a server collecting radiological images in a hospital, the architecture prepared for handling multiple connections (workload) proves important. Handling memory/system performance leaks, which can rule out server failure is also important.

Another example may be applications supporting medical diagnostics, which will not do their job when the execution of algorithms takes too much time in relation to the benefits.

Common medical standards

Quality assurance in a medical project requires knowledge of project specifications. As the standards for a medical project impose numerous requirements, it is necessary to know them within the scope of the system.

Of course, it is not necessary to know everything by heart, but it is worth knowing where to look for information. It is worth mentioning the following standards for quality assurance in medical projects:

• DICOM – the least recognized standard, but very important. Generally, use mainly in radiology, unifies the way of data transmishion, working with, storing on a disk and browsing.
• HL7 – Health Sever International. A standard used in medical environments to exchange clinical, administrative and financial information between healthcare information systems.
• ISO and IEC standards for the design of medical products. We should pay attention to such standards as:

1. 1. 1. 1. ISO 13485 – Quality System for the medical devices industry
         2. ISO 14971 – Risk Management for medical devices
         3. IEC 62304 – Software lifecycle for medical devices
         4. IEC 62366 – Usability in medical devices
         5. IEC 60601-1 – Programmable electric medical devices

When working with standards it’s worth looking for their briefs or prepare them yourself. They should contain key requirements, concepts, essential information. This makes it much easier to work in the project, design test cases as well as simplifies the testing itself.

Quality assurance in a medical project – improvement

Testing the design and assessing its quality does not in itself make the medical product attractive to the end user and does not make it meet all necessary standards. To get closer to this goal, it is necessary to analyze the test results and consider what action will bring the greatest benefit. For the role of a QA engineer to be effective in improving quality, the cooperation of the whole team is therefore required.

The work of a QA can help:

• determine whether the product corresponds to the stakeholder, and the reality of the proper institutions
• focus on systems and the process.
• analyse the system delivery process.
• encourage the team to solve problems and improve quality of the project.
• create “lessons learned”, especially for unobvious implementations of the standard, exceptions, ways of using reference tools, creating the project tutorials.

References:

[1] https://tsh.io/blog/why-do-you-need-quality-assurance-engineer/ [09.03.2020]

[2]  HL7 101: A Beginner’s Guide [09.03.2020]

[3] https://blog.cm-dm.com/post/2011/11/01/ISO-and-IEC-standards-explained-to-software-engineers-and-quality-managers [09.03.2020]

Contact us if you have any questions!

See the prevoius post: Software outsourcing partner and organizational culture – success factors