Quality assurance in a medical project
Understanding the needs of the industry is crucial when it comes to quality assurance in a medical project. What is crucial here is to meet the relevant norms and standards, while taking care of the attractiveness of the system. Ensuring all these conditions are met is not an easy task and validating the correctness of such a system depends on the cooperation of many people.
Quality Assurance (QA) may include the concept of a process or role where the scope of the role is not clearly defined. There are many definitions, but in this post I would like to focus on the role of the “engineer”. David Krala describes it as follows: “Quality Assurance engineers make absolutely sure that the developed software operates on the highest quality level and doesn’t contain any elements discouraging from using the service”. . This is what I deal with every day in quality assurance in a medical project and I hope to explain in this article what exactly this work is about, what benefits it can bring, and where to gain its deeper understanding.
Is the product adequate?
The answer to this question results from testing the system. It enables verification and validation, which in the medical system means:
- Verification – checking if the product meets the norms, standards and specifications. It is usually carried out on the test environment.
- Validation – checking if the product meets the expectations of the stakeholder or the end users. It often involves testing the system in the circumstances in which it will be used (for example, tests in a medical facility).
The level of complexity of the tests depends on the product we are dealing with. The more a product can put the health or life of a patient at risk, the more demanding the testing process is. A web application that displays the patient’s test results might have to meet fewer standards than the one used to control the X-ray dose. For this reason, the extent of the required actions and processes is determined depending on the class of a project. The second important issue is to define the responsibility for product testing in a team. “Quality is everyone’s responsibility”. (W. Edwards Deming), therefore, although the main role for testing the system is that of a QA, the responsibility for testing a particular aspect should be defined for each individual in the team respectively.
Let’s look at the means of system verification:
- Testing – designing, managing and performing tests. Tests are designed on the basis of system requirements and standards that the project must meet. This process includes both the work of a QA engineer (mainly tests on an environment similar to the one in which the system will be used) and programmers (mainly unit tests).
- Code review – a review of source code performed by other team members, usually programmers. It allows to eliminate errors, implementation inaccuracies or missing areas. It can also contribute to improving the quality of the code, which makes it more transparent and more effective.
- Documentation review – review of documents such as system requirements, risk analysis or system architecture design. As medical projects require a lot of formalization, good preparation of these documents often turns out to be one of the key aspects that facilitate further work.
It is good practice to engage Quality Assurance engineers from the very first phases of the medical project. This person can take part in verifying product specifications, user requirements or risk analysis. This allows inaccuracies, missing elements, shortcomings or erroneous assumptions to be noticed from the very beginning of the project. Additionally, QA is able to gain the knowledge that is needed to design the tests much faster. Another outcome is the ability to start testing faster.
Once the product specification is complete, you can start designing tests based on it. Important elements to consider are:
- Test data – data used during system testing.
- Reference tools – emulators, alternative solutions, examples from trusted sources.
- Automation – it is worth considering the automation of system functions, which result from medical standards. These standards are rarely updated, and their verification often
requires a lot of time and meticulousness.
Access to medical test data is a common cause for concern. One of the biggest problems is their availability and dissemination. Data presenting actual patient’s personal data must be anonymised, i.e., sensitive data must be deleted, and it is often difficult to obtain such data because the patient’s consent is required.
It is worthwhile to obtain data from systems and devices with which the created software will integrate. In this way you can verify how well the application will handle the processing of data with which it will work on a daily basis.
You can also prepare the test data yourself, but often it is a process that requires a lot of time and attention. There are libraries that allow you to change the data in such a way as to adjust it to the needs of the tests. Although medical standards provide a fairly accurate specification of how the data is to appear, in the real world there are simplifications or certain parts of the standards are not used. There are also ambiguities and interpretation problems. New versions of the standards also do not take into account outdated requirements, which in some cases, however, should be provided.
Many medical facilities are equipped with obsolete hardware and it is impossible to replace its software with newer versions. Without taking into account the old requirements of standards in the system, communication with older devices may be impossible. Creating test data on our own may expose us to omitting these issues, which in turn may bring about troublesome moments during system integration with external software used in the hospital.
Usually the option that brings the most benefits is to have anonymised data from different devices/systems, and a tool to view and modify it. Working on the data in circulation will allow to meet real problems occurring in the medical systems environment and editing tools will generate missing data or unusual cases.
Where to look for test data and tools to generate it?
- Open-source libraries implementing a given medical standard. For example, PyDicom and Pynetdicom python libraries have some interesting test data, and DicomLibrary provides some demonstration images.
Medical data collections for testing, algorithm development (e.g. BraTS dataset).
Medical document collections.
Cooperation with a medical facility/product/customer.
Medical data should be subject to special attention. It is important to protect it in terms of preventing leakage or misrepresentation. In the case of systems operating in a hospital network, a lot of dangers are eliminated, or a lot of responsibility is redirected to IT administration, but it should never be assumed that this completely solves the problems. If the team is lacking a person with high security qualifications, it is worth having it tested by a qualified external team.
Performance and quality assurance in a medical project
The performance of an application can be measured in comparison to alternative solutions, determined by the experience of the software developers or end users.
Each system has its own specificity which is the focus of performance testing. For example, a system for assigning and administering drugs in a hospital facility can be based on optimizing database queries. In case of implementation of a server collecting radiological images in a hospital, the architecture prepared for handling multiple connections (workload) proves important, or handling of memory/system performance leaks, which can rule out server failure.
Another example may be applications supporting medical diagnostics, which will not do their job when the execution of algorithms takes too much time in relation to the benefits.
Common medical standards
Quality assurance in a medical project requires knowledge of project specifications. As the standards for a medical project impose numerous requirements, it is necessary to know them within the scope of the system.
Of course, it is not necessary to know everything by heart, but it is worth knowing where to look for information. It is worth mentioning the following standards for medical projects:
DICOM – the least recognized standard, but very important. It is used mainly in radiology, unifies the way data is transmitted, worked with, stored on a disk and browsed.
HL7 – Health Sever International. A standard used in medical environments to exchange clinical, administrative and financial information between healthcare information systems.
ISO and IEC standards for the design of medical products. Attention should be paid to such standards as:
ISO 13485 – Quality System for medical devices industry
ISO 14971 – Risk Management for medical devices
IEC 62304 – Software lifecycle for medical devices
IEC 62366 – Usability in medical devices
IEC 60601-1 – Programmable electric medical devices
When working with standards it’s worth looking for their briefs or prepare them yourself. They should contain key requirements, concepts, essential information. This makes it much easier to work in the project, design test cases as well as simplifies the testing itself.
From evaluation to improvement – quality assurance in a medical project
Testing the design and assessing its quality does not in itself make the medical product attractive to the end user and does not make it meet all necessary standards. To get closer to this goal, it is necessary to analyze the test results and consider what action will bring the greatest benefit. For the role of a QA engineer to be effective in improving quality, the cooperation of the whole team is therefore required.
The work of a QA can help:
- determine whether the product corresponds to the stakeholder, the reality of the institutions in which it will be used.
- focus on systems and the process.
- analyse the system delivery process.
- encourage the team to solve problems and improve quality of the project.
- create “lessons learned”, especially for unobvious implementations of the standard, exceptions, ways of using reference tools, creating the project tutorials.
 https://tsh.io/blog/why-do-you-need-quality-assurance-engineer/ [09.03.2020]
 HL7 101: A Beginner’s Guide [09.03.2020]
 https://blog.cm-dm.com/post/2011/11/01/ISO-and-IEC-standards-explained-to-software-engineers-and-quality-managers [09.03.2020]
Contact us if you have any questions!
See the prevoius post: Software outsourcing partner and organizational culture – success factors