Automatic and seamless connections for brain aneurysm mesh
By Piotr Fröhlich
Earlier this year, we were faced with a great challenge that consisted of several other, hidden challenges. For our client – BIOMODEX® – we were supposed to, automatically, connect the inflows and outflows of a segmented 3D model of a brain aneurysm to points in a 3D space – and do so seamlessly.
Let’s talk about
our 3D modelling solutions!
To add on top of that challenge, we only had mesh as an input reference (no indication as to where the endings of the vessel were, where the aneurysm was located, etc.). What is more, we had to follow specific requirements for where such a cutting point can be located in relation to the aneurysm and the 3D points. Let’s also talk about what exactly is the segmented 3D model of a brain aneurysm – firstly, let’s see some visuals, fig. 1. As you can see, that is a mesh, created from a medical image of choice that is an exact model of a person’s organ, vessel or whatever is being segmented. In our case, this is an aneurysm with vessels that are flowing in and out of it. Since we are operating in a brain, that mesh is composed of micro-triangles with vessel diameter being up to 5 [mm]. To see that in scale, refer to fig. 2. This scale makes it even more difficult due to the small margin of error while we operate in such a microscale models where +/- 1 [mm] can be almost 100% of the vessel diameter.
We divided this solution into several steps that will be described in further paragraphs. We start by calculating the centerlines of the model – this enables us to know several things about the presented mesh, such as diameters along the centerline points. Then we divide the provided mesh into separate branches, so that we are able to establish which part of the mesh is an aneurysm and which part is an ending of the model. Then we proceed to the optimization part of the automatic connection – when we decide which point to choose for cutting the branch and connecting it to the 3D point in space. Lastly, we cut the branches and seamlessly connect them to the 3D points. This division was also enriched with safety checks. Since we are working in the environment of a medical product, such safety checks must be introduced to ensure safety and compliance with the norms. All of that was done in order to introduce the optimal way to connect brain aneurysm branches with as little human interaction as possible, thus minimizing the chance of a man-made mistake.
Figure 2: Segmented aneurysm model and a sphere with 5[mm] diameter
To summarize, the steps are:
- Centerlines calculation
- Inflow confirmation
- Splitting into branches
- Calculation of aneurysm location
- Aneurysm confirmation
- Selection of cutting points
- Cutting and connecting the branches in cutting points
2.1 Centerlines calculation
Centerlines are a set of points leading from point A to point B going through the center of the model. As you can see in fig. 3, the centerlines calculated for the model incorporate the aneurysm and go through the middle of all the branches that go in and out of the aneurysm. What is more, centerlines give us a lot of insight into the topology of the model – for each point on the centerline we obtain the information on what is the diameter of the vessel at that point.
Figure 3: Calculated centerlines for the mesh
Figure 4: Mesh after splitting into branches
That is a lot of information – we obtain it by utilizing vmtk module, which collects this information by pulling spheres through the mesh. This is actually what we obtain as the branch radius – a maximal radius of an inscribed sphere. Even if we have very small, thin branches, this information is still enough because there is a separate centerline for that small branch. Due to the fact that the vmtk does not provide a stable enough solution, we introduced a set of fallback mechanisms to obtain the centerlines. Upon detecting an incorrect centerline (e.g. consisting only of the first and the last point) we fall back on another algorithm to calculate the centerlines. That algorithm is based on the network extraction and provides more consistent results, with the computation time being the tradeoff. Initially, we introduced a raycast based algorithm to fall back on, but due to the biological nature of the provided meshes it did not work with satisfactory precision – it was the fastest alternative though.
2.2 First safety check
After the initial centerline calculation, a safety check must be introduced to confirm that the inflow was estimated in the correct place. This is one of two human interactions that our algorithm requires (both are optional and are introduced solely to increase safety).
2.3 Branch splitting
The centerlines alone are not enough to estimate where exactly the aneurysm is. We characterized the aneurysm to be short and thick branch – this is why we decided to use these features in our heuristics to estimate where the aneurysm is located. We need to split the mesh and the centerlines into branches to calculate the placement of the aneurysm – to do so we are dividing them into groups – where one group is a piece of mesh that is in between two crossroads, like in fig. 4. Let’s focus on the blue piece of the separated mesh from fig. 4 – we can see that this piece represents a piece of the mesh in between the aneurysm and the bifurcation to two branches. We can also observe that the aneurysm is perfectly marked as a separate group (dark green in upper right side corner).
2.4 Calculation of brain aneurysm location
The aneurysm has a lot of distinctive features that can separate it from the regular branches in the provided model. Since we already have the mesh split into branches – we will utilize that as well as some features of the aneurysm – we select the thickest and shortest branch and mark it as an aneurysm. If there is no aneurysm in the mesh, it simply does not pass the proposed threshold, thus not estimating a short and thick regular branch as the aneurysm.
2.5 Second safety check
After estimation of brain aneurysm placement, a second safety check must be introduced to confirm that the aneurysm is indeed in the correct place.
2.6 Selection of cutting points
Up to this point, we have calculated where the endings of the model are (with the division into inflow and the rest). We also know which one of the branches are not in-between any crossroads. And of course where the aneurysm is. We now have to calculate the optimal way of connecting the endings to the points in 3D space in such a way that they are not in collision with the model, are far enough from the aneurysm and are as short as possible. Additionally, the connection cannot intersect with itself. As we see in fig. 5, the points in 3D space (in pink) are located as shown on the animation. The algorithm will now try to select a point on each of the branches with the endings to create an optimal tube connection between this selected point and the point in 3D space. It will do so by iterating over N points on the branch and calculating the tube (with all the restrictions and optimizations) for all of those points and selecting the best options for all the endings. This way, we know that the created tubes are the best possible tubes that can be created in this setting.
2.7 Cutting and connecting the branches in cutting points
Having the cutting points calculated for the optimal tubes, we cut the mesh using planes that are perpendicular to the tangent vector created by consecutive points of the centerline in the cutting point. After the branches were cut in the correct places, we used our own algorithm or connection sweeping (that will be described in another article) to create a seamless connection between the shape of the branch and the created circular tube. The result can be observed in fig. 6.
3. Conclusion: automatic and seamless connections for brain aneurysm mesh
As we described in the article, we were able to solve the problem at hand. Our algorithm can generate seamless tube connections between the segmented 3D model of brain aneurysm and the points in 3D space. Utilization of heuristics and algorithms enabled us to provide a way to lower man-made mistakes and profoundly increase the speed of creating such connections. We believe that the challenges we needed to overcome while tackling this problem made us a much stronger team in terms of complex problem solving, mesh-related operations and risk management.
See the previous post: Our contribution to ITK-SNAP open-source software.