Mobile Applications in Healthcare

How to support healthcare professionals using AR technology?

Case Study of the Hand Anatomy app we've built

Augmented reality is one of the rising trends in healthcare. However, just as every breakthrough technology, before its widespread adoption, it has to go through an experimental phase, during which new ways of use are discovered and the technology is perfected. For the best results, close cooperation between the tech industry and healthcare professionals is essential. It can benefit both parties: tech companies can receive valuable feedback and get to know the point of view of end-users, and medical personnel can take advantage of the innovations in their daily work.

We have experienced the many benefits of this approach to MedTech product development ourselves, during the creation of an experimental augmented reality app aimed for hand surgeons. Read our case study to learn how the experience was and its outcome!

Coming up with the idea

Having in mind the growing importance of AR technology in healthcare, our research & development team was looking for a way to develop a product that would improve this industry (or at least a part of it). Why healthcare? In a nutshell, we realized that it’s a crucial aspect that affects lives all around the world, and can greatly improve the quality of life of millions.

Maybe we won’t cure cancer in a few weeks, but we can still make a difference for healthcare professionals and their patients at some point, we thought.

The idea itself came from our CEO, who learned from a healthcare professional that an AR app could be of great help in hand surgeries, in particular as learning material. Due to our practical approach to product development based on customer research, we knew that working on the solution for the sake of a pure experiment, without end-user feedback wasn’t an option. We wanted to create an app that would be a real helping hand (pun intended!) for doctors, and we knew that only collaborating directly with healthcare professionals we could ensure that our app responds to their needs and is of real value. That’s why we teamed up with a hand surgeon that supported us along the way, testing the solution, giving feedback, evaluating our work and suggesting improvements.

Choosing the tech stack for augmented reality app development

The technology that plays a crucial role in our augmented reality surgery app is the Unity’s AR Foundation framework which helped us build both iOS and Android applications, using ARKit and AR Core platforms, respectively. Thanks to this framework we were able to make sure we support both operating systems, which was quite a demanding task due to completely different approaches to AR.

Choosing an adequate hand tracking framework was actually one of the major challenges we had to face during the development phase. We also considered using Google’s MediaPipe hand tracking library: a cross-platform framework based on machine learning. However, although it works great for purposes such as understanding sign language, it turned out to be still in an early development stage and wasn’t sufficient for our project in terms of performance, which would negatively impact user experience, and it didn’t integrate well with the app we’d developed so far.

Another hand tracking library that we evaluated was Manomotion, but its pricing scheme that we considered too complicated for our purposes stopped us from evaluating it further. Eventually, the feedback gathered from end-users made us realize that the quality of hand tracking and animations should not be the primary concern: it turned out that a good user experience achieved with the current AR technologies was way more important.

Besides the issues with hand tracking, we also faced a challenge with finding an anatomically accurate human hand model. After evaluating our options, we chose to purchase one, but even so, we had to spend more time than expected trying to make sure it renders correctly and nicely inside our app.

The outcome? A fully functioning augmented reality surgery app!

After all the hard work, we finally managed to deliver a fully functioning MVP within the expected deadline. The application can be a convenient learning material for doctors in practice or even experienced surgeons and support their work. Now we are looking forward to keep collecting more feedback from end-users and continue improving our MVP by implementing new features.

Our highly hands-on and user-oriented approach to AR healthcare app development proved to bring value to medical professionals because it ensures achieving the product-market fit and creating a solution that truly makes a difference for doctors and patients.

Do you like our approach to AR app development? Learn more about our expertise in designing seamless and purposeful interactive experiences or find out how we would approach your idea: estimate your own project!

Other projects

Data visualization software for New Economy Forum.
Trending mobile app for selling and buying pre-owned clothes