scroll

Mersive Technologies

Restructuring screen capturing mobile app

Our client aimed to revamp their existing screen-capturing mobile app to regain availability on the Google Store and update it for the App Store. They provided us with an external device (the pod) that needed to be connected to a screen (TV or computer display). This setup would allow multiple users to share their mobile device screens simultaneously on the connected display. The goal was to create a seamless and reliable screen-sharing experience that could be used in various business and educational environments, enhancing collaboration and productivity.

Expertise

Mobile development

Frontend development

Programming native functionalities

QA

Industry

Retail

Timeline

2 months

Technology

React

React Native 

webRTC

Expo

Zustand

JavaScript

JSX

Typescript

Swift 

Kotlin

Java

The challenge

Our team faced several significant challenges during the project. 

Firstly, we needed to navigate the strict permissions requirements for screen recording on both Android and iOS platforms. For Android, the app needed to record the screen and audio, whereas iOS only permitted screen recording for third-party software, posing an initial obstacle.

The disparity in screen recording capabilities between iOS and Android added another layer of complexity. With the introduction of Android version 14, permissions for MediaProjection were altered significantly. If we requested screen recording permissions before displaying a notification, the result was a black screen recording. Conversely, if we displayed the notification before obtaining recording consent, the app would crash onto the home screen.

The restrictions on the iOS side were equally challenging. iOS generally does not allow apps to remain active in the background, which poses a significant limitation for continuous screen recording. Additionally, iOS does not permit audio capture from the screen, further complicating our goal of providing a seamless screen-sharing experience.

The challenge
Requirements
Challenge
Process
Technical Requirements

For Android, we successfully used a native library for WebRTC to record the screen without issues. The solution to the permissions issue discussed in the challenges section involved a clever workaround. During the app's first use, we displayed the recording consent twice: first to get the initial permission, then clearing the recorded stream, displaying a notification, and finally requesting recording permission again. This double-consent approach resolved the issue, as native WebRTC/notifee libraries had not yet addressed it. 

On the other hand, iOS allowed screen recording without significant hurdles, but we needed to create an extension for the existing app using a native screen recording module. Since the main app could not predict whether the user had started recording or simply closed the modal window of the native module, we implemented a workaround. Users could not exit the screen-sharing interface until native sharing had started, ensuring smooth operation. The same modal window allowed users to stop screen sharing.

Additionally, iOS typically does not permit apps to run in the background for extended periods. As a result, streaming would last only up to 15 seconds after the app transitioned to the background. To circumvent this, we implemented an artificial VoIP call connection. When users initiated screen sharing, we triggered a VoIP call, automatically placing it on hold and muting it. This kept the app active in the background. Users saw the call interface in the app's background and could end screen sharing by disconnecting the call.

Outcome?

Our innovative solutions and dedication to overcoming technical challenges paid off. The revamped app became a top performer, reaching number 7 on the bestsellers list in the business category on Google Play.

The Challenge

Our journey was meticulously planned and executed, ensuring we met all client requirements while overcoming the unique challenges of both Android and iOS platforms.

Planning:

The planning phase began with a detailed scope definition based on the client's requirements and user flow. We created structured tasks in Jira to keep the project on track and established weekly meetings to ensure continuous communication and alignment with the client. Additionally, we coordinated the logistics of picking up the physical devices from the client's office, which were essential for testing and development.

Execution:

We kicked off the execution phase by setting up all necessary environments and configuring Firebase, laying the groundwork for seamless development. Our team then began coding the design screens for both smartphones and tablets, ensuring a responsive and user-friendly interface.

Developing custom solutions for iOS required in-depth research and innovation to navigate the platform's strict permissions and background operation limitations, and we successfully accomplished this.

As development progressed, we also focused on understanding the requirements for publishing the app in the app stores. This involved creating marketing materials, instructional videos demonstrating how the device works, comprehensive app descriptions, and test versions.

Our Process

what partners say about us?

The code and the work were good quality and really what we were looking for. They were able to bring technical design thinking to the project. Project management was tight and I always knew what was happening.
Daniel Kiyoi
CEO & Founder Magic Dust
Apptension was flexible and professional. When I needed to quickly add capacity, it took a week or two at worst - often days. The cooperation enabled me to slowly scale up my own IT team, and the company was very helpful until the last moments of the transition.
Mateusz Oleksiuk
CEO LESS_
The technical creativity delivered by the team at Apptension was invaluable in the formation of our mixed reality start-up Hyper.
Nathan Sparshott
Co-Founder & CEO of Hyper
The SaaS Boilerplate Apptension built was a huge reason we were so successful, because all these little seemingly unrelated tasks and integrations needed to happen. If we had been working with anybody else, It would have probably taken months to do the same work.
Kwame Nyanning
CEO blkbx.
We needed their help to build the backend services and capabilities needed to deliver a production level on demand. Their ability to produce high-quality work with consistency while dealing with a new type of project was impressive.
Kelly O’Conor
Product Lead, Siberia
The project’s success has resulted in a long-term partnership for design and development. Apptension is a fantastic partner, who is willing to go above and beyond in order to deliver what the client needs.
Catarina Rocha
p(r)oud solutions
Apptension fared well in our project, working with our bespoke CMS and complex requirements. The designs were implemented well and the schedule was kept tight.
Christian Marc Schmidt
Partner at Schema
Looking for similar outcomes?

recent case studies

Mersive Technologies
Restructuring screen capturing mobile app
About
Our client aimed to revamp their existing screen-capturing mobile app to regain availability on the Google Store and update it for the App Store. They provided us with an external device (the pod) that needed to be connected to a screen (TV or computer display). This setup would allow multiple users to share their mobile device screens simultaneously on the connected display. The goal was to create a seamless and reliable screen-sharing experience that could be used in various business and educational environments, enhancing collaboration and productivity.

Expertise

Mobile development

Frontend development

Programming native functionalities

QA

Industry

Retail

Timeline

2 months

django
flask
react native
node.js
next.js
python
gatsby.js
vue.js
react.js
javascript
aws
docker
serverless
figma
photoshop
illustrator
after effects
firebase
blender
graphql
nuxt.js
Scss
typescript
apollo
saas boilerplate
styled components
D3
bigcommerce
bigquery
Wagtail CMS
Django Rest
redux
gsap
i18next
kubernetes
Google Cloud
Platform
rabbitmq
celery
Before
After
Prerequisites
The challenge

Our team faced several significant challenges during the project. 

Firstly, we needed to navigate the strict permissions requirements for screen recording on both Android and iOS platforms. For Android, the app needed to record the screen and audio, whereas iOS only permitted screen recording for third-party software, posing an initial obstacle.

The disparity in screen recording capabilities between iOS and Android added another layer of complexity. With the introduction of Android version 14, permissions for MediaProjection were altered significantly. If we requested screen recording permissions before displaying a notification, the result was a black screen recording. Conversely, if we displayed the notification before obtaining recording consent, the app would crash onto the home screen.

The restrictions on the iOS side were equally challenging. iOS generally does not allow apps to remain active in the background, which poses a significant limitation for continuous screen recording. Additionally, iOS does not permit audio capture from the screen, further complicating our goal of providing a seamless screen-sharing experience.

Challenge
The process

Our journey was meticulously planned and executed, ensuring we met all client requirements while overcoming the unique challenges of both Android and iOS platforms.

Planning:

The planning phase began with a detailed scope definition based on the client's requirements and user flow. We created structured tasks in Jira to keep the project on track and established weekly meetings to ensure continuous communication and alignment with the client. Additionally, we coordinated the logistics of picking up the physical devices from the client's office, which were essential for testing and development.

Execution:

We kicked off the execution phase by setting up all necessary environments and configuring Firebase, laying the groundwork for seamless development. Our team then began coding the design screens for both smartphones and tablets, ensuring a responsive and user-friendly interface.

Developing custom solutions for iOS required in-depth research and innovation to navigate the platform's strict permissions and background operation limitations, and we successfully accomplished this.

As development progressed, we also focused on understanding the requirements for publishing the app in the app stores. This involved creating marketing materials, instructional videos demonstrating how the device works, comprehensive app descriptions, and test versions.

Requirements
The solution

For Android, we successfully used a native library for WebRTC to record the screen without issues. The solution to the permissions issue discussed in the challenges section involved a clever workaround. During the app's first use, we displayed the recording consent twice: first to get the initial permission, then clearing the recorded stream, displaying a notification, and finally requesting recording permission again. This double-consent approach resolved the issue, as native WebRTC/notifee libraries had not yet addressed it. 

On the other hand, iOS allowed screen recording without significant hurdles, but we needed to create an extension for the existing app using a native screen recording module. Since the main app could not predict whether the user had started recording or simply closed the modal window of the native module, we implemented a workaround. Users could not exit the screen-sharing interface until native sharing had started, ensuring smooth operation. The same modal window allowed users to stop screen sharing.

Additionally, iOS typically does not permit apps to run in the background for extended periods. As a result, streaming would last only up to 15 seconds after the app transitioned to the background. To circumvent this, we implemented an artificial VoIP call connection. When users initiated screen sharing, we triggered a VoIP call, automatically placing it on hold and muting it. This kept the app active in the background. Users saw the call interface in the app's background and could end screen sharing by disconnecting the call.

Outcome?

Our innovative solutions and dedication to overcoming technical challenges paid off. The revamped app became a top performer, reaching number 7 on the bestsellers list in the business category on Google Play.

Roadmap
Solution

WHAT The CLIENT Said ABOUT this project?

what partners say about us?

The code and the work were good quality and really what we were looking for. They were able to bring technical design thinking to the project. Project management was tight and I always knew what was happening.
Daniel Kiyoi
CEO & Founder Magic Dust
Apptension was flexible and professional. When I needed to quickly add capacity, it took a week or two at worst - often days. The cooperation enabled me to slowly scale up my own IT team, and the company was very helpful until the last moments of the transition.
Mateusz Oleksiuk
CEO LESS_
The technical creativity delivered by the team at Apptension was invaluable in the formation of our mixed reality start-up Hyper.
Nathan Sparshott
Co-Founder & CEO of Hyper
The SaaS Boilerplate Apptension built was a huge reason we were so successful, because all these little seemingly unrelated tasks and integrations needed to happen. If we had been working with anybody else, It would have probably taken months to do the same work.
Kwame Nyanning
CEO blkbx.
We needed their help to build the backend services and capabilities needed to deliver a production level on demand. Their ability to produce high-quality work with consistency while dealing with a new type of project was impressive.
Kelly O’Conor
Product Lead, Siberia
The project’s success has resulted in a long-term partnership for design and development. Apptension is a fantastic partner, who is willing to go above and beyond in order to deliver what the client needs.
Catarina Rocha
p(r)oud solutions
Apptension fared well in our project, working with our bespoke CMS and complex requirements. The designs were implemented well and the schedule was kept tight.
Christian Marc Schmidt
Partner at Schema

other success stories

Conversational AI Avatar
Building a real-time conversational AI avatar
Riyadh Air
Building a microsite with a 3D plane model
Feelit
A SaaS mobile app designed to bridge the gap between event attendees and event makers
Creative agency
Developing an eCommerce website for a luxury brand
Mojo
A website solution for an AI-based fertility tester
mTab
Building a full-stack data analysis, visualization and storytelling platform