Augmented Reality (AR) shines when it forms meaningful links with the real world, and in this article, we'll explore how to do so by giving 3D life to a coloring book.
🖼️ Augmented reality coloring books.
Spawning 3D objects in augmented reality often feels very basic there’s usually not much to do besides looking at them, and that’s because their only relation to the real world is their position. Establishing multiple relations between virtual and real is where AR apps need to focus.
This concept was put to the test when a Peruvian school approached us to create an AR coloring book app in 2 weeks. The idea was simple: have kids color pages and then use AR to transfer the colors into animated 3D objects. This approach allowed us to go beyond merely positioning virtual objects in the real world; it connected the act of coloring with the digital realm.
Design
With a very tight schedule, we opted for simple designs for the user interface (UI) and a user experience (UX) stripped down to the bare minimum. A key consideration was having an area on the coloring pages that wouldn’t change to allow the image recognition to work out. Aided by DALL-E, we created richly detailed frames that wouldn’t be colored, serving as a strong and distinct element for image recognition.
🖥️ User interface and coloring pages.
Development
The primary development package used was Vuforia, a software development kit (SDK) specialized in augmented reality (AR). Ricardo Villanueva, XR developer, guides us through this process and shares some of the key factors in choosing Vuforia:
△ Object and Marker Recognition: Vuforia offers a robust object and marker recognition system. In this case, Vuforia’s detection demonstrated superior image recognition and tracking compared to other SDKs like EasyAR, Maxst AR, and AR Foundation.
△ Broad Device Compatibility: Vuforia supports slightly more devices than AR Foundation, including a wide array of smartphones and tablets, enhancing its user accessibility.
△ Cross-Platform Development: It enables AR app development for various platforms, including iOS, Android, and select Windows devices.
△ Easy Integration: Finally, it easily integrates with Unity through a package manager and has extensive documentation and an active developer community.
Additionally, the application was based on an augmented reality coloring library developed by the Korean company AirAR. This library allows the detection of colors from images captured by the camera and applies them to a Unity material, which can then be applied to virtual 3D objects.
🎨 AR coloring in action.
How it Works
The step-by-step process is as follows:
1. The device’s camera detects the coloring book page, which has been set as the marker for spawning the 3D objects without colors.
2. Users press the “Color” button, and the app uses color mapping to detect and process the image’s colors captured by the camera.
3. The detected colors are applied to a texture using UV mapping, where the 3D model is represented in 2D to determine which sectors of the model get painted.
4. Finally, the texture is applied to the 3D model, displaying the colors of the color page. The number of vertices and polygons was kept low to avoid potential issues or lag.
🎨 More AR coloring fun.
An Immersive Future for Education
The enthusiastic response from the students at the school was a testament to the potential of the intersection of technology and education. Their deep involvement and joy as they watched their colored pages come to life in 3D models highlighted an important lesson: technology like AR can greatly enhance learning by making it interactive, immersive, and fun.
📈 Engagement with the activity was greatly enhanced.
Looking forward, the potential applications of AR in educational settings are vast and largely untapped. History lessons in which students can explore historical civilizations in 3D, or biology classes in which they can interact with complicated ecosystems, all powered by AR, promise to transform traditional learning into an immersive, interactive journey.
⚛️ Boom!
