Songbird

The Concept

Songbird began from a simple question: what if reading a book could be a fully immersive, multisensory experience? A physical book that, when seen through a phone or tablet camera, comes alive — characters animate from the illustrations, songs pour from the page, and the story expands into the room around the reader.

The book works on two levels. As a traditional reading experience, it stands alone — a complete, illustrated story for children. Through the AR layer, it becomes something else: a performance space where Catherine's vocal interpretations of each character bring emotional depth, and Luis's animations bring visual drama.

Creative Collaboration

This project required unusually close collaboration between Catherine's performance work and Luis's technical implementation. Each character needed to feel continuous across its visual animation and vocal interpretation — the same personality expressed through both movement and voice.

Catherine's Contribution

• Character voices — each with distinct vocal personality and emotional range
• Narrative singing that advances the story
• Musical compositions that accompany animated sequences
• Vocal performance recorded at professional quality for spatial audio integration

Luis's Contribution

• AR framework development (image tracking, scene management)
• Custom character rigging and animation
• Spatial audio system — 3D positioned sound that follows characters
• Cross-platform implementation: iOS and Android
• Offline capability so the core experience works without internet

AR Technical System

The system uses computer vision to recognise specific pages and illustrations, triggering corresponding 3D animations and audio.

Recognition and Tracking

• High-precision image tracking for individual book pages
• Robust tracking under varying lighting conditions
• Multiple simultaneous target recognition
• Smooth scene transitions as pages turn

Audio and Animation

• Stylised 3D rendering optimised for mobile AR performance
• Real-time lighting adaptation to match physical environment
• Spatial audio positioning — sound moves with animated characters
• Adaptive playback responding to reader engagement

Educational Design

The project was designed with learning outcomes in mind. Extended engagement with story content, exposure to vocal performance and musical storytelling, and an accessible introduction to augmented reality technology make the project valuable beyond entertainment.

Initial testing revealed significantly longer reading sessions, high return rates, and children reporting inspiration to create their own stories and songs.