• A Look at Ocarina - Interview with Smule CEO Jeff Smith

    Smule last Thursday released Ocarina, "the first true musical instrument created for the iPhone." Ocarina is an iPhone implementation of an ancient instrument (you may know it best from Zelda). The notable thing about Ocarina - its used just like any wind instrument. You use your fingers to choose the notes, and simply blow into the mic to play it.

    Tilting the iPhone down adds depth and vibrato to the pitch of the Ocarina. Simply pressing combinations of the 4 "buttons" allows you to play full scales of notes, which are easily configured via the settings menu. You can even see where other people are playing Ocarina, and listen in to their playing!

    We chatted with Smule CEO and co-founder Jeff Smith about the application.

    MMi: Did you start developing for the iPhone on jailbroken devices, or only once the SDK was released? If you worked with jailbroken devices, which methods did you jailbreak with? ModMyi readers are all about modding their iPhones.

    JS: We started with the SDK release.

    MMi: Is Smule planning on developing for Android, or other mobile platforms?

    JS: Not at this time.

    MMi: Smule has 4 applications in the App Store currently. Which is your personal favorite?

    JS: Ocarina. I'm a musician and I love to share this music with my friends, family, and people all over the world. I've performed as a concert pianist, and so it has been a bit of a challenge to learn about wind instruments and tonguing, etc. But it's a lot of fun. And of course, if you saw the Stairway video, that is me in the center with the great hair (and no, that is not really my hair).

    MMi: What were some of the unique issues that arose when developing Ocarina for the iPhone?

    JS: Audio is fairly challenging on all mobile devices, and the iPhone is no exception. Imagine attempting to do real-time audio input analysis and output synthesis while concurrently generating 60 frames/second in OpenGL graphics, and on top of that, doing real-time network upload of musics so that Ocarina can capture your live performance on a server to share with others around the world. We've had several users notice that we are using a lot of battery on the phone. This is because we are typically at around 98% CPU usage for this application.

    A special challenge was in providing backward compatibility with the first generation iPhones. We've found that full duplex audio is quite challenging on this platform. The new 3G iPhones, by contrast, have an amazing audio pipeline. We even think the 44Khz speaker is pretty good, and recommend using our product without headphones.

    MMi: Ocarina is pretty easy to learn - what was the process like for developing the UI?

    JS: Much credit to our CTO and co-Founder, Dr. Ge Wang. Ge is one of the leaders in the field of computer music and is a full-time professor at Stanford University in their Center for Computer Research in Music and Acoustics (CCRMA). Ge, along with another member of our team, Dr. Perry Cook, co-founded the Princeton Laptop Orchestra -- a group that recently performed in Carnegie Hall. And so Ge has an amazing insight into what is possible, but perhaps more importantly, what is desirable for untapping the creative potential of users.

    MMi: Did you find your work with the Stanford Laptop Orchestra helpful in creating Ocarina? How so?

    JS: I defer to Ge here. But yes, the SLOrk development, and before that PLOrk, were opportunities to understand computer music performance in a formal setting. What Ge & Perry found is that the computer can be an expressive instrument. And what we have found at Smule is that the iPhone is truly an optimal platform for the creation of social and expressive music. Myself, I'm a PhD candidate with Dr. Wang at Stanford, and so believe what we are doing is very important for the field of music.

    MMi: How long did Ocarina take to develop?

    JS: We founded Smule in June of 2008. We had a pretty nice head start in Audio given that Ge developed the Chuck audio engine while at Princeton for his PhD thesis. Audio is fundamental to everything we do at Smule. And so we began building our application framework and design our audio engine for the iPhone (CHiP or Chuck on the iPhone) in the summer. We shipped our first application, the Sonic Lighter, in September. By October, our audio engine was robust enough to build the first musical instrument, the Ocarina, which we just launched last Thursday.

    MMi: Can you give us any hints on what Smule has got coming in the future?

    JS: The future will be Smulean. I mean this in the most positive but serious way. But briefly, we want plan to break down barriers with sound: we aim to realize the immense potential of audio on mobile platforms, to inspire users to reach new levels of self-expression, and to provide the medium for the new network based on proximate and physical collaboration (versus purely virtual collaboration) between people around the world.

    MMi: Smule applications are based on sound and the iPhone microphone - what made you interested in that niche? What gave you the idea?

    JS: Dr. Perry Cook, a member of our team and the head of the Princeton Sound Lab, had developed some innovative algorithms to approximate wind on microphones. He should me a demo of an Andean flute on his MacBook over the summer. And so the Ocarina perhaps started with Perry. But it didn't take long for Ge to redefine an entirely different model for music engagement as an expressive social instrument. Ge designed the Ocarina.

    MMi: Have you used any Ocarina in a live performance?

    JS: Yes. As recently as last Saturday night. Stay tuned. I expect several ensembles and orchestras to begin to use this instrument.

    MMi: Is there anything else you'd like to add?

    JS: Only that we think it is time for the world to understand the potential for audio. Of course we all appreciate video. But, it would seem audio has taken a back seat to video, in particular as it relates to games. And yet our ears are as sophisticated as our eyes, right?

  • Connect With Us

  • Twitter Box

  • Facebook