Coding Audio Apps. Or: drop all your beloved programming paradigms
updated about 1 year ago; latest suggestion about 1 year ago
Coding realtime audio apps, like synthesizers, often requires to question - and possibly drop - many paradigms that seem hardwired in our brain.
Code running in the audio thread has to deliver a constant and uninterrupted stream of data in realtime, which makes it problematic to invoke operating system calls or perform memory allocation. Languages with inherent memory allocation, like Swift and Objective-C, are thus a poor choice for realtime digital signal processing.
Requirements, solutions and common pitfalls of audio coding will be explained and the importance of latency is addressed. Problems and methodes for communication with the UI thread are discussed.
If time permits, the special aesthetics of UI for musical apps is explained, and the love of DSP coders for overflowing unsigned data types is shown.
Throughout the talk, and when in context, things can be spiced up by briefly running an app with live audio output.
The author is an experienced App Developer and DSP engineer. He/She (it's anonymous here, remember?) has given two talks so far, so not a very experienced speaker, but also not a complete newbie.
Thanks for these suggestions - regarding low level, although I also spent a considerable time coding audio algorithms entirely in assembly, I'll restrict myself to a subset of C++ for realtime audio on iOS. Furthermore I'd like to dive a bit into iOS Core Audio and why it is so fantastic for realtime.
Regarding Audio File formats, as this is not related to realtime signal processing, I won't cover this. iOS provides excellent built-in support for handling all kinds of these.
Great topic! I have worked a little bit with audio myself and found that there is not much information available on how to encode and decode specific audio formats like MP3, OGG, AAC etc. Would love to hear more about how this works and how to obtain the required licenses for these encoders.
As I started my career programming assembly, I love a good low-level coding talk.
Thanks for the suggestion, but I think realtime audio signal processing for musical purposes and video do not have much in common, neither from the user perspective or for the programmer. Also, I don't know anything about realtime video processing, so this is nothing I could talk about even if I wanted.
This sounds really interesting. Would be nice to add an aside about real-time video.