Coding Audio Apps. Or: drop all your beloved programming paradigms

updated 11 months ago; latest suggestion 9 months ago

Coding realtime audio apps, like synthesizers, often requires to question - and possibly drop - many paradigms that seem hardwired in our brain.

Code running in the audio thread has to deliver a constant and uninterrupted stream of data in realtime, which makes it problematic to invoke operating system calls or perform memory allocation. Languages with inherent memory allocation, like Swift and Objective-C, are thus a poor choice for realtime digital signal processing.

Requirements, solutions and common pitfalls of audio coding will be explained and the importance of latency is addressed. Problems and methodes for communication with the UI thread are discussed.

If time permits, the special aesthetics of UI for musical apps is explained, and the love of DSP coders for overflowing unsigned data types is shown.

Throughout the talk, and when in context, things can be spiced up by briefly running an app with live audio output.

The author is an experienced App Developer and DSP engineer. He/She (it's anonymous here, remember?) has given two talks so far, so not a very experienced speaker, but also not a complete newbie.

Suggestions

  • The proposal author responds 9 months ago

    Thanks for these suggestions - regarding low level, although I also spent a considerable time coding audio algorithms entirely in assembly, I'll restrict myself to a subset of C++ for realtime audio on iOS. Furthermore I'd like to dive a bit into iOS Core Audio and why it is so fantastic for realtime.

    Regarding Audio File formats, as this is not related to realtime signal processing, I won't cover this. iOS provides excellent built-in support for handling all kinds of these.

  • Ed8b3fcf16c3bf26d8e604113849c7b940bd36a0?size=100x100 ed8b3fcf16c3bf26d8e604113849c7b940bd36a0 suggests 9 months ago

    Great topic! I have worked a little bit with audio myself and found that there is not much information available on how to encode and decode specific audio formats like MP3, OGG, AAC etc. Would love to hear more about how this works and how to obtain the required licenses for these encoders.

  • 8b2f2d3e6553c2b4048b93300959895ab76b4154?size=100x100 8b2f2d3e6553c2b4048b93300959895ab76b4154 suggests 9 months ago

    As I started my career programming assembly, I love a good low-level coding talk.

  • The proposal author responds 10 months ago

    Thanks for the suggestion, but I think realtime audio signal processing for musical purposes and video do not have much in common, neither from the user perspective or for the programmer. Also, I don't know anything about realtime video processing, so this is nothing I could talk about even if I wanted.

  • 007dca74c613a8f4ab1ed47c7562a1cae68df25a?size=100x100 007dca74c613a8f4ab1ed47c7562a1cae68df25a suggests 10 months ago

    This sounds really interesting. Would be nice to add an aside about real-time video.