Writing shaders with Metal for user interface
updated about 1 year ago; latest suggestion about 1 year ago
This proposal has been withdrawn...
Have you ever been interested in Metal but don’t know where to start or where to use in your application? Shader programming with Metal would be a good starting point for learning the framework and understating the basic concept of parallel computing. In this presentation, I would like to introduce shader programming with Metal and how fun and easy it is.
Shaders are used widely in computer graphics field and Metal enables them to be integrated easily to the UIKit based application. Shaders can enable animations which would be very complicated or difficult to draw with the existing animation frameworks such as
UIView animations or
CALayer animation, and, I believe, it brings more visually appealing user interface design.
In this presentation, I will cover the following points:
Brief introduction to Metal
Probably one of the difficulties of learning Metal is the number of APIs to use. In my experience, writing shaders on Playground, reflecting the change and playing around with
MTKView in realtime was a very fast way to understanding how the framework works. In this section, I will explain the basics of Metal and the shortest path to play with it.
- What is Metal, and advantages over OpenGL
- Requirements for Metal
- Main APIs of Metal
- Basic idea of Shader
- Metal on Playground
Drawing UI parts with fragment shader
Even the way shaders draw lines or curves is completely different from the one of UIKit. You probably would use
UIBezierPath or other vector based APIs for UIKit. Shaders define functions for each paths and decide the color of every pixels. I will show examples that combines those functions and draw complicated figures such as downloading indicator with an animating texture or pop ups with wavy outlines like below:
Fluid simulation with Metal
The examples above use only the graphics pipeline shaders such as vertex shader or fragment shader. Basically, they use only one
MTLTexture and animate the figures changing the color of pixels with the time input.
In this second example, I will show a fluid animation modeled with Navier-Stokes equations as below and explain the concept of GPGPU. The image is converted from
UIImage bitmap to
MTLTexture and animated disappearing like smoke. The fluid simulation model requires several
MTLTextures that might not be directly rendered on the screen but be used to preserve scalar or vector fields such as advection or pressure and the fluid model computes the next state based on the previous state using kernel functions. I will show one of the advantages of Metal, the integrated API for graphics rendering and data-parallel computation through this example.
Thanks for your comment! I am inspired this idea from some shader-ish 3D-ish userinerfade of Apple’s official applications such as Digital touch, Siri’s waveform animation or Animoji. Although there are some difficulties to use the framework in a production app (I would love to mention it in the presentation), I think it definitely can give some hints for more advanced UI/UX and broaden design options for designers.
OK. This is seriously cool. I don't have any idea how I would use this in my app, but I could see finding a place for it where I could never justify using Metal to build a game.