Live coding GLSL shaders with IPython

I recently started to learn some OpenGL and was looking for a nice way to use Python for this. I wanted to be able to update the shaders quickly so that I can quickly try out different code. This is one of the results. I was heavily inspired by the live coding video of onomo.jp, in fact I practically copied and adapted his code to get a start:

http://vimeo.com/51993089

The ideas and code of Cyrille Rossant was also very helpful, to get a start with OpenGL in Python and IPython:

http://cyrille.rossant.net/2d-graphics-rendering-tutorial-with-pyopengl/

To get a start you can just run my gl_livecode.py script that I also use in the beginning of the video. It is available as gist:

https://gist.github.com/pbouda/5531821

To process the audio I just split the stream in two frequency bins, more or less randomly. I didn’t rely on anything sophisticated but just used what gave the most beautiful visualization in the end. The update function currently looks like this:

def move_stuff(self):
    a = self.stream.read(chunk)
    indata = np.array(wave.struct.unpack("%dh"%(chunk), a))*self.window
    fft_data = abs(np.fft.rfft(indata))
    total_sum = np.sum(fft_data)
    spec_x = np.sum(fft_data[40:100])/(len(fft_data[40:100])*chunk*2048)
    spec_y = np.sum(fft_data[250:])/(len(fft_data[250:])*chunk*32)
    gl.glUniform2f(self.spec, spec_x, spec_y)

You can easily change the move_stuff function while the programm is running, just use something like this in IPython:

def move_stuff(self):
    a = self.stream.read(chunk)
    indata = np.array(wave.struct.unpack("%dh"%(chunk), a))*self.window
    fft_data = abs(np.fft.rfft(indata))
    total_sum = np.sum(fft_data)
    spec_x = np.sum(fft_data[:220])/(len(fft_data[:220])*chunk*2048)
    spec_y = np.sum(fft_data[220:])/(len(fft_data[220:])*chunk*32)
    gl.glUniform2f(self.spec, spec_x, spec_y)
f_move = types.MethodType(move_stuff, win.widget, GLPlotWidget)
win.widget.move_stuff = f_move

This example changes the location and the size of the two frequency bins. To update the fragment shader for example:

FS = """#version 330
uniform vec2 resolution;
uniform vec2 spec;
uniform float time;

void main(void) {
  vec2 uv = 2.0 * (gl_FragCoord.xy / resolution) - 1.0;
  float col = 0.0;
  //uv = abs(uv);
  for (float i = 1.0; i<23.0; i++) {
      uv.y += sin(i*20.0 + spec.x*5.0 + time*6.0 + uv.x*.5) * spec.y*2.5;
      col += abs((.5*spec.x)/uv.y) * spec.y;
  }
  gl_FragColor = vec4(col,col,col,1.0);
}
"""
win.widget.fs = FS
win.widget.link_shaders()
win.widget.get_uniforms()

Then iterate. :-) Have fun!

About me

My name is Peter Bouda and I am and Angular/JavaScript coach and consultant at ng-lisbon.com with more than 18 years of professional experience in web application development. I help you to bring your web product to market as quickly as possible.

Join my next Angular workshop at TUGA IT, May 18th, in Lisbon.

Read about my work
or
Contact me

Send me a message and I will get back to you.