Peter Coombs

How it Works

The A.I.I works by using three patches made in Max 7. The first two of these are ones that were found online to help with the routing of the Kinect data, as the Kinect doesn’t output values that are read natively by Max 7 or Mac.

The first patch is Kinect-Via-Synapse made in 2011. This patch routes the tracking data from the programme Synapse and displays it for the user. It does this by using certain OpenNI applications which translates the data taken from the Kinect to open sound control (OSC) data, and this is how the Max/MSP interface communicates.

The second patch is Force-System published by Ben B in 2012 for Cycling ‘74’s Physics Patch-a-Day tutorial page. This patch uses four jit.phys.ghost objects which act as force points. These allow the generated particles to gravitate towards and around each force point. The particles are generated by jit.phys.multiple and jit.gl.multiple objects where the number of particles, their torque, colour, scale and mass are determined by a jit.noise object. This patch also contains a moveable force point which can interact with the particles.

This part of the patch was modified by duplicating the moveable force point and by using the scaled values from Kinect-Via-Synapse to cause the points to move based on those values. Also included is a colour changing parameter for the moveable force points which uses the RGB data from an external webcam placed above the Kinect.

The third patch is built to combine the first two patches by using the data from Kinect-Via-Synapse to run the Force-System patch in a number of different ways. It does this by taking the tracking data from Kinect-Via-Synapse and scaling it to the resolution of the screen being used in order to assign a suitable range for the moveable force points to be rendered around the display. It does this by using the $ argument in the message box which describes the moveable force points. This allows for the argument to be changed, and in this case this would be the X and Y coordinates. This is achieved by scaling the output of the X and Y outputs from Synapse, to the resolution of the screen being used to display the visuals. The same technique was used to change the colour information of the moveable force points. This uses colour information from the external webcam, by using a suckah object over a jit.pwindow which receives the colour data of a specific area of the jit.pwindow via another message box.

The audio is generated by three channels each of which contain two audio generators. One uses white noise generators which are individually processed with a band pass filter with a preset Q of ‘23’, but the center frequency is controlled by the Kinect’s output. The other half of the audio is a series of four, sin, saw or triangle wave generators whose frequencies are being controlled by a ‘random’ object. However, the range of randomised values are also controlled by the Kinect. This gives an overall impression of frequency change, but with a desired ambience. To add ambience, each of the three tracks are fed through individual reverbs, and a delay before the signals are added together and sent to a digital to analog converter. Each audio track has an overall volume which is controlled by the RGB data fed to it.This allows a colour that is pure red to produce a signal from the ‘red’ audio generator, so that the timbre would contain only a sin wave and white noise.