Author: Oeyvind

Simple crossadaptive mixing template for Logic

Next week we’ll go to London for seminars at Queen Mary and De Montfort. We’ll also do a mixing session with Gary Bromham, to experiment with the crossadaptive modulation techniques in a postproduction setting. For this purpose I’ve done a simple session template in Logic (as it is a DAWs that Gary uses regularly).

logic_screen1
Logic session with two tracks, EQ being applied to track 2, controlled by pitch analysis of the signal on track 1. Reaper (running the analysis and modulator routing plugins) can be seen in the background. The jack patching can also be seen in the lower right part of the image.

To keep the Logic session as clean as  possible, we will do the signal analysis and preparation of the modulation signal in Reaper. This also allows us to use the VST versions of the plugins for analysis and modulation mapping we’ve created earlier. (It should be possible to use these as AU plugins too, but that is somewhat more experimental as of yet). Signal routing between Logic and Reaper is done via Jack. This means that both Logic and Reaper use Jack as the audio driver, while Jack communicates with the audio hardware for physical I/O. The signal to be analyzed (guitar on track 1 in our Logic session) is fed to a separate bus track in Logic, and the output from this track is sent to Reaper via Jack. We analyze the pitch of the signal in Reaper, send the pitch tracking data to the vstMIDIator plugin, and route the MIDI signal from Reaper back to Logic via the IAC bus. In Logic, it is relatively straightforward to map an incoming MIDI signal to control any parameter. You can find this in Logic Preferences/Automation, where there are Learn and Edit buttons towards the bottom of the dialog. To learn a mapping you will touch the desired destination parameter, then send the midi message to be mapped. In our example, we use an EQ on vocals (track 2), and control the frequency of a narrow band in the EQ with the MIDI modulator signal. This way the pitch of the guitar controls the EQ for the vocals.

logic_screen2
The analyzer and MIDIator plugins running in Reaper, with the Logic mixer in the background

Conversation with Marije

We had a Skype meeting between me (Oeyvind) and Marije Baalman today, here’s some notes from the conversation:

First, we really need to find alternatives to Skype, the flakyness of connection makes it practically unusable. A service that allows recording the conversation would be nice too.

We talked about a number of perspectives on the project, including Marijes role as an external advisor and commentary, references to other projects that may relate to ours in terms of mapping of signals to musical parameters, possible implementation on embedded devices, and issues relating to the signal flow from analysis to mapping to modulator destination.

Marije mentioned Joel Ryan ( * * ) as an interesting artist doing live processing of acoustic instruments. His work seems closely related to our earlier work in T-EMP . Interesting to see and hear his work with Evan Parker . More info on his instruments and mapping setup would be welcome.

We discussed the prospect of working with embedded devices for the crossadaptive project. Marije mentioned the Bela project for the BeagleBboard Black, done at Queen Mary University. There are of course a number of nice tthigns about embedded devices, making self-contained instruments for easy of use for musicians. However, we feel that at this stage, our project is of such an experimental nature that is easier explored on a conventional computer. The technical hurdles of adapting it to an embedded device is easier tackled when the signal flow and processing needs are better defined. In relation to this we also discussed the different processes involved in our signal flow: Analysis, Mapping, Destination.

Our current implementation (based on the 2015 DAFx paper ) has this division in analysis, mapping/routing and destination (the effect processing parameter being controlled). The mapping/routing was basically done at the destination in the first incarnation of the toolkit. However, lately I’ve started reworking it to create a more generic mapping stage , allowing for more freedom in what to control (i.e. controlling any effect processor or DAW parameter). Marije suggested more preprocessing to be done in the analysis stage, to create cleaner and more generically applicable control signals for easier use at the mapping stage. This coincides with our recent experiences in using low level analyses like spectral flux and crest for example; they work well on some instruments (e.g. vocals) as an indicator of balance between tone and noise, however on other instruments (e.g. guitar) they sometimes work well and sometimes less so. Many of the current high level analyses (e.g. in music information retrieval) seems in many respects to be geared towards song recognition, while we need some way of extracting higher level features as continuous signals. The problem being to define and extract what is the musical meaning of a signal, on a frame-by-frame basis. This will most probably be different for different instruments, so an source-specific preprocessing stage seems reasonable.

The Digital Orchestra and IDMIL , and more specifically the libmapper might provide inspiration for our mapping module. We make a note of inspecting it more closely. Likewise, the Live Algorithms for Music project, and also the possibly the Ircam OMAX project. Marije mentioned 3Dmin project at UDK/TU Berlin using techniques of exploration of arbitrary mappings as a method of influencing rather than directly controlling a destination parameter. This reminded me of earlier work by Palle Dahlstedt , and also, come to think of it our own modulation matrix for the Hadron synthesizer.

Later we want Marije to test our toolkit in more depth and comment more specifically on what it lacks and how it can be improved. There are probably some small technical issues in using the toolkit under Linux, but as our main tools are crossplatform (Windows/OSX/Linux), these should be relatively easy to solve.