Simple analyzer-modulator setup for Reaper

reaper_screen1
Simple example Reaper project with Analyzer and MIDIator. Spectral flux of the input signal will control Reverb Room Size Since we use only one input and it modulates its own reverb, the setup is not crossadaptive, but rather just an adaptive effect. The basic setup can be extended by adding more input channels (and Analyzers) as needed. We do not need to add more MIDIators until we’ve used up all of its modulator channels.

Following up on the recent Ableton Live set, here’s a simple analyzer-modulator project for Reaper. The routing of signals is simpler and more flexible in Reaper, so we do not have the clutter of an extra channel to
enable MIDI out, rather we can select MIDI hardware output in the routing dialog for the MIDIator track. In Reaper,  the MIDIator plugin will start processing all by itself (no need to open its editing window to kick it to life). You need to enable a virtual MIDI device for input (remember to also enable input for control messages) and output. This is done in Reaper preferences / Midi hardware setttings.

The analyzer does not send audio throu, so we still need to use two input tracks; one for the analyzer and one for the actual audio input for processing. The MIDIator is set up to map the spectral flux of Analyzer channel 1 to Midi controller 11, channel 1.  The audio input is sent to a Reverb channel, and we’ve mapped the MIDI controller (11 on channel 1) to the Room size control of the reverb.

Simple analyzer-modulator setup for Ableton Live

live_screen1
Simple example Live set with Analyzer and MIDIator. Spectral flux of the input signal will control Reverb Decay Time. Since we use only one input and it modulates its own reverb, the setup is not crossadaptive, but rather just an adaptive effect. The basic setup can be extended by adding more input channels (and Analyzers) as needed. We do not need to add more MIDIators until we’ve used up all of its modulator channels.

I’ve created a simple Live set to show how to configure the analyzer and MIDIator in Ableton Live. There are some small snags and peculiarities (read on), but basically it runs ok.

The analyzer will not let audio through, so we use two audio input tracks. One track for the actual audio processing (just send to a reverb in our case), and one track for the analyzer. The MIDI processing also requires two tracks in Live, this is because Live assumes that a MIDI plugin will output audio so it disables the MIDI out routing when a plugin is present on a track. This is easily solved by creating a second MIDI track, and select MIDIator as the MIDI input to that track. From this track we can route MIDI out from Live. You will want to set the MIDI out to a virtual MIDI device (e.g. loopMIDI on windows, IAC bus on OSX). Enable midi input from the virtual midi device (you can enable Track, Sync and Remote for a MIDI input. We want to enable Remote )

In our example setup, we’ve enabled one modulator on the MIDIator. This is set to receive spectral flux from Analyzer 1, and send this modulator data to midi channel 1, controller 11. We’ve mapped this to Reverb Decay Time on the effect return track. Input sounds with high flux (which means that the sound is probably a bit noisy) will have long reverb. Sounds with low flux (probably a tonal or stable sound) will have short reverb.

On my computer (windows), I need to open the MIDIator editing window to force it to start processing. Take care *not* to close the MIDIator window, as it will somehow stop processing when the window is closed (This only happens in Ableton Live, not sure why). To get rid of the window, just click on something in another track. This will hide the MIDIator window without disabling it.

Simple crossadaptive mixing template for Logic

Next week we’ll go to London for seminars at Queen Mary and De Montfort. We’ll also do a mixing session with Gary Bromham, to experiment with the crossadaptive modulation techniques in a postproduction setting. For this purpose I’ve done a simple session template in Logic (as it is a DAWs that Gary uses regularly).

logic_screen1
Logic session with two tracks, EQ being applied to track 2, controlled by pitch analysis of the signal on track 1. Reaper (running the analysis and modulator routing plugins) can be seen in the background. The jack patching can also be seen in the lower right part of the image.

To keep the Logic session as clean as  possible, we will do the signal analysis and preparation of the modulation signal in Reaper. This also allows us to use the VST versions of the plugins for analysis and modulation mapping we’ve created earlier. (It should be possible to use these as AU plugins too, but that is somewhat more experimental as of yet). Signal routing between Logic and Reaper is done via Jack. This means that both Logic and Reaper use Jack as the audio driver, while Jack communicates with the audio hardware for physical I/O. The signal to be analyzed (guitar on track 1 in our Logic session) is fed to a separate bus track in Logic, and the output from this track is sent to Reaper via Jack. We analyze the pitch of the signal in Reaper, send the pitch tracking data to the vstMIDIator plugin, and route the MIDI signal from Reaper back to Logic via the IAC bus. In Logic, it is relatively straightforward to map an incoming MIDI signal to control any parameter. You can find this in Logic Preferences/Automation, where there are Learn and Edit buttons towards the bottom of the dialog. To learn a mapping you will touch the desired destination parameter, then send the midi message to be mapped. In our example, we use an EQ on vocals (track 2), and control the frequency of a narrow band in the EQ with the MIDI modulator signal. This way the pitch of the guitar controls the EQ for the vocals.

logic_screen2
The analyzer and MIDIator plugins running in Reaper, with the Logic mixer in the background

Mixing example, simplified interaction demo

When working further with some of the examples produced in an earlier session , I wanted to see if I could demonstrate the influence of one instrument’s influence of the the other instruments sound more clearly. Here’ I’ve made an example where the guitar controls the effects processing of the vocal. For simplicity, I’ve looped a small segment of the vocal take, to create a track the is relatively static so the changes in the effect processing should be easy to spot. For the same reason, the vocal does not control anything on the guitar in this example.

The reaper session for the following examples can be found here .


Example track: The guitar track is split into two control signals, one EQ’ed to contain only low frequencies, the other with only high frequencies. The control signals are then gated, and used as sidechain control signals for two different effects tracks processing the vocal signal. The vocal signal is just a short loop of quite static content, to make it easier to identify the changes in the effects processing.


Example track: As above, but here the original vocal track is used as input to the effects, giving a more dynamic and flexible musical expression.

Cross adaptive mixing in a standard DAW

To enable the use of these techniques in a common mixing situation, we’ve made some example configurations in Reaper. The idea is to extract some feature of the modulator signal (using common tools like EQs and Compressors rather than more advanced analysis tools), and use this signal to affect the processing of another signal.

By using sidechaining we can allow the energy level of one signal to gate/compress/expand another signal. Using EQ and/or multiband compression on the modulator signal, we can extract parts of the spectrum, for example so that the processing will be applied only if there is energy in the deep frequencies of the modulator signal. Admittedly, this method is somewhat limited as compared with a full crossadaptive approach, but it is widely available and as such has a practical value. With some massaging of the modulator signal and creative selection of effects applied to the affected signal, this method still can produce a fairly wide range of cross-adaptive effects.

By using automation techniques available in Reaper (we will investigate how to do this in other hosts too) , we can map the signal energy directly to parameter changes. This allows cross-adaptive processing in the full sense of the term. The technique (as implemented in Reaper) is limited by the kind of features one can extract from the modulator signal (energy level only) and by the control signal mapping being one-to-one , so that it is not possible to mix several modulator sources to control an effects parameter. Still this provides a method to experiment easily with cross-adaptive mixing techniques in a standard DAW with no extra tools required.

Example projects:

Sidechaining used in an unconventional cross-adaptive manner is demonstrated in the CA_sidechain_pitchverb Reaper project.

Signal automation based on modulator energy is demonstrated in the CA_EnvControl Reaper project

Terms:

Modulator signal: The signal being used to affect processing of another signal
Affected signal: The signal being modulated