Convolution demo sounds

The convolution techniques we’ve developed in this project are aimed at live performance and are so far mostly used for that purpose. The manner in which the filters can be continuously updated might also allow for other creative applications within convolution and filtering. As a very basic demo of how the filters sound with prerecorded sounds, we’ve done two audio examples. Do note that the updating of the filters are still realtime, it is just the source sounds playing into the filters that are prerecorded.

Source

The source sounds are prepared as a stereo file, with sound A in the left channel and sound B in the right channel. Sound A is assembled from sounds downloaded from freesound.org , all made by user Corsica_S (files “alpha.wav”, “bravo.wav”, “charlie.wav”, “delta.wav”). Sound B is a drumloop programmed by Oeyvind Brandtsegg some time in the late 90’s.

conv_sources
 conv_sources

Source sounds for the convolution demo.

 

Liveconv

Sound A is recorded as the impulse response for liveconv. The length of the filter is approx 0.5 seconds (the duration of the spoken word used as source). It is replaced approx every 5.5 seconds, manually automated to line up with when the appropriate word appears in the source. This way, the first IR contains the word “alpha”, then it is replaced after 5.5 seconds with the word “bravo”, and so on until all 4 words have been used as an IR.

conv_liveconv
 conv_liveconv

Liveconv demo using above source sounds.

tvconv

Sound A and sound B runs continuously through the filter. No freezing of coefficients is applied. The filter length is 23768 samples, approximately 0.74 seconds at a sample rate of 44.1 kHz.

conv_tvconv
 conv_tvconv

tvconv demo using the above source sounds.

 

 

The entrails of Open Sound Control, part one

Many of us are very used to employing the Open Sound Control (OSC) protocol to communicate with synthesisers and other music software. It’s very handy and flexible for a number of applications. In the cross adaptive project, OSC provides the backbone of communications between the various bits of programs and plugins we have been devising.

Generally speaking, we do not need to pay much attention to the implementation details of OSC, even as developers. User-level tasks only require us to decide the names of messages addresses, its types and the source of data we want to send. At Programming level,  it’s not very different: we just employ an OSC implementation from a library (e.g. liblo, PyOSC) to send and receive messages.

It is only when these libraries are not doing the job as well as we’d like that we have to get our hands dirty. That’s what happened in the past weeks at the project. Oeyvind has diagnosed some significant delays and higher than usual cost in OSC message dispatch. This, when we looked, seemed to stem from the underlying implementation we have been using in Csound (liblo, in this case). We tried to get around this by implementing an asynchronous operation, which seemed to improve the latencies but did nothing to help with computational load. So we had to change tack.

OSC messages are transport-agnostic, but in most cases use the User Datagram Protocol transport layer to package and send messages from one machine (or program) to another. So, it appeared to me that we could just simply write our own sender implementation using UDP directly. I got down to programming an OSCsend opcode that would be a drop-in replacement for the original liblo-based one.

OSC messages are quite straightforward in their structure, based on 4-byte blocks of data. They start with an address, which is a null-terminated string like, for instance, “/foo/bar”  :

'/' 'f' 'o' 'o' '/' 'b' 'a' 'r' '\0'

This, we can count, has 9 characters – 9 bytes – and, because of the 4-byte structure, needs to be padded to the next multiple of 4, 12, by inserting some more null characters (zeros). If we don’t do that, an OSC receiver would probably barf at it.

Next, we have the data types, e.g. ‘i’, ‘f’, ‘s’ or ‘b’ (the basic types). The first two are numeric, 4-byte integers and floats, respectively. These are to be encoded as big-endian numbers, so we will need to byteswap in little-endian platforms before the data is written to the message. The data types are encoded as a string with a starting comma (‘,’) character, and need to conform to 4-byte blocks again. For instance, a message containing a single float would have the following type string:

',' 'f' '\0'

or “,f”. This will need another null character to make it a 4-byte block. Following this, the message takes in a big-endian 4-byte floating-point number.  Similar ideas apply to the other numeric type carrying integers.

String types (‘s’) denote a null-terminated string, which as before, needs to conform to a length that is a multiple of 4-bytes. The final type, a blob (‘b’), carries a nondescript sequence of bytes that needs to be decoded at the receiving end into something meaningful. It can be used to hold data arrays of variable lengths, for instance. The structure of the message for this type requires a length (number of bytes in the blob) followed by the byte sequence. The total size needs to be a multiple of 4 bytes, as before. In Csound, blobs are used to carry arrays, audio signals and function table data.

If we follow this recipe, it is pretty straightforward to assemble a message, which will be sent as a UDP packet. Our example above would look like this:

'/' 'f' 'o' 'o' '/' 'b' 'a' 'r' '\0' '\0' '\0' '\0'
',' 'f' '\0' '\0' 0x00000001

This is what OSCsend does, as well as its new implementation. With it, we managed to provide a lightweight (low computation cost) and fast OSC message sender. In the followup to this post, we will look at the other end, how to receive arbitrary OSC messages from UDP.