Tuesday, September 21, 2010

Procedural music with PyAudio and NumPy

Combining two of my favorite pastimes, programming and music... This is the hacky "reduced to it's basic components" version of a library I've been working on for generating music and dealing with music theory.

Tweaking the harmonics by changing the shape of the harmonic components and ratios can produce some interesting sounds. This one only uses sine waveforms, but a square / saw generator is trivial with numpy.

It takes a second to generate, so don't turn your volume up too loud in anticipation (it may be loud).

import math
import numpy
import pyaudio
import itertools
from scipy import interpolate
from operator import itemgetter

class Note:

NOTES = ['c','c#','d','d#','e','f','f#','g','g#','a','a#','b']

def __init__(self, note, octave=4):
self.octave = octave
if isinstance(note, int):
self.index = note
self.note = Note.NOTES[note]
elif isinstance(note, str):
self.note = note.strip().lower()
self.index = Note.NOTES.index(self.note)

def transpose(self, halfsteps):
octave_delta, note = divmod(self.index + halfsteps, 12)
return Note(note, self.octave + octave_delta)

def frequency(self):
base_frequency = 16.35159783128741 * 2.0 ** (float(self.index) / 12.0)
return base_frequency * (2.0 ** self.octave)

def __float__(self):
return self.frequency()

class Scale:

def __init__(self, root, intervals):
self.root = Note(root.index, 0)
self.intervals = intervals

def get(self, index):
intervals = self.intervals
if index < 0:
index = abs(index)
intervals = reversed(self.intervals)
intervals = itertools.cycle(self.intervals)
note = self.root
for i in xrange(index):
note = note.transpose(intervals.next())
return note

def index(self, note):
intervals = itertools.cycle(self.intervals)
index = 0
x = self.root
while x.octave != note.octave or x.note != note.note:
x = x.transpose(intervals.next())
index += 1
return index

def transpose(self, note, interval):
return self.get(self.index(note) + interval)

def sine(frequency, length, rate):
length = int(length * rate)
factor = float(frequency) * (math.pi * 2) / rate
return numpy.sin(numpy.arange(length) * factor)

def shape(data, points, kind='slinear'):
items = points.items()
keys = map(itemgetter(0), items)
vals = map(itemgetter(1), items)
interp = interpolate.interp1d(keys, vals, kind=kind)
factor = 1.0 / len(data)
shape = interp(numpy.arange(len(data)) * factor)
return data * shape

def harmonics1(freq, length):
a = sine(freq * 1.00, length, 44100)
b = sine(freq * 2.00, length, 44100) * 0.5
c = sine(freq * 4.00, length, 44100) * 0.125
return (a + b + c) * 0.2

def harmonics2(freq, length):
a = sine(freq * 1.00, length, 44100)
b = sine(freq * 2.00, length, 44100) * 0.5
return (a + b) * 0.2

def pluck1(note):
chunk = harmonics1(note.frequency(), 2)
return shape(chunk, {0.0: 0.0, 0.005: 1.0, 0.25: 0.5, 0.9: 0.1, 1.0:0.0})

def pluck2(note):
chunk = harmonics2(note.frequency(), 2)
return shape(chunk, {0.0: 0.0, 0.5:0.75, 0.8:0.4, 1.0:0.1})

def chord(n, scale):
root = scale.get(n)
third = scale.transpose(root, 2)
fifth = scale.transpose(root, 4)
return pluck1(root) + pluck1(third) + pluck1(fifth)

root = Note('A', 3)
scale = Scale(root, [2, 1, 2, 2, 1, 3, 1])

chunks = []
chunks.append(chord(21, scale))
chunks.append(chord(19, scale))
chunks.append(chord(18, scale))
chunks.append(chord(20, scale))
chunks.append(chord(21, scale))
chunks.append(chord(22, scale))
chunks.append(chord(20, scale))
chunks.append(chord(21, scale))

chunks.append(chord(21, scale) + pluck2(scale.get(38)))
chunks.append(chord(19, scale) + pluck2(scale.get(37)))
chunks.append(chord(18, scale) + pluck2(scale.get(33)))
chunks.append(chord(20, scale) + pluck2(scale.get(32)))
chunks.append(chord(21, scale) + pluck2(scale.get(31)))
chunks.append(chord(22, scale) + pluck2(scale.get(32)))
chunks.append(chord(20, scale) + pluck2(scale.get(29)))
chunks.append(chord(21, scale) + pluck2(scale.get(28)))

chunk = numpy.concatenate(chunks) * 0.25

p = pyaudio.PyAudio()
stream = p.open(format=pyaudio.paFloat32, channels=1, rate=44100, output=1)

The scale in use is Harmonic Minor.
It's encoded via halfsteps as: "[2, 1, 2, 2, 1, 3, 1]"
Try changing it to natural minor: "[2, 1, 2, 2, 1, 2, 2]"

I'd be interested in teaming up with anyone willing to write some actual input methods (matrix sequencer, tab reader, however else it could be done) as well as an interface for tweaking synthesized sounds. My present code base is a bit much for a blog post, so contact me if interested.


Grant Thomas said...

I'd be interested in seeing ( or maybe working on ) a method for reading MusicXML or ABC notation.

Wybiral said...

Those both look pretty promising. That's a better approach to what I had in mind (I was thinking to implement an interface for tabbing / matrix sequencing) but reading a standardized format would be the best angle. I've gotten quite a bit of feedback on this snippet of code, so I'll probably end up setting up a project page for it soon and at least get what I have out there.

Anonymous said...

How does writing the raw audio data to the sound card work? I can't see how you get from your maths/data to audio.

stuaxo said...

Aw this is disappointing I just get a segmentation fault when I run it.

stuaxo said...

@Adam The stream.write bit at the bottom.

Wybiral said...

@stuaxo: A segfault? Do your pyaudio / numpy / scipy modules all work without issues on their own?

stuaxo said...

I'll have a look when I'm in Linux next time, they all installed fine though.

BTW, it ran in Windows fine (startup takes a while, but thats the same for all Python in Windows).

Eliasvan said...

For me it gives the following message in Linux (Fedora12), while I hear no sound (pyaudio demos are working properly):

"*** glibc detected *** python: malloc(): memory corruption: 0x090fa038 ***"

I debugged the script and this error came from the command "stream.write(data)". I found this error emerges if the length of data is 8200 or greater (equals 2**13+8). (The length of your data is 5644800 = ~5.64MB).

So I changed the line
and the error was gone. However I still don't hear any sound - perhaps because I reduced the length to only 8.2KB?

Help or suggestions are appreciated. I'm actually very interested about this subject, once, I planned to do something similar but much bigger (creating instruments and notes on-the-fly and multi-threaded controlled by input types like emotions and music styles), but I planned to do it with C++ because I didn't know about the pyaudio module (which seems to be very promising).

Wybiral said...

To avoid using just pyaudio, here is a little playback module that allows you to just pass the audio chunks to the best available playback mechanism. Then you can play a numpy audio chunk with just play(chunk)

Eliasvan said...

Thank you very much! This was exactly the thing I searched for.

PS: One more note: your code tries to import "encode". By me, that generates an ImportError, however the script runs perfectly without "import encode".

Wybiral said...

Oops... It was a module I wrote for converting between bit formats. I ended up just copy & pasting the functions I used and forgot to remove that import.

I'm glad that helps. It should play any numpy array encoded as floating point samples between -1.0 and 1.0 provided the memory to hold that audio chunk is available

I've found it very useful for my own numpy audio experiments, because the playback modules seem to not be so universal.

etienned said...

Do you know pyo? http://code.google.com/p/pyo/ That could interest you.

Eliasvan said...

Thank you for the link! The realtime part (thanks to py-C library) especially interests me!

Eliasvan said...

etienned, pyo does not work properly with pulse-jack combination. It's seems to be based on portaudio. ;(

Prizm said...

EAV Pro Audio stock a wide range of PA Speakers and PA Systems, Mixing desk, Power Amplifiers, audio interfaces, Wireless microphones and wired microphones along with a massive range of home and professional recording equipment.

Joaquin Abian said...

works perfect in win7 64-bits. And sounds great !

Marko H said...

Great! Many thanks!

Giotherobot said...

Very cool and simple to understand! I revised the code to work in python 3.4 and sound awesome!
If anyone wants the updated code email me!