Side thread: Sndobj, Python, and RT-safe

Everything and anything, but nothing about the LinuxSampler project.
User avatar
Consul
Moderator
Posts: 189
Joined: Wed Jan 23, 2008 11:19 pm
Location: Port Huron, Michigan, USA
Contact:

Side thread: Sndobj, Python, and RT-safe

Post by Consul » Sat Mar 01, 2008 1:47 am

Okay, here's another question that is going to reveal my ignorance even more.

Anders' recent adventures with Sndobj has me curious about the library, it's ability to be used by Python, and what it means for an application and language to be real-time safe. Specifically, why is Python not suitable for real-time use? And what does it mean to be RT-safe anyway? Even C++ code can be non-RT-safe, apparently (I'm speaking of some comments I read about GStreamer).

I may have the passion and some math knowledge, but I still need some help in the computer science department. Thanks again!
Darren Landrum

User avatar
dahnielson
Moderator
Posts: 632
Joined: Wed Jan 23, 2008 11:25 pm
Location: Linköping / Tranås, Sweden
Contact:

Re: Side thread: Sndobj, Python, and RT-safe

Post by dahnielson » Sat Mar 01, 2008 2:06 am

Ok, I'm really from the humanities department. Let me start with a quote from the ladspa.h defining hard real-time for plugins:
Property LADSPA_PROPERTY_HARD_RT_CAPABLE indicates that the plugin
is capable of running not only in a conventional host but also in a
`hard real-time' environment. To qualify for this the plugin must
satisfy all of the following:

(1) The plugin must not use malloc(), free() or other heap memory
management within its run() or run_adding() functions. All new
memory used in run() must be managed via the stack. These
restrictions only apply to the run() function.

(2) The plugin will not attempt to make use of any library
functions with the exceptions of functions in the ANSI standard C
and C maths libraries, which the host is expected to provide.

(3) The plugin will not access files, devices, pipes, sockets, IPC
or any other mechanism that might result in process or thread
blocking.

(4) The plugin will take an amount of time to execute a run() or
run_adding() call approximately of form (A+B*SampleCount) where A
and B depend on the machine and host in use. This amount of time
may not depend on input signals or plugin state. The host is left
the responsibility to perform timings to estimate upper bounds for
A and B.


Basically it must execute with minimum effort and don't have any locking code to be RT safe. And as anyone who have ventured into more advanced python coding know: Python is crowned with one big lock, the GIL, a General Interpreter Lock. For instance it means that Python can not use SMP threads natively because all of them will be using the same lock and attempts to refactor it to more atomic locks have all turned out to be less efficient, so the solution is to use Parallel Python to run and manage several job-instances of python.

In the case of SndObj the GIL isn't really a problem since all audio stuff run in its own SndObj non-python thread.
Anders Dahnielson

Ardour2, Qtractor, Linuxsampler, M-AUDIO Delta 1010, Axiom 61, Korg D12, AKAI S2000, E-MU Proteus 2k, Roland R-5, Roland HP 1300e, Zoom RFX-1000, 4GB RAM x86_64 Intel Pentium Dual 1.80GHz Gentoo Linux

User avatar
Consul
Moderator
Posts: 189
Joined: Wed Jan 23, 2008 11:19 pm
Location: Port Huron, Michigan, USA
Contact:

Re: Side thread: Sndobj, Python, and RT-safe

Post by Consul » Sat Mar 01, 2008 5:55 am

So could I then use SndObj with Python, doing the inner loop with SndObj and doing the GUI in Python? Presumably, the inner loop would actually be in Python, but would call only SndObj functions within, right? Or maybe not, since the GUI would need to be in a different thread. So much to learn...
Darren Landrum

User avatar
dahnielson
Moderator
Posts: 632
Joined: Wed Jan 23, 2008 11:25 pm
Location: Linköping / Tranås, Sweden
Contact:

Re: Side thread: Sndobj, Python, and RT-safe

Post by dahnielson » Sat Mar 01, 2008 2:37 pm

Well, you could do an inner (audio) loop in python in a different thread using Parallel Python for true multi-threading. But there's really no point in doing so since the SndObj thread manager already will do that for you, just add the SndObj objects to the manager. All the SndObj objects are implemented in C++ and if you want to add your own (but I want to point out that SndObj was intended for prototyping and experimentation and already cover most bases) you would probably do that in C++.

Now, I haven't investigated if it is possible to implement new SndObj directly in Python. But doing so is usually not a good idea if you really want RT at normal sample rates (of course it depends on your CPU and how many python-implemented objects you run at the same time). Python code is compiled to interpreted byte code that will always execute slower than if implemented in C/C++ (but in many cases the difference doesn't matter and is a small cost to trade in for the convenience of using python), but a rule of thumb when coding for performance in Python (cPython to be specific) is to use the built in types and data structures, because they are all implemented in highly optimized compiled C code instead of interpreted byte code, or use Python packages implemented in C/C++. The first step in making Python code run faster is to hunt down code using extensive looping and implement it in C/C++ (or use Pyrex).

Something that was implied in my earlier reply but wasn't explicitly stated is that the qualification for RT is pretty simple: E.g. if your code generates/process one sample at the time and the sample rate is 48 kHz, can you guarantee your code will be able to execute at least 48000 times per second for real time performance? (If your buffer is 256 then the code has to execute 187.5 times per second, but the difference is academic because you usually have a loop in your code to iterate over the buffer.) And if your code use something that can possibly block it, then you can't guarantee RT performance even if you can time it executing 48000 times per second on your computer, because the execution time for each execution can the vary wildly and RT is all about having a constant execution speed.
Anders Dahnielson

Ardour2, Qtractor, Linuxsampler, M-AUDIO Delta 1010, Axiom 61, Korg D12, AKAI S2000, E-MU Proteus 2k, Roland R-5, Roland HP 1300e, Zoom RFX-1000, 4GB RAM x86_64 Intel Pentium Dual 1.80GHz Gentoo Linux

User avatar
Consul
Moderator
Posts: 189
Joined: Wed Jan 23, 2008 11:19 pm
Location: Port Huron, Michigan, USA
Contact:

Re: Side thread: Sndobj, Python, and RT-safe

Post by Consul » Sat Mar 01, 2008 5:43 pm

dahnielson wrote:Now, I haven't investigated if it is possible to implement new SndObj directly in Python.


This isn't quite what I had in mind, actually. What I'm really wondering is if I can write the actual engine in C++ with SndObj and then implement the GUI and controls for it in Python. (EDIT: Blargh, this still isn't quite what I have in mind, although it's closer. I need to think about this some more.)

As for the event handler, which would process incoming MIDI events, I don't know exactly what to do about that yet. That part doesn't need to run at the sample rate. Handling multiple voices of polyphony is also an issue. Can I write my event handler in Python and have it fork threads of the engine per voice? I might just have to try that out and see. I agree that I'm doing way too much asking and not enough coding and trying out, but my system is having extraordinary trouble taking to a Linux distro because of the Emu card.

dahnielson wrote:And if your code use something that can possibly block it, then you can't guarantee RT performance even if you can time it executing 48000 times per second on your computer, because the execution time for each execution can the vary wildly and RT is all about having a constant execution speed.


THAT sentence has explained more to me than anything else so far! Also, I have a friend with a great deal of experience in soft and hard real-time programming, and he's been giving me lessons. Thanks!

EDIT: WOOHOO! I actually hit the "Quote" button! :D
Darren Landrum

User avatar
dahnielson
Moderator
Posts: 632
Joined: Wed Jan 23, 2008 11:25 pm
Location: Linköping / Tranås, Sweden
Contact:

Re: Side thread: Sndobj, Python, and RT-safe

Post by dahnielson » Sat Mar 01, 2008 6:03 pm

Consul wrote:This isn't quite what I had in mind, actually. What I'm really wondering is if I can write the actual engine in C++ with SndObj and then implement the GUI for it in Python. As for the event handler, which would process incoming MIDI events, I don't know exactly what to do about that yet. That part doesn't need to run at the sample rate. Handling multiple voices of polyphony is also an issue. Can I write my event handler in Python and have it fork threads of the engine per voice? I might just have to try that out and see. I agree that I'm doing way too much asking and not enough coding and trying out, but my system is having extraordinary trouble taking to a Linux distro because of the Emu card.


Sure. There are python examples in the SndObj distribution that use Tk or wxWidget in the main thread for GUI and run the audio in the SndObj thread.

And yes, handling MIDI can be done entirely in Python. What it comes down to is how many loops and list comprehensions the MIDI event handler have to execute per event in addition to the frequency of incoming events. It's a matter of scaling. The mapper I posted in the design thread has a O(N) complexity depending on the number of defined regions.

SndObj provide a MIDI mapper that use a LUT to translate MIDI note numbers into base frequencies that can be fed to an oscillator.

About polyphony: It shouldn't matter if you run 10 or 20 voice threads if you only got two cores/processors because in reality only two threads are then executed simultaneously if it's parallelism you're after, most of them would execute in sequence anyway, to produce output for the next buffer/frame in time.

And when it comes to threading the rule of thumb is to let every thread be domain specific: Just use one GUI thread, one audio thread and so on. Otherwise you might end up having two audio threads with a racing condition and sample accurate timing problems that will prompt your head to explode.

Consul wrote:THAT sentence has explained more to me than anything else so far! Also, I have a friend with a great deal of experience in soft and hard real-time programming, and he's been giving me lessons. Thanks!


You're welcome. ;)

Consul wrote:EDIT: WOOHOO! I actually hit the "Quote" button! :D


Good for you.
Anders Dahnielson

Ardour2, Qtractor, Linuxsampler, M-AUDIO Delta 1010, Axiom 61, Korg D12, AKAI S2000, E-MU Proteus 2k, Roland R-5, Roland HP 1300e, Zoom RFX-1000, 4GB RAM x86_64 Intel Pentium Dual 1.80GHz Gentoo Linux

User avatar
Consul
Moderator
Posts: 189
Joined: Wed Jan 23, 2008 11:19 pm
Location: Port Huron, Michigan, USA
Contact:

Re: Side thread: Sndobj, Python, and RT-safe

Post by Consul » Sat Mar 01, 2008 6:33 pm

dahnielson wrote:Sure. There are python examples in the SndObj distribution that use Tk or wxWidget in the main thread for GUI and run the audio in the SndObj thread.

And yes, handling MIDI can be done entirely in Python. What it comes down to is how many loops and list comprehensions the MIDI event handler have to execute per event in addition to the frequency of incoming events. It's a matter of scaling. The mapper I posted in the design thread has a O(N) complexity depending on the number of defined regions.

SndObj provide a MIDI mapper that use a LUT to translate MIDI note numbers into base frequencies that can be fed to an oscillator.


Thanks! This just might make life easier.

About polyphony: It shouldn't matter if you run 10 or 20 voice threads if you only got two cores/processors because in reality only two threads are then executed simultaneously if it's parallelism you're after, most of them would execute in sequence anyway, to produce output for the next buffer/frame in time.

And when it comes to threading the rule of thumb is to let every thread be domain specific: Just use one GUI thread, one audio thread and so on. Otherwise you might end up having two audio threads with a racing condition and sample accurate timing problems that will prompt your head to explode.


Ah, this makes things more interesting (which is to say, difficult). So I'll have to figure out how to handle polyphony with only one thread, as a fundamental part of my application. No cheating allowed, I guess. Just launching a new engine thread for new incoming events seemed too easy.
Darren Landrum

User avatar
dahnielson
Moderator
Posts: 632
Joined: Wed Jan 23, 2008 11:25 pm
Location: Linköping / Tranås, Sweden
Contact:

Re: Side thread: Sndobj, Python, and RT-safe

Post by dahnielson » Sat Mar 01, 2008 6:46 pm

Consul wrote:Ah, this makes things more interesting (which is to say, difficult). So I'll have to figure out how to handle polyphony with only one thread, as a fundamental part of my application. No cheating allowed, I guess. Just launching a new engine thread for new incoming events seemed too easy.


Well, you execute the code for the first voice, add the result to the output buffer, then execute the second voice, add the result to the output buffer, and so on. If the sampling frequency is 48 kHz and the buffer 256 samples long you got 5.5 milliseconds to execute the code for all voices to fill the buffer and spit it out.

Just think about it. If you have two threads and only one CPU the total time of executing both threads in parallel (well, quasi-parallel by switching between them) is the same as executing them in sequence.

And when doing threading, you usually have a pool of threads, "worker threads", and a manager that fetch them jobs because launching brand new threads always come with an overhead.
Anders Dahnielson

Ardour2, Qtractor, Linuxsampler, M-AUDIO Delta 1010, Axiom 61, Korg D12, AKAI S2000, E-MU Proteus 2k, Roland R-5, Roland HP 1300e, Zoom RFX-1000, 4GB RAM x86_64 Intel Pentium Dual 1.80GHz Gentoo Linux

User avatar
Consul
Moderator
Posts: 189
Joined: Wed Jan 23, 2008 11:19 pm
Location: Port Huron, Michigan, USA
Contact:

Re: Side thread: Sndobj, Python, and RT-safe

Post by Consul » Sat Mar 01, 2008 6:56 pm

This is my fourth attempt at a reply as my head spins around this information. I think what I'll concentrate on now is designing my synth's architecture, then I'll design and build the GUI (I've always been a "design the interface first" kind of developer). From there, I'll figure out one problem at a time what I need to solve. Maybe I can take a look at some source code to see how others have done it.

It's quite clear to me that I'm most severely lacking in the computer science side, rather than the math or electronic side.
Darren Landrum

User avatar
dahnielson
Moderator
Posts: 632
Joined: Wed Jan 23, 2008 11:25 pm
Location: Linköping / Tranås, Sweden
Contact:

Re: Side thread: Sndobj, Python, and RT-safe

Post by dahnielson » Sat Mar 01, 2008 7:03 pm

If you use SndObj then you just create a bank of oscillators, filters and amps for the voices. Usually you decide on a fixed number of maximum voices that you think your synth should handle, like 120, and then create 120 ocillators, 120 filters and 120 amps and hook up the output of each voice (from the voice amp) to a mixer object with say 120 stereo inputs and 1 stereo output that is fed to SndJackIO (or whatever IO object you decide to use).

Btw, you can live 28 years and 22 of them in the company of a computer without ever write a threaded application by design.* It wasn't until I wrote a client for Last.fm that I really needed more than one thread because the HTTP code, fetching meta data and the mp3 stream, would block the GUI code if it self was blocked on the server side until the connection timed out. The application appeared to freeze because and the GUI become unresponsive until the HTTP code got unblocked or timed out if it ran in a single thread.

* Of course, a simple bash script will be multi-threaded if you use background jobs.
Last edited by dahnielson on Sat Mar 01, 2008 7:24 pm, edited 1 time in total.
Anders Dahnielson

Ardour2, Qtractor, Linuxsampler, M-AUDIO Delta 1010, Axiom 61, Korg D12, AKAI S2000, E-MU Proteus 2k, Roland R-5, Roland HP 1300e, Zoom RFX-1000, 4GB RAM x86_64 Intel Pentium Dual 1.80GHz Gentoo Linux

Post Reply