ls16: another take on a LS DSSI plugin

You name it!
luisgarrido
Newbie
Posts: 22
Joined: Wed Mar 25, 2009 12:16 pm

ls16: another take on a LS DSSI plugin

Post by luisgarrido » Fri Jun 25, 2010 11:39 pm

Hi there.

ls16 is a "proof of concept" type of project in my investigations about audio plugin guis for Linux.

Still, it is not completely useless, so I decided to release it. Read the webpage, take good note of the caveats and give it a whirl to see if it is something you can find a use for.

http://ls16.sourceforge.net/

Cheers,

Luis

luisgarrido
Newbie
Posts: 22
Joined: Wed Mar 25, 2009 12:16 pm

Re: ls16: another take on a LS DSSI plugin

Post by luisgarrido » Mon Jun 28, 2010 12:33 am

Hi.

A few users have identified a couple of issues when installing ls16.

If after installing it ls16 is not detected, it is most likely because the linuxsampler installation package didn't add liblinuxsampler to the dynamic loader cache. You can test this by observing the results of

Code: Select all

ldd /usr/lib/dssi/ls16.so
or

Code: Select all

jack-dssi-host -v ls16.so
It will complain that liblinuxsampler can't be found. The solution is creating as root a file named /etc/ld.so.conf.d/linuxsampler.conf that contains the path to LS libraries, typically:

Code: Select all

/usr/lib/linuxsampler
And then updating as root ld's cache:

Code: Select all

ldconfig
Another issue that happens in Fedora distributions is that no DSSI GUI can be used because of a problem with OSC and the network configuration. The solution is to add your hostname (the output of the command 'hostname') to the line that describes the loopback interface in /etc/hosts, so it reads something like:

Code: Select all

127.0.0.1              localhost.localdomain localhost mycomputer.mydomain
ls16 seems to really shine when used as an output plugin in a midi bus in qtractor.

Cheers,

Luis

Alex
Moderator
Posts: 316
Joined: Wed Jan 23, 2008 9:08 pm

Re: ls16: another take on a LS DSSI plugin

Post by Alex » Tue Aug 31, 2010 8:43 am

Luis, ls16 works pretty well in Muse too.

I would add here, that having 2 instances in a project can be problematic on startup, so maybe you could have brief glance at the DSSI in Muse, and check how ls16 reacts with the muse startup sequence. (Ideally, user with larger templates might employ 8 or 10 ls16 instances in a default template, as an example)

A decent plugin, that with a little tweak and polish, could prove to be a decent addition to the musician's toolbox.

Thanks for sharing,

Alex.

lgarrido
Newbie
Posts: 4
Joined: Wed Jan 23, 2008 11:51 pm

Re: ls16: another take on a LS DSSI plugin

Post by lgarrido » Tue Aug 31, 2010 12:30 pm

Hi, Alex.

As we already discussed by email, I need to be able to reproduce the condition you reported before I am able to do anything about it. ls16 is just a small link in a fairly complex toolchain which hasn't seen many use thus far, we are treading some new ground here: DSSI support in MusE is rather new and ls16 is, as far as I know, the first 3rd party liblinuxsampler client ever published.

The bug could be anywhere: first and foremost candidate is ls16 itself, of course, but it could also lurk in MusE, liblinuxsampler, libgig or perhaps somewhere else. If you can provide a consistent use case that triggers the bug (preferably built upon free GIGs I can download) I'll have a look at it when time permits.

Did you find the same problem in qtractor?

Thanks for the feedback. Cheers,

L

Alex
Moderator
Posts: 316
Joined: Wed Jan 23, 2008 9:08 pm

Re: ls16: another take on a LS DSSI plugin

Post by Alex » Tue Aug 31, 2010 1:53 pm

lgarrido wrote:Hi, Alex.

As we already discussed by email, I need to be able to reproduce the condition you reported before I am able to do anything about it. ls16 is just a small link in a fairly complex toolchain which hasn't seen many use thus far, we are treading some new ground here: DSSI support in MusE is rather new and ls16 is, as far as I know, the first 3rd party liblinuxsampler client ever published.

The bug could be anywhere: first and foremost candidate is ls16 itself, of course, but it could also lurk in MusE, liblinuxsampler, libgig or perhaps somewhere else. If you can provide a consistent use case that triggers the bug (preferably built upon free GIGs I can download) I'll have a look at it when time permits.

Did you find the same problem in qtractor?

Thanks for the feedback. Cheers,

L

Luis, i installed qtractor briefly, and it seemed to work the same.

I've been testing this fairly consistently over the last 2 weeks, but i've yet to find a clue.

I'm unlikely to use ls16 for larger templates, but i thought i'd do you the courtesy of testing it and trying it out with a larger resource use case.

And i'll get back to it if and when time permits.

Alex.

lgarrido
Newbie
Posts: 4
Joined: Wed Jan 23, 2008 11:51 pm

Re: ls16: another take on a LS DSSI plugin

Post by lgarrido » Tue Aug 31, 2010 5:16 pm

Great, I appreciate your effort and I'll be glad to have a look at it when you or someone else can isolate this pesky troublesome condition.

For completeness, I'll describe Alex's find here: when you create a complex project in MusE that uses two ls16 instances all seems to work fine, but after you save it and try to load it back later on it will crash during the GIG loading stage.

I have tried to reproduce this condition without success, so it is difficult to pinpoint where the problem might lie.

Alex
Moderator
Posts: 316
Joined: Wed Jan 23, 2008 9:08 pm

Re: ls16: another take on a LS DSSI plugin

Post by Alex » Tue Aug 31, 2010 7:09 pm

Luis, maybe a clue.

In a standalone instance of LS, only a fraction of the gig sample lib is loaded into memory. So if i have 12 gig files representing my 1st violins (each one with multiple articulations), they're still going to be really efficient, and use negligible memory in relation to their size.

It seems that ls16, as a DSSI, (when the user is adding gigs to an ls16 instance), is loading the entire gig file instead. If i attempt add my 12 gig files for 1st violins to an instance of ls16, i'm chewing nearly a third of my memory already, and that's without loading anything into channels. ls16 is slightly expensive as a DSSI as it is (10-12% mem increase without libs, per ls16 instance), but add complete gig files to that (at least, that's what seems to be happening), and 2 instances is going fairly close to mem limit. (4gb of ram here, and 2 instances of ls16 with about 12 gig files per instances eats about 70-78% of ram, not including system ram requirements)

It might be worth looking at 2 areas, imho.

1.) The efficiency of an ls16 DSSI instance on it's own, without gigs. Can the process in muse be more memory friendly? Could others try loading 2, 4, 6, 8 ,10 instances, without gig files loaded, and relay the result of mem usage, etc.....?

2.) The second is of course, finding out if ls16 is indeed trying to load entire files, instead of pre-samples.

I hope this sheds some light, or least pinpoints to some degree possible challenges that can be dealt with or improved on.

And i stress here that i'm not a coder, so all of the above may be wrong.....

Alex.

lgarrido
Newbie
Posts: 4
Joined: Wed Jan 23, 2008 11:51 pm

Re: ls16: another take on a LS DSSI plugin

Post by lgarrido » Wed Sep 01, 2010 7:57 pm

Hi, Alex.

That is an intriguing hypothesis.

ls16 opens up 16 sampler channels upfront, which pre-allocates a hefty chunk of memory which may be used or not afterwards. I definitely should include some optimizations there (lazy initialization of channels, offering the user the possibility to choose one of the three available memory lifetime models for an instrument, etc.)

Anyway, I made a comparison with qsampler using the RSS figure offered by ps and its derivatives:

- Owned by qsampler the linuxsampler process uses up some 16 MB RSS. After loading one channel with the tiniest of gigs (a 140 KB JazzBass.gig I found at Worra's) the linuxsampler process raised to 213 MB RSS.

- jack-dssi-host + one single instance of ls16 uses 201 MB RSS. I guess that high figure comes from putting LS in a later stage of initialization and buffer acquisition than qsampler does. But after I loaded one channel with JazzBass.gig it went to 202 MB RSS.

I tried the same with maestro piano (about 1 GB). RAM usage is 279 MB for qsampler and 267 MB for ls16 (in both cases the LS back-end seems to cache about 66 MB of the full 984 MB of that file).

At first glance there seems not to be much difference in RAM usage with qsampler or ls16.

However, it is true that every new ls16 instance adds up an extra 100 MB RSS, since the plugin creates a new AudioOutputDevice for every instance with all it entails, like an extra disk thread. I really should try to optimize this.

Could it be that these extra 100 MB are the ones that destabilize Alex's setup? I wonder.

Alex
Moderator
Posts: 316
Joined: Wed Jan 23, 2008 9:08 pm

Re: ls16: another take on a LS DSSI plugin

Post by Alex » Thu Sep 02, 2010 11:16 am

Luis, i guess my first question as a reply is, why does ls16 need so much memory to create a thread, and the bits? If i load only 10mb of gig files, for example, i lose a large chunk of memory i can't use.
I would also ask, if standalone LS can use so little memory, not only for startup, but for it's sample portion ram use, then is ls16 actually trying to load files into that 100mb, instead of just seeking the path to the file on the computer, and letting the backend LS libs handle the sample piece/memory allocation?
Next question is, does ls16 need to create a thread at all? (I ask because i don't know.) Isn't there existing threads in muse already? If so, can these be used?

Finally, do the LS libs impose a penalty in any way to have a DSSI option?

Thanks for the comprehensive feedback,

Alex.

lgarrido
Newbie
Posts: 4
Joined: Wed Jan 23, 2008 11:51 pm

Re: ls16: another take on a LS DSSI plugin

Post by lgarrido » Thu Sep 02, 2010 1:02 pm

Alex,

ls16 only calls liblinuxsampler, as qsampler does through LSCP. It is, if you want, a plugin-ish way to launch a linuxsampler process and encapsulate some LSCP calls. ls16 doesn't load any gig file, it just asks LS to do it.

All that memory usage comes from LS code, and what I demonstrated through the previous test case is that after a gig has been loaded there is no difference between qsampler and one single instance of ls16 in that respect.

The only difference is that ls16 initializes LS stuff before loading any gig, so it allocates the memory sometime earlier down the line, but in the end you have to load a gig sooner or later, and then the linuxsampler libs will eat up that RAM independently of whatever frontend are you using for.

Now, if you add more than 16 channels you need extra instances of ls16, and then is when there is indeed a difference. Every ls16 instance will ask LS to create a new AudioOutputDevice and, as a consequence, LS spawns a new disk thread and allocates a new set of buffers, apparently eating up an extra 100 MB.

I designed ls16 this way for two reasons:

1) To provide for the possibility that a host would ask two instances with different sample rates. I am not aware of any hosts that needs this, but the dssi specification allows it, and an AudioOutputDevice can only work at a predetermined sample rate set up on creation. Quoting liblinuxsampler docs, "the sampler engines currently assume this to be a constant value for the whole life time of an instance of the implementing audio device."

2) Justifiable laziness. To optimize this, all the plugin instances could share a pool of reusable AudioOutputDevices classified by sample rate and create a new one only when a new sample rate does appear. However, then I need to create a different set of audio buffers for every ls16 instance so the host can individualize them. This complicates my life substantially, and ls16 in its current incarnation is only a proof of concept project that already stretches the dssi specification beyond its limits, so I didn't want to spend much time in premature optimizations for a project that is still in a very fluid state. Anyway, I plan to address this issue in future developments.

Summarizing, up to 16 channels ls16 is equivalent to standalone. Beyond that, there appears to be a 100 MB penalty for every new ls16 instance. It certainly looks a lot, I didn't know that an AudioOutputDevice was so RAM hungry. If Christian or Andreas have any further ideas on how to optimize this I am definitely open to suggestions.

Cheers,

Luis

Post Reply