A draft whitepaper of my parameterization idea

Everything and anything, but nothing about the LinuxSampler project.
User avatar
Consul
Moderator
Posts: 189
Joined: Wed Jan 23, 2008 11:19 pm
Location: Port Huron, Michigan, USA
Contact:

Re: A draft whitepaper of my parameterization idea

Post by Consul » Thu Mar 27, 2008 2:13 am

I definitely wasn't thinking of doing this with a network editor, though that does make for a nice way to patch modular functions together. Again, Dave Robillard's FlowCanvas might be the ticket: http://wiki.drobilla.net/FlowCanvas -- I even thought about how to use FlowCanvas as a way of making FAUST functions, and then linking them together to make synths. In short, an open-source Reaktor on steroids. :)

For a sampler, though, I have a very different idea for an interface in mind, one based upon the way Melodyne works. I'll try to draw it up, but I'm not the greatest artist in the world. Do you or anyone else here know of a simple way to create decent GUI mockups? I have the QT4 designer, which I guess would get me going.

EDIT: Maybe we can do it both ways, a network editor (a FlowCanvas widget) AND a Melodyne-inspired view of the sample data. I'll have to play around with that.

Of course, FlowCanvas is a GTK widget, so either we'd need to make a Qt version, or use GTK.
Darren Landrum

User avatar
dahnielson
Moderator
Posts: 632
Joined: Wed Jan 23, 2008 11:25 pm
Location: Linköping / Tranås, Sweden
Contact:

Re: A draft whitepaper of my parameterization idea

Post by dahnielson » Thu Mar 27, 2008 1:33 pm

Melodyne-likeness
Please expand on your Melodyne-like idea for a sampler.

Melody is at heart just a time/pitch shifter (well, a very good one) with a extremely friendly interface. In addition to time/pitch shifting it analyzes the content and plot it out on a staff like grid (showing not only the individual notes but also their transitions) and can even display it as regular notation making it very friendly to musicians. Peter Neubäcker is to me the quintessential "mad scientist".

For example Kontakt 3 contain time/pitch shifting playback called "time machine", iirc.

Chris Cannam have blessed our free software souls with the Rubber Band Audio Time Stretcher (not to mention Sonic Visualiser and VAMP) which concept like these can be built upon.

General notion
What I find important, as I previously expressed elsewhere, is the metability ("What, are we just making up words now!? To inventize our vaporware!?" :D ) of an engine. That it's possible to emulate other samplers, e.g. in our sampler create a sampler that can load SFZ or AKAI programs. In addition to being able to create something beyond them.

Maybe SynthEdit should get mentioned in this discussion (ok, done).

Yes, Reaktor on some kind of terpenoid lipid with a carbon skeleton might be a good description...
Anders Dahnielson

Ardour2, Qtractor, Linuxsampler, M-AUDIO Delta 1010, Axiom 61, Korg D12, AKAI S2000, E-MU Proteus 2k, Roland R-5, Roland HP 1300e, Zoom RFX-1000, 4GB RAM x86_64 Intel Pentium Dual 1.80GHz Gentoo Linux

User avatar
Consul
Moderator
Posts: 189
Joined: Wed Jan 23, 2008 11:19 pm
Location: Port Huron, Michigan, USA
Contact:

Re: A draft whitepaper of my parameterization idea

Post by Consul » Thu Mar 27, 2008 4:25 pm

Well, by "Melodyne-inspired", I'm referring to the way that Melodyne visualizes the wave data, the big difference being we won't really have the same kind of time dimension. The same way they do it would work great for a sampler. (By the way, did you know Peter Neubäcker is an ex-Csound guy?) Of course, I'm all for integrating Rubber Band into a sampler.

So, each lane represents the note, which may or may not contain one or more wave files, as it all depends on what the sound designer is up to. There presumably has to be at least one wave somewhere.

Hey, this just brought up some interesting ideas.

Okay, imagine what I've outlined so far. Now, here are some feature ideas, some interface level, and some engine level:

1) Auto-determination of the frequency of the imported wave (select a folder full of waves, and BANG! Automapped! :) ) I'm not sure Rubber Band can do this, but I'm sure something out there can.
2) Display wave data in every note lane, even if there are fewer waves than lanes (ie, if we're pitch-shifting waves up and down). For the note lanes that do not have their own wave and are being shifted to from above and/or below, draw the waveforms a different color or style to indicate.
3) Each lane can consist of multiple layers, each containing a wave file.

I will now define a "slot" as being a single layer of a single lane. A slot can contain wave data, or it can have wave data shifted into it. Each slot has its own set of parameters which can be controlled and modulated, whether through some standard system or the fancier Omnibus system. (Thanks for that name, Anders!)

I use the term wave data because I don't want it to be limited to a single wave. However, multiple waves can exist in a single slot only serially (say that nine times fast!). In other words, you can load multiple waves into a slot, but only one after another.

This engine plus Omnibus should allow for our microsample synthesis kind of stuff. This could also allow for some nice beat-slicing functionality. The control of the Omnibus system is where a canvas-based network editor would come in handy. The key, of course, is for the engine to expose all of its parameters to the outside.

All of the data point between each slot (whether across lanes or layers) can be interpolated in some fashion. This would allow for things like velocity and positional cross-fading, as well as portamento effects. Maybe this is something that Omnibus could handle. I'll have to think about that some more. Also, keep in mind that the layers need not be just velocity layers, but any sort of division per note that the sound design calls for.

Anyway, from an interface standpoint, it could be made easy to switch between the lanes view and the layers-per-lane view. In all cases, you'll instantly see where all the waves are and where they're being shifted to.

So how does it all break down?

In the final analysis, the engine doesn't really need anything more than:

A) Streaming playback of a sample
B) Control over the start point of playback
C) Pitch and time shift capability
D) All of these with appropriate parameters exposed

Everything else can be handled by Omnibus and other middleware. The interface would then serve as way to control and visualize everything.

Sorry for the stream of consciousness. Hopefully, you can make sense of my ramblings.
Darren Landrum

User avatar
dahnielson
Moderator
Posts: 632
Joined: Wed Jan 23, 2008 11:25 pm
Location: Linköping / Tranås, Sweden
Contact:

Re: A draft whitepaper of my parameterization idea

Post by dahnielson » Thu Mar 27, 2008 5:00 pm

Consul wrote:Well, by "Melodyne-inspired", I'm referring to the way that Melodyne visualizes the wave data, the big difference being we won't really have the same kind of time dimension. The same way they do it would work great for a sampler. (By the way, did you know Peter Neubäcker is an ex-Csound guy?) Of course, I'm all for integrating Rubber Band into a sampler.
No, I did not know about Neubäcker's background. Cool.
Consul wrote:1) Auto-determination of the frequency of the imported wave (select a folder full of waves, and BANG! Automapped! :) ) I'm not sure Rubber Band can do this, but I'm sure something out there can.
Aubio, also available as VAMP plugins (onset detection, pitch tracking, note tracking and tempo tracking plugins).
Consul wrote:2) Display wave data in every note lane, even if there are fewer waves than lanes (ie, if we're pitch-shifting waves up and down). For the note lanes that do not have their own wave and are being shifted to from above and/or below, draw the waveforms a different color or style to indicate.
3) Each lane can consist of multiple layers, each containing a wave file.
Ok, you're talking about an interface that let you build new sound textures using different samples by arranging and timestretch/pitchshift them and then be able to manipulate it all by changing parameters?

As you say, the engine part of it can be expressed in the scope of what the Reaktor-like idea should be capable of. It all comes down to being able to create the interface for it.
Anders Dahnielson

Ardour2, Qtractor, Linuxsampler, M-AUDIO Delta 1010, Axiom 61, Korg D12, AKAI S2000, E-MU Proteus 2k, Roland R-5, Roland HP 1300e, Zoom RFX-1000, 4GB RAM x86_64 Intel Pentium Dual 1.80GHz Gentoo Linux

User avatar
Consul
Moderator
Posts: 189
Joined: Wed Jan 23, 2008 11:19 pm
Location: Port Huron, Michigan, USA
Contact:

Re: A draft whitepaper of my parameterization idea

Post by Consul » Thu Mar 27, 2008 5:08 pm

Sweet! It sounds like I'm finally figuring out how to communicate all of this. In other words, you got it. :)

Part of the problem is my descriptions are mixing up all of the different layers: engine, Omnibus (what a great name), and interface. At this point, it might be best to describe each in detail and decide what functions will go where. I'll think on that on my walk to and from class today and post later.
Darren Landrum

User avatar
dahnielson
Moderator
Posts: 632
Joined: Wed Jan 23, 2008 11:25 pm
Location: Linköping / Tranås, Sweden
Contact:

Re: A draft whitepaper of my parameterization idea

Post by dahnielson » Thu Mar 27, 2008 5:14 pm

He he.

Granular synthesis is essential to timestretch/pitchshifting algorithms. Now we're putting another granular layer onto this trifle. :D

Omnibus (LGOC X-type):

Image
Anders Dahnielson

Ardour2, Qtractor, Linuxsampler, M-AUDIO Delta 1010, Axiom 61, Korg D12, AKAI S2000, E-MU Proteus 2k, Roland R-5, Roland HP 1300e, Zoom RFX-1000, 4GB RAM x86_64 Intel Pentium Dual 1.80GHz Gentoo Linux

User avatar
Consul
Moderator
Posts: 189
Joined: Wed Jan 23, 2008 11:19 pm
Location: Port Huron, Michigan, USA
Contact:

Re: A draft whitepaper of my parameterization idea

Post by Consul » Thu Mar 27, 2008 9:09 pm

The Rubber Band library is based on a phase vocoder, not on granular resynthesis. Basically, it uses the Emu method of pitch- and time-shifting. For granular synthesis and resynthesis within our sampler, I'd imagine Omnibus could do most anything we'd want as long as it runs at audio rates.

So, at this point, I'm going to go over what each part of our new sampler will do. This is just brainstorming, so feel free to jump in.

CLASS: Sample Voice
Needed functions (italicized letters represent function parameters):
  • Load Wave File (filename - if one or more is already loaded, append the next one - preread n samples into memory)
  • Set Index (set the "playback head" at sample number n)
  • Play Sample (start playback at pitch m and speed n from Index Position)
  • Calculate Zero-Crossings (it can then cache these to a file)
  • Find Previous Zero-Crossing (find nearest zero-crossing previous to sample n)
  • Find Next Zero-Crossing (find nearest zero-crossing after sample n)
Those last two functions will greatly aid in loop setting and loop point modulation.

That'll do for a start, I think.
Darren Landrum

User avatar
dahnielson
Moderator
Posts: 632
Joined: Wed Jan 23, 2008 11:25 pm
Location: Linköping / Tranås, Sweden
Contact:

Re: A draft whitepaper of my parameterization idea

Post by dahnielson » Thu Mar 27, 2008 10:55 pm

Consul wrote:The Rubber Band library is based on a phase vocoder, not on granular resynthesis. Basically, it uses the Emu method of pitch- and time-shifting.
Aaah. How stupid of me, I actually knew that once. :D
Anders Dahnielson

Ardour2, Qtractor, Linuxsampler, M-AUDIO Delta 1010, Axiom 61, Korg D12, AKAI S2000, E-MU Proteus 2k, Roland R-5, Roland HP 1300e, Zoom RFX-1000, 4GB RAM x86_64 Intel Pentium Dual 1.80GHz Gentoo Linux

User avatar
Consul
Moderator
Posts: 189
Joined: Wed Jan 23, 2008 11:19 pm
Location: Port Huron, Michigan, USA
Contact:

Re: A draft whitepaper of my parameterization idea

Post by Consul » Fri Mar 28, 2008 12:33 am

I think my current class in C++ down at the college is actually serving my well finally. We've been learning about classes, and how to use them as specialized data types with built-in functions to manipulate that data. So the idea behind the voice class would be to control the playback of a wave file in a variety of ways without the rest of the program needing to know anything about wave files.

Now that I've demonstrated I've actually learned something in college, let's continue designing classes!

The wave playback class is easy. Now, we'll need to figure out how to create the functionality for Omnibus. The best way to do this is likely by way of a class defining all of our various data types, and then creating new classes inheriting from that master for the various functions. That would make it easier for any programmers to create new Omnibus blocks, a block being simply a function of one of these subclasses.

So, what would be the best way to handle the various kinds of data in Omnibus? Please feel free to brainstorm, everyone.

EDIT: Well, we've been talking about control signals as audio-rate signals, so there's no reason why they shouldn't be the basic 32-bit float like the audio signals. What other kinds of data would be handy for any Omnibus processing blocks?
Darren Landrum

User avatar
dahnielson
Moderator
Posts: 632
Joined: Wed Jan 23, 2008 11:25 pm
Location: Linköping / Tranås, Sweden
Contact:

Re: A draft whitepaper of my parameterization idea

Post by dahnielson » Fri Mar 28, 2008 12:54 am

Have they mentioned policy classes (as in Meyers/Alexandrescu) yet? The C++ equivalent of Javas interfaces, Pythons protocols (which is extremely informal in comparison) and UMLs generalisation/realisation relationships? :)

However, I feel it's a little bit early to start hammering out classes without having a clear idea about how the applications functionality will come together, what concepts it will be working with. Better to wireframe it and think about it in general conceptual terms instead of implementation specifics.

Not that I will stop you from writing code!
Last edited by dahnielson on Fri Mar 28, 2008 1:25 am, edited 1 time in total.
Anders Dahnielson

Ardour2, Qtractor, Linuxsampler, M-AUDIO Delta 1010, Axiom 61, Korg D12, AKAI S2000, E-MU Proteus 2k, Roland R-5, Roland HP 1300e, Zoom RFX-1000, 4GB RAM x86_64 Intel Pentium Dual 1.80GHz Gentoo Linux

Post Reply