Draft of Tomorrow's 05/21 Print Interview w/Lennart Poettering

Paul W. Frields stickster at gmail.com
Wed May 20 14:41:05 UTC 2009


Awesome. I made a couple minor spelling and other trivial fixes
throughout.  New version below:

* * * 
*The Sound of Fedora 11 - Audio Control with Lennart Poettering*

Where would we be without sound?  It's the most primitive of 
communication methods, and yet it has spawned so much technology around 
it.  Whether you're a musician, a DJ, riding a bus to work, or even just 
stuck in a cubicle listening to the radio somewhere, sound has become an 
integral part of your daily experiences.  When Fedora 11 lands, along with 
it will land a number of enhancements to the sound subsystem, 
including unified volume control, per stream and per device monitoring, 
and proper Bluetooth audio support.  I recently caught up with Lennart 
Poettering, Red Hat Desktop Team Engineer and resident audio guru.  
Here's what he had to say about the upcoming improvements and what the 
future holds:

*1. Please introduce yourself and give us a brief intro to how you 
started working on the upcoming audio improvement in F11.*

I am Lennart Poettering and have been working for Red Hat in the Desktop 
Group for two years now this month. I live in Berlin, Germany.

PA has been part of Fedora since F8. Since then we used to ship two 
volume control appications: the GNOME volume control and a PA specific 
tool (pavucontrol). The latter was mostly a showcase what can be done 
with PA and I wrote it mostly as a demo, not because I thought it was 
any good as an UI.

Of course having these two volume control UIs in Fedora was a situation 
that badly needed fixing. Especially since both UIs exposed too many 
unnecessary options: the GNOME volume control exposed a lot of low-level 
hardware-specific features that only a tiny minority of people actually 
really understood, and the PA volume control exposed a lot of low-level 
software features that a slightly larger minority of people only 
actually really understood.

Now during the last year we reached a point were the feature set of PA 
for volume controls became very complete (with such things as arbitrary 
meta data on every stream/device, per-stream and per-device monitoring, 
hardware volume range extension, "flat" volumes and lots of other stuff) 
and Jon McCan with help from Bastien Nocera finally took up the work to 
fix the UI situation.

They basically designed the new UI from scratch with input from 
usability experts. It implements many of the features the old 
pavucontrol tool did, but in a much nicer, streamlined way. Also it 
integrates sound theme/event sound control with general audio 
configuraton and volume control in a single UI tool.

*2. Can you give us some background on the upcoming changes to the audio 
subsystem in the Fedora 11 Release?*

If you want to know more about the Volume Control, I'd just refer to the 
Feature page:

https://fedoraproject.org/wiki/Features/VolumeControl

We moved PA 0.9.15 into F11, a nice overview over the new features you 
can find here:

http://0pointer.de/blog/projects/oh-nine-fifteen.html

However that overview is a bit out-of-date. There are quite a few 
additional features that went into 0.9.15, most prominently full 
Bluetooth Audio support: Together with Bastien Nocera and the BlueZ guys 
I worked to make Bluetooth audio easily accessible -- the bluetooth 
applet now exposes an easy dialog that allows you to pair and activate a 
bluetooth headset. After that is done it will automatically appear in 
PulseAudio. If you need to reactivate it later, you can do that with a 
simple click in the applet menu. It works surprisingly well. It even 
works fine for lip-sync video. Which is kind of magic, given that 
Bluetooth Audio doesn't actually offer any timing interfaces, so syncing 
up audio with video is not really possible. I spent a lot of time to 
make sure it does work nonetheless, and it seems to work on the majority 
of headphones although I cannot say for sure if it does for all of them.

*3. Where did the ideas to change all this stuff come from? Didn't audio 
always work in Fedora?*

Depends what you mean by 'work'. Sure, basic audio output worked. But in 
many ways what we had on Linux was not comparable to what MacOS or 
Windows supported. And it still isn't in many ways. However in other 
ways we have now surpassed those competitors.

A lot of the changes we introduced with PA are not directly visible to 
the user. For example the so called 'glitch-free' logic in PA is very 
important for a modern audio stack, however the normal user will never 
notice it -- except maybe because when we introduced it initially a lot 
of driver bugs got exposed that people were not aware of before because 
that driver functionality (usually timing related) was not really 
depended on by any application. In fact even now many of the older 
drivers expose broken timing that makes usage with PA not as much fun as 
it could be.

A more detailed explanation of this 'glitch-free' logic you may find here:

http://0pointer.de/blog/projects/pulse-glitch-free.html

Both Windows Vista and MacOS X have similar g-f logic in their audio 
stacks, however with PA we brought it to the next step. For example, we 
implemented this logic in a zero-copy fashion and with arbitrary sample 
types. This allows us to pass PCM data through our pipelines without 
ever having to copy/convert it unless we really have to.

So yes, as you might noticed I spend a lot of time to get low-level 
internals right. And I like to speak about it, even though most people 
are not aware of all those technical details and how awesome this all 
is.  ;-) That said, this stuff isn't perfect yet and could need more 
improvements.

But it's not all just in the low-level details. Also on higher levels we 
got inspired by how our competitors do things. For example the new 
"flat" volume logic was pioneered in Vista, and we have now adopted a 
similar logic in PA. It's a great way to reduce the complexities of 
volume control by 'merging' a few of the sliders in the pipeline. It 
thus solves the "So which slider is now causing my volume to be too 
low?" a bit. But also here, there's more work to be done.

It's not all just getting inspired by our competitors. There are a lot 
of genuinely new features in PA that none of them have (at least to my 
knowledge). For example, in PA we have 'spatial' event sounds. I.e. if 
an event sound sound is triggered by a mouse click/dialog at the left 
side of the screen the sound is generated more from the left speakers, 
and similar for the right side. This is of course mostly a toy. But I 
think a useful one  ;-) .

Listing all the fancy features PA has would certainly be a bit too much 
for your interview. So I'll leave it with this...  ;-)

Generally, we get inspiration from everywhere. And sure, as long as the 
most basic music playback was enough for you audio did always work in 
Fedora. But OTOH, when we started with the integration of all of these 
new audio features into Fedora two years ago the audio stack was still 
at a point of what was modern in the 90's. With the new features of the 
new volume control and PA we are working on bringing Linux audio to what 
is modern today.

*4. Can you also give us a comparison of our new audio framework in 
reference to other audio frameworks and audio subsystem models that are 
out there?*

There are many frameworks out there. On Free Software systems PA doesn't 
really have any competitor. Some people think that JACK is one, but it 
actually is not. JACK is clearly focussed on audio production and not 
very useful on the desktop otherwise. For example, it is strictly 
designed to provide very low-latency at the price of power consumption. 
This is the right thing to do for audio production but not on the 
general desktop. Logic like 'glitch-free' (see above) makes a lot of 
sense for the usual desktop audio since it allows flexible adjusting of 
the latency to what is needed. If used properly it can be used to 
decrease the interrupt rate to 1/s, while still allowing instant 
reaction to user input. Since most PCs these days are laptops theses 
kind of power consumption related features are very important.

One of the current weaker points of Audio on Linux is that we have this 
clear seperation of JACK for audio production and PA for 
desktop/embedded. Other operating systems have managed to make this a 
bit smoother by having a single stack for both. This however actually 
has both advantages and disadvantages.

To improve the sitatuion f now we focussed on making PA and JACK 
cooperate better. In F11 when JACK needs low-level access to an audio 
device it will tell PA so and PA will comply and release the device. 
This should make switching between the two sound systems easier though 
of course this is no perfect solution. Given the lack of manpower 
further integration is unlikely to happen anytime soon -- though both 
the JACK guys and I seem not generally opposed to something like that.

Now, if you compare our audio stacks with those of the big other 
operating systems (Windows and MacOSX), then besides the fact that they 
usually integrate desktop audio and audio production better than we do 
(as mentioned) there are many things we are better in and many they are 
better in. We certainly have more flexibility: i.e. depending on your 
application you can access audio on a lot of different levels: you can 
access ALSA directly if you need very low-level control, or via PA for 
desktop level control. You have APIs like GStreamer for media streaming 
and so on.

This flexibility however translates to more complexity in many ways, and 
a hodgepodge of API styles. (OTOH Apple's CoreAudio actually isn't as 
streamlined as many MacOS proponents like us to think.) The 
documentation for our APIs is usually much worse then theirs. We really 
need some improvements in that area. Featurewise, PA usually has better 
networking related features then those counterparts. But there's a lot 
of features they have right now we lack.

Other Unixes, such as FreeBSD and OpenSolaris are still stuck with OSS 
(Open Sound System) audio. In F11 we finally switched OSS off by default 
(though you can still reenable it via some minor hackery). OSS was the 
predecessor of ALSA. Thankfully it is now fully obsolete on Linux. OSS 
is mostly a design from the early nineties. It has received only minor 
updating since then. It is no way comparable to what we now have on 
Linux or even what MacOS or Windows provide. (Although is has some very 
vocal fans which like to write me hate mails because I say things like this)

*5. This work all started in earlier releases dating all the way to even 
Fedora 8, if I am correct. How has all this stuff progressed and evolved 
from then? What was done in previous releases that enabled building upon 
for this release?*

Fedora 8 was the first release where we integrated PA. In Fedora 9 we 
stabilized PA support. In F10 we integrated the 'glitch-free' logic 
which turned out to be quite a bumpy ride given that it exposed a lot of 
timing related driver bugs. In F11 g-f has now been made more robust and 
most of the more modern audio drivers should now be fixed. Also we have 
now started to push PA support more into the UI, like with this new 
volume control.

*6. What are the plans for the future, if any, in this particular space 
in the distro?*

I am working on multiple things for F12. Firstly there will be a couple 
of more low-level changes to PA. The core will be made more threaded. 
Right now, we run most things in one 'main' thread and do low-level 
audio I/O in one thread for each audio card. My plan for F12 is to split 
that one 'main' thread up into as many threads as possible. THis should 
make PA more robust for a couple of operations, and make latencies more 
reliable.

Then, I am working on considerably beefing up PA's usage of the 
low-level hardware volume controls. For example, many cards have 
seperate low-level volume sliders for "Speaker", "Master", "PCM" (and 
more) that are in the line from the PCM data we stream to the speakers. 
PA currently exposes only one of those sliders (usually "Master"). My 
plan is to 'multiply' those sliders and create a single 'product' 
virtual slider from them that has a better granularity and a larger 
range. This rework will also introduce input/output switching and 
probably more.

What has already landed in PA's git repository is support for UPnP A/V. 
When used in conjunction with Zeeshan Ali's Rygel UPnP MediaServer 
implementation this allows streaming any application's audio to a any 
UPnP MediaRenderer (including PS3/Xboxes and all those 'Internet Radio' 
devices). This is actually pretty neat. Later on we hope to make PA a 
MediaRenderer as as well as a MediaServer. This nicely compliments our 
current Apple RAOP support.

And there's a lot of other things planned. We'll see how much of that 
will be ready for F12. I don't like to talk too much about upcoming 
features and planned code if I don't have anything to show yet, so I'll 
leave it at this.

And then there's always a little project of mine that is called 
'libsydney' that is intended to be a portable, modern and friendly PCM 
API. During the last months I focussed more on PA itself though.

*7. Do you feel that work like this helps enhance the desktop experience 
on Linux in general and strengthens the cause of the Linux Desktop, or 
is it more all in day's work?*

I think that PA is the way forward for audio on the Linux desktop. It 
may have its deficiencies -- but everything has. We still have some way 
to go, but I believe that a modern audio layer is really important for 
the Linux Desktop to succeed.

And no, it doesn't feel at all in a day's work. It always is a great 
feeling to see how PA got incorporated into so many distributions and 
how it is now used by so many people. I am pretty sure that only if you 
hack on Linux software you get this in this ways.

*8. Speaking of all in a days work, what are things do you usually work 
on? What do you most enjoy doing outside of work.*

RH basically hired me to help improving audio on Linux. So that's what I 
am doing during work.

Outside of work spend my time with photopgraphy. And I am trying my best 
to travel to interesting places as much as I can and my time off allows.

* * *

-- 
Paul W. Frields                                http://paul.frields.org/
  gpg fingerprint: 3DA6 A0AC 6D58 FEC4 0233  5906 ACDB C937 BD11 3717
  http://redhat.com/   -  -  -  -   http://pfrields.fedorapeople.org/
  irc.freenode.net: stickster @ #fedora-docs, #fedora-devel, #fredlug




More information about the Fedora-marketing-list mailing list