New year means some snow and cold noses here in Warsaw, and a new job to me, this time in the mobile industry, what has rather diversified my day, and that's good. Happy 2009 to you, to your friends and family.

But I am on the KDE board too, it's not going to change. What's recently time-consuming for me is refactoring of the Native Kexi Forms. The most serious and anticipated decision is dropping the (implemented in 2004..2005) idea of the forms component reusable at a rich API level for other applications. The idea has introduced too man layers after months of development, too many to have things maintainable, with just proof-of-concept KFormDesigner being the only app using the framework except Kexi.

Since 2005 things have changed, Qt Designer gained its own reusable libraries. Before someone asks - we don't use them in Kexi Forms not just because that would affect the licensing (LGPL) or because of backward compatibility required by the current Forms' XML format (which is ~95% the same as Designer's but the 5% makes the difference). My point (or read it as feelings) is that while there would be no technical overhead in reusing Qt Designer, we would have overhead counted in man-months.

The first step, mostly finished now was to remove Qt3 Support dependencies. A bit late? True, but not too late. The code using the meta objects and properties is too fragile to receive massive changes in one go. Unlike KoProperty that received rather massive rewrite to Qt4's model/view API (I am not 100% happy with the results yet ) since last summer, Forms are more complex and I am employing strategy of incremental improvements for them. But after all, did I mention that continuous advances in many other parts of KOffice (if not just entire KDE 4) work as true motivation?

Before I resume working on PAM (I need to implement a change to pam_lastlog to fix a pernicious bug), I wanted to just write a quick entry for the paranoid of you who still use PAM for system login.

Since, as you most likely already know, MD5 is once again considered insecure, one obvious concern would be the fact that passwords saved in MD5 on a system are not secure either. For this reason if you’re using Linux-PAM, you can make use of the SHA512 hashing of system password keys, which I already wrote about.

Remember that to use that you have to make sure your Linux-PAM (sys-libs/pam) is built against a recent enough version of glibc. Unfortunately the version of pambase with this feature hasn’t hit stable yet, the bug above is blocking it, and I’m going to have to hack at pam_lastlog to fix that.

What I didn’t write last time, is that you can easily spot if your system is using md5 passwords by using this simple command from root:

# fgrep '$1$' /etc/shadow

Of course one has to access your /etc/shadow file to breach your passwords, so your system has to have been compromised before, but it’s still not nice if they can find out what your basic passwords are.

Moving on.

And well deserved too. 'There are times when the phrase ?Absolutely, totally, gobsmackingly, mindbogglingly amazed? just doesn?t cover it.' ... 'This means that fans, while not calling me Sir, must now refrain from throwing things. Regrettably, no sword is included in the box :)'

And for anybody who hasn't discovered his work yet, start with The Color of Magic and continue until you get to Nation.

[Technorati Tag: Fiction]

[Technorati Tag: Humor]

> Read More... | Digg This!

This year, I'd really like to swap over to using OpenSolaris as my default desktop at home if possible, but there are several things that are just not there yet, and are therefore preventing me from doing so.

The first one is being able to listen to my favorite Internet radio station. If I click on any of the "listen" links (such as the 128k MP3 one in my Firefox browser, I get the "Opening rp_128.m3u" popup, with Totem Movie Player as the default application. If I try to run that, Totem bitches that:

An error occurred The playback of this movie requires a MPEG-1 Layer 3 (MP3) decoder plugin which is not installed [ OK ]

Unfortunately it doesn't just ask me if I'd like to install one (like Ubuntu does the first time I try this on that platform). Also my Mac and Windows XP machine "just work", right out of the box.

I then tried Songbird. Same problem.

I found a LifeHacker entry on Radio Beta that looked promising. I entered "Radio Paradise" in the Quick Search field, and then clicked on the Play icon for the entry that it found. It then bitches that:

In order to listen to this radio, please download VLC and Firefox plugin (for MacOS X and Linux users.)

I clicked on that download link, and found that for Solaris (and therefore presumably OpenSolaris), that there are no precompiled binaries and that I would have to get the source code and build it myself.

My interest started to wane at this point.

Am I missing something? Is there a package I can just install that will allow me to do what I want on OpenSolaris?

[Technorati Tag: OpenSolaris]

[Technorati Tag: Internet Radio]

[Technorati Tag: Radio Paradise]

> Read More... | Digg This!

For once, "Tokyo" refers in fact to a physical place, not some code project. Shocking but true.

Just prior to Christmas, I took a week-long trip to Tokyo Japan on Joyent business. This was interesting for me because it was both my first time to Japan and in fact first time to leave the country. Given that I am a California native, I've had little reason to leave. We commonly say here "Your within a 4 hour drive of almost any environment on earth". California is just a great place and I figured if I ever did leave the country, it should be some place particularly interesting, not just Mexico or Canada.

The first thing about traveling to Japan is that jet lag sucks and travel is painful. Sure, like everone has a tragic "I took a 36 hour flight" story, but a 12 hour flight in coach just sucks. The flight is 12 hours there and 9 hours back, thanks to trade winds... shocking that they take a full 3 hours off, but its true. When you take all travel concerns into account (including, in my case, a connection through LAX) I lost about 3 days to travel. I wanted to go to the Tokyo OpenSolaris Users Group OpenSolaris 2008.11 release event, and it was funny that I was scrambling Wed morning (PST) to make it in time for a Thursday evening (JST) event.

Once you get there, "jet lag" takes on a new meaning. Typically I think of "jet lag" as a minor diviation of your sleep schedule, like going coast-to-coast. But in Japan the time is so off, that you get hit hard about 5PM (JST) and then get a second wind around 7PM and then have trouble sleeping till 3-4AM. The first morning I was there I woke up at 5AM and by 6 gave up on trying to sleep.

While I can't talk much about my work there, I was in a data center for 2 days straight, then did a hand off to our other staff at home while we were on standby for another 2 days. We used that time for customer meetings and taking in as much of Tokyo as possible. Lesson to my fellow administrators, when your in a strange place and up against a deadline... pre-stage, pre-stage, pre-stage. I actually took a 2.5" USB powered drive with ZFS Datasets ready for mount and use. ZFS rules.

Anyway... I thought I'd share some miscellaneous thoughts in general about Tokyo for those who've never ventured to Japan:

1. They say that going to Japan is like going to another planet. Not true. It was very much like any other large metropolis... people just don't speak English.
2. I was told, that in a city like Tokyo which does a lot of international business that most people know english pretty well. Bullshit. In the large hotels in Shinjuku, ya, but everywhere else they don't know english. Due to the ammount that Japanese culture has integrated english words, they might know a couple of words, but it really comes down to hand jestures. If you walk into McDonalds and say "how are you today?" you get a blank smile. In my hotel (a really nice one actually, in Ariake, even the front desk barely knew any english.)
3. "Large Coffee" in Japanese is "Oh-key ko-he"... life was difficult before this.
4. You always see Japanese crowded into packed areas in the media, so you think Japanese like being crowded. Wrong. They like space too... but when you need to rely on public transportation to get anywhere and you can squeeze into a train, you bear it and cram.
5. Japanese don't look at anyone else. At least, young people don't. In America we're constantly sizing up everyone around us, looking, thinking, perhaps even commenting.... not in Japan. In America if you walk past someone that is alone, you commonly say something like "hey", "yo", "hows it going?", nod, or otherwise acknowledge their existence. In Japan you can be around hundreds of people and feel absolutely isolated and alone. Consequently, its a really depressing and lonely place if your alone.
6. ...unless you wear a kilt. I wore a kilt one day when there and people couldn't believe what they were seeing, women especially. After 3 days of feeling like I didn't exist this was a welcome reaffirmation of my humanity. :)
7. Elderly Japanese (70+?) are much more friendly... they'll commonly give you a smile or say something back if you say hello (in Japanese obviously).
8. The American understanding of "Hello" in Japanese is "Konichiwa"... but in fact, that means "Good Afternoon". There are variations for morning, afternoon and evening. Commonly this is followed with the word "gozaimasu", which adds some formality, like saying "Good morning sir" instead of "Morning" ("Ohayoo gozaimasu").
9. Japanese pronunciation is more important than even the words themselves. I asked the from desk where I could find a "Key-mo-noh" (Kimono)... this turned into a confusing number of jestures and ultimately a dash for a Casio pocket translator. The word was right but due to my bad pronunciation we could not connect.
10. In America we give people a hard time about "butchering our language"... if felt somehow redeeming to have people giving me a look of dispare and amusement as I butchered theirs.
11. Learning Japanese is really tough. Pronunciation is the key to spoken Japanese... but writing is a whole seperate problem, as they have 3 seperate major writing systems Kanji (iconic, drawn from Chinese), Katakana (syllabic, meaning characters that you can sound out), and Hiragana (the American equivalent is cursive). The kick in the teeth is that commonly in Japanese they will use all 3 in a single sentence.
12. Tokyo is huge. Taxi's are expensive, especially if your traveling more than a couple miles. Supposedly a taxi ride from the Narita airport on the edge of town (feels way out of town actually) to the heart of the city will run you US$500 and takes about an hour.
13. Navigating trains in Tokyo is really complex. There are hundreds of stops and the kicker is that unlike most places there is not a single central train authority that runs all the trains.... there are several different train companies with their own lines, so you commonly cross over from one to another. As a result there were many people who have lived there for 5+ years and had considerable trouble navigating the train system unless they were familiar with that particular route.
14. Tokyo is clean. Super clean. And, ironically, finding a trash can is hard to do. All the taxi's and buses have clean white doily things on the head-rests, and people just don't litter. You see the occasional cigarette butt, but thats about it.
15. Bathrooms are fun in Japan. They use electric dryers exclusively, commonly a "toaster" like contraption in which you insert your hands, and a stream of high-pressure air blows across your hands as you slowly pull them up... bone dry hands, totally awesome. Even bathrooms in Japan don't have trash-cans.
16. Toto toilets are scary and wonderful things. You know, you've seen those images of Japanese toilets with an instrument panel right? I could write a whole series just on those things, but needless to say the first time you sit down on a toilet seat thats warm, it freaks you out.
17. The ability to order Sushi like a pro in the US doesn't mean jack sh*t in Japan.
18. All Japanese are short. Totally wrong. I'm 6'4", everyone wanted pictures of me towering over the little Japanese. Just plain wrong, I didn't notice any difference between California and Japan in terms of variation in height. In fact, there were several Japanese construction workers that were massive and definitely not to me messed with.
19. If the Toyota released all their japanese cars in the US, GM and Ford would be out of business. I saw several Toyota's that put Mercedes to shame. You have to see it to believe it.
20. Japanese quality is awesome. If I traveled there regularly I'd probly buy all my clothes in Japan.
21. Adjusting to coinage is odd. The smallest Japanese bill is 1,000 yen (round it to US$10; less due to conversion, but ballpark). $5 and down is all coinage. In the US we tend to discard change (collected in jars, or whatever)... but there, you have to adjust to using coinage frequently or you walk around with a bulging pocket all the time.
22. Mint... apparently mint isn't big in Japan, you don't hardly see it. If its green its almost certainly green tea flavored. Strawberry, however, is very popular.
23. Japanese aren't big on candybars or chocolate in general. At least, not like we are in the US. In a mini-mart in the US we have one or more isles dedicated just to chocolate, commonly in candy-bar form. Over there you find only a couple varieties. Kit Kat and Snickers are the only US bars I saw.
24. Yes, Hentai is as common as they say. Also, Japanese Manga is telephone book sized, not little things like we read in the US.
25. Strange observation... I was hard pressed to find a Japanese magazine about business or computers. I found one magazine about PC's, but most were about TV or culture. I wanted to pick up some economic/news magazines but couldn't find 'em.
26. Vending Machines. You hear that they are everywhere. This is true, there is almost always one within eye shot... however the notion that you can "buy anything in a vending machine" is overblown. Most of the vending machines were just drinks and maybe a can of nuts or something. I didn't see any vending machines for portable electronics, or books, or all the wierd stuff you hear about. I'm sure they exist, but some people make it sounds like you can buy a Sony Walkman in a vending machine in the middle of a park.
27. Dress. Dress varies based on what area ("Ward") of Tokyo you are in, but in general they dress much nicer than in the US. Men most commonly wear a 2 button suit. Young women wear short skirts with knee or thigh high tights and either leg-warmers or tall boots. Teenage boys tend toward jeans and a tshirt.
28. Video Games. If you walk into an arcade, all the arcades are played sitting down! What we commonly consider an "up-right" game, has a little bench. The "crane-pickup" games are really popular and have kool prizes. One arcade had these games filled with food items like ice-cream bars and such.
29. Couples. I was really amazed at how many couples I saw! In the US its generally difficult to tell who is a couple because we've lost the tradition of holding hands. A man and women in San Francisco exiting a restaurants may be a couple, or brother-sister, or friends, or co-workers... its hard to tell. In Tokyo there were tons of couples holding hands and cuddling on trains.
30. Gambling. Gambling is big in Tokyo. Commonly in the form of slot machines and a game called "Pachinko". They don't have card games, and thus most people didn't seem to think of it as gambling, but these things are eveywhere!
31. Mini-marts. Mini-marts are big there, particularly 7-11 and Circle K. People buy lunch, breakfast, and dinner at these places, typically before or after getting on a train. They sell a lot of Ramen (yes, they do sell "Cup o' Noodle" in Japan) and provide hot-water to fill it up before leaving. Other meal items include every variation of rice and seafood you can think of, including sushi.
32. Sushi. I wondered how much better sushi was there than here. I wasn't shocked, the sushi in Japan is unlike anything you've had in the US. I've eaten at some of the high-end places in San Francisco and they don't come close to your average box-lunch sushi there.

    I could go on for a while but will leave it there. I commonly reflected on the movie "Lost in Transation" while in Tokyo. I even got to quickly venture into Shinjuku to the Tokyo Hyatt where it was largely filmed (the "bar" that he hangs out in has a 1,000 Yen cover charge JUST to sit there. A Guinness in a pub can cost me 1800 yen. But man oh man it was a beautiful lounge.) The theme of being disconnected and alone in Tokyo rings true from the film.

    I didn't get to see as much of the city as I wanted to. I especially wish I'd had time to see the legendary Akihabara (Japanese Geek Central), but time didn't permit. None-the-less I'm happy with what I was able to take in. We spent one day without a guide just taking the train some place and exploring around the station, the other day with a guide in between customer meetings.

    I'm absolutely indebted to Alain Hoang who helped guide us and answer our questions. He's an amazing sysadmin and one of the nicest guys I've ever met. If it weren't for his help we would have probly never ventured further than we can walk. Besides that, he deserves a metal for helping me stumble through some basic Japanese and better understand the culture.

    I don't know if I'll ever have reason to return to Tokyo. I certainly would enjoy being able to, especially if it weren't so close to Christmas (I returned the day before Christmas Eve), but given the cost I doubt I would ever return to vacation. Never the less, I've picked up an odd desire to continue learning Japanese and katakana... I've got an odd feeling I'll be back again one day. Who knows.

    So, in short, if you ever have the opportunity to visit Tokyo I encourage you to take it, but make sure you pad the trip with at least 5 days to take in as much as possible.

    
    
    > Read More... | Digg This!

After a longish time, here for you a new chapter of my widely read series For A Parallel World, improving buildsystems to reduce build time on modern multiprocessor, multicore systems.

This time, rather than the usual build failures, I’m going to speak of a parallel install failure. Even though one can think of install as a task that rarely can fall into problems like race conditions and the such, and even though it’s probably the part that gets less boost when using parallel make on a multicore system (since it’s usually I/O bound rather than CPU bound), it’s actually one very fragile part of many packages.

One of the common failures is due to old install-sh script used to simulate the install command on systems too old to have a POSIX-compatible one, and which is also used to create directories recursively if mkdir -p is missing. For a series of reason, this hits pretty often on FreeBSD, but this is beside the point. This can be easily solved by replacing the old faulty script with an updated copy out of automake or libtool, which does not have problems at all.

A few times, the problem is instead due to a broken Makefile.am. Let’s take a practical example from some software I fixed recently after being called in action by nixnut: gramps . Please note that if you look at the bug now you’re going to spoil the post, since it contains the solution straight away, while I’m going to explain it step by step.

Let’s start from the reported build log:

test -z "/usr/share/gramps/docgen" || /bin/mkdir -p
"/var/tmp/portage/app-misc/gramps-3.0.3/image//usr/share/gramps/docgen"
 /usr/bin/install -c -m 644 'gtkprintpreview.glade'
'/var/tmp/portage/app-misc/gramps-3.0.3/image//usr/share/gramps/docgen/gtkprintpreview.glade'
 /usr/bin/install -c -m 644 'gtkprintpreview.glade'
'/var/tmp/portage/app-misc/gramps-3.0.3/image//usr/share/gramps/docgen/gtkprintpreview.glade'
/usr/bin/install: cannot create regular file
`/var/tmp/portage/app-misc/gramps-3.0.3/image//usr/share/gramps/docgen/gtkprintpreview.glade':
File exists
make[3]: *** [install-docgenDATA] Error 1
make[3]: *** Waiting for unfinished jobs....

As usual, the first thing we’re looking for when there is a parallel build (or install) failure are repeated commands. As I’ve shown in Case Study n. 2, when the same command is repeated multiple times it’s often due to mistakes in the Makefiles, thus before thinking of a problem with the dependencies, I check for that. It’s way more common especially on automake-based build systems.

So indeed we can see there are two calls to the install command for the file gtkprintpreview.glade (this also shows us that it’s not a problem of old and faulty install-sh script since the call is directly to the system command). Contrary to what happens when it’s a build rule that is wrongly expressed in the makefile, the double-call during install phase is usually present both using parallel jobs and not. The difference is that when the two calls happen sequentially, the second just overwrites the results of the first; wastes time but it’s successful. On the other hand when parallel jobs are used, the two calls are often enough happening at the same time, and thus we have a race condition.

Okay so next step as usual is to look at the incriminated Makefile.am:

[snip]
docgen_DATA = 
        gtkprintpreview.glade

dist_docgen_DATA = $(docgen_DATA)
[snip]

Here we’re at the core of the problem. The gtkprintpreview.glade file is part of the sources, and it has to be installed as part of the docgen class of files (thus in $docgendir). But the data installed in that path is listed twice, once in the docgen_DATA variable and one in dist_docgen_DATA, causing the file to be installed twice on two independent targets. Since the two targets are independent, when using parallel jobs they both will run at the same time the same command.

Let me try to explain what the mistake has been. By default the sources are packaged up in the final tarball, if they are not generated by rules from the make process; sometimes you wish files that are built by make to still be distributed, and thus you either have to use EXTRA_DIST or prefix dist_ to the class of the installed files, to explicit that the files have to be distributed. Unfortunately the gramps developers didn’t know automake well enough, and thought that dist_docgen_DATA worked quite a lot like EXTRA_DIST (maybe it actually used EXTRA_DIST in the past, for what I know), and thus duplicated the variable content.

The solution? Just replace the use of docgen_DATA with dist_docgen_DATA and remove the second definition, the problem is solved at the source.

Just like Jim Rohn said: Miss a meal if you have to, but don't miss a book, these days I force myself to read more books more often than before. Here is a list of English fictions that I managed to read fortnightly, in the order of my preference, just in case you are looking for good books and want to read some of them as well.

• Paulo Coelho's The Alchemist. I always wanted to finish this one and finally I did. Needless, it is only one of its kind. At one point in your life, you simply need to read and digest this book, because there is just Santiago in each and everyone of us.
• The Kite Runner by Khaled Hosseini. I wrote the short review before. I also watched the movie, but the movie was not even close to the book.
• The Curious Incident of the Dog in the Night-time, an award winning book from Mark Haddon. You may finish this small book in a day, but that day will be the day that changes your life. The story is about an intelligent (lots of excellent math references in the book) but autistic young boy who tried to find the villain that murdered a dog. However, the story unfolds into something a lot bigger than that, something that changed his life significantly.
• The Witch of Portobello, again from Paulo Coelho.The witch, who was murdered, was a woman named Athena. She was very special and her life story was narrated by her relatives and friends. A trivia: find a sentence that exists both in this book and The Alchemist.
• Conn Iggulden's Emperor series: The Gates of Rome, The Death of Kings, The Fields of Swords, The Gods of War. A heavily fictionalized story of arguably the greatest leader ever in the whole Roman Empire, Julius Caesar. I really like these books and want to write a separate short review for this series.
• Marina Lewycka's very funny A Short History of Tractors in Ukrainian. Both entertaining and educating. I wish someday it will be made into a movie. Set in modern day England, the story of the tractors is told by one of the protagonists, a brilliant old immigrant from Ukraina.
• A Thousand Splendid Sun by Khaled Hosseini (again). A sad story about Mariam and Laila, two Afghan women. It is a very nice read, especially after The Kite Runner. I am, however, a bit spoiled because fews of the plots become more predictable once you recognize The Kite Runner's patterns.
• The Book Thief, an award-winning bestseller, written by Markus Zusak. A sad but vivid story of a young German girl's experience during the second world war. She was closed to Death several times, and it was Death who narrated her story of life.
• The Last Templar by Raymond Khoury, that draws the commonly used theme: the adventure to find out hundred years old relic, supposed to be guarded by the Templar knight in the old time, which may change the foundation of the religion.
• Sandstorm, a wonderful thriller from ex-SAS Michael Asher. George Sterling, upon a hint that his son who was missing in the desert some time ago, mounted a journey to try to find back the lost son. Right from the beginning, it was not smooth at all and soon he would be involved in a mystery larger than just a family reunion.
• A collection of short stories, The Veteran by Frederick Forsyth. The highlight is the short story of the same title, a detective investigation of a murder that looked boring and usual at the beginning but then revealed a much larger matter in the end.
• Captain Corelli's Mandolin. Everyone knows the Nicholas Cage version (of the movie), but trust me, the book is much much better. Set in the second world war, the story is about the struggle of an Italian captain in an island in Greece.
• Another one from Raymond Khoury: The Sanctuary. Again about a secret, a centuries-old relic, that was supposed to change the way we look at the life and the universe.
• The Apocalypse Watch, written by Robert Ludlum. A top-notch agent disappeared after successfully sneaked into a supersecret military facility. What did happen to him? His brother was set to find that out.
• With a backdrop of second world war, The Black Order, a spy-thriller by James Rollins, is a story of the search for the cause of life. Introducing the concept of quantum evolution, apparently it was also possible to abuse the technology to "modify" the life itself.
• A reporter was missing and someone was supposed to find him. All of sudden, a mystery and a big conspiracy were the stake. Andy McNab's Crossfire.
• East of The City by Grant Sutherland. Yet another thriller (albeit with a slow start) about an underwriter which, when doing an investigation, was dragged into his dark past.
• What if you try to trace the mysterious girl who lived in your place before? That is what Michael Connelly wrote in the tech-thriller Chasing the Dime. At first, it looked just normal, but soon it became an important part of a dangerous game.
• Ultimate Weapon from Chris Ryan. A story of two men fighting in war-time Iraq to find their loved one.

I may forget some other books. But maybe those are not really important after all.

I`ve always wanted to try this side-by-side bit. I think I like black and white much better. Yes? No?

Kein Sieger glaubt an den Zufall.

-- Friedrich Nietzsche

(English translation: No winner believes in coincidence)

Photo by this girl.

If you’ve been following my blog for a while you probably remember how much I fought with VirtualBox once it was released to get it to work, so that I could use OpenSolaris. Nowadays, even with some quirks, VirtualBox Open Source Edition is fairly usable, and I’m using it not only for OpenSolaris but also for a Fedora system (which I use for comparing issues with Gentoo), a Windows install (that I use for my job), and a Slackware install that I use for kernel hacking.

Obviously, the problem is that the free version of VirtualBox come with some disadvantages, like not being able to forward USB devices, having limited type of hardware to virtualise and so on. This is not much of a problem for my use, but of course it would have been nicer if they just open sourced the whole lot. I guess the most obnoxious problem with VirtualBox (OSE at least, not sure about the proprietary version) is the inability to use a true block device as virtual disk, but rather having to deal with the custom image format that is really slow at times, and needs to pass through the VFS.

For these reasons Luca suggested me many times to try out kvm instead, but I have to say one nice thing of VirtualBox is that it has a quite easy to use interface which allows me to set up new virtual machines in just a few clicks. And since nowadays it also supports VT-x and similar, it’s not so bad at all.

But anyway, I wanted to try kvm, and tonight I finally decided to install it, together with the virt-manager frontend although there are lots of hopes for this, it’s not yet good enough, and it really isn’t usable for me at all. I guess I might actually get to hack at it, but maybe this is a bit too soon yet.

Continue reading on my blog for the reasoning, if you’re interested.

I’ve been told quite a few times that my posts tend to be too long, and boring for most basic users, so for a time I’ll try to use the “extended content” support from Typo, and see how people react. What this means is that the blog post is just summarised on feeds, and on aggregators like Planet, while the complete text can be read by accessing the article directly on my blog.

When I started my work reporting bundled libraries almost an year ago, my idea had a lot to do with sharing code and just to the side to do with the security issues related to bundled libraries. I had of course first hand experience with the problem, since xine-lib has (and still in part had) bundled a lot of libraries. When I took over maintainership of it in Gentoo, it was largely breaching policy, and the number of issues I had with that was huge. With time, and coordination with upstream (to the point of me becoming upstream), the issues were addressed, and nowadays most of xine-lib bundled libraries are ignored in favour of the system copies (where possible; some were largely modified to the point of not being usable, but that’s still something we’re fighting with). Nowadays, the 1.2 branch of xine-lib already doesn’t have a FFmpeg copy at all, always using the system copy (or an eventual static copy built properly).

But nowadays I started to see that what is obvious to me about the problems with bundled copies of libraries is not obvious to all developers, and even less obvious to “power users” who proxy-maintain ebuilds and just want them to work for them, rather than complying with Gentoo policies and standards. Which is why I think that sunrise and other overlays should always be scrutinised carefully before being added to a system.

At any rate, for this reason I’m going to explain in this post why you should not use bundled internal copies of libraries for packages added to Gentoo, and why in particular these packages should not be deemed stable at all.