In the IT world we’re obviously full of practises that, albeit working, are very much hinted against because risky, broken on different setups or just stupid. Many of these practises are usually frequent enough either because they can be easy to apply without knowing, or because they were documented somewhere and people read and spread it.

These practices, in most compiled programming languages, when using optimising compilers, are guarded by warnings, almost-errors that are printed on the error stream of the compiler itself when it identifies a suspect construct. If you use Gentoo, you know them very well as you certainly see lots of them (unless you have -w in your CFLAGS).

Lots of people ignore warnings, because either fixing them is too much work as it would require changing a huge part of the code, or because they are not stopping them from compiling. Much more rarely it happens that the code actually works fine and the warning is bogus; it’s not unheard of though. Also the more advanced the warning, the more probabilities it might be implemented wrong.

On the other hand, the vast majority of warnings are put there for a good reason, and should actually be properly taken care of. These warnings could have been used to make a program 64-bit safe years before 64-bit systems started to be widespread, or might have made sure the code for a project written years before GCC 4.3 were to build correctly with the latest version of the compiler. Of course they are not the one and absolute solution, as many changes might not have had warnings before (like the std:: namespace change), but it could have helped.

But I don’t want to talk about compiler warnings today, but rather about Portage warnings. Since a few versions, thanks also to the availability of Zack and Marius, Portage started throwing warnings after a successful merge, giving you insight with possible problems with an ebuild or with the software the ebuild uses. These are pretty useful as they can catch for you if a ./configure switch was renamed, or removed, after a version bump; and they might tell you if the software is doing something risky and you should warn upstream about that. (Why should you warn upstream? Well, packagers often see lots more code than the daily programmer it’s not uncommon that a programmer might not know about an issue that a packager might know (because of the distribution policy about the problem).

In addition to these that might be setup-dependent, repoman also started warning about suspect (R)DEPEND and other issues with the ebuilds. Hopefully, even if repoman will probably become slower by piling up checks in it, it will be nice to make sure developers know what they are committing in.

This is particularly important because there are quite a few sub-optimal ebuilds in the tree already, and while it’s difficult to find and fix all of them, it’d be quite nice if we could avoid introducing new ones.

Unfortunately, I start to worry that it might not be as feasible as I hoped, because there is a huge fault in my idea that adding warning will keep people away from the mistakes: there is lack of documentation on these problems. As much as I wish I could count my blog as a source of documentation I know this is far from the truth, but I haven’t been able to start writing docs again yet because I was following this world rebuild closely, at least to understand how to follow my priorities. I know I’ll be working on quite a few things in the future, especially once the hospital is just a memory, and hopefully I’ll be able to write enough doc so that the warnings become clear enough that the whole tree will be safe for everybody to use under whichever circumstances.

Still in the same Sushi Factory. As usual, the fried prawn rolls quickly become my all-time favorites.

For the version of Thunderbird I'm running under OpenSolaris 2008.05, when you receive an email with images in it, I get a little "banner" at the top of the email where I have to click the "Load Images" button in order to see the images in that email.

Well today it got to the point where it had annoyed me enough that I wanted to adjust this, so that images where automatically loaded. Seems simple right?

After spending about five minutes looking at all the Preferences options and Account Settings options and not finding it, I decided to google for it.

The solution is to bring up Edit->Preferences, click on the Advanced panel, go to the General tab and click on the "Config Editor..." button. Then look for the "mailnews.message_display.disable_remote_image" setting and click on the "true" word to turn it to "false".

Why is this so hard? Why couldn't there be some way of doing it directly from the little banner with the "Load Images" button is? Maybe a little "Change this..." link.

Anyway, hopefully this post will save somebody else a few minutes if they are struggling with the same problem.

[Technorati Tag: Thunderbird]

> Read More... | Digg This!

While browsing through kde-look.org I found a cool idea for visualizing a daily agenda.

This reminds me of the Akonadi architecture diagram and I even have code (probably not up to date) for drawing this kind of diagrams. So I guess it would be doable without too much effort to implement a daily agenda viewer like this. Would be a fun project.

I almost sat in this puddle tonight. It's only water, of course, so it would be harmless. Well, I assume it's only water. But anyway. Can you image sitting in a little puddle of blood? Now, that's a great shot, Jon.

As I was standing on the train tonight holding my camera, I noticed that I was looking at my reflection in the window as we dove into dark tunnels. So, what else would you do, right? Click. The more and more images I shot, though, the more and more people slowly moved away from me. I wonder why? Anyway, I just found a nice way of getting some breathing room on a crowded train. Cool.

A while back I blogged about querying dbpedia with sparql. The queries in that blog were pretty simple. Today, I present a more complicated example.

You could also look for stuff near Gran Canaria or Berlin.

I noticed that four of the book lists that I have near the top left corner of my main blog page, have acquired bit-rot. Most of the images of the book covers were no longer being found.

Now I don't know whether this is a result of Amazon changing their API's earlier this year, or the cache URL's are just no longer valid, but I thought I'd fix them up. Rather than taking the URL that Amazon supplies when you search for a particular ISBN programmatically, I'm now using the "standard" medium size image URL that seems to nicely work with most books at Amazon with a recent publication date.

The four lists are:

• 1001 Books You Must Read Before You Die.

• Pulitzer Award Winning Novels.

• 100 Favorite Mysteries of the 20th Century.

• Salon.com Reader's Guide to Contemporary Authors.

I'm one of those people that finds it easier to remember an unfamiliar book by its cover than its title. That doesn't always help as they seem to need to change the art-work for the same book with each new release. It did enable me to find a few of them at the Los Altos library book sale last Friday evening.

I also highly recommend the Salon.com Reader's Guide to Contemporary Authors. The list only gives the suggested book to start with for each of the authors, but the actual book goes much further than that with mini biographies and bibliographies plus various essays and digressions under a multitude of topics.

[Technorati Tag: Books]

[Technorati Tag: Lists]

> Read More... | Digg This!

Since I now have a true SMP system, it’s obvious that I’m expected to run parallel make to make use of this. Indeed, I set in my make.conf to use -j8 where I was using -j1 before. This has a few problems in general and it’s going to take some more work to be properly supported.

But before I start to get to those problems, I’d like to provide a public, extended answer to a friend of mine who asked me earlier today why I’m not using Portage 2.2’s parallel emerge feature.

Well, first of all, parallel emerge is helpful on SMP system during a first install, a world rebuild (which is actually what I’m doing now) or in a long update after some time spent offline; it is of little help when doing daily upgrades, or when installing a new package.

The reason is that you can effectively only merge in parallel packages that are independent of each other. And this is not so easy to ensure, to avoid breaking stuff, I’m sure portage is taking the safe route and rather serialise instead of risking brokenness. But even this, expects the dependency tree to be complete. You won’t find it complete because packages building with GCC are not going to depend on it. The system package set is not going to be put in the DEPEND variables of each ebuilds, as it is, and this opens the proverbial vase to a huge amount of problems, in my view. (Now you can also look up an earlier proposal of mine, and see if it had sense then already).

When doing a world rebuild, or a long-due update, you’re most likely going to find long chains that can be put in parallel, which I sincerely find risky, but they don’t have to be. When installing a new package, on a system that is already well installed and worked on for a few weeks even, you’ll be lucky (or unlucky) to find two or three chains at all. If you’re doing daily updates, finding parallel chains is also unlikely, as the big updates (gnome, kde, …) are usually interdependent.

Although it’s a nice feature to have, I don’t find it’s going to help a lot on the long run, I think parallel make is the thing that is going to make a difference in the medium term.

Okay, so what are the problems with using -j8 for ebuilds then?

• we express the preference in term of (GNU) make parameters, but not all packages in Portage are built with make, let alone GNU’s;
• ebuilds that use a non-make-compatible build system will try to parse the MAKEOPTS variable to find out the number of parallel jobs to execute; this does not always work right because there can be other options, like -s (which I use) that might make parsing difficult;
• even -s option can be useful to some non-make-compatible build systems, but having to translate every option is tremendously pointless and boring;
• some people use a high number of jobs because they have multiple box building as a cluster, using distcc or icecream; these won’t help with linking though, or with non-compile jobs; forcing non-compile tasks to a single job is going to discontent people using SMP systems, using a job count based on network hosts for non-compile tasks is going to slow down people with single-cpu and multi-host setups;
• some tasks are being serialised by ebuilds when the could be ran in parallel;

And this is not yet taking into consideration buildsystem-specific problems!

What should be doing then? Well, I think the first point to solve is the way we express the preferences. Instead of expressing it in term of raw parameters to make, we should express it in term of number of jobs, and of features. For instance, a future version of portage might have a make.conf like this:

BUILD_LOCAL_JOBS="8"
BUILD_NETWORK_JOBS="12"

BUILD_OPTIONS="ccache icecream silent"

And then an ebuild would call, rather than simply emake, two new scripts: elmake and enmake (local and network), which would expand to the right number of jobs, for make-compatible buildsystems, that is. For other build systems eclasses could deal with that by getting the number of jobs and the features from there.

More options might be translated this way without having to parse the make syntax in each ebuild, or in each eclass. The ebuilds could also declare a global or per-phase limit to jobs, or a RESTRICT="parallel-make", that would make Portage use a single job.

The last point is probably the most complex one. Robin already dealt with a similar issues in the .bdf to .pcf translation of fonts, and solved it by having a new package provide a Makefile with the translation rules, the conversion could then be parallelised by make, instead of being serialised by the ebuild. I think we should do something like this in quite a few cases; the first one I can think of is the elisp compile for emacs extensions, and I don’t know whether Python serialises or execute in parallel the bytecode generation when providing multiple files to py_compile. And this is just looking at two eclasses I know doing something similar to this. But also Portage’s stripping and compressing of files should probably be parallelised, where there are enough resources to do so locally.

I guess I have found yet another task I’ll spend my time on, especially once I’m back from the hospital.