Freedesktop.org is all about getting things done, and most of the time that's just a matter of finding people to do whatever work needs doing, be that development, system administration or adding documentation to one of the wikis.

Sometimes, we need to buy hardware or services with actual money, and for that we constructed an LLC last year; prompted by the generous donation from Google of funds sufficient for most of our current hardware stack. Through this experience, it became clear that some ?real? organization that offered greater transparency and structure to our limited financial dealings would be helpful. However, none of us felt like we could afford the time and energy necessary to construct such an organization on our own.

Software in the Public Interest was created to hold the assets of the Debian project, and has since expanded to fulfill a similar role for many other projects, among them PostgreSQL, wxWidgets and Drupal. They provide precisely the kind of structure that freedesktop.org needs right now; focusing purely on helping manage the financial resources of an organization without being involved in the political or social structure. Their sole responsibility is to ensure that any tax-exempt donations are managed in accordance with various legal obligations, including the disposition of funds should the project cease to exist.

I think the affiliation of freedesktop.org with SPI will benefit us and make it possible to keep our project running smoothly into the future and help continue our advance towards world domination.

> Read More... | Digg This!

With Google's generous help, freedesktop.org has three fun new machines to play with. Part of the migration process involves moving all of the wikis from gabe to annarchy and I spent the day figuring out how that will work.

One of the big features of this move is that we can (I hope) afford to run more current software on the machine used to house the web server and user home directories. For our wiki, which is running MoinMoin, that means moving from version 1.2.4 to 1.4.99 or so. Oddly, there have been some changes in how the wiki is configured.

I've got a small demo wiki running at http://annarchy.freedesktop.org. That shows the new version running with canned content from the MoinMoin installation. Migrating the wiki configurations to the new format is pretty easy; requiring changes to the virtual host entries used for apache and replacing the per-host configuration files with one generated from the template provided with MoinMoin.

MoinMoin also has some migration scripts to get from 1.2.4 content format to 1.4.99, I need to figure out how to use those so we can move content across. I'm hoping these will 'just work', and there's a reasonable chance they will. Of course, that's only half the battle; I'm sure there are hard-coded assumptions in many of our wikis about what host the wiki software is running on. I guess we'll get to find that out.

> Read More... | Digg This!

Our new host, annarchy, is destined to become the place where we provide user accounts and web hosting. User accounts currently share the same machine, gabe, as cvs and mail. Moving from this configuration should improve performance and reduce the impact of security compromises to our web server or user accounts.

I'm just copying all of the user accounts from gabe to annarchy, and then at some point (soon), I'll flip the big switch and 'users.freedesktop.org' will suddenly point at the new machine. I'll leave logins to gabe enabled until the web services are transferred, but home directory contents will disappear from gabe and reappear on annarchy. That machine has a separate partition for user accounts, which should eliminate all of the 'fun' we've occasionally had when people filled up the partition shared with CVS and the web server.

I've already done a preliminary rsync of user directories to annarchy, so the remaining work is mostly to switch DNS entries and move home directory content out of the way on gabe. I'd like to get that done this week, but I don't know whether to wait for the web content to move all at the same time or not.

> Read More... | Digg This!

The blackberries, Marrionberries and Boysenberries are all doing a bang-up job this month and I've made a dozen pies from their fruits in the last couple of weeks.

Laura brings in a bucket and says ?freezer or pie? at which point I have to decide whether I can stand to stop hacking on xlib for a half an hour to get a pie in to the oven. Fortunately, the answer has been ?pie? almost all of the time so far.

I haven't quite figured out how to make a decent raspberry tart yet, so they're all destined for jam this year. Gotta work on that sometime soon.

> Read More... | Digg This!

> Read More... | Digg This!

As a part of my education about graphics, it seemed about time to intentionally select and use fonts with an eye to producing readable output.

To start this endeavour, I found a replica of one of the first typefaces which our modern eyes can readily access. Designed around 1470 by Nicolas Jenson, this face adapts the miniscule letters used in calligraphic writing to the majuscule lettering used in inscriptions (and as seen in Gutenberg's publications) to produce what we know of as upper and lower case (modern terms taken from the location of the two drawers in the typesetter's case). This, and similar faces, known as Venetian style, form the foundation of modern western font design.

The particular face I chose was as close as I could find to Jenson's original, it's called Cloister Old Style. Although bold face lettering wouldn't appear for nearly 400 years after this face was designed, this particular family does have four styles (roman, italic, bold and bold italic).

Getting this font into X applications was trivial, but after dumping them into ~/.fonts, I discovered that the four styles were given different family names. We can thank Windows for that, I think -- with only four different styles available for each family, font foundaries have mixed the style into the family name to ensure the user would be able to select each desired font. I ended up editing the fonts with Fontforge to fix their family and style names.

Getting this font into LaTeX was rather more involved. After about four hours, I had found sufficient LaTeX, dvips and psnfss documentation and deduced enough about the debian changes to those to try and get custom fonts into a LaTeX document. What a horror. I did write a shell script to help with this process -- once one has named the font files appropriately (yes, the names must follow the nfss conventions), one can run this script over the resulting .ttf files to add them to one's LaTeX/dvips environment. I create Type42 files as intermediaries as this (should) bring the TrueType hints along to the output file, making it potentially more legible on a screen. The result, however, is that 'pdflatex' can't handle the font files, so one has to run dvips and then dump that through ghostscript to generate pdf (using dvipdf).

To set some text, I chose a sample paper that Carl and I wrote for OLS two years ago about cairo (then Xr). I started by using the document class file written by Bart Massey for last year's Usenix annual conference. That cleaned up a lot of the header problems -- they were too small before and had way too much mandatory white space around them. After that, I started thinking about what to do for a monospaced font. Courier was what the document used originally, but that's a poor color match for Cloister Old Style. Looking through my own limited selection of fonts, I identified Bitstream Vera Sans as the closest in color, but the lack of serifs made each instance stand out like a sore thumb.

So, it was back to the net to find something that looked nicer when set with Cloister Old Style. The number of decent monospaced fonts ever made can probably counted on one's fingers, so the choices were limited. I settled on Bitstream's Prestige 12 Pitch. It offers a color similar to Cloister Old style with slab serifs, and even a few beveled serifs on the ascenders. Of course, the x height was too large, but that is adjustable in TeX by hacking the t1bpr.fd file put in the /usr/local/share/texmf/tex/latex/psnfss directory by my shell script. I found a cute hack which lets this scale factor be configured in the document (or other style file):

expandafterifxcsname Pr@scaleendcsnamerelax
 letPr@@scale@empty
else
 edefPr@@scale{s*[csname Pr@scaleendcsname]}%
fi

With this, one inserts Pr@@scale after the <-> in each DeclareFontShape invocation and then a defPr@scale{0.8} into the style file and now all instances of this family are scaled by 0.8 which matches the x height of Cloister Old Style.

With compatible fonts in hand, I turned to line spacing. Traditionally, a font with a small x height needs extra vertical spacing between lines to keep things readable. Looking back at the first book published with Jenson's typeface, Eusebius, De Evangelica Prćparatione, I note that the leading is actually quite small, similar to what we use today with the typical largish x height faces. When trying that spacing, I found it rather dense, so I ended up setting the leading to 3.5 points for the 12 point size used in the body of the document, for an interline spacing of 15.5 points.

The resulting output seems an improvement over the document as published at OLS.

> Read More... | Digg This!

Ok, so nickle isn't the worlds most popular language. So why would I spend a couple of days building a binding to cairo for it? Mostly because I like to code in nickle and I wanted some way to get simple graphics up on the screen for plotting and the like. However, it's also been a good way to get back into the heart of cairo and see how things look.

Instead of a "direct" binding to cairo, I've created a slightly higher level mechanism where nickle exposes none of the underlying window system details and instead simply provides a 'window' upon which things can be drawn. This means that the process of drawing is as simple as:

	autoimport Cairo;
	cairo_t cr = new ();
	scale_font (cr, 12);
	move_to (cr, 10, 20);
	show_text (cr, "Hello, World!");

The back end code uses gtk+ instead of raw Xlib because that way I get all of the ICCCM hints set and a whole lot of help with threading. I run the gtk+ code in a separate thread from nickle and have nickle just paint bits through cairo to a pixmap. Wake up the gtk+ thread and have it copy the pixmap to the window. Resize is handled by creating a new pixmap of the new size, copying the old data (or as much as will fit) to the new pixmap and repainting the screen.

The gtk+ thread also receives input, and writes simple string events to a pipe which can be read by the nickle code without a lot of hassle. Right now, it just does mouse input, keyboard input is a lot harder, so I'll leave that until I really need it. Probably I'll send key up/down events and also UTF-8 strings; that way apps can get what they need without a lot of work.

The other piece of this work is in learning how to implement a usable foreign function interface in nickle. I've put off this work in the hope that I'd have a sudden flash of brilliance about how to make this all just work without a lot of per-library stub writing. After doing all of that stub generation by hand, I learned that there is a lot of repetitive stuff, but also a huge amount of semantic knowledge about how both cairo and nickle work needed to successfully glue things together.

The biggest problem (no surprise here) was the difference of memory management methods. Nickle, like any sensible language, has automatic storage allocation with a garbage collector. cairo, like any sane C libary uses reference counts with no recursive data structures. It's not hard to get a finalizer called from nickle to dereference the cairo object. The trick is knowing what other places need references added or subtracted. And that takes an intimate knowledge of the cairo internals, something which can't be automatically abstracted from the headers. So, the goal of automatically gaining access to cairo from nickle without writing any cairo-specific code doesn't seem entirely possible.

Still, it seems like at least some of the work needed to generate a language binding can be automated; the busy work of creating nickle datatypes cooresponding to the exposed cairo structures and enums could surely be streamlined somehow. And, the constant nickle?C datatype conversions should permit some automation. But, if that automated result needs to be hand-edited to make it usable, the utility of automatic generation is greatly reduced; it wouldn't aid maintenance of the resulting library at all. Future experimentation is clearly needed here.

For those with an experiemental bent, the bits needed for all of this are available from CVS on either nickle.org or cairographics.org.

> Read More... | Digg This!

I've spent much of today reading through fontconfig bugzilla and trying to fix bugs. Many of them had already been fixed or had patches attached. That certainly made things easier.

But, the big changes in 2.2.97 are a result of a pleasent development effort via email over the last week resulting in definitive action on how to handle multi-lingual names in fonts (style, family and fullnames). I'd been stalled on this since last year, trying to figure out a way to make existing applications ?just work?, and somehow present nice localized names wherever possible.

As usual, the right answer was to punt and let some other software sort things out. Fontconfig isn´t the right place to try and localize font names; it doesn´t know what language should be used to present each font name. For most users, the right font depends more on the expected target languages of the font than the locale the user is running in:

• For users not facile with a particular language, presenting fonts designed primarily for that language with an obviously foreign name helps them sort ?useful? fonts from ?useless? fonts.
• For multi-lingual users, presenting fonts for each language in that language ensures that the font names are as accurately represented as possible.

This logic relates solely to family names. For style names, it's necessary to always present them in a language understood by the user. For now, that probably means using the current locale as we have no other notion of user languages. I think some of this demonstrates a fundemental limitation of the locale system?it can't represent users facile with more than one language. I'd like to know whether we can fix this somehow; it seems wrong.

In any case, fontconfig now presents applications with two parallel lists, one containing the names (in UTF-8 encoding, of course), the second containing the language of each name. I used parallel lists because the alternatives all seemed a lot worse; the data is nominally informative, not normative, and has no place in the list or match process.

Applications are free to grub through the language list to select an appropriate name to present to the user. I'm hoping developers will propose some interfaces to add to fontconfig to make this easier; I'd rather let the users figure out the interface than attempting to produce one without an actual use case (or two) in front of me.

There are lots of other changes in fontconfig now:

• it includes 'fullname' information where different from family or family+style.
• it digs out the underlying font format (using the X-specific FreeType API added for X font servers).
• it grubs through the silf and gsub/gpos TrueType tables to find out what layout support is present in each font.
• the selectfont configuration element was added to let people control which fonts are available for applications to choose from.

Change logs and source are available (as usual) from the fontconfig.org website.

> Read More... | Digg This!

Ok, so it's really only two machines at present, but we have big plans.

As Daniel described in some detail, the machine was compromised and we've spent the last week rebuilding it. But, I wanted to describe our current setup and what we'd like to see in the near future.

The worst effect of the breakin was to disable *all* of our services, from web to email and even CVS. That's clearly not acceptable, and I think we can expect future security issues with our vast collection of random web services. This points to the usual solution -- use separate boxes for separate services. Based on conversations with other people running similar sites, I think we need:

• A Web hosting machine. Running apache2 and as little else as possible. User accounts would normally have access to this machine.
• A CVS machine. User accounts would not normally have access to this machine. CVS access is via SSH only. Project admins would have shell access to this machine to fix CVS repositories.
• An Email machine. Sysadmin access only.
• A security bastion for LDAP. LDAP updates can be managed by GPG-signed emails. Again, sysadmin access only.

We've got three machines available at present:

• gabe, aka freedesktop.org. This is the HP server that Jim and I wrangled into our budget last year. Dual 2.8G Xeon processors with 512K caches (the 1M cache chips were not available). 3.5G of memory, six 38G scsi disks in a hardware raid array. The only "real" server we own, it's currently serving as web, mail and cvs server.
• tycho, aka fooishbar.org. This is an older dual PIII server that I got from VA Linux Systems back in the day. It's got a couple of reasonable disks, but no RAID setup. It's certainly not powerful enough for either HTTP or CVS. But, it should make a fine mail server or perhaps LDAP. It needs a reinstall job, and we need to kick daniels off of it.
• kara. This is a recent addition, a dual opteron box with IDE drives. It's really a desktop machine, and has the (current) distinct disadvantage of not being rebootable. We're using it as the main LDAP server for now, but I think that better be temporary until we can get tycho rebuilt.

We've stuck in a budget request for another "real" server. If that happens we can use that for CVS. It would be nice to get another small server for mail or ldap so that we could avoid using the dual opteron box; it's a desktop machine and isn't likely to be as reliable as we'd like to see.

X.org is also looking into building another site; I imagine we'll be able to piggy-back on that effort and mirror the fd.o content whereever x.org lands.

> Read More... | Digg This!

Most window systems do window management by having the applications essentially manage their own windows. They have shared code which the applications invoke when appropriate. This seems like a good model for cooperative and well behaved applications as it reduces the complexity of the process considerably.

So, unlike X, I'll use this and see if I get stuck at some point. I'm not expecting to need very sophisticated window management, so we'll hope it all works out. To allow a variety of different window styles, I've hard coded a few different window types:

• Plain - no decorations at all, useful for menus and the like
• Application - full decorations
• Full screen - like plain, but pinned to fill the screen
• Dialog - temporary window associated with an application window
• Alert - system modal window, always on top.

I'll be surprised if I don't end up adding more.

Event Dispatch

To get events connected from the hosting system, I have a simple dispatch model. There's no queueing within the system, although there may be some within the event device itself. Events are dispatched from the device to a specific screen.

Pointer device event are delivered by searching through the viewable pixmaps on the target screen from top to bottom and sending the event to the first one with a non-transparent pixel covering the pointer location.

A ButtonDown event causes the pointer device to be 'grabbed' by the target pixmap; further pointer device events are delivered to that target irrespective of where the pointer is located. When the matching ButtonUp event is received, the pointer is ungrabbed.

Like Windows, I've split out Key events from Text events. That is to say, you can look at the raw KeyUp/KeyDown events if you like, or ignore them and let the system convert them to Unicode values. This allows applications to handle non-Unicode keys but not have to deal with the complexity of whatever conversion system is used to generate Unicode. I'm hoping this will permit the addition of non-keyboard based text input methods for devices using Twin. Hmm. If I add some editing commands as well, it should be possible to drive text input from devices which have imprecise conversion which needs correction.

With this event dispatch model, I've hooked button motion to a simple routine which moves the windows around on the screen. As usual, you can see what things look like to day in the obligatory screen shot.

> Read More... | Digg This!

The biggest problem with using the Hershey fonts is that they are constructed from sequences of line segments. At large sizes, they tend to look blocky. Besides, why spend all of that time adding Bézier splines to twin only to not have glyphs take advantage of them.

I tried to come up with an automatic mechanism for identifying sequences of strokes in each glyph which could be replaced by a spline, but I eventually gave up and figured that with a reasonable tool, I could manually identify the points. It turns out that this wasn't sufficient; I had to also insert new points and delete existing points before fitting splines. That's because the glyphs all have nice flat spots on the horizontal and vertical edges of each circle, a Bézier spline wants to terminate in the middle of these flat spots.

Fortunately, I have cairo to help build applications now. What a relief. Constructing the font editor took a couple of hours, with incremental adjustments as I started to edit each glyph. The new glyphs use quite a bit less memory than the old, enough so that I could add pre-computed snapping arrays and metrics to each glyph which makes the glyph painting code a lot simpler.

The twin clock also continues to improve in appearance; take a look at the latest incarnation, which includes the pretty new text.

> Read More... | Digg This!

Well, I had an adventure today. I wanted to find out whether twin was going to run acceptably fast on my target platform, so I decided to get some profiling data. Unfortunately, gprof doesn't (natively) work on multi-threaded applications. Fortunately, google found 'gprof-helper.c' which did manage to get reasonable profiling data.

As usual, profiling uncovered many bugs in my program, as well as a probably bug in some cairo code.

I saw trig functions being called hundreds of thousands of times when painting a simple clock face and went to investigate. Turns out the function Carl and I wrote to compute the number of vertices needed to approximate a circle within a specified error tolerance was almost completely bogus. After figuring out a suitable replacement, the clock was actually keeping up with time. There are still the expected performance issues of software compositing, but given that the target CPU is not an x86, I'll have to wait until I've got hardware and a compiler to try and speed those up.

I'd also like to fix the text so that curves weren't roughly approximated with line segments. For small text, it's not bad, but for larger characters, it looks pretty awful. I'll have to go find some spline fitting code and see what I can manage.

> Read More... | Digg This!

Twin is targetting a system without an MMU (and without much in the way of process control), so "applications" are more likely to look like threads than separate processes. As such, I'm using pthreads to simulate that environment.

Building applications is largely a matter of creating enough infrastructure so that you can write a new application without having to go change all of the existing ones. I've had a brief chat with Owen Taylor who had some simple suggestions:

• Mark dirty and queue a recompute/redraw instead of attempting to update the display in place
• Nested windows aren't a feature
• Keep behaviour simple
• Keyboards and text input are a pain

With this in mind, I've done a few simple things. The first was to split out the screen update into a separate thread. Applications paint into their windows and just ignore the screen update process. Mostly. The one thing we've learned with Composite is that applications really need to tell the system when they're drawing and when the image is ready for publication. Given that information, it's trivial to ensure that the display always presents a consistent picture.

Under Composite, we'll probably need some kind of double buffering so that areas of the screen can be repainted even while windows occupying those areas are busy drawing. Otherwise, a busy application could "lock out" the screen in those areas for a long time.

In Twin, I expect applications to be "well behaved", so I'm waiving the ability to update at any time in exchange for saving piles of memory. Instead, whenever an application is busy drawing, the screen won't get updated. When all of the applications are idle again, the screen will get updated.

That much was easy enough to get working; now it's on to event distribution. I've already got a separate thread reading events from the X connection. I'll try constructing event queues and having that thread dump events into those queues so that the application thread can process them synchronously with respect to output stuff.

> Read More... | Digg This!

Well, getting the basic drawing stuff done in twin was good fun, and I learned a lot about the algorithms we're already using in cairo to do path stroking. But, a PostScript-ish API without affine transforms is like pie without the crust -- tasty, but lacking substance.

Given that our target platform doesn't support floating point numbers, I couldn't just use those for the twin API. Instead, I'm using 16.16 fixed point numbers. That doesn't provide a huge dynamic range, but it should make "reasonable" drawings possible. Those 16.16 values are run through a 3x2 matrix and converted to the 12.4 fixed point values use in the rendering layer. I still need to figure out what to do with out of range values. Ideally, I could clip lines to the bounds of the coordinate space and preserve (approximate) slopes. That may be more work that it's worth for this system; time will tell.

Given affine transforms, all of the usual 2D drawing tricks are now possible. As text is just strokes, even that can be transformed without problem.

One remaining issue is to re-implement the text snapping code under the transformations. I'm not quite sure how I'm supposed to do that though, I'd like to figure out some reasonable invarients so that text doesn't look too funky.

> Read More... | Digg This!

Recall that the main goal of the twin window system is to fit in no more than 50KB. For text, that meant that I couldn't just take an existing TrueType font and use the FreeType rasterizer to scanconvert each glyph. (FreeType weighs in at over 400KB on an x86 machine)

Instead, some significantly smaller solution was needed. One easy solution was to just take outlines from some font, convert them to a compact format and rasterize them with the existing twin drawing code. I gave this a whirl with the Bitstream Vera Sans Roman type face. The results of that experiment are available here. As you can see, the lack of hinting makes the glyphs very fuzzy all over. It's a long ways from the quality available on my laptop screen, but it is workable and provides access to a huge set of fonts.

Converting the outlines was easy enough, and without any compression, the ASCII subset of vera fits in about 7KB of memory, using 1-byte for each coordinate. Increasing coverage to Latin-1 and it uses 16KB.

So, that's a workable solution, but not ideal because of the poor readability of the text. I had another idea, but needed to google for some really old data...

In 1979, I built a graphics library for an atmospheric sciences group that I worked with. The goal was to provide an interactive graphing environment on our VAX 11/780 which included a 512x512 8-bit color display, as well as produce hard-copy plots for publication on an HP pen plotter. Drawing the images on the screen was simple enough, but I needed a way to get text on the screen. I found some stroke fonts, originally developed in the 1960s by Dr. A. V. Hershey at the US Naval Weapons Laboratory. These worked just fine for my little graphics application.

Fortunately, it wasn't hard to locate the original Hershey data set. Armed with that, it was a short nickle script from there to C structures containing the entire repertoire of glyphs.

You can see most of the Hershey glyph set here

I was surprised to realize that the simple sans serif face didn't include all of the symbols from ASCII. So, I borrowed a few from other face and built the remaining from pieces of other glyphs.

Just drawing the glyphs and convolving them with a circular pen produced quite reasonable results. However, any significant scaling produced lots of fuzzy bits again. Because the glyph data consists of strokes, I thought I could try to 'snap' positions to the pixel grid and eliminate a lot of the gray matter at glyph edges.

Integerizing the glyph coordinates yielded a surprisingly readable result as seen here. Of course, the letter forms are badly distorted in places, but at least the horizontal and vertical elements are nice and sharp

Armed with that encouraging result, I set out to snap only the elements which would actually benefit from pixel alignment -- those horizontal and vertical edges. That turned out to be quite easy; stretching and shrinking the character in each direction between the various fixed alignment points. The initial results for that test are here.

With simple stroke data, it's easy to change the pen width to make the glyphs 'bolder', and easy enough to slant the text to make it appear oblique. Neither of these changes require any additional glyph data. The final set of glyphs can be seen here.

There is still some issue with glyph widths that I haven't quite solved; I think it has to do with normalizing the space from the right-most point in the glyph to the next glyph origin.

> Read More... | Digg This!