OSGalaxy

published by brad hards on 2009-11-26 09:56:16 in the "Development" category

Not really a KDE related post, but instead one about the OpenChange project.

OpenChange is a project to implement the Microsoft Exchange / Outlook protocols, and we're creeping up on the 0.9 release. For those not familiar with it, the aim is to be wire-level compatible, so that you can use a FOSS client (such as Evolution or an Akonadi client) with an unmodified Exchange server.

OpenChange depends on some underlying Samba4 libraries, so we normally work with the Samba project to get releases that basically match up - we don't want to rely on building bits of Samba4 from the git repository, and we don't want to depend on really on versions of Samba4. Not too far ahead, and not too far behind. Just right...

The next release of Openchange (0.9 "COCHRANE") will rely on Samba 4 alpha9. So the main thing to do is to make sure that we can work with the current state of Samba 4, to provide patches to Samba for stuff that needs to be fixed on the Samba side, and to have changes for OpenChange ready to go.

We expect to release within two weeks of Samba 4 alpha9, but OpenChange 0.9 will be released when it is considered done.

We anticipate a 0.10 release in early 2010, and probably a subsequent 0.11 release prior to 1.0.

I've gone through the trac tickets, and moved anything that can't reasonably be achieved for 0.9 to a 0.10 milestone. Large changes have to wait for 0.10 - it is getting very late to risk destablising libmapi.
So what we still have left:
- integrate exchange2ical work from Ryan Lepinski into trunk
- ensure that OpenChange works with FreeBSD/OpenBSD
- ensure that OpenChange works with OpenSolaris
- libmapi/Samba4 API updates
- mapiproxy/Samba4 API updates and association group API evolution
- a couple of tickets that may not be too hard to fix, related to pkg-config support
- a ticket related to Free / Busy time support that has a crash.
- a patch that checks some return values that we current don't handle.
- some documentation stuff.

Details are on http://trac.openchange.org.

Known problems:
- openchange development server will not be working "as it is". Resolving this requires revision of the OpenChange schema updates (to match Samba4 changes). This may not be resolved for 0.9.

If I've dropped a patch, or there is something you really need for 0.9, now is your chance to make yourself known. Probability of a patch being applied is inversely proportional to complexity, and also inversely proportional to how close I think the release is. That is, simple and soon is good.



> Read More... | Digg This!

published by brad hards on 2009-10-04 10:51:43 in the "Development" category

Once again, its been a long time since I blogged. I have been doing a bit of OpenChange development though.

Mostly its been minor bug fixes, cleanups and so on. This weekend I decided to take on something a bit more substantial. Email rules handling between Microsoft Outlook and Microsoft Exchange are pretty sophisticated, and OpenChange didn't do too much of it. I ended up having to get all the way down to the unmarshalling data structures from the RPC calls to understand why things didn't work. Its starting to come together now, with the condition part of the rule mostly under control (although not complete) and the actions part of the rule hopefully not too hard once I get conditions sorted.

I'm a long way from done, but the current diffstat:

 exchange.idl             |   97 ++++++-----
 libmapi/conf/mparse.pl   |    4
 libmapi/emsmdb.c         |   10 +
 libmapi/mapidump.c       |  384 +++++++++++++++++++++++++++++++++++++++++++++++
 libmapi/property.c       |  152 ++++++++++++++++++
 utils/openchangeclient.c |   84 ++++++++++
 6 files changed, 686 insertions(+), 45 deletions(-)

Most of the IDL changes are just reorganising the order of things. The hard bits were in libmapi/property.c.

OpenChange is a library, so there aren't nice GUI screenshots. However I can show the results of a new openchangeclient operation, to list the rules on a mailbox folder:

bradh-dev@saxicola:~/openchange/oxorule$ ./bin/openchangeclient --ruleslist
[10] "which is an Out of Office message" (0xE1FCF59014000001)              
        rule provider: RuleOrganizer                                       
        state: 0x00000011 ( ST_ENABLED ST_EXIT_LEVEL )                     
        All of the following are true:                                     
                Content of PR_MESSAGE_CLASS_UNICODE matches (case insensitive) IPM.Note.Rules.OofTemplate.Microsoft                                                                                                                 
                All of the following are true:                                                                    
                        PR_MESSAGE_SIZE is greater than 12288                                                     
                        PR_MESSAGE_SIZE is less than 20480                                                        

[11] "where my name is not in the To box" (0xE2FCF59014000001)
        rule provider: RuleOrganizer                          
        state: 0x00000001 ( ST_ENABLED )                      
        All of the following are true:                        
                PR_MESSAGE_TO_ME is equal to false            
                Comments contain 4 tagged values:Unknown tag: 0x60000003 (1), Unknown tag: 0x00010102 (<>), Unknown tag: 0x0001001f (priority@example.com), PR_DISPLAY_TYPE (0),  Restriction present:                      
                        PR_SENDER_SEARCH_KEY is equal to <>                                               
                All of the following are true:                                                                    
                        PR_FLAG_STATUS is equal to 2                                                              
                        Unknown tag: 0x802a001f is equal to Call                                                  

[12] "test user1 or foo@example.com" (0xE3FCF59014000001)
        rule provider: RuleOrganizer2                    
        state: 0x00000001 ( ST_ENABLED )                 
        Any of the following are true:                   
                Comments contain 4 tagged values:Unknown tag: 0x60000003 (1), Unknown tag: 0x00010102 (<>), Unknown tag: 0x0001001f (test user1), PR_DISPLAY_TYPE (0),  Restriction present:                                
                        PR_SENDER_SEARCH_KEY is equal to <>                                               
                Comments contain 4 tagged values:Unknown tag: 0x60000003 (1), Unknown tag: 0x00010102 (<>), Unknown tag: 0x0001001f (foo@example.com), PR_DISPLAY_TYPE (0),  Restriction present:                           
                        PR_SENDER_SEARCH_KEY is equal to <>                                               

[13] "'Old project' or 'Really old project'" (0xE0FCF59014000001)
        rule provider: RuleOrganizer
        state: 0x00000011 ( ST_ENABLED ST_EXIT_LEVEL )
        Any of the following are true:
                Content of PR_SUBJECT_UNICODE contains (case insensitive) Old project
                Content of PR_SUBJECT_UNICODE contains (case insensitive) Really old project

[14] "Project X" (0x15B3492E12000001)
        rule provider: RuleOrganizer
        state: 0x00000001 ( ST_ENABLED )
        Content of PR_SUBJECT_UNICODE contains (case insensitive) Project X

[15] "'Viagra' or 'herbal'" (0x14B3492E12000001)
        rule provider: RuleOrganizer
        state: 0x00000011 ( ST_ENABLED ST_EXIT_LEVEL )
        Any of the following are true:
                Content of PR_SUBJECT_UNICODE contains (case insensitive) Viagra
                Content of PR_SUBJECT_UNICODE contains (case insensitive) herbal

[16] "Administrator" (0x16B3492E12000001)
        rule provider: RuleOrganizer
        state: 0x00000001 ( ST_ENABLED )
        All of the following are true:
                Comments contain 4 tagged values:Unknown tag: 0x60000003 (1), Unknown tag: 0x00010102 (<>), Unknown tag: 0x0001001f (Administrator), PR_DISPLAY_TYPE (0),  Restriction present:
                        PR_SENDER_SEARCH_KEY is equal to <>
                PR_IMPORTANCE is equal to 2

That mostly converts the data structure into something intelligible (although there are a few things that I'm not able to sensibly represent in natural language yet, as you can see. There are other variations within the outlook rules, and my code cannot yet do them all. It isn't too hard to complete, but I'm trying to make sure I have test cases for each case before implementing. If you have a really sophisticated set of rules within outlook and would like to share them, feel free to export them and mail it to me.



> Read More... | Digg This!

published by oever on 2009-09-07 21:33:25 in the "Development" category

While sipping from a Vignes de Nicole and nibbiling on some Heukäse, I am thinking about Wine. Not the liquid version, but the software project.

As a reader of Linux Weekly News, I noticed that the Wine project makes very frequent releases. I looked up its release history and saw that Wine has made a developer release every fortnight for the last four years. The 11 years before that the releases were approximately monthly. Each of these development releases comes with an announcement with a long list of the changes that happened in these two weeks. This dedication is due to Alexandre Julliard who has made all of those releases.

Releasing so regularly is a sign of health. Still, within the Wine team, people were dissatisfied with the occurance of regressions in their code. So patchwatcher was written. Patchwatcher is a set of scripts that do pre-commit checking of patches. There is a good overview that explains the design of the system.

The system picks patches from the patch mailing list and tests them. The results of the tests are available publicly and the maintainer will only commit a patch when no tests fail due to the patch. This conservative approach pays off. Not all developers are subject to this regime. If they introduce a regression, git bisect is used to find the problem.

The rigor of Wine development is inspiring. I will browse the Wine developer wiki more often.



> Read More... | Digg This!

published by bille on 2009-08-24 15:50:31 in the "Development" category

The main openSUSE users' mailing list are a demanding bunch who know what they want. Over the last few months the KDE group have been asking them what they still miss from KDE 3 in KDE 4, and one of those things has been the ability to add a submenu of the main app launcher, whether Kickoff or traditional, to the panel as a button in its own right.

I thought Chani's recent blog about a generic app menu dataengine for KDE 4 could be useful in reimplementing this feature, so I sat down for a quick Friday afternoon hack. However, after a while I realised that I could do better than just hacking up another menu applet implementation. I realised that we already have all the code needed for building the menu tree, drawing the menu, handling the clicks, and configuring the menu in the existing launcher menus ('start buttons'). Looking at the code, it just came down to changing the point in the menu used for the menu's root. So I added a way to enable the context menu for submenus, identify the path of the submenu, then tell the panel or the desktop to add a new 'simplelauncher' applet using the original submenu as the menu root. The changes were only a few lines of code to the existing applets so we have a nice minimal yet fully-featured solution.

That didn't take too long so I even had time to make a movie:

OGG Theora version

movie of adding a submenu

Flash version



> Read More... | Digg This!

published by krake on 2009-08-16 12:04:53 in the "Development" category

There has been some confusion about Google data capabilities around the KDE 4.3 release.

The 4.3 Feature Plan has an entry for that and it is marked as "Completed".

What it meant is that the Google Data resoures by Adenilson Cavalcanti would be available at the time of the 4.3 release, however it got, understandably, interpreted as being part of KDE PIM in 4.3.

Lesson learned: don't put features which are not part of the main modules on the feature plan.

Anyway.
The Google Data resources are currently developed and maintained as part of KDE's extragear effort, basically a "third party" module hosted on KDE's infrastructure.
Applications in there follow their own development and release cycles, though I think it is possible to put tags on certain versions to indicate which version will work best with the newest KDE release.

In case of the Google Data resources, the original plan of releasing simultaniously with KDE 4.3 got scrapped in favor of an urgent fix in one of the two resources and, IIRC, in the Google Data access library used by both.

Since neither the maintainer nor any of us from the KDE PIM had previous experience with extragear style releases, we seem to have failed doing a proper release announcement or doing it at the proper channels.

The source archive for the resources can be found here, depending on the libgcal version 0.9.2 which can be found here.

We might decide to move their code into the main KDE PIM module (kdepim-runtime actually) at a later version. For now the extragear style has the advantage of allowing a fast paced development with released in between KDE releases.



> Read More... | Digg This!

published by krake on 2009-07-30 18:36:01 in the "Development" category

Once in a while we come across rumors, urban ledgends and myths about all kinds of things.

For example you might have read, heard or otherwise encountered wild claims about Akonadi's dependencies, maybe even as ridiculous as "depends on KDE".

Don't get me wrong, I don't have the slightest problem with stuff depending on KDE, but one of Akonadi's goals is to be a viable option for all PIM applications and a dependency on KDE would make that quite hard.

So I spent a couple of days updating my Java skills by writing a proof-of-concept Akonadi client in pure Java (re-using the Unix socket adapter written for the Java D-Bus bindings).

Granted, it is not an extremely impressive client. Its functionality is to "walk" through the Akonadi collection tree, print collection properties to its console output and list each collection's items.

The output for each collection is formatted like this:

Name
    identifier (remote identifier)
    list of possible content MIME types
      item identifier (item remote identifier) item MIME type

On my test system the Akonadi folder structure looks like this
Folders in Akonadi Console

In the output of the demo app it looks like this:

Creating Akonadi data connection at /home/kevin/.akonadi-test/.local/share/akonadi/akonadiserver.socket
Birthdays & Anniversaries
    5 (akonadi_birthdays_resource)
    application/x-vnd.akonadi.calendar.event
      72 (b19) application/x-vnd.akonadi.calendar.event
      73 (b29) application/x-vnd.akonadi.calendar.event
      74 (b52) application/x-vnd.akonadi.calendar.event
      75 (b55) application/x-vnd.akonadi.calendar.event
      76 (b64) application/x-vnd.akonadi.calendar.event
      77 (b65) application/x-vnd.akonadi.calendar.event

Search
    1 ()

akonadi_googledata_resource_0
    6 (google-contacts)
    text/directory
      78 (http://www.google.com/m8/feeds/contacts/abcd%40gmail.com/full/0) text/directory
      79 (http://www.google.com/m8/feeds/contacts/abcd%40gmail.com/full/32b8b2c80dd7cb2d) text/directory

std.ics
    3 (file:///home/kevin/std.ics)
    text/calendar
    application/x-vnd.akonadi.calendar.event
    application/x-vnd.akonadi.calendar.todo
    application/x-vnd.akonadi.calendar.journal
    application/x-vnd.akonadi.calendar.freebusy
      2 (KOrganizer-874394642.152) application/x-vnd.akonadi.calendar.event
      3 (KonsoleKalendar-1123156469.102) application/x-vnd.akonadi.calendar.event
      4 (KOrganizer-313723522.982) application/x-vnd.akonadi.calendar.event
      5 (KOrganizer-1709699137.995) application/x-vnd.akonadi.calendar.event
      6 (KOrganizer-756482139.219) application/x-vnd.akonadi.calendar.event
      7 (libkcal-598331247.452) application/x-vnd.akonadi.calendar.todo

Kolab
    15 (akonadi_kolabproxy_resource)
    inode/directory

    demo.kolab.org/kevin.krammer@demo.kolab.org
        16 (7)
        inode/directory

        My Data
            17 ( 8 )
            inode/directory

            Calendar
                20 (10)
                inode/directory
                application/x-vnd.akonadi.calendar.event

            Contacts
                19 (11)
                inode/directory
                text/directory
                application/x-vnd.kde.contactgroup
                  88 (81) text/directory
                  89 (82) text/directory
                  90 (83) text/directory
                  91 (84) application/x-vnd.kde.contactgroup
                  92 (85) text/directory
                  93 (86) application/x-vnd.kde.contactgroup
                  94 (87) text/directory

            Journal
                18 (12)
                inode/directory
                application/x-vnd.akonadi.calendar.journal

            Notes
                21 (13)
                inode/directory
                application/x-vnd.akonadi.calendar.journal

            Tasks
                22 (14)
                inode/directory
                application/x-vnd.akonadi.calendar.todo

std.vcf
    4 (file:///home/kevin/std.vcf)
    text/directory
      8 (2bpus4eE8c) text/directory
      9 (2efjr21aA) text/directory
      10 (2uun8V066z) text/directory
      11 (3abf3YIm3w) text/directory
      12 (4w20Vzaqu) text/directory
      13 (566ynkrpNY) text/directory
      14 (5JBqjwr9D4) text/directory

demo.kolab.org/kevin.krammer@demo.kolab.org
    7 (imap://kevin.krammer@demo.kolab.org@demo.kolab.org/)
    inode/directory

    Inbox
        8 (imap://kevin.krammer@demo.kolab.org@demo.kolab.org/INBOX)
        message/rfc822
        inode/directory
          80 (imap://kevin.krammer@demo.kolab.org@demo.kolab.org/INBOX-+-1) message/rfc822

        Calendar
            10 (imap://kevin.krammer@demo.kolab.org@demo.kolab.org/INBOX/Calendar)
            message/rfc822
            inode/directory

        Contacts
            11 (imap://kevin.krammer@demo.kolab.org@demo.kolab.org/INBOX/Contacts)
            message/rfc822
            inode/directory
              81 (imap://kevin.krammer@demo.kolab.org@demo.kolab.org/INBOX/Contacts-+-1639) message/rfc822
              82 (imap://kevin.krammer@demo.kolab.org@demo.kolab.org/INBOX/Contacts-+-1640) message/rfc822
              83 (imap://kevin.krammer@demo.kolab.org@demo.kolab.org/INBOX/Contacts-+-1641) message/rfc822
              84 (imap://kevin.krammer@demo.kolab.org@demo.kolab.org/INBOX/Contacts-+-1643) message/rfc822
              85 (imap://kevin.krammer@demo.kolab.org@demo.kolab.org/INBOX/Contacts-+-1649) message/rfc822
              86 (imap://kevin.krammer@demo.kolab.org@demo.kolab.org/INBOX/Contacts-+-1651) message/rfc822
              87 (imap://kevin.krammer@demo.kolab.org@demo.kolab.org/INBOX/Contacts-+-1652) message/rfc822

        Journal
            12 (imap://kevin.krammer@demo.kolab.org@demo.kolab.org/INBOX/Journal)
            message/rfc822
            inode/directory

        Notes
            13 (imap://kevin.krammer@demo.kolab.org@demo.kolab.org/INBOX/Notes)
            message/rfc822
            inode/directory

        Tasks
            14 (imap://kevin.krammer@demo.kolab.org@demo.kolab.org/INBOX/Tasks)
            message/rfc822
            inode/directory

    shared.common playground
        9 (imap://kevin.krammer@demo.kolab.org@demo.kolab.org/shared.common playground)
        message/rfc822
        inode/directory


> Read More... | Digg This!

published by tstaerk on 2009-06-21 15:13:42 in the "Development" category

Some weeks ago, someone posted a question on the KDE PIM mailing list "Which IDE do you use" or so. This reminded me of the ideals of my youth when I believed that the better your IDE - the more efficient your programming work - the more you get done in a given time for your software development.

Well, since some years I have not been using any IDE any more. First because I was too lazy to integrate KDE 4 into kdevelop. Then Thomas wrote this cool howto and I got my ktimetracker to kompile from within kdevelop quickly. Then I was frustrated by several glitches of kdevelop. In my eyes, there is no tremendous difference to kwrite. However, I gave up again.

OK, on this mail to the KDE PIM mailing list, I saw eclipse mentioned. I thought it would be a good idea to pass a weekend evaluating it and writing a techbase article on it. And I can only say - I am overwhelmed. Apart from some minor glitches (to import a project, do not choose "File -> Import", but "File -> Open") I just get what I wanted:

  • Code completion works
  • Running applications with one click works
  • You can carry on svn'ing
  • Classes are detected dependably
  • Quickly switch between implementation and declaration (meaning you can get to the apidox-explanation of a function quickly)

And, the killer feature for me: Call graphs work. Yes, those little graphs telling you what function calls which! That do not even work well in the stand-alone program doxygen!

If you now want to know how you can use eclipse for your KDE development, here is my tutorial.



> Read More... | Digg This!

published by lubos lunak on 2009-06-16 11:49:04 in the "Development" category

Since Kendy's blog has somehow disappeared from Planet KDE, let me copy&paste one entry (http://artax.karlin.mff.cuni.cz/~kendy/blog/archives/monthly/2009-06.html#2009-06-15T14_37_23.htm):

Thanks to the heroic efforts of Éric Bischoff, Bernhard Rosenkränzer, and Roman Shtylman, the OpenOffice.org KDE Integration has been ported to KDE4. It still has some rough edges (currently the detection does not work out of the box?you have to export OOO_FORCE_DESKTOP=kde4 to get it), but it is safe because it co-exists nicely with the existing KDE3 integration; so if you are not satisfied, you are able just swich back to KDE3 (export OOO_FORCE_DESKTOP=kde).

For KDE4, Roman is also changing the out-of-process implementation of the KDE file picker to an in-process implementation, so you'll probably notice some performance improvement of the KDE file dialog launch too Smiling

End of copy&paste. For people who'd be interested in helping with this, I've been told those should join #go-oo on Freenode.



> Read More... | Digg This!

published by brad hards on 2009-05-31 10:52:10 in the "Development" category

As reported in a previous blog, I've spent some time working with Taras Glek on Dehydra and Treehydra.

It is stabilising and getting to be easier to build. However I thought I'd show a simple example of actually using GCC and Dehydra to check for a policy decision, along the lines of the things that EBN does.

Lets say that we want to avoid use of the strcpy() function. Instead, all developers are supposed to use a xstrcpy() equivalent.

Unfortunately, some developers forget, and use the strcpy() version anyway. It would be useful if the compiler could produce a warning or error message for each use of strcpy().

So we'll create a dehydra script that produces an error message (and stops compilation) when the strcpy() function is used.

There are a few basic callbacks that our script could make use of:
- process_decl(decl), which dehydra calls for every global variable, function, or template declaration.
- process_function(decl, body), which dehydra calls for each function definition (declarations without bodies are not included), including both top-level functions, class member functions, and inline class member functions.
- process_type(type), which dehydra calls for each class, struct, enum, union and typedef declaration.

In the situation we are looking at, we're going to need to look at places where strcpy() might be used, which is in function definitions. So we'll use the process_function callback.

Now since we're not yet very familiar with how dehydra works, it might be useful to look at the various things we'll be working with. Here is an example of a simple javascript script to feed to dehydra:

function process_function(decl, body)
{
  /* print out the function name and location */
  print("Function found: " + decl.name + " at location: " + decl.loc);

  /* Loop over the function body */
  for each (let bodyItem in body) {
      for each (let stmtItem in bodyItem.statements) {
        print("Statement item: " + stmtItem.name + " at location: " + bodyItem.loc);
      }
  }
}

If we had a little test program, such as:

#include "tutorial1.h"

char* do_something(bool useFirst, char* first, char* second)
{
    char* my_copy;
    if (useFirst) {
        xstrcpy(my_copy, first);
    } else {
        strcpy ( my_copy, second);
    }
    return my_copy;
}

and ran the dehydra script as $ g++ -Wall -fplugin=gcc_dehydra.so -fplugin-arg-gcc_dehydra-=error_strcpy_initial.js -c tutorial1.cc -o /dev/null then the output will look something like:

tutorial1.cc: In function ?char* do_something(bool, char*, char*)?:
tutorial1.cc:7: warning: ?my_copy? may be used uninitialized in this function
Function found: do_something(bool, char*, char*) at location: tutorial1.cc:3:7
Statement item: my_copy at location: tutorial1.cc:5:11
Statement item: useFirst at location: tutorial1.cc:6:5
Statement item: xstrcpy(char*, const char*) at location: tutorial1.cc:7:32
Statement item: strcpy(char*, const char*) at location: tutorial1.cc:9:34
Statement item: my_copy at location: tutorial1.cc:11:12

Note that the first two lines are the usual warnings from g++ - the compiler is still doing normal checks.

By looking at the output, we can see how each line in the source code is presented. So to detect use of strcpy(), all we have to do is to compare the statement name to strcpy(char*, const char*), and emit an error message.
So by updating the test script to be:

function process_function(decl, body)
{
  /* Loop over the function body, which is an array of {loc:, statements:array of Variable Types} */
  for each (let bodyItem in body) {
    for each (let stmtItem in bodyItem.statements) {
      if (stmtItem.name == "strcpy(char*, const char*)") {
        error("Use of strcpy() at " + bodyItem.loc + " (in " + decl.name + ")");
      }
    }
  }
}

and running it as g++ -Wall -fplugin=gcc_dehydra.so -fplugin-arg-gcc_dehydra-=error_strcpy.js -c tutorial1.cc -o /dev/null we will get the required error message:

tutorial1.cc: In function ?char* do_something(bool, char*, char*)?:
tutorial1.cc:7: warning: ?my_copy? may be used uninitialized in this function
tutorial1.cc: At global scope:
tutorial1.cc:12: error: Use of strcpy() at tutorial1.cc:9:34 (in do_something(bool, char*, char*))

This technique is more powerful than using grep (or similar application, such as ack) or perl, because it actually understands the code. So it doesn't suffer from false positives where you try to grep for strcpy and hit comments:

/* don't use strcpy here */

and it doesn't suffer from false negatives where you try to exclude xstrcpy:
strcpy(xstrcpy, ystrcpy);

It is still just a slightly more sophisticated version of grep (and ack), but we can come up with more complex tests, which I'll look at in a future blog entry.



> Read More... | Digg This!

published by brad hards on 2009-05-24 12:11:23 in the "Development" category

Based on a comment from Taras Glek on my feeble attempts to get a GCC plugin going, I did some work on Dehydra / Treehydra. Its an interesting approach, and one that benefits from the GCC plugin API.

Basically, you can write the checks that you want to do in Javascript (using the Spidermonkey engine from Mozilla). That is more powerful than the declarative approach that mygcc uses, although I'm still struggling a bit with it. Hopefully I can get the rest of the porting to GCC 4.5 done, and start writing checks that are actually useful.

It isn't the easiest thing to get into - the build steps involve a separate checkout from Mozilla of Spidermonkey (using Mercurial), some minor patches to GCC (which has to be built from svn), and then building dehydra and treehydra. Hopefully we can make that package-friendly by the time GCC moves to its release stage.



> Read More... | Digg This!

published by brad hards on 2009-05-19 03:28:25 in the "Development" category

Its been a while since I did anything productive in KDE land, so thought I'd try to do something in a morning. Its hard freeze time, so that should be bug fixing. Bug 185532 was something I'd been thinking over for a while.

That bug basically deals with XPS documents that didn't render correctly with RTL text. It looked like a fairly standard Right-to-Left text problem, and I assumed there was just something I hadn't handled because none of the documents I look at are Right-to-Left. I am, after all, ignorant of both Hebrew and Arabic.

The fix turned out a bit more complicated, and I think also fixes up some other XPS rendering issues. If you have a collection of XPS documents, especially anything with Arabic or Hebrew text, and are OK with building KDE and applying patches, I'd appreciate any feedback. You can find the patch on KDE's ReviewBoard

Guidance:
1. If your document works differently to an unpatched Okular, please add a comment at Bug 185532
2. If your document doesn't work well, and is about equally bad in both patched and unpatched Okular, raise a new bug.

Thanks!

Brad



> Read More... | Digg This!

published by brad hards on 2009-05-09 06:28:05 in the "Development" category

In a previous blog entry, I discussed some initial work on GCC plugins. Since then, the GCC gurus (in particular, Rafael Avila de Espindola) have made sure headers get installed correctly.

I was a bit confused about how you can find the header location (since it isn't part of the normal header paths), but David Korn pointed out the -print-file-name= command line option for gcc. If you ask for -print-file-name=plugin/include/, then this could be a good path. Turns out if you get back plugin/include/, then gcc can't find the file name / path.

So I stretched my cmake skillz, and came up with this:

# - Find GCCPlugins
# Find the include path for GCC plugin headers
#
# This module defines
#  GCCPLUGINS_FOUND - whether the GCC Plugin support was found
#  GCCPLUGINS_INCLUDE_DIR - the include path for GCC plugin headers

if (GCCPLUGINS_INCLUDE_DIR)

  # Already in cache
  set (GCCPLUGINS_FOUND TRUE)

else (GCCPLUGINS_INCLUDE_DIR)

  # Ask the compiler where to find the paths
  IF(NOT CMAKE_COMPILER_IS_GNUCC)
    SET(GCCPLUGINS_FOUND FALSE)
    RETURN()
  ENDIF(NOT CMAKE_COMPILER_IS_GNUCC)

  EXECUTE_PROCESS( COMMAND ${CMAKE_C_COMPILER} "-print-file-name=plugin/include/"
                   OUTPUT_VARIABLE _gccplugins_path
   ERROR_VARIABLE _gccplugins_error )

  IF(_gccplugins_error)
    # perhaps not really gcc
    SET(GCCPLUGINS_FOUND FALSE)
    RETURN()
  ENDIF(_gccplugins_error)

  STRING( REGEX REPLACE "[n]" "" _gccplugins_path "${_gccplugins_path}" )
  IF(_gccplugins_path STREQUAL "plugin/include/")
    # we don't have this path - it needs some suffix
    SET(GCCPLUGINS_FOUND FALSE)
    RETURN()
  ENDIF(_gccplugins_path STREQUAL "plugin/include/")

  # OK, so it looks like we have a real plugin path. Check the headers are there
  FIND_PATH(GCCPLUGINS_INCLUDE_DIR
            NAMES gcc-plugin.h
    PATHS ${_gccplugins_path}
    NO_DEFAULT_PATHS
    DOC "GCC plugins include path")

  IF(GCCPLUGINS_INCLUDE_DIR)
    SET(GCCPLUGINS_FOUND TRUE)
  ELSE(GCCPLUGINS_INCLUDE_DIR)
    SET(GCCPLUGINS_FOUND FALSE)
  ENDIF(GCCPLUGINS_INCLUDE_DIR)

endif (GCCPLUGINS_INCLUDE_DIR)

(The STRING( REGEX REPLACE ) had me going for a while. I couldn't figure out why the FIND_PATH didn't work - turns out there is a newline at the end of the _gccplugins_path and that stops the FIND_PATH.)

Then the CMakeLists.txt can just become

project( MyGccPlugin )

# Probably nothing that needs 2.6, but that is what I tested with
cmake_minimum_required(VERSION 2.6)

# Find the GCC plugin headers
set( CMAKE_MODULE_PATH ${CMAKE_SOURCE_DIR}/cmake/modules )
find_package( GCCPlugins REQUIRED )
include_directories( ${GCCPLUGINS_INCLUDE_DIR} )

# Source files for the plugin
set( my_gcc_plugin_SRCS my_gcc_plugin.c )

# build the plugin (share library object)
add_library( my_gcc_plugin MODULE ${my_gcc_plugin_SRCS} )
# have the name set to my_gcc_plugin.so (on Linux) rather than
# libmy_gcc_plugin.so
set_target_properties( my_gcc_plugin PROPERTIES PREFIX "" )

(Dear lazyweb: why doesn't find_package( GCCPlugins REQUIRED ) cause the cmake run to fail if the headers aren't found?)

Anyway, enough about buildsystem.

The plugin I've been experimenting with is to check the system call. I currently have code that checks each function call, and can produce a warning if the name of the function call is system. However that will produce a lot of false positives. Consider a function that looks like:

void bar(char * variable)
{
    const char* const appname = "df";

    system("ls");
    system(variable);
    system(appname);
}

It isn't too hard to eliminate the first case (where the first argument is a constant expression), which is an improvement over just using grep or ack. What I'm aiming to do is to warn on only the second case (where the argument really isn't const) and not on the third case. To do that, I need to keep track of the arguments, and that might involve understanding how gcc tracks aliasing.



> Read More... | Digg This!

published by brad hards on 2009-04-27 08:03:01 in the "Development" category

One of the new features proposed for GCC 4.5 is the ability to use plugins.

I'm a big fan of plugin architectures, especially in open source software. I think that plugins provide a really nice starting point for potential developers. So you can start with something simple and well defined, and grow into the rest of the system. Personally, the idea of understanding all of GCC is just overwhelming. But perhaps I could do a really basic plugin that can do an additional static check.

The idea is based on mygcc which allows extensible checkers to be written for gcc. As an example, consider a (C) API that looks something like

foo_init();
foo_use1();
foo_use2();
foo_cleanup();

where you have to init() before using any of the use() functions, and then cleanup() when you're done. Its an error / bug if you forget to init(), or cleanup(), or if there is any path that results in the sequence being broken. Its an error if you cleanup() twice. An example of this is something like malloc/free or varargs (va_start() / va_arg() / va_end()), but I'm less interested in general cases (which might be able to be added to gcc) and more interested in cases from specific libraries (e.g. crypto/security stuff like gnutls and libgcrypt, or alternative memory manager libs like talloc). It would be nice if you could get gcc to (optionally) check that you're using the library roughly right when you're not so familiar with the API.

The plugins branch has been merged into gcc trunk, so I spent some time getting it set up over the weekend (hint: it really is supposed to be built with builddir != sourcedir). There are a few surprises (or, I'm still doing it wrong), like the headers not getting installed, but I'm confident that will get sorted before release.

In the mean time, here is some starting points. CMakeLists.txt to build the plugin (without having installed headers):

project( MyGccPlugin )

# Probably nothing that needs 2.6, but that is what I tested with
cmake_minimum_required(VERSION 2.6)

# We need a range of header files (perhaps more than are here)
# Some are relative to the build directory, and some to the source directory
set( GCC_SRC_DIR "/home/bradh/devel/gcc-svn" )
set( GCC_BUILD_DIR "/home/bradh/devel/gcc-build" )
include_directories( ${CMAKE_CURRENT_SOURCE_DIR} )
include_directories( ${GCC_SRC_DIR}/gcc )
include_directories( ${GCC_SRC_DIR}/libcpp/include )
include_directories( ${GCC_SRC_DIR}/include )
include_directories( ${GCC_BUILD_DIR}/gcc )

# Source files for the plugin
set( my_gcc_plugin_SRCS my_gcc_plugin.c )

# build the plugin (share library object)
add_library( my_gcc_plugin MODULE ${my_gcc_plugin_SRCS} )
# have the name set to my_gcc_plugin.so (on Linux) rather than libmy_gcc_plugin.so
set_target_properties( my_gcc_plugin PROPERTIES PREFIX "" )

A framework for a module, heavily based on one I extracted from gcc's testsuite:

/* A sample plugin example that shows how to use the GCC plugin mechanism.  */

#include <stdlib.h>
#include "config.h"
#include "system.h"
#include "coretypes.h"
#include "tree.h"    
#include "tree-pass.h"
#include "intl.h"    
#include "gcc-plugin.h"

/* A really simple callback handler */
void handle_end_of_compilation_unit (void *event_data, void *data)
{                                                                
        warning (0, G_("my gcc plugin: End of compilation unit"));
}                                                                

/* This presumably could be used to enable / disable the plugin */
static bool gate_my_gcc_plugin_example (void)                    
{                                                                
        return true;                                             
}                                                                

static struct gimple_opt_pass pass_my_gcc_plugin_example =
{                                                        
        {                                                
                GIMPLE_PASS,                             
                "my_gcc_plugin_example",              /* name */
                gate_my_gcc_plugin_example,           /* gate */
                NULL,                                 /* execute */
                NULL,                                 /* sub */   
                NULL,                                 /* next */  
                0,                                    /* static_pass_number */
                0,                                    /* tv_id */            
                PROP_cfg,                             /* properties_required */
                0,                                    /* properties_provided */
                0,                                    /* properties_destroyed */
                0,                                    /* todo_flags_start */   
                TODO_dump_func                        /* todo_flags_finish */  
        }                                                                      
};                                                                             

/* Initialization function that GCC calls. This plugin takes an argument
   that specifies the name of the reference pass and an instance number,
   both of which determine where the plugin pass should be inserted.   
*/                                                                     
int plugin_init (const char *plugin_name, struct plugin_gcc_version *version, int argc, struct plugin_argument *argv)
{                                                                                                                   
        struct plugin_pass      pass_info;                                                                          
        char                    *ref_pass_name = NULL;                                                              
        int                     ref_instance_number = 0;                                                            
        int                     i;                                                                                  
        struct plugin_info info = {"0.0a",  "my_gcc_plugin help should go here" };                                  

        /* Process the plugin arguments. This plugin takes the following arguments:
           ref-pass-name=<PASS_NAME> and ref-pass-instance-num=<NUM>.  */         
        for (i = 0; i < argc; ++i) {                                              
                if (strcmp (argv[i].key, "ref-pass-name") == 0) {                 
                        if (argv[i].value) {
                                ref_pass_name = argv[i].value;
                        } else {
                                warning (0, G_("option '-fplugin-arg-%s-ref-pass-name' requires a pass name"), plugin_name);
                        }
                } else if (strcmp (argv[i].key, "ref-pass-instance-num") == 0)  {
                        if (argv[i].value) {
                                ref_instance_number = strtol (argv[i].value, NULL, 0);
                        } else {
                                warning (0, G_("option '-fplugin-arg-%s-ref-pass-instance-num' requires an integer value"), plugin_name);
                        }
                } else {
                        warning (0, G_("plugin %qs: unrecognized argument %qs ignored"), plugin_name, argv[i].key);
                }
        }

        if (!ref_pass_name) {
                error (G_("plugin %qs requires a reference pass name"), plugin_name);
                return 1;
        }

        /* Set up the pass information. This controls where we get inserted in gcc's processing */
        pass_info.pass = &pass_my_gcc_plugin_example.pass;
        pass_info.reference_pass_name = ref_pass_name; /* from the command line argument */
        pass_info.ref_pass_instance_number = ref_instance_number; /* from the command line argument, defaults to 0 */
        pass_info.pos_op = PASS_POS_INSERT_AFTER;

        /* Register a callback for the pass_info setup */
        register_callback (plugin_name, PLUGIN_PASS_MANAGER_SETUP, NULL, &pass_info);

        /* Register a callback for our meta-data (version and help information) */
        register_callback (plugin_name, PLUGIN_INFO, NULL, &info);

        /* Register a callback for the end-of-file handler */
        register_callback (plugin_name, PLUGIN_FINISH_UNIT, handle_end_of_compilation_unit, NULL);

        /* Add other register_callback functions here */

        return 0;
}

You can build it using the normal cmake path-to-sourcedir followed by make.

You can then run gcc/g++ as /opt/gccsvn/bin/g++ -fplugin=./my_gcc_plugin.so -fplugin-arg-my_gcc_plugin-ref-pass-name=cfg my-program-src.c
(with changes the pass name depending on what you want, e.g. if you want the SSA form, you can just use ssa instead of cfg.

Clearly that doesn't actually check anything (it just outputs a warning message at the end of each file being built), but I need to understand a lot more gcc before I can actually do a checker.



> Read More... | Digg This!

published by spstarr on 2009-04-16 20:30:45 in the "Development" category

There will be some new weather providers coming soon. I am currently working on adding a Netherlands source (current conditions only provided free). Someone is working on a German source and I plan on adding 7 day forecast info to the NOAA provider hopefully soon.

Just be aware, I do read what's out there from googleland and yes we do need more weather providers for you folks complaining Eye-wink
As for using Marble, I'd like to, as NOAA provides .kml files which would make radar display very nice to have. But I'm only one person you know, help me out and things happen faster!

Thanks to aseigo, we have Solid Network awareness support for the dataengine.

Thanks to Petri Damstén, we have a cleaner interface for weather data handling.

It's very nice to see the weather wallpaper!

More to come... Smiling



> Read More... | Digg This!

published by bille on 2009-03-31 21:50:12 in the "Development" category

A promising student was talking with me about working on Network Management in GSoC 2009, but decided to concentrate on his studies this summer. Out of the discussion I've created this idea proposal. In case anyone is interested in making mobile broadband connections really easy to do in Network Management, see the KDE Google Summer of Code 2009 ideas page.



> Read More... | Digg This!