Planned to spend today with my new toy, the Canon 9900F Scanner, and so I did. It wasn't quite the way I had intended, though. I have gradually given up on programs like SANE (which, I contend, contains a typo in the expansion: it should read Scanner Access Not Easy). On the one hand, it severely limits the choice of hardware, and on the other hand scanners already come bundled with lots of software. Yes, it's in Microsoft eye candy format, but it should be easy to use.

The truth is a little different. Just installing the software must have taken 100 clicks, including requiring me to accept eight end user license agreements, presumably all different. I suppose they do illegal things like disclaim all notion of “fitness for purpose”. But I could only see a little of them, and the parts that I saw seemed innocuous enough.

Running the software was another matter: it looks as if it's broken. The user interface certainly is. To read in a set of negatives or slides, you place them with the emulsion side up (unlike in an enlarger, and a detail that was obviously too technical for the superficial manual to mention) and start a new program (“scanner driver”, I suppose), which blocks the previous program until it's finished. This driver decides for itself where the boundaries of the images are—with about 70% accuracy, meaning that in any one set of 8 slides about 2 are incorrectly framed. I can't see any way in this toy interface to influence this action.

I had been warned about the scan speed, but without accurate estimates. I haven't tried timing it, but my best bet is that each slide takes about 3 minutes to scan, about 25 minutes total for 8 slides (this is at 1600 bpi, which gives resolutions roughly equivalent to my other photos). When they've finally finished scanning, you have to save them—one at a time, with multiple mouse clicks for each photo. By default it tries to save them in some proprietary format, and you have to select JPEG (“JPG”) format every time you start the program. It also forgets its numbering scheme when you stop it, so if you run the program multiple times, you will end up having to store the files in different directories or rename each individual image for the subsequent scans. At least it remembers resolution parameters, and I was able to set up Samba to store the images directly on a real computer.

Got so frustrated that I checked if I could use SANE with the device. FreeBSD recognized the Firewire connection, but of course SANE doesn't support it.

The whole thing reminds me yet again of how bad software has become of late. The “Microsoft Mentality” seems to accept software that has bugs of this nature, as long as it's “intuitive” (i.e. follows the eye-candy conventions). The lack of published standards means that I can't just go and build a driver that will work properly. This problem is probably more important than all the issues that people are trying to legislate about “Open Source”: free software can't be much good if you have to reverse engineer everything you do.

As if that wasn't enough, finally had a serious paper jam with my Brother HL-2700CN colour laser printer. I had accidentally slipped a wire between the duplex unit and the main frame, and it apparently blocked the paper path. After clearing the resultant jam, it kept jamming, and I had to remove the duplex unit (obviously something that Brother hadn't intended) to remove the paper jammed in between. Putting the duplex unit back in is significantly more difficult than putting it in the first time round, because the screws provided collapse after it's installed, and they're no longer available the second time round.

Today was the day that Yvonne had planned for her Gymkhana, and so of course it had to rain. It's not that we're unhappy about rain—it's been quite a dry autumn—but it always seems to have to happen when it's least needed. Up to yesterday we had been wondering whether to cancel or not, and eventually decided to continue. We did get a little rain, but in fact it was really what the weather forecast claimed:

Sunday     :   Cool and cloudy with a few showers.  Mainly moderate southwesterly
winds.

...
Mount Barker   Min  6   Max 15

Normally the term “a few showers” means that it will rain all day long, but today it was accurate. Had a good time, and unlike last time, Darah behaved herself very well, and we went through the entire field in record time. Took a few photos, though most was on video.

After that, the barbecue, which was attended by about half the people who had confirmed they would come, leaving us with a lot of food left over. I think next year we'll let people bring their own food; at least that way we don't have to eat barbecue food for the rest of the week.

More work scanning slides. The software is Just Plain Broken. It has great difficulty recognizing boundaries, and even the online manual is wrong: it's in HTML, and the main file has the intuitive name canoscan.htm (sic). In it is stuff like:

    <TD WIDTH="95"><IMG SRC="CanoScan/img/header_logo.gif" WIDTH="95" HEGHT="34" HEIGHT="28"></TD>
    <TD WIDTH="631"><IMG SRC="CanoScan/img/hd_01.gif" WIDTH="631" HEGHT="34" HEIGHT="28"></TD>

HTML tidy finds 53 errors in this document, including proprietary extensions. The real problem, though, is that there is no directory CanoScan; it's called canoscan, so nothing worked until I put in a symlink. I'm continually appalled by the poor quality of software nowadays.

The manual did tell me how to get around incorrectly recognized negatives, though: turn off thumbnails, which gives the entire area. Unfortunately, contrary to the documentation, and probably due to a bug, it also turns off retouching, which left some very dirty negatives. What a pain. Despite everything, managed to scan in two 36 exposure slide films, which, at an average scan time of 5 minutes per slide, took all day. The first film was taken on the Bay of Bengal on the trip from Penang to Madras, and the second one from Madras to Agra.

Back to work again, or at least that was my intention. Working with pthreads is a real pain, as I had feared. Andrew had done some work under Linux, but under FreeBSD it didn't work the same way. In fact, we're not sure it will under Linux: Andrew doesn't have an SMP box. With zaphod now running with 768 MB of main memory, I ended up with the application crashing randomly somewhere in the KSE library. Worse, it didn't seem to be using both CPUs. Confirmed with the Ozlabs people that pthreads does, indeed, use multiple CPUs under Linux, but I don't think that it does under FreeBSD. Our target platform is probably Solaris, and I have no idea what that's like, but I could well imagine it will be the same as FreeBSD. At the very least it looks like I'll have to install Yet Another Linux system. At least I should gain some routine...

Off to Sydney in the late afternoon for an AUUG board meeting. Things are no longer the same as in the “old” days (i.e. four years ago). I was the only person there, and I was put in the North Sydney Harbourview Hotel, not the nicest hotel in the world: a railway line goes underneath it, and it's severely in need of refurbishing. My room included the only air conditioner thermostat I've seen in Australia that still has a Fahrenheit scale, and the fan was very noisy. Off to a Malaysian restaurant past which I had been on previous visits, but it was pretty rough.

Woken by the first train of the morning at 5:36, but managed to get back to sleep to a more humane time. Off to the meeting, where we discussed—yet again—the future of AUUG, taking all day. Lunch at a Thai place which would have been excellent had it not been for their emphasis on creamy sauces. At least they had nice presentation, including chilis cut in the shape of a flower.

In the evening was the AUUG NSW chapter meeting, with John Terpstra, who is doing a road show over the next couple of weeks. Started off with an interesting viewpoint on software marketing economics, using such fashionable terms as “mercantilism” and “feudal”, but I had to leave for the airport before he was done.

  struct sigaction newdisp = {&caught, 0, 0};
...
utility-lib.cc:42: warning: aggregate has a partly bracketed initializer

struct sigaction {
    union {
        void    (*__sa_handler)(int);
        void    (*__sa_sigaction)(int, struct __siginfo *, void *);
    } __sigaction_u;        /* signal handler */
    int sa_flags;       /* see signal options below */
    sigset_t sa_mask;       /* signal mask to apply */
};

  struct sigaction newdisp = {{&caught}, 0, 0};
...
utility-lib.cc:42: warning: aggregate has a partly bracketed initializer
utility-lib.cc:42: warning: aggregate has a partly bracketed initializer

While doing that, also looked at some of the problems I was having on Monday. Upgraded zaphod—yet again—to the latest FreeBSD-CURRENT, and also revived beeble (dual processor Celeron machine) as brynhild, a Debian box that I set up last July and upgraded the kernel to Linux 2.6.5 in the hope that we can get some useful information about the effectiveness of threading under Linux.

Since that wasn't keeping me busy either, also took a look at the SPARCstation 5 that Peter Cassidy gave me on Friday, and tried to install Solaris 8 on it. It proved that the CD-ROM drive he gave me had the wrong SCSI ID (7, which looks dangerous), but even after setting it to the regulation ID 6, I couldn't read from it. Fortunately I had an identical one myself, and it worked. After over 20 minutes booting from the CD-ROM, I got the message:

64Mb of memory is required for Solaris
32 Mb was found.
Exiting
#

Took the box apart and confirmed that the memory chips looked identical to the SDRAMS of a few years back, of which by coincidence I had a 128 MB chip over:

On trying to insert the 128 MB chip, discovered that the index grooves are offset by approximately a millimetre, presumably for a good reason. Put the box together again and tried to boot NetBSD, somewhat hampered by lack of documentation, in particular the name of the file to boot. Tried again with OpenBSD, and was able to mount the root file system of the existing installation (Solaris 2.5, which apparently is able to get by with less memory. Fixed the /etc/passwd and /etc/shadow files, and was then able to boot and log in to the machine.

That didn't help much: there's no compiler or anything on the box, and so I couldn't do very much. I suppose I could try installing the compilers from Solaris 8, but who knows what problems I'd have there. Time for a bigger box, preferably with multiple processors.

Still fighting this brain-dead Canon 9900F Scanner. It still only recognizes about 70% of the slides I put into it, and puts ridiculous frames around the ones it doesn't recognize:

This is a real pain to use.

Less activity than yesterday, but still got some useful work done. Finally worked my way through the C++ code and got it to work, not without running into another problem:

cc -o fconv fconv.o -g -L.  -lmonkey -lm
fconv.o:/home/monkey/monkeyfs/fconv.cc:190: undefined reference to `__gxx_personality_v0'
./libmonkey.a(utility-lib.o):/home/monkey/monkeyfs/utility-lib.cc:42: undefined reference to `__gxx_personality_v0'
./libmonkey.a(physio.o):/home/monkey/monkeyfs/physio.cc:24: undefined reference to `__gxx_personality_v0'
./libmonkey.a(cache.o):/home/monkey/monkeyfs/cache.cc:29: undefined reference to `__gxx_personality_v0'
./libmonkey.a(search.o):/home/monkey/monkeyfs/search.cc:22: undefined reference to `__gxx_personality_v0'
./libmonkey.a(rows.o): In function `Monkey::AddAltKey(char*, unsigned short, unsigned short, int, int*, key_compare_type)':
/home/monkey/monkeyfs/rows.cc:109: undefined reference to `operator new(unsigned)'
./libmonkey.a(rows.o): In function `Monkey::Open(char*, int)':
/home/monkey/monkeyfs/rows.cc:813: undefined reference to `operator new(unsigned)'
./libmonkey.a(rows.o):/home/monkey/monkeyfs/rows.cc:78: undefined reference to `__gxx_personality_v0'
./libmonkey.a(audit.o):/home/monkey/monkeyfs/audit.cc:24: undefined reference to `__gxx_personality_v0'
./libmonkey.a(utility.o):/home/monkey/monkeyfs/utility.cc:35: undefined reference to `__gxx_personality_v0'
./libmonkey.a(modify.o):/home/monkey/monkeyfs/modify.cc:20: undefined reference to `__gxx_personality_v0'
*** Error code 1

This proved to be due to the rather unobvious fact that I was using the cc driver for the compiler instead of the g++ driver. This is a real pain: all this code used to work. After changing the link driver, it did work, but now I need to adapt it to where it needs to go.

More work on Monkey today, and I'm now at the point where I can start modifying code. The big change is from C++ to C, which proved to be less difficult than I thought, mainly because I got used to the idea of not using many C++ extensions fairly early on. It's a pity, though: C++ has a number of features, notably the implicit use of a class instance, that would work equally well in C.

Finally got fed up with the Canon 9900F Scanner, which has a driver that can't identify where the slides are in the frame. You'd think that it would be able to do at least that, since it's apparently written exactly for this scanner. After establishing that I did, indeed, have the latest version of the driver, called up their support line. I would much rather have sent an email, but they didn't give me that choice. Spoke to Ejaz, who initially had difficulty understanding what the problem was, but who asked me to “uninstall” the driver and reboot. OK, this is Microsoft XP, so I did that, and was astounded by the fact that the driver wasn't gone after all. I wish I knew what these Microsoft terms really mean; I've given up hoping that they might use terms at their face value.

Of course, the problem didn't go away, and so Ejaz first suggested that I return the scanner, because it was obviously defective (because the driver can't identify the edges of the slides!). When I suggested that he was misunderstanding the situation, he suggested that I should install Adobe PhotoElements, a stripped down version of PhotoShop which also came with the scanner, and which I hadn't installed because it asked for a serial number. Finally found that and installed it, but of course it didn't make any difference, since it called the same driver. The only thing I did get to notice is that PhotoElements require even more mouse-pushing: for every single image I have to tell it to save in JPEG format and not in its own proprietary format. There are preferences, but such an idea as specifying a preferred directory and save format doesn't seem to be one of them.

After that, Ejaz suggested that I should try connecting the scanner to another machine and see if it worked there. About this time I got completely fed up and asked whether, if that also didn't work, he would then ask me to try running it in a different State to see if that made any difference. It seems that second-level support won't even look at it unless people go through this ridiculous rigmarole. Finally agreed with him that I'll document the whole issue (amongst other things, here on this web site) and send him an email, which he can then forward to their second-level technical support.

Came into the office this morning to discover that wantadilla had paniced:

This GDB was configured as "i386-undermydesk-freebsd"...(no debugging symbols found)...
panic: bremfree: bp 0xd2d79490 not locked

I no longer had the original kernel tree for this system—it dated back to October 2002—so gave it up as a bad idea and upgraded to a modern -CURRENT, which worked better than I expected.

Apart from that, spent most of the day scanning in old photos and documenting them. Interestingly, though it takes about 4 minutes to scan a single photo, I wasn't idle. Setting up this scanner to recognize the slides is an iterative process: I have never yet had it recognize the frames of all 8 slides correctly, though in some cases it just adds a black frame along one or two sides. In other cases, it gets things completely broken, as the incorrect frame in the following image shows:

Follow the link to get something visible. The final screen shot is in original size, but that's only 1024x768, and the images are made much smaller than that, so it's very difficult to recognize them. This image, of the uninterpreted slides, shows clearly that the software has placed the frame incorrectly. The corresponding interpreted (“thumbnail”) view follows this frame, making it impossible to use.

Also spent some time playing around with the Makefile for building my photo pages, and it now creates thumbnail images and updates the index file. Still, it's very frustrating to use this scanner.

Another day spent seemingly only scanning in photos, though I spent a couple of hours working out a Problem Report for the Canon 9900F Scanner, in the process discovering a few more issues about the nature of the problem: it has a box for selecting the kind of film, but only in raw display mode. Unfortunately, it ignores the box, and tends to reset it. What a crock. I wonder if anybody else has had problems of this nature; Ejaz says no, but that's to be expected.

In the afternoon out riding in Kuitpo Forest, and once again had only a short ride. It always seems to be the same: Yvonne's horse tires after about 20 minutes, and we have to return. It doesn't seem to matter which horse (today it was La Tigre), nor the fact that I ride at most every other week and Yvonne rides every day. Why do her horses tire and not mine?

Spent most of the day converting my Monkey subset from C++ to C. It's painful business, and it shows that C++ really does have a number of advantages over C. If only there were a way to use the clever object semantics without the environmental bloat, something like a C+. In particular, it's a real nuisance that there's no way to incorporate a structure into another structure without leaving its wrapper behind. In C++, I can write something like this:

struct fcb
{
  friend int Tarzan (Monkey *, int, void *);     /* magic name of function friendly to monkeys */
  friend int Tarzan (Monkey *, int, int);        /* magic name of function friendly to monkeys */

                                                 /* --------------------------------|    */
  int32 flags;                                   /* |           flags               |  0 */
                                                 /* |                               |    */
                                                 /* --------------------------------|    */
...

struct Ks_info: public fcb
{
...

class Monkey: public Ks_info
{
...

struct fcb
{
                                                 /* C has no friends */
                                                 /* --------------------------------|    */
  int32 flags;                                   /* |           flags               |  0 */
...

struct Ks_info
{
...
  struct fcb fcb;
  };

struct Monkey
{
...
  struct Ks_info ksinfo;

To refer to the fields at all, I need an explicit additional argument this to the functions, and I need to refer to the same flags field as this->ksinfo.fcb.flags. There's probably no difference in what the compiler generates, so it's not a question of efficiency, simply legibility.

In the evening spent some time updating my Canon 9900F problem report. Discovered that there is a way of telling the driver what kind of films are in the scanner. The problem is, it doesn't just ignore the information, it changes it.

This is obviously an interesting matter: I've been getting a lot of web site hits on the subject. I wonder how many other people have similar experiences but have been fobbed off by the support people.

More work on converting Monkey to C today. It's getting faster, but it still takes forever. By the evening I had only one source file to go.

I haven't made up my mind on how bad it looks in C. I end up with lots of things like this:

  if (! (M->K.flags & OPENFLAG))                            /* not open, */
    return errno = MONKEY_NOTOPEN;                          /* error */
  startreq (M);
  Monkey_FlushBuffers (M);                                  /* flush the cache */
  Monkey_cache_dealloc (M);                                 /* deallocate the cache structures */
  M->K.flags &= ~ (OPENFLAG | CORRUPT | DIRTY);             /* no longer open or inconsistent */
  if (M->K.flags & FCB_CHANGED)                             /* FCB changed since the file was opened, */
    Monkey_flush_fcb (M, 1);                                /* write it back to disk */

  if (! (flags & OPENFLAG))                                 /* not open, */
    return errno = MONKEY_NOTOPEN;                          /* error */
  startreq ();
  FlushBuffers ();                                          /* flush the cache */
  cache_dealloc ();                                         /* deallocate the cache structures */
  flags &= ~ (OPENFLAG | CORRUPT | DIRTY);                  /* no longer open or inconsistent */
  if (flags & FCB_CHANGED)                                  /* FCB changed since the file was opened, */
    flush_fcb (1);                                          /* write it back to disk */

The big changes here are the implicit pointers and the naming of the functions. The question is, is this a good idea? It's very likely that the code generated by both these fragments would be identical, but the C++ hides a lot of the detail. I can't make up my mind whether it's better hidden or not. And yes, I've deliberately shortened the pointers and coalesced a structure; for example, originally I had written Monkey->ksinfo.fcb.flags instead of flags, but that seemed too much, so now it's M->K.flags.

In fact, the original was much larger. I'm stripping out lots of functionality. Monkey does all sorts of nice things that we don't need in the current application, including multiple alternate keys, compression, field descriptions, auditing and user-defined collation sequences. In fact, the original C++ code looks like this:

  if (! (flags & OPENFLAG))                                 /* not open, */
    return errno = MONKEY_NOTOPEN;                          /* error */
  startreq ();
  FlushBuffers ();                                          /* flush the cache */
  cache_dealloc ();                                         /* deallocate the cache structures */
  if (flags & (ICOMPR | DCOMPR))                            /* we have a compression buffer */
    free (compbuf);                                         /* give it back */
  flags &= ~ (OPENFLAG | CORRUPT | DIRTY);                  /* no longer open or inconsistent */
  if (! (flags & ISALTFILE))                                /* not an alternate key file */
    {
    if (altkeys)
      {
      altkeyeof = altfile->eof;                             /* save alt key EOF and */
      altkeysindexlevel = altfile->indexlevels;             /* number of index levels */
      }
    if (flags & FCB_CHANGED)                                /* FCB changed since the file was opened, */
      flush_fcb (1);                                        /* write it back to disk */
    close (fcbfile);                                        /* and FCB file */
    if (fdsize)                                             /* we have field descriptors */
      free ((char *) fd);                                   /* free them */
    if (altkeys)                                            /* we have alternate keys */
      {
      altfile->Close ();                                    /* close the altfile first */
      free ((char *) ak);                                   /* free the key storage */
      }
    if (flags & (MY_COLLATION | ASCII_COLLATION))           /* then we have a collation table, */
      free ((char *) collation);                            /* free it */
    }

That has nothing to do with C++, of course, but it's interesting to strip down a program for once. Usually you incrementally add things, and you don't see how much bigger it gets all at once. You do when you remove the things again.

Playing around with Microsoft, trying to print from “Word”. I had forgotten that Microsoft needs a “driver” (apparently a definition file) for printers, and I had mislaid the CD-ROM that came with the printer, so had to print to files instead and get FreeBSD to print.

Setting up networked printers under Microsoft is really strange, though: as the following image (click for larger versions) shows, “To set up a printer that is not attached to a [Microsoft] print server, use the "Local printer" option”:

I also found this image amusing: “Always trust content from Microsoft Corporation”.

Spent more time than I would like working on administrative stuff today. One of the things about my new job is that I'm coming more in contact with the Microsoft world, and I keep getting documents in Microsoft “Word” format. For some reason, possibly related to the last update of wantadilla, OpenOffice decided to crash on me with a SIGSEGV, so had to upgrade that. Jonathon Coombes tells me that it's possible to get it to do things in command-line mode, but I haven't seen much evidence yet. In any case, was obviously too stupid to set up printing on OpenOffice as well, so ran that through my FreeBSD file system as well.

In the afternoon to a board meeting of the IT Council of South Australia. Looks like we have an interesting year ahead of us, with some big changes planned.

In the morning, spent some time thinking about data structures, and came to a breakthrough in understanding how Monkey could be the solution to just about every problem. Then continued working on the Monkey conversion, and finally got it to compile. Even had time to spend a bit of time rearranging things to better fit the environment in which it will be living.

In the afternoon into town for the first of what I expect to be weekly meetings of the Storage Systems Group. Today it was at Ross' house, and we spent three hours talking about a number of things, including of course the use of Monkey. That topic proved to be less successful: without a white board, it was difficult to explain the concepts. I think it would have been difficult even with the white board, and I left with the action item to write up Monkey, which I'm going to defer until I have something to show for myself. I did come up with a couple of interesting suggestions, though.

How many things I have to do! Getting this Monkey prototype working looks so simple now, though inevitably it will take longer than it seems.

Unfortunately, didn't get very far down that road. Follow-ups to yesterday's meeting took up a surprising amount of time. Still, spent some time going through the remaining Monkey code and reviewing it. The original Monkey used three files:

• The main file, which had a Microsoft-like convention of ending in the characters .KSF, for example Monkey.KSF.
• A separate file for the alternate key file, structurally identical to the main file, but with the ending replaced by .AKF, for example Monkey.AKF.
• An FCB file with basic information about the other two files, including block and record lengths, and key information. It had the base name of the other files stripped of the endings, i.e. Monkey.

In particular the FCB file seems to have outlived its usefulness. I developed Monkey initially under Microsoft MS-DOS, and it's a hangover from those times. The alternate key file still does have a justification, since it contains different records from the main file, and the block sizes can be different. I think it's too hard to store different block sizes in a Monkey file, but if the block size is the same, there should be no reason why we can't store more than one tree in a single file.

It's a rather unusual feeling to do different things on weekends, and it's taking getting used to. Today spent some time preparing for my first “real” all-grain brews, including newly-built equipment such as the gold-plated wort cooler that Bob Hay built for me. Also noticed some unpleasant effects of leaving the said cooler in the boil pot in pure water overnight: the electrolytic effect caused noticeable corrosion and left a lot of crud in the water:

That's a timely reminder not to boil the cooler in the wort the whole time, but to add it just before the end.

Spent almost the whole day brewing two batches of beer from grain using the tools that I had already prepared. The fun started when trying to sparge the first brew: nothing came out. Expecting that my primitive sparge manifold was to blame, found a tool to extract it from the hot mash, with no difference. Finally traced it to the tap, which was blocked. The taps I'm using come with a little plastic strainer in the nozzle:

Unfortunately, it's relatively easy for a little grain to get in there, and they clog up very easily. The photos here were taken later, when the other tap got itself clogged with nothing more solid than the scum from hop pellets. Here's what it looks like after removing the strainer:

Finding the problem and fixing it was a lot more difficult for the mash, and it took me over an hour. On replacing the manifold, discovered that there are more reasons than uniformity to decide how many slits to put in the manifold: the flow rate was still too low, and I had to remove the thing again and cut three times as many slits in it. After that, mercifully, the sparge worked, and I had no problems with the rest of the brew. In particular, the wort cooler worked nicely, and I was down to pitching temperature in about 20 minutes.

The second brew went better, of course, and I got a really nice, clear wort out of the sparge. I'll be interested to see what effect that has on the flavour of the beer. Certainly the problems have jeopardized my original intention of making two beers which vary only slightly.

It all took a long time. It's probably worth formalizing the time it takes to do a brew like this:

So, a total of about 4½ hours for the operation. Admittedly, it shouldn't require much presence, but it's longer than I expected.

As if that weren't enough frustration for one day, also tried to install the GIMP on my system, which failed with too old a version of X, not the first port to do so; it seems that something has changed in the X rendering libraries. Somehow the Ports Collection is in trouble; all sorts of dependencies failed to compile, and I ended up recompiling the entire X distribution several times, without getting past the problems.

Adelaide Cup day today, a public holiday. Spent most of the day catching up on things that I had let slip yesterday, and a good week's worth of lower-priority mail. As my priorities shift, keeping up with mail is becoming even more difficult than it has always been.

Also continued with the rebuild of my ports; there has to be an easier way. Several of them didn't build, and I ended up installing the GIMP from a package. Spent the entire day running portupgrade, unfortunately in interactive mode, and ended up having to answer questions about ports I had never heard of. When it finally came to upgrade gcc, I hit^C and broke things completely:

/usr/local/sbin/portupgrade:35:in `require': No such file to load -- pkgtools (LoadError)
        from /usr/local/sbin/portupgrade:35

My beer has been interesting: for the first time ever, I aerated the wort of Brew 28 for over half an hour, and it rewarded me by fermenting nearly twice as fast (and noticeably warmer) than I've ever had before. Brew 29 didn't get as much air and fermented at less than half the rate. I'll have to investigate how to aerate the wort without it foaming out of the fermenter; looks like I'll have to aerate about half at a time.

Finally got round to starting on my userland test tool. Made relatively good progress in setting up a framework, incorporating a command language parser that I had originally used in Vinum, and which I had used in other programs too. The problem was that the keywords appear in different places in different guises, and getting them in sync was a pain. Solved that with a little shell script which generates the necessary bits and pieces, and by evening I had the framework set up and compiling and “running” cleanly. All I need now is for it to do something.

Sent another message to Ejaz about the Canon scanner problems, and asking for immediate action. This did bring some reaction: a reply from Ejaz, stating that he had received my message, but not addressing the other issues (isn't that so often the way when people answer the wrong way round?), and a call from Mitchell in Sydney, who told me that he had been able to reproduce the problem, that he had escalated it to Japan, and that he hoped for a response within a week. Interestingly, he drew a lot of attention to the fact that he had tried it on two different computers with two different mother boards (both presumably Microsoft-based) and also on an Apple. Maybe they do have a lingering suspicion that it could be a problem with the computer hardware.

Also managed to get my ports sorted out, sort of. Whatever breakage I did with portupgrade yesterday seems to be unrecoverable, and I ended up building packages on adelaide and installing them on wantadilla. There must be an easier way: one would seem to be to start from scratch with the ports when installing a new system, and only installing the top-level ports (i.e. the ones you explicitly install in the first place), as opposed to all the myriad support ports. There should be a way to note which are which; I'll do some thinking.

They're doing some roadside tidy-up work on Battunga Road (the road past our property), and as a result they have a lot of soil left over. A week or two ago we agreed that they could dump a lot of it in our creek, which is suffering badly from erosion (in fact, that's probably where a lot of the soil came from, so they're just putting it back). Today they started, and the logistics proved to be more complicated: the people who run these operations are not rocket scientists, and just explaining that we wanted to limit the creek, rather than to fill it in completely, seemed beyond their grasp. Spent a lot of time trying to sort that out, not entirely successfully.

On the work front, continued with the coding today, and made fair progress. The original Monkey has now shrunk to about 10% of its initial size, reminding me of the quotation:

Hoare's Law of Large Problems:

Inside every large problem is a small problem struggling to get out.

Still a number of global changes happening, in particular to the cache layer, which will probably require further attention. It proves to be interesting to put the entire source for Monkey (5 files) into a single file for editing. When it's more or less in the shape I want, and I actually get round to testing it, I can take it apart again.

One of the first things I do every morning is to check my beer stocks, and when brewing also the temperature and fermentation rates of the beer. For that I use brewmaster.lemis.com, an ancient Dell laptop prototype that I keep in the kitchen and connect to the network via wireless. It runs OpenBSD, mainly because that was the first system that would understand its PCMCIA bus. The files are all stored on wantadilla and accessed via NFS. Today, the network hung, and I couldn't get the interface back up again. Checked with firefly, Yana's machine, which was connected to the same access point, and that worked fine. Rebooting brewmaster didn't help, nor did changing the wireless card. Finally I rebooted the access point, and things came back to life. It seems that the AP only failed partially.

The AP runs Linux, as do two other machines in its immediate physical proximity: tivo.lemis.com, a TiVo, and sat-gw, which runs my satellite downlink. I don't know what it is about them, but they're three of the biggest problem machines I have.

Had intended to continue with my new program today, but realized that the cache abstraction was still far too closely coupled with Monkey data structures, and spent the day tidying that up as well. It looks a lot cleaner now, but I'm still left with the conviction that it's too primitive. On the other hand, the kernel already provides buffer cache, so we're not looking for absolute performance here; as long as the cache search times don't distort the relationships, there should be no problem.

Continued working on my program today, spending most of the time looking at how to modernize Monkey. It was based on Tandem's Enscribe product, which must be 30 years old. Enscribe was good, but looking at things from a modern standpoint makes it look a little old-fashioned in some areas. Spent all day redesigning and reimplementing the data and index block structures.

Gradually I'm getting into the routine of relaxing at weekends, and did fairly well today. In the morning bottled some beer. I'm still surprised how much difference the aeration of the wort seems to have made: the beer I bottled today had been pitched less than 6 days before, but the fermentation had completely finished. The wort I pitched a couple of hours later, and which was less aerated and otherwise almost identical, is still fermenting away in primary fermentation and looks like taking a few more days.

Apart from that, did little. Spent some time watching TV, scanned in even more photos, and took it easy.

Another day much as yesterday, catching up on mail and scanning photos. Also thinking about controlling my brews more carefully: the difference in fermentation speeds and temperatures between Brew 28 and Brew 29 was not only instructive, it was also a temperature control catastrophe. Ordered some electronics to measure temperature and control heating and cooling. More work ahead.

We had intended to go riding, but somehow didn't have enough time. I wonder how other people manage to get bored.

On with the work on my program today, and spent more time tweaking the cache implementation. I must resist fixing things that don't make much difference; the important thing is to get the thing running (or at least stumbling) at all. Made reasonable progress nevertheless, and got as far as starting to test the initial functionality. It currently looks like I overwrite something malloced almost before I start. Maybe it's time to investigate Valgrind.

Continued work on my initialization code today and found that my malloc corruption was non-existent: I had forgotten to initialize a member of the superblock, so called malloc() with a 0 length—and it returned a pointer 0x800. Strange, and non-intuitive, but it seems that the standard allows it.

Apart from that, a typical day of debugging, and by the end of the evening had some data structures on disk to show for it. Also created some gdb macros to look at the data structures. Programming in gdb macro language used to be like pulling teeth. Now it's a lot better: it's just annoying.

Further work on my project today, and made faster progress than intended: I hadn't planned what to do next, and spent some time doing that. Nothing much else of interest.

On with my project today. Ran into what appears to be a day 1 bug in Monkey: if a record exists with a key shorter than the total key length (i.e. the record is short and the key at the end is only partial), it was not possible to insert a record with the same initial key but of different length. This means that if the file contains a record with the key foo, it was not possible to create entries fo or foox. I don't understand why I didn't fix it: there was a XXX as a comment at the comparison, and it was easy enough to fix (if the keys are the same for their common length, compare their lengths instead).

Also ran into some problems I had caused when mutilating Monkey into shape, so didn't get as far as I had expected. Mañana.

More work on the functions I had created yesterday, and got them working quite nicely—I thought. Then decided that it would be a good test of the system to have detailed list functionality. It turned out I was correct: it was a good test, and the program failed. Spent the rest of the day investigating the block structure, which I suspect I broke during the code changes of the last couple of weeks. In the process worked out a number of useful gdb macros, so I can now do things like this:

(gdb) pcache
$22 = {
  blockcount = 1024,
  blocksize = 65536,
  alloccount = 19,
  first = 1021,
  Block = 0x8055000,
  stats = {
    reads = 0,
    writes = 0,
    updates = 0,
    flushes = 1,
    hits = 79,
    misses = 18,
    blockin = 0,
    blockout = 0
  }
}
    Slot     RBN     block       flags
    1021       3   807c000       dirty data block, 5 records
    1019       5   809d000       dirty data block, 5 records
    1020       4   808d000       index block, level 1, 1 records
    1022       2   806c000       index block, level 1, 1 records
(etc)
(gdb) block 1019
$15 = (struct Block *) 0x809d000
$16 = {
  h = {
    rbn = 5,
    nrecs = 5,
    ilevel = 0 '\0'
  },
  sl = {
    nextrbn = 4294967295,
    prevrbn = 4294967295
  },
  data = "\004"
}
Block at RBN 0x5, level 0, 5 records:
block: 809d000, bptr: 80acffe, offset: 16
Rec 0, offset 10, length 72, absaddr 0x809d010
Contents: ~~~~~~~~~~~~~~~~~~~~~~~@~~~~~~~~~~~~~~~@~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
$17 = {
  inode_num = 4,
  mode = 493,
  nlink = 1,
  uid = 1004,
  gid = 1000,
  atime = 1085720266,
  atimensec = 0,
  mtime = 0,
  mtimensec = 0,
  ctime = 1085720266,
  ctimensec = 0,
  flags = 0,
  birthtime = 0,
  birthtimensec = 0,
  eof = 0,
  tlib = 0,
  indexlevels = 0
}
Rec 1, offset 58, length 72, absaddr 0x809d058
(etc)

That shows a nice, clean inode; unfortunately, what the list function gets to see is complete junk. Monday.

In the afternoon, Phil of Barker Air Conditioning came along to look at our old air conditioner, which hasn't given us any trouble for nearly five years. A couple of days ago it went back to its old tricks of icing up. Phil took a look, but since it was functioning correctly, there wasn't too much he could do. He refuted his earlier assumption (on the phone) that there had been a refrigerant leak, but decided now that the reversing valve had been sticking. He could be right; it's a $700 bill if he is, not to mention a wait of a week or two.

A day with a difference today. Started by bottling Brew 29, which went pretty quickly. I used to spend about two hours bottling a brew of beer, but with practice and streamlining I'm now down to over an hour.

Then out and discovered that the air conditioner was icing up again:

This photo, looking in from the top of the air conditioning unit, shows the two halves of the cooling coil. The bottom half is covered in ice. The rusty cylinder at bottom left is the motor housing for one of the fans.

Spent some time investigating this, and placed the de-icing sensor at the end of one of the pipes in the bottom half of the coil, which didn't help much:

This one shows the sensor wire at the right, iced up. After a call to Phil came to the realization that during the de-icing cycle two things should happen: the reversing valve should reverse (difficult to see), and the fans should turn off (obvious). The fans weren't turning off, so after some perusal of the circuit diagram and more talk with Phil, came to the conclusion that there was something wrong with the de-icing switch. It's normally closed, so disconnecting one of the connectors should start a de-icing cycle. It did.

There's still the possibility that the way I connected the switch sensor wasn't good enough, so replaced it in the middle of the bottom half of the coil in the same manner as it was previously mounted in the top.

In the afternoon turned my attention to a temperature logger kit that I had bought from Ozitronics kits, and which I had seen at Linux.conf.au in January. In the process, realized that I had forgotten a lot of common knowledge about electronics components, and that the kit instructions didn't help. Which is the positive pole of an electrolytic capacitor? Which way round does a diode go? Spent some time confirming that my suspicions were correct (the capacitors I have have a marking next to the negative lead, which is also shorter; diodes have a bar at the cathode end, the one that is shown as a bar in the circuit symbol). Didn't take long to put the kit together, but connecting up the temperature sensors, which look like small transistors, is terrible. There just don't seem to be any components that you can use for this sort of thing. I need to find a better solution to this issue, but for the time being just kludged it by soldering things together, along with some lengths of (German) telephone wire, which has the most confusing colour coding I've ever seen: all conductors red, most with varied-spacing blue stripes. The spacing is such that you can't tell them apart without stripping at least 10 cm of the outside insulation, so I ended up using a continuity meter to find the ends. The result works, but looks terrible:

More investigation is required before I can really use these things, but at least I'm getting an output like this:

=== root@sydney (/dev/ttyp1) ~ 21 -> cu -s 2400 -l /dev/cuaa0
Connected.
R V1.0 2002-01-06 20:37:37 C
1 0024.25
3 0023.50
4 0024.68
1 0024.25

The first number on each line is the sensor number (note from the photo that 2 isn't connected), and the second is the temperature in °C. Looking at the values, it seems that the device uses Fahrenheit internally (resolution 0.10 °F) and converts to °C. We can live with that.

Up early this morning and out riding in the Razorback Road section of Kuitpo Forest with Yvonne and Michael Hickinbotham. As with our ride of three weeks ago, La Tigre didn't hold the distance. Yvonne can say what she wants, but I've yet to see a Paso Peruano that can keep moving at a reasonable speed for any length of time. It seems that La Tigre had some infection last time, and indeed she held out better this time, over an hour, but that's still not very impressive.

Back home and looked at the air conditioner, which hasn't iced up since repositioning the ice sensor, so put it back together. Hopefully that one is sorted out, and at a lot less expense than changing the reversing valve.

Spent most of the rest of the afternoon writing a program to control fermentation temperatures. ending up with over 1600 lines of code which could actually do something, though of course not control temperatures (yet). Still, things are looking a lot more obvious now.

What a waste of a day! After getting through my mail and documentation in the morning, got a phone call from Yvonne on a land line, because she couldn't get through to me on the mobile. There was a good reason for that: I couldn't find it, and in fact I couldn't recall seeing it after returning from yesterday's ride. Searched the house, but couldn't find it, so started preparing for a new SIM card. That's always more difficult than expected, of course: the phone is still registered in the name of Linuxcare, who do not exist any more in Australia. I've tried to transfer it into my name, but for that I require the agreement of Linuxcare: catch 22. Spent some time on the phone trying to negotiate things, but in the end I was left with the recommendation just to forget the account and get a new number. I can't terminate the account, of course, so that would leave Telstra out of pocket. What a ridiculous setup.

In the afternoon, off to town for a meeting of the IT Council Open Source Committee. While getting into the car, found something lying between it and the air conditioner outside unit, where I had been working yesterday: it was my phone.

At the meeting, we came to the conclusion that the best thing we could do would be to recommend that people consider Open Source when shopping for software, but that our real issue should be Open Standards. With any luck we'll get a new policy on Open Standards in the not too distant future.

Do you have a comment about something I have written? This is a diary, not a “blog”, and there is deliberately no provision for directly adding comments. It's also not a vehicle for third-party content. But I welcome feedback and try to reply to all messages I receive. See the diary overview for more details. If you do send me a message relating to something I have written, please indicate whether you'd prefer me not to mention your name. Otherwise I'll assume that it's OK to do so.