Greg's diary--May 2002

Wednesday, 1 May 2002	Echunga
		Top of page
		next day
		last day

It's strange that a 40 minute teleconference in the morning should completely confuse the day, but it really seems to do so. I'm gradually getting into a routine which enables me to tend to my mail and other routine things in the morning and do some Real Work in the afternoon.

This afternoon finally got the lookup code for JFS 1 written. It's only about 15% of the length of the corresponding ext2 code, and I'm still left wondering whether I've missed something.

Thursday, 2 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

Into town this afternoon to visit the IST people at Greenhill Road. They had sphinx running, but they didn't know it because it didn't produce any login prompt on the serial console, and they didn't have a monitor which would display the frequencies involved, though the G50 they brought out tried for a while.

Had a bit of a chat with the people. They have a great need of understanding Linux, and we discussed what I could do in the way of a presentation for them. On reflection I think the best I could do would be a “what's the difference” presentation for people used to Microsoft who want to use a real operating system, which could just as well be FreeBSD as Linux. There's a lot of potential (and work) there.

Friday, 3 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

Spent just about all day working on JFS1, a lot of it trying to understand the Linux lookup code. There are two basically very similar directory search routines: lookup, which looks for a file name in a directory (for example, used by open(2)), and readdir, which returns the “next” entry in a directory to the (userland) caller. These two functions both need to read the directory and find an entry: it should almost be possible to write them as two calls to a single common function. Instead, they're written in very different ways, and I spent most of the day trying to find out why. By the evening had my version of readdir written, and almost, but not quite, got a kernel built. Ah well, not long to go now. I've certainly learnt a lot.

Saturday, 4 May 2002	Echunga	Images for 4 May 2002
		Top of page
		previous day
		next day
		last day

I had meant to get back to work on the book today, but first I had to get the AUUG Election paperwork sorted out, and that ended up taking most of my energy.

Finished by 4 pm, but then Yana needed help with cooking the dinner. We've been doing a bit of experimentation while Yvonne's away, and today included German potato dumplings (Knödel), made from a prepared mixture from a German company trading as “Panni”. The instructions required the content of the packet to be mixed with two cups of cold water. How I hate this stupid use of “cups”. I've established that a cup can have at least one of the following volumes:

70 ml (an Italian Espresso cup).
100 ml (in the instructions of the most recent coffee machine I bought, “normal cups”). This seems to be standard misconception amongst coffee machine makers. I've never seen a cup to match, and I suspect it's so that they can sell a 6 cup coffee maker as a 12 cup device.
125 ml (in the same coffee machine instructions, “European cups”). I've never seen a cup of this size, either.
200 ml (a real tea or coffee cup, made in “Europe”). This bears a good relationship to the smaller coffee machine cups: for every cup you want to drink, you select two in the coffee machine.
225 ml. I forget where this is, but I found it in one cookbook.
238 ml, an “American Cup” (half a pint). Noted in multiple cookbooks.
250 ml, an Australian “metric cup”.
284 ml, a “British Cup” (half a pint). Noted in multiple cookbooks. This is a little more than the volume of a coffee mug, which I consider to be considerably larger than a “cup”.

This is a uniquely English thing; none of my cookbooks in other languages use cups, and why should they? Given the potential inaccuracies, it's just plain stupid.

It would have been tempting to go for the Australian cup, since we're in Australia, but the package showed the weight in ounces, something that wouldn't happen here. In the end we made a toss-up between US and British, and went with 500 ml. That proved to be wrong: we had to add more water. I suppose we should have taken the British cups. But why does “Panni” do something that stupid? In the German version I'm sure they have the correct volumes.

Aaaaah, that feels better.

Sunday, 5 May 2002	Echunga	Images for 5 May 2002
		Top of page
		previous day
		next day
		last day

More activity on the mailing lists today, including stuff I can't divulge yet. Watch this space.

Finally got some more work done on the book, though not as much as I had intended. It's about time I took a bit of a rest, rather than working through all day, every day. Did a bit of reading, lazing around and music. I should do so more often.

Monday, 6 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

Having finally got a kernel to boot, I was planning to finally test the JFS1 code I wrote. Frustratingly, I ran into multiple hardware problems and ended up installing yet another Linux system. The last one was called pizza, and I still have hopes of resurrecting it, so I stayed with bread and called this one pita. At any rate, by the end of the day I had a machine up and running, this time the Mandrake 8.1 that refused to install last month. These Linux installations are surprisingly fragile.

Tuesday, 7 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

Carried on today playing around with Linux installations. Say what you will, this Mandrake stuff is buggy. Ended up installing four times, with the installer going crazy in different ways on each occasion. It seems that this machine is minimal because it only has 32 MB memory, but even if that were valid, it should be no excuse.

Finally, by the evening, had the system more or less in the condition I wanted in, so installed my new kernel. Not only would it not boot, it stopped anything else from booting. This is really puzzling, not to mention frustrating, especially as one of the bugs of the Mandrake install was to not create a rescue disk.

Wednesday, 8 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

Up early this morning for a teleconference with Austin, then later another with Canberra, where we discussed my failing kernel boots. I don't know whether I should be happy or not that they haven't come up with much of an explanation either.

Spent the rest of the day reinstalling Linux. By the evening I had Yet Another Installation done, and installed my new kernel ...with the same results. I'm completely baffled.

In the evening with Yana to the AUUG meeting, where the attendance was disappointing, probably because of the way in which it had been announced. Gave my “Why BSD is better than Linux” talk, unfortunately mainly to the converted, then off yet again to the Mongolian Barbecue, where I ate far too much. Late to bed.

Thursday, 9 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

Off to the airport early this morning to pick up Yvonne. Qantas in Melbourne had managed to make her life a misery; after over 24 hours flight they refused to give her a business class seat on to Adelaide, and even tore off her priority stickers, though she had a first-class ticket and seats were available. I'd complain again, but it's pretty clear that Qantas don't care. I think I'll forego frequent flyer points and fly with the competition next time I fly to Melbourne.

Apart from that, she seemed in good enough spirits, but with this boot problem still unsolved, I didn't have much time to talk to her. In addition, Mike Smith announced his retirement from the FreeBSD core team, so we had more than the usual flood of mail.

Another day dedicated to finding out why I can't boot Linux. No solution. In deference to a very tired Yvonne, early to bed.

Friday, 10 May 2002	Echunga –> Sydney
		Top of page
		previous day
		next day
		last day

Up early this morning without really having wanted to, worrying about this damned Linux boot issue. Took the last disk I had had, which had not run either in the old sat-gw nor in the old kimchi, and put it into zaphod. It booted straight off, as did all the other disks I installed during the week. Obviously there was something in two different machines, probably a BIOS issue, which stopped Linux from booting but made no problems for FreeBSD. It's not an issue with the drive geometry, so I really can't make out what it could be. In any case, it's clear that I need newer hardware. The time I spent working on this problem this week cost more than a mid-size PC.

The rest of the day wasn't too good either. Had a number of other problems, including wantadilla getting another disk hang. If FreeBSD-CURRENT weren't in such a volatile state at the moment, I'd have upgraded, but as it is I just rebooted. Grrr.

In the afternoon, finally did some JFS 1 testing. It didn't work, of course, but if that's the only problem, we'll soon have that fixed.

In the evening to Sydney for the AUUG board meeting, where I met up with Luigi Cantoni and Michael Paddon. Stayed up late and ended up with a respectable bar bill of $108, which fortunately Michael paid.

Saturday, 11 May 2002	Sydney
		Top of page
		previous day
		next day
		last day

After last night's late night up far too early this morning. Luigi had already been out to the airport to pick up Malcolm Caldwell. Had breakfast together and arrived at Luigi's office, where we had the meeting, nearly an hour early. Got off to a good start, and despite the plethora of agenda items—or maybe because of it—were finished on time. Amongst other things, we prepared for the AUUG Elections, the first in five years. Liz had never been involved in one before.

After that, into town with Conrad Parker to pick up Sue Blake and Patryk Zadarnowski, and off to the same place in Coogee that we went to on 9 March, where again we met up with horms and all of the outgoing AUUG board officers. I had a laptop for Sue from Jerry Dunham, which Yvonne had brought back with her. Quite a nice machine, which Jerry had given her particularly cheaply, and Patryk was quite envious.

Sunday, 12 May 2002	Sydney –> Echunga
		Top of page
		previous day
		next day
		last day

Up not too early this morning, despite the attempts of the concierge. On paying the bill, discovered that I got a 15% discount on the bar drinks. That'll make Michael Paddon happy that he paid for the drinks on Friday.

Monday, 13 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

Geoff Smith in today to finally grade the driveway and do some drainage work in the paddocks. We've been talking about this for over a year, and finally we were ready when the gate went in last month, but Geoff didn't have time. To his credit, he came by himself this time after noting that the autumn rains were apparently due on Thursday.

More panics for some reason today. Had another incidence of the disk hang problem that I had tried to work around on 7 April, and spent some time investigating it. It turns out not to be a hang at all, but a loop. It only ever occurred running Emacs, and it seems to be transferring the same data over and over again; at any rate, the disk transfer rate was a continuous 45 MB/s, which couldn't happen any other way. A ktrace showed that the process wasn't leaving the kernel, so it was probably tickling some driver bug. Took a dump and installed a more recent version of Emacs, but the latter is unlikely to help, and I didn't have time for the former.

More work on JFS1 today. I'm tantalizingly close: I can almost mount a file system, but not quite. Looks like a weird endian problem reading the inodes.

Tuesday, 14 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

More work on JFS1. It seems that the problem with the inodes was that both Linux and AIX use the type mode_t to refer to file modes, but in Linux it's the traditional UNIX 16 bit value, and in AIX it's 32 bits. Fortunately, the meaning of the lower half of the value in AIX (permissions and file type) is apparently the same, so I was able to handle that relatively easy by redefining the field as two 16-bit fields. After that, I was able to mount the file system, but any attempt to read a directory caused a loop in the kernel. It seems that data isn't getting read. Ah well, it's progress, anyway.

Geoff Smith finished his earthworks round lunch, and it started to rain in the evening. Not a minute too soon.

Wednesday, 15 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

More rain today, and had ample chances to check how the drainage is working. We still need some fine tuning.

Spent most of the day on JFS 1 again, but didn't get much further. There's still a part of the Linux VFS that I don't understand.

As the result of Mike Smith leaving the FreeBSD core team, we're now in an election campaign again. Everything seems to be happening at once.

Thursday, 16 May 2002	Echunga	Images for 16 May 2002
		Top of page
		previous day
		next day
		last day

So near and yet so far! A day or two ago I thought I had broken the back of the JFS1 file system, but today I discovered all sorts of stuff in the virtual memory interface that (a) I don't understand (b) nobody I know understands (c) I can't make head or tail of. I'm torn between thinking the corresponding ext2 code is very stupid and thinking that maybe I'm very stupid, and the code is very clever. sigh

Friday, 17 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

26 years ago today I started working for Karstadt, and coincidentally 20 years ago today I started working for Tandem. How the world has changed in that time.

More election-related activities today. For some reason people are getting unusually vehement about the whole issue. Despite all the analyses I've been making of the project dynamics over the last few years, this surprises me.

JFS1 stuff took a while to handle, but gradually I'm making sense of it. Well, at least I understand what it's doing, and some of the potential race conditions which it's avoiding, but the way it's done still puzzles me. I thought that gotos went out of fashion before Linux was started.

Saturday, 18 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

More rain today, and took things fairly easy. I really need to get a bit of time off work for a while.

They're translating “The Complete FreeBSD” into Chinese. Amusingly, the translator is in the same building at Tsinghua University where I held some Tandem courses in 1997.

Also spent some time reading about DSLD (Degenerative Suspensory Ligament Desmitis), from which at least one of our horses suffer. It's amazing how many different views exist, and it's obvious that some people are in denial mode, so for example Kathy Rodriguez, the breeder of ERB Magia Negra whom I visited near Sacramento a couple of years ago. Magia has been diagnosed with DSLD; you'd think the breeders would be interested in understanding the issues, rather than making unsubstantiated statements about the nature of the affliction.

Sunday, 19 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

Still no motivation to do real work, so spent today adding old UNIX documentation to the FreeBSD source tree. As far as I know this is the first use of old UNIX code since Caldera, the new owners of the UNIX source code, released Research UNIX under a Berkeley license. It's amazing how much cruft it contains, though. O'Reilly published the information as part of their 4.4BSD manual page set about 8 years ago, but it wasn't until today that I discovered that the documents have formatting errors, the same ones that I fixed today.

Monday, 20 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

More work on JFS1 today, though I was significantly distracted by the elections going on both in the FreeBSD and AUUG camps. I'll be glad when it's all over.

JFS1 continues to be frustrating. I have now deciphered the meaning of the get_block paradigm, but in the middle of it all my machine died on me. I'm coming to the conclusion that it has hardware problems. As a result, possibly because it died while running lilo, I wasn't able to boot: I ended up with the same unbootable syndrome I had a couple of weeks ago. The Linuxcare Bootable Business Card, which is intended for such purposes, didn't help: the version of lilo on it is obviously too old, and the lilo on disk is dynamically linked and didn't like the glibc on the BBC. In the end grabbed another of the disks I made a couple of weeks ago, booted from that, and fixed things. What frustration.

Tuesday, 21 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

More work on the JFS1 stuff today. It read in the directory, but then looped in the kernel. More head-scratching, after which I was finally Enlightened—I think. What I've discovered:

What looks like an error is not always an error. sys_getdents64(unsigned) finishes with:
```
        out:
        return error;
```
(don't you love these gotos?). You need to look up a few lines to find out what it is, though:
```
            copy_to_user(&lastdirent->d_off, &d.d_off, sizeof(d.d_off));
            error = count - buf.count;
```
In other words, it's a residual count.
In another example, ext2_readdir has:
```
                over = filldir(dirent, de->name, de->name_len,
                        (n<<PAGE_CACHE_SHIFT) | offset,
                        le32_to_cpu(de->inode), d_type);
                if (over) {
                    ext2_put_page(page);
                    goto done;
                }
```
This is in the body of a loop, which is exited by non-standard means. I read this to mean that over states when the transfer is complete. Looking at the filldir function (which is really filldir64), I find:
```
    if (reclen > buf->count)
        return -EINVAL;
...
    return 0;
}
```
These are the only returns in the function. It took a long time to realise that this was really saying “when the buffer is full, we're done”.
Finally, the description of the functions is inadequate to the point of being misleading. According to Documentation/filesystems/vfs.txt, the readdir entry point is
```
  readdir: called when the VFS needs to read the directory contents
```
Presumably out of this statement, books like O'Reilly's otherwise excellent Understanding the Linux Kernel state:

readdir(dir, dirent, filldir)

Returns the next directory entry of a directory in dirent; the filldir parameter contains the address of an auxiliary function that extracts the fields in a directory entry.

In fact, the function returns a whole page full of directory entries.

With that knowledge, set to rewrite jfs1_readdir, trying not to be as ugly as ext2_readdir, but didn't finish.

Wednesday, 22 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

Two teleconferences today, interspersed with more work on JFS1. I can now read the top level directory, but I can't stat any files, so there's still a fair amount to do. Still, it's encouraging to get some sense out of ls(1).

Thursday, 23 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

Into town to the IBM office today for a meeting with Hugh, which knocked the entire morning out of whack. Sadly, IBM is scaling back, and we have to reduce personnel in Ozlabs as well. I'm the lucky one. I get at least two months to find another job within IBM, but I can't see much likelihood of getting a job here in Adelaide. If I have to make the choice, I'll choose Adelaide, not IBM.

Didn't get through my mail until mid-afternoon, by which time a nagging tooth was turning into real pain, so off to the dentist, who decided he needed to do some root canal work. That should be fun.

Friday, 24 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

Oh, the pain of it! This tooth or jaw infection has really come out in force, and though I started on antibiotics yesterday evening, it got worse during the day, as the dentist promised. I wish he had diagnosed it correctly last November.

Nonetheless, finally made some progress with the JFS1 stuff, and I can now read the top level directory and cd into it:

=== root@pita (/dev/pts/0) / 2 -> ls -l /jfs1
total 21
lrwxrwxrwx    1 root     root            8 Jun 17  2001 adm -> /var/adm
drwxr-xr-x    3 root     root          512 Jun 17  2001 aix
drwxr-xr-x    4 root     root        14848 Jul 10  2001 bin
drwxr-xr-x    4 root     root          512 Jun 17  2001 ccs
(etc)

There's still something funny, though: there are some strange delays, and any attempt to access a lower-level directory locks up the machine. Ah well, it's progress, anyway.

Saturday, 25 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

Thank God the pain is going! Spent most of the day today working on articles for next month's Daemon News. These audio programs are a can of worms.

Sunday, 26 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

Somehow managed to spend all day today playing around with audio ports for my article in Daemon News. One thing that's really obvious is that there's no adequate overview of the ports, and I had a lot of trouble finding things which worked well together, and which installed without problems. Why is it that so many people consider documentation is superfluous? audio/linux-realplayer even manages to install the help files in the wrong place.

Finding streaming audio sources was just as much trouble. What I did find gave me the distinct impression that it's not worth it. It'll be a while before MP3 replaces CDs in my house, anyway.

Monday, 27 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

More work on JFS1 today, slow but not completely impossible. For some reason it takes 20 seconds or so to read in each directory block. It's true that I wasn't aiming for performance in this file system, but this is probably overstating the case. Spent all day without finding out why the process isn't being woken up when the I/O completes.

Tuesday, 28 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

Spent more time today looking at the I/O completion issue. The real thing I don't know now is how best to approach the problem. Decided to think about it and install the latest version of Linux 2.4 in the meantime. That meant patching the 2.4.18 kernel to bring it up to 2.4.19-pre8. For reasons I don't understand, the patch didn't apply cleanly:

$ grep failed patch.log | wc -l
     309

All of the failures were in non-Intel architectural files, but I still don't understand. That shouldn't happen.

To the dentist's this afternoon to have my root canals drilled out. Not nearly as painful as I feared; in fact, most of the pain was from keeping my mouth open for such a long time. Looks like it's going to be relatively straightforward.

Wednesday, 29 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

Why do conference calls always take so much out of the day? Admittedly, at 82 minutes today's call was longer than normal, but I find that on Wednesdays I hardly got anything done.

Still working on source trees today. I really find it annoying that I'm getting so many patch errors. In addition, found a bug in mklinks, my tree cloning program, and spent some time fixing that. I really need to do some more work on it.

Thursday, 30 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

Spent most of the day trying to build a kernel. This is ridiculous. Sure, I could have done it faster, but I want something which is repeatable. Dropped back to the 2.4.18 tree, and found that if I built a kernel on sat-gw and tried to install it on pita, I'd run into build problems. I also find it astounding that the default configuration file includes almost no (or possibly absolutely no) Ethernet card support.

Spent some time working on articles. I wish I understood what's going on in the FreeBSD project at the moment.

Another tape drive bites the dust. These DDS drives are really unreliable.

Friday, 31 May 2002	Echunga
		Top of page
		previous day
		next day
		last day

More work on tidy Linux installs. It's amazing how difficult it is to get things repeatable. I wonder if the average Linux user just accepts the “some assembly required” approach.

Finally got my articles finished, in the nick of time considering that we'll be away all weekend.

Yana's birthday today, 17 years old already. Where has the time gone? In honour of her birthday went to a Japanese restaurant, the Matsuri in Gouger St. At any rate, we didn't leave hungry; in fact, none of us were able to finish our food, and we took a lot back home with us. Quite authentic food and surroundings, but just a little run-down.