|
| Greg's diary |
| June 2006 |
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Thursday, 1 June 2006 | Echunga | Images for 1 June 2006 |
| Top of page | ||
| next day | ||
| last day |
Yes another day where I can't see where the time went. I'm suspecting mail messages more than before. Sure, writing a serious technical mail message can take a while, but the research that goes into the answer takes even longer. Finally got round to answering the review messages I have had, but it took most of the day. I've decided to estimate that an in-depth mail message or reply takes an hour. That's much too high for my normal load, of course:
Most of those outgoing messages are either trivial or automated, though; I only sent three or four that took more than a few seconds to write.
The day started early enough with the arrival of Shane Adcock, the electrician, to change the switch that had blown up in January. This time he didn't manage to change the wiring to a single switch before the UPS for teevee, my current video machine, died. It had been up for six months. sigh.
Also did a first cut at a macro set for using nroff to produce output text for MediaWiki. As expected, it's not much work, and I had something that looks almost correct in an hour or so. Made one discovery in the process: I had a macro P (paragraph), which in normal nroff simply inserts vertical space. In MediaWiki you don't need vertical space, but if you're in a list environment, you need to prepend the line with :, one per level of list. So the macro looks like this:
The first time round, I had a typo: the third-to-last line read:
(in other words, a space after the variable name). This apparently causes groff to treat the name as containing a space, i.e. a different identifier. This, of course, meant that the macro looped, and it took a while to understand where it was dying.
In the evening, into town to the Adelaide MySQL Meetup Group meeting. On the way picked up the camera from Yana, who arrived 40 minutes late, making me late from then on; I also picked up a DVB-T tuner card from Ben Kramer, in a tearing hurry, and then back into town with far too little time. On the way, found Unley Road blocked (I later heard that there had been a fatal car accident there 45 minutes before), but somehow managed to make it on time anyway.
Presented the current state of the API document. Going through it again made me justify some of the decisions I have made, and I left with two pages of handwritten notes. There's still a lot of writing to do.
| Friday, 2 June 2006 | Echunga | Images for 2 June 2006 |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Lars still doesn't have VoIP working properly, so he's asked me to install Skype. I have certain reservations about using a proprietary, non-standard solution when there are perfectly good free, standards-based solutions available, but on the other hand, we need to communicate. Didn't even try to install on FreeBSD; for one thing, I don't have a machine with a sound card in easy reach, and I have had enough issues with multimedia under FreeBSD already. Installing from the web onto eucla (in its Fedora Core 5 incarnation) was an interesting experience. Just clicking almost worked:
|
|
||||||||
Unfortunately, to do it right you need to download the signature as well; otherwise you get this message:
|
|
||||||||
The web form allows you to download the signature, but gives absolutely no help installing it. I don't even know where to put it. That could definitely be improved on.
After installation, you're on your own. The web site doesn't even tell you what to do next. After some searching, discovered that it had installed it in the GNOME “Desktop”, where it finally brought up a far-too-small window (eucla has a 1920x1200 display). And it's not where I need it. Checking with ps confirmed that it's started simply as skype, so tried that. The result, on my lowest-resolution monitor, was a barely legible window with characters slightly less than 1mm wide (about 3 pt), and with input fields black on dark grey:
|
|
||||||||
For some reason it wouldn't let me enter more than 4 letters of the password I had chosen. It seems to be related to the fact that I entered a space earlier, but that doesn't make any sense.
This is Yet Another application which claims that all names I choose are taken. I can't really believe this. I tried groggy, groogle, gurgle, Ihateskype before finally being given givemevoip. Then I found that I have done this before with the name groggled, so used that instead. Finally logged in and got a screen with more tiny characters:
|
|
||||||||
This is the sort of window that leaves me feeling probably very much the way the “modern” GUI user feels when he sees:
It's really not “intuitive”, and the strange icons say nothing to me. About the only thing that is better than the login prompt is the offer of help. Selecting Help did help; it connects me to the web site that I should have been pointed to in the first place. For some strange reason it started a Konqueror, though I normally use firefox, and one instance was already running. I was also able to select sizes from the Tools bar. After changing from small to large, the text size changed from flyspeck to small (characters now 1.35 mm wide, nearly 4 pt). Why do so many programs have such enormous difficulties with text size?
Tried to set preferences, and once again was given a Konqueror. Stopped that and started a firefox. This time Skype made up for the tiny windows it normally presents and gave me a screen-filling, overlapping page with enormous texts which I couldn't shrink. After selecting “Sign in” it looked like this:
Note the overlapping text and truncated texts. Also, why does Skype set the title of all windows to “Skype?”?
As if that wasn't enough, this page decided that my password was no longer valid—though I had just logged on with it! Went through the “recover password” process yet again.
I finally got things working—almost. I called up somebody I know and got the dreaded one-way audio. I hadn't used this headset on this machine for a while, so there was a possibility that there was something wrong with it. Rebooted into FreeBSD and tried linphonec, which I had successfully used in the past. In the meantime, though I had “upgraded” the FreeBSD installation—from 7-CURRENT to 6.1-RELEASE—and I didn't have some of the libraries I needed. As so often, installing from the Ports Collection failed for reasons I didn't want to investigate, so spent some time setting up the library paths for ldconfig to include the libraries for 7-CURRENT. After that, and after setting the mixer controls, linphonec worked as well as it ever did. So it wasn't a problem with the headset. Probably it was something to do with the mixer. I know how to do that in FreeBSD; like most other Linux users, I don't know how to do it in Linux, and the instructions don't just not tell you, they don't even mention the issue.
So looked at Skype on FreeBSD. It installed out of the box, and worked. Even the tiny characters are gone, so maybe this is a GNOME issue and not a Skype issue. The only problem is that this is my Linux box, and I have to reboot it into FreeBSD to run Skype. That will need to be improved on.
Long (Skype) phone call with Lars in the evening. It looks as if we might be very close to creating code.
| Saturday, 3 June 2006 | Echunga | Images for 3 June 2006 |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Woke up in the night with a flash of inspiration and the complete detailed design of the online backup API in my head; got some of it down in writing later in the day, but was overtaken by other events. At least I have the warm, fuzzy feeling that I'm making headway.
Other events included installing the DVB-T tuner that I borrowed from Ben Kramer on Thursday, along with c't magazine's Debian “VDR” software. I think this stands for “Video Disk Recorder”, a term which doesn't make much sense to me: where's the video disk? I prefer to use the more descriptive term “CVR” (“Computer Video Recorder”). That hung up a surprising number of times; I wonder if the motherboard is getting even flakier. And why does Debian load the floppy driver three times in the course of installing on a system that doesn't have a floppy drive?
That, too, was interrupted by the arrival of Yana to pick up her new camera.
This was a Canon EOS 20D, the first DSLR I had ever bought.
Spent quite a bit of time setting that up and taking a number of random photos. Also gave her tomato, my Apple G3 Powerbook, to run it with.
Installing software showed yet more nonsense in contemporary software: this really expensive camera comes with a USB cable, of course, but it doesn't offer a SCSI interface. Instead, you have to use the software they provide, and it's “user-friendly”: the files on the camera show up as a “folder” (i.e. directory) in a GUI-style lists with lots of triangles representing “open” or “closed” “folders”. Yana was trying to copy to the directory /Users/yana/Photos/20060603. To do that, you select the directory on the camera and drag it to the target directory. Of course, that means you need to “open” the /Users directory, the yana directory in that display, and finally the Photos directory in that to display the directory 20060603.
The problem is that this list is so long that it can't be displayed on the screen at the same time as the camera's directories. And since you're dragging, you can't influence the scroll bar. What a stupid method! To add insult to injury, it didn't install them in the directory /Users/yana/Photos/20060603; it created a new directory and installed them in /Users/yana/Photos/20060603/100CANON/ instead. That's completely broken.
Gave Yana a compact flash adaptor for the card, after which she can do (the easy way, from the command line):
=== yana@tomato (/dev/ttyp9) ~ 77 -> cd Photos/20060603
=== yana@tomato (/dev/ttyp9) ~/Photos/20060603 78 -> cp -p /Volumes/TOSH1G0/100CANON/* .
I never cease to be amazed how complicated “user-friendly” software is.
This appears to have been a particular problem with Canon. Other makes provide a file system interface on request.
| Sunday, 4 June 2006 | Echunga | |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Spent the morning writing up some recipes, then continued trying to install c't magazine's “VDR” software. I think there must be a conspiracy amongst authors of multimedia software to avoid documentation; once again, and unexpectedly for c't, the documentation was almost completely missing. During the installation I had to choose a transmitter profile, and of course they were only available for Germany (I didn't even see Switzerland or Austria). The on-screen instructions (all that I ever found) said that they could be changed afterwards, so chose one at random. At the end of the installation, the terminal under Alt-F8 was supposed to have the application; but it just produced a text-mode message saying (roughly) “this screen reserved for DVR”.
Spent a lot of time investigating and discovered:
I found no, really no documentation whatsoever. There's stuff on the web at http://www.vdr-wiki.de/, but it's not related to the c't project, and even there it's a real pain to piece together. To quote (and translate) the configuration page:
Configuration information is spread widely throughout the wiki, so this page is just first aid for finding where to start searching.
It then points to the user handbook, which starts by describing how to use the software in everyday situations. I still haven't found anything that describes initial setup.
It installs a German keyboard. That may come as no surprise, but a large number of people in Germany use the US layout, since the German layout is really bad for programming. In particular, the | symbol is on a key which is physically not present on US-style keyboards. And of course there was no information on how to deactivate it.
The installation was very bare-bones: almost none of the software I wanted was there. The /etc/apt/sources.list file didn't even contain any useful entries; I had to copy them from another system before I could even install Emacs.
Installing X failed, possibly because of things I had done to recover from other deficiencies.
The strange behaviour of the Alt-F8 combination appeared to be because the application was having trouble initializing, to interpret the messages appearing in/var/log/messages. Possibly these were due to an incorrect transmitter profile; but nothing I found explained how to set it up, or even the name of the file.
On the positive side, I did find out how to override DHCP and use manual setup when installing a Debian system: let it do its thing, then press the back button.
Also tried KnoppMyth again. This time the installation failed consistently with an error in an installation script. I'm not sure what I did differently, but given my previous experiences, I couldn't be bothered trying more than 4 times. Gave up in disgust. This is all far more complicated than it needs to be.
| Monday, 5 June 2006 | Echunga | Images for 5 June 2006 |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Usual Monday routine in the morning, then more work on my backup design. Lars and others have complained that the “low-level design” section of the online backup API isn't detailed enough. Continued with the work I started on Saturday and discovered that I was making immense headway—but in writing the code, not the “low-level design”.
Is this the right way to go? All texts on programming state that you should design first, then code. I agree, but there's a point that many people miss: don't use the wrong tools. The purpose of a design is to understand what you're going to do. Then the code itself becomes trivial, as indeed it is proving to be. The issue is that a “low-level design” that looks more like pseudo-code is just that: pseudo. If you write a spec down in C, you can put it through a compiler and perform at least formal checks. With pseudo-code, you can't.
So I think that there's no particular point to write a low-level design as a coding specification; it's written in a badly defined language, and it's bound to have some issues that wouldn't have happened in the first place if it had been written in C (or some other real programming language). This doesn't mean that the design is superfluous: it can be a great form of documentation for the program. But it should be derived from the program, rather than the other way round. And with good comments in the source code (something that's far too seldom), the derivation should be simple.
Another phone call with Lars in the evening. He still doesn't have his VoIP working, and so I had to reboot eucla to FreeBSD to call him with Skype. Then I rebooted to Linux to send him the work I had done today and—it was gone! Investigation of the file system showed that the last three files updated in the directory were:
I saved backup.c at about 17:15; the previous timestamp was where I had finished working on Saturday. Spent a lot of time looking over all machines I have for the file, without success. Rebooted Linux in single user mode to see if there was something in the file system journal, but found nothing. Only much later did it occur to me what must have happened:
On Saturday, just before Yana arrived, I sent the current state of the file to Lars. That's the date in the listing above. To send it to him, I had to move it from eucla to wantadilla. Normally I'd do this with:
=== grog@eucla (/dev/pts/3) /home/MySQL/5.1-Online-Backup/sql 3 -> cp backup.c /wantadilla/var/tmp/backup.c
On this occasion, I must have used Emacs to save to the new location.
I then carried on editing /wantadilla/var/tmp/backup.c with the Emacs running on eucla.
After rebooting eucla after the phone call, I copied the file the way I normally do it, as above. Bang! Gone are all my changes.
What's the moral of the story? Rebooting is bad? Certainly. More backups? Not really. I back up my systems every evening at 21:00. All the changes were done today, so that wouldn't have helped. About the only consolation is that the changes are fresh in my mind, so it won't take too long to reproduce them.
Yvonne tried loading an impatient horse in the afternoon; not a good idea. It banged her head against the bar at the front of the float, giving a nice bulge over her left eye:
|
|||||||||||||||||||||||
Fortunately it wasn't too serious.
| Tuesday, 6 June 2006 | Echunga | |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Spent today catching up with the lost data from yesterday. Somehow it's difficult to maintain the same enthusiasm when you're repeating work you've already done, and at the end of the day I wasn't much further than yesterday. Well, not as much further as I had hoped.
I'm still considering how to make the “low-level design” useful. Another problem is that the real issues with most algorithms in complicated products like the MySQL server is not the algorithm itself, but how it interfaces with the rest of the system. Consider a low-level design snippet like this:
This nicely captures what I want to do; we select a database, stop DDL operations, write a header to the output stream, and spawn one worker thread for each storage engine.
This gives a completely misleading feeling that I've achieved something. What I have really done is expand the previous paragraph to several lines of useless pseudo-code. The real issues are:
How do I convert the SQL statement USE dname into calls inside the server?
Where are the definitions of stop_all_ddl () and write_backup_header ()? Do they really not take any parameters? Yes, this is borderline; they're also also coincidentally pretty much the closest thing to real C in the example. But stopping DDL requires interfacing with parts of the engine that I don't yet know. Writing the header requires going back to the client through the MySQL network protocol; that's where the real work lies, and it doesn't fit into this framework.
How do I iterate through the “atend” engines? Even in the example, should I not be more specific and write it like this?
But that's becoming more implementation specific, and that's not what's requested.
| Wednesday, 7 June 2006 | Echunga | Images for 7 June 2006 |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Winter is here! It doesn't happen often, but we had a (mild) frost in the night:
|
|||||||||||||||||||||||
Out to Grumpys in mid-morning to watch them prepare some of their “Grumpys One” wort concentrate. 250 kg of malt in 1000 litres of wort (OG about 1060). After all my step infusion and decoction mashes, their single infusion mash looks relatively primitive:
|
|||||||||||||||||||||||
Still, the resultant beer is good, and that's what counts. Spent quite some time talking technical details with Thomas, including heat considerations. Their HLT (Hot Liquor Tank) holds 1000 litres and has 30 kW in heating elements, and it takes hours for it to reach a boil. A HERMS (Heat-Exchanging Recirculating Mash System) isn't much use if it can't change temperature relatively quickly.
Continued work on the backup API. I'm pretty much done with the logic; now I need to look at interfacing, which could take longer. Set up a test server with debug tracing enabled to investigate what happens when the client issues specific commands:
Build a tree (in this case, the most recent 5.1).
=== grog@eucla (/dev/pts/1) /home/MySQL/5.1-Online-Backup 7 -> cd mysql-test/
=== grog@eucla (/dev/pts/1) /home/MySQL/5.1-Online-Backup/mysql-test 8 -> ./mysql-test-run \
--skip-ndb --start-and-exit --debug
=== grog@eucla (/dev/pts/1) /home/MySQL/5.1-Online-Backup/mysql-test 9 ->../client/mysql \
-Svar/tmp/master.sock -uroot
(etc)
From another terminal, save the debug log:
=== grog@eucla (/dev/pts/3) /home/MySQL/5.1-Online-Backup/mysql-test 22 -> cd var/log
=== grog@eucla (/dev/pts/3) /home/MySQL/5.1-Online-Backup/mysql-test/var/log 23 -> cp master.trace \
/home/MySQL/Backup/master.trace.0
In the first window, do some things with the client.
=== grog@eucla (/dev/pts/3) /home/MySQL/5.1-Online-Backup/mysql-test/var/log 24 -> cp master.trace \
/home/MySQL/Online-Backup/master.trace.1
In a third terminal, look at what it did.
=== grog@eucla (/dev/pts/4) /home/MySQL/Online-Backup 235 -> diff -wu master.trace.[01] | less
When I find something interesting in the trace, I can then put the server into the debugger and look more carefully. For example:
In the client:
In the corresponding trace, I find:
I can then start a gdb and attach it to the running server, then run the query again:
=== grog@eucla (/dev/pts/2) /home/MySQL/5.1-Online-Backup 6 -> ps aux | grep mysqld
=== grog@eucla (/dev/pts/2) /home/MySQL/5.1-Online-Backup 7 -> gdb sql/mysqld
| Thursday, 8 June 2006 | Echunga | |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
More work on the API code today; things are slowing down, as I expected: I'm now looking at how to interface to the storage engines. That means a lot of code reading, in the process adding a few comments to explain what is going on. That took up most of the day, but at the end of it I had a feeling that I understood it much better.
Another 2 hour call with Lars in the evening.
| Friday, 9 June 2006 | Echunga | Images for 9 June 2006 |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Long mail message from Lars in the morning, with a long list of things that needed to be fixed in the Online Backup specification. I've been pushing changes ahead of me for quite a while, and today I found one good reason why: it took me all day to review about a third of the document. Part of that is, of course, that it's so much in need of review, but it's still a pain.
Another long phone call with Lars in the evening. Looks like this will be a daily thing for the next couple of weeks at least.
While working on the design, was interrupted by some loud laughing from a pair of kookaburras in the gum tree just outside the house:
|
|||||||||||||||||||||||
The last time this happened to me was when I was designing the RAID-5 code for Vinum, and at the time I wondered if it was an omen. In any case, though the RAID-5 code was some of the most complicated code I have ever written, it has also been some of the most reliable; I've never heard of any error there. If it is an omen, it might be positive. On the other hand, I'm wondering whether they're planning to nest in the tree, and if so, whether I like the idea or not. They're noisy animals.
| Saturday, 10 June 2006 | Echunga | |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Why am I so tired? Looking back at the last few weeks, I suppose that's obvious. Finally I seem to have had enough time to catch up on my sleep, so today got up late and went to bed early.
In between that, bottled some beer and tried to install Ubuntu “Dapper Drake” on ceeveear. It hung during install—not the first time that this has happened. I'm coming more and more to the conclusion that many of these problems are with the motherboard. Or is it the processor? Or memory?
Looking at the history of this motherboard, I note:
I had lots of trouble with this board when I got it.
The sound device has never worked.
From time to time, my overnight backup job died with a SIGSEGV, which conventional wisdom blames on bad memory.
Things got worse a few months ago, so I swapped the motherboards of wantadilla and ceeveear. They still didn't seem bad enough to throw anything away.
In the last few weeks, a lot of my work on ceeveear has been thwarted by random hangs, including during installation.
For some reason, the disk controller has great difficulty with disks jumpered to Master or Slave; it's only really happy with “Cable Select”. I don't know if this is a change from previous behaviour.
So what's defective? Motherboard? Processor? Memory? I've had all three in the past. The easy one to test is memory. Downloaded Memtest86+, burnt an ISO and let it run. By the time I went to bed, some hours later, it had not found any errors.
| Sunday, 11 June 2006 | Echunga | |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Three Yemeni hostages at the American Concentration Camp in Cuba have committed suicide. Given the inhumane conditions, it's not surprising. What really galled me, though, was a statement by Rear Admiral Harry Harris, the commander of the camp, who is reported to have said: "I believe this was not an act of desperation, but an act of asymmetrical warfare waged against us".
On a superficial level, this is silly, of course. The whole justification that the USA use for the camp is that these are terrorists, not soldiers, so they claim that the Geneva Convention does not apply. In that case, as Yvonne observed, it would have had to have been an act of terrorism.
On a more serious note: I don't often comment on politics, but this really turns my stomach. 70 years ago the Germans ran concentration camps, and after their defeat, who hunted down and prosecuted everybody involved? First and foremost the Americans. Where is their concern now, when they are doing the same things themselves? Where are the ideals which they claim to hold so dear? And where is the plain common sense that should tell the US military that persecuting people isn't a way to end terrorism?
I had intended to brew today, but I'm still so tired, so decided to postpone it. It's not as if I didn't do anything; I still had some kimchi to make, and in the process discovered lots of duplicate packages of herbs and spices. Tidied that up; some of this stuff is really ancient. Found some dried sage with a “best by” date of December 1993, and some mustard seeds with a label written by my first wife, Doris, with whom I split up 24 years ago. The seeds must have been quite a bit older than that.
After that, instead of watching TV, I somehow ended up writing some significant new stuff for Wikipedia about the Citroën DS 19. Why can't I just relax for once?
| Monday, 12 June 2006 | Echunga | |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Queen's Birthday holiday today, spent mainly brewing beer. We had intended to go riding, but it was too cold.
| Tuesday, 13 June 2006 | Echunga | Images for 13 June 2006 |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Back to work after a long weekend is even more of a drag than most Mondays. In addition, I seem to be getting nowhere with my review of the Online Backup specification. Somehow I'm reminded of what I know of Brooks' “The mythical man-month”. I suppose it's about time I read the book.
| Wednesday, 14 June 2006 | Echunga | |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Up early this morning to complete my worklog stuff ahead of a meeting of the ICT Council for South Australia. Yesterday Microbits, a local systems integrator, closed down, and we had the ABC news team in to take some video and interview David Raffen; it appeared on the evening news.
Back home and caught up on my review mail. Things seem to be calming down again.
| Thursday, 15 June 2006 | Echunga | Images for 15 June 2006 |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
How I hate broken web sites! But I seem to be one of the few. Once again had to access the FreeBSD web site, which was “modernized” about 8 months ago (i.e. layout made to overlap, ridiculous font sizes, text lines a multiple of the screen width). Mentioned this on a BSD IRC channel and found little agreement. It seems that people have given up on traditional values and find it good to have to mess around to read things. Here some of my objections and the responses I received:
Groggy: The text is too big.
Answer: Change your browser settings.
Groggy: I've changed my browser settings. Now I have flyspeck:
|
||||||||||||||||||||||
Answer: Upgrade your version of Opera.
Groggy: The text is too small.
Answer: Your eyes are fading.
Answer: Your monitor has too high resolution.
Groggy: The text is too wide.
Answer: You don't want to wrap computer output, do you?
This one is almost OK. When sending mail, I recommend not to wrap text. The problem here is really that browsers won't wrap properly.
Groggy: The left margin is too wide.
Answer: I guess he wants dynamic margins.
Sigh We're such a long way from having good presentation on the web that it's not funny. As long as people don't understand that it's a problem, it won't be fixed. And until it is, we're not going to get monitors that can display digital camera images in native resolution (currently from 6 MP to 11 MP).
Finally gave up on my memory test on ceeveear and sent Yvonne into town for a new motherboard. The memory test had been running for 115 hours, done 183 passes and found no errors; a far cry from the results I got on Opterons last year.
In fact, a “new” motherboard for a 32 bit machine (in this case, Athlon XP) is almost impossible to find. Almost overnight everything has become 64 bit. That's a serious issue if you have a 32 bit processor, of course; and the sorry state of multimedia seems to be even worse on 64 bit processors.
Apart from the motherboard, Yvonne brought back a couple of wireless mice, USB devices with very small dimensions—and despite promises, no middle button. Well, you can press on the scroll wheel, but for that you first need your middle finger cut off at the second joint. When will mouse manufacturers adapt to human hands?
| Friday, 16 June 2006 | Echunga | Images for 16 June 2006 |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Things are gradually quietening down; as usual in such times, spent much of the day catching up on other issues. Also spent far too much time trying to code in pic a diagram of the control flow between the threads of the backup server. pic reminds me of Nehru's comment about democracy:
Democracy is good. I say this because other systems are worse.
Still, within limits it does allow me to do both a PostScript version and a plain text version:
|
|
||||||||
I still need to work out why the plain text version looks so bad. In particular, the arrowheads are missing. That shouldn't be the case with this input:
| Saturday, 17 June 2006 | Echunga | Images for 17 June 2006 |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Into the office this morning to find that I had 33 MB of mail:
|
||||||||||||||||||||||
The daily output from echunga was 30 MB in size, mainly a printout of the mail queue. After stopping the MTA, ran mailq, which ran for 22 minutes before printing at the end:
The entries in /var/log/maillog showed that they had mainly come from addresses that don't resolve, or that resolve to ADSL lines. Looks like a DDoS attack; I wonder how hotmail is handling this: they'll be getting all the bounces. After some consideration about how to get rid of the bounce messages, decided to let them take their course.
Many of these messages were from systems with no reverse DNS, so I'm now once more rejecting host names which don't map. I suppose a few “innocent” people with misconfigured mail systems will be hit too, but this is ridiculous. It's only a partial solution, too. One of the worst offenders does have reverse lookup:
Wouldn't it be nice to have a way to recognize ADSL lines and reject mail from them?
Put the new motherboard in ceeveear. It's not at all new; in fact, it's so old that it can't correctly identify the Athlon XP+ 2500 processor that I'm using, and it runs at half speed. I've also had to scrape up some SDRAM because it doesn't take DDR. But it runs, and the hangs I had before no longer occur. I still couldn't get the “VDR” software to do anything useful, though, and this brain-dead setup is such a pain that I gave up after installing X and discovering that the configuration routines couldn't recognize my nVidia 6200-based board and wanted all the horrible configuration rigmarole that you used to have to supply years ago.
Tried KnoppMyth, which installed this time (another difference from two weeks ago), but it forgot to install the boot loader. What a pain this stuff is! Installed FreeBSD 6.1-PRERELEASE and started building -STABLE: at the speed this processor is running, it took the rest of the day before hanging on NFS access to echunga. That never used to happen.
In the evening, did some experimental cookery, pancakes with meat filling. Not uninteresting, but I'm not done with experimentation yet.
| Sunday, 18 June 2006 | Echunga | Images for 18 June 2006 |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Finally I've gone riding again! I suppose it's an indication of how busy I've been that I haven't been on a horse since 2 January, and even then only briefly. It shows, too; Darah was completely out of condition, and once again we only rode for an hour. Time to ride more often.
More work on ceeveear, the computer video recorder that I'm trying to build. Got a fractionally newer version of KnoppMyth, which refused to exit the partition menu during a manual install. Gave up on that, grabbed another (“empty”) disk and installed on that using the “automatic” install. That worked, and it even recognized my nVidia 6200-based card and installed a driver on it which set contrast so flat that I could barely recognize anything. Put in an nVidia 4000-based card and it worked. MythTV still has one of the strangest user interfaces I have ever seen, and I still wasn't able to get it to “open” the video card. xawtv found it (though at the time I didn't have an antenna connected), so it looks like my next job is to learn how to set up MythTV.
| Monday, 19 June 2006 | Echunga | Images for 19 June 2006 |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
I'm getting to hate Mondays; there's just too much routine work to do. Maybe I should move some of it to Fridays. After getting the administrative stuff out of the way, spent some time finishing the documentation of the handler interface. I had hoped to define the additional functions, but that's going to have to wait until tomorrow.
Wyeast supplies its yeast in “smack packs”, sealed metal foil packets with yeast and nutrient in two separate parts of a separate plastic sachet inside. To start the yeast off, you “smack” the pack to cause the inside package to rupture and the yeast to mix with the nutrient. Depending on the age of the yeast, it can take from a day to a week for the yeast to come to life, causing the outside package to swell up. I really don't like the method; it's very difficult to decide whether the interior package has ruptured or not.
On Saturday, I smacked a package of Weihenstephan W68, and it didn't swell as much as usual. Finally decided to put it in a starter and discovered that I had only half opened the package:
|
|||||||||||||||||||||||
It doesn't seem to have done the yeast any harm not to have the nutrient, though.
| Tuesday, 20 June 2006 | Echunga | |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
I've been trying to resolve a problem auction with PayPal for over a month now. First they asked me to get a quote stating that the camera was defective, then asked me to send it back at my cost—both fees that they would not refund. After sending it back, I had to send them the tracking number, by fax. That happened at the end of last month, and I got a message back stating:
So I did that, and got a reply which appears to have been computer generated. All attempts to contact a human being at PayPal failed, and finally today I got a message:
What nonsense! PayPal seems to work well enough under normal circumstances, but they seem to have people in the claims department that fail the Turing test. Finally found a phone number and called them up; Felix at the other end was definitely human, so far the first evidence of humanity I've seen in their resolution department. He didn't understand it either, and so I had to do more faxing (with the line dropping every time after the first page) of the proof of receipt. What a pain! Is this the way of the 21st century?
While removing a USB device from wantadilla, it spontaneously reset. I've been meaning to upgrade to FreeBSD 6.1 for some time, but I had had problems with a display card. Now I had a new card, so decided to finally take the plunge. With the new nVidia 6200 card (the one that showed inadequate contrast with KnoppMyth on Sunday), X ran fine, and spent the rest of the day trying to install ports. I still can't install firefox: it depends on some GNOME components, and somehow the build doesn't work right.
Despite everything, managed to get my first cut of the design of the handler interface for the online backup written up; possibly a little too hastily: Lars tore it to shreds during the evening. sigh.
| Wednesday, 21 June 2006 | Echunga | |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Things are working relatively well with the new wantadilla, but firefox still refuses to build. Attempts to install some of the myriad GNOME components didn't work either. Finally tried a pkg_add -r, which finds a remote binary package and installs it. That worked! Somehow the Ports Collection isn't what it used to be, though probably cans of worms like GNOME don't help.
Another message from PayPal in the evening:
The obvious conclusion: PayPal's guarantees are worse than useless. They have caused me to return the merchandise, and they now refuse to accept the proof. This means that I no longer have the merchandise, and that I can't get the money back. Tomorrow I'll investigate the legal implications.
Work on the backup code is getting into detail; hopefully there'll be some kind of milestone by the end of the week.
| Thursday, 22 June 2006 | Echunga | |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Much time on the phone this morning with PayPal. Spoke first to “Gurkha” (is that really his name?) who told me that the case had been closed because I had not provided suitable evidence. I pointed out that I had provided all evidence required in a court of law, and asked him if he would prefer the vendor's head on a pike, but of course that exceeded his vocabulary by even more than the concept that a complaint about their actions can't be resolved by describing their actions. Got an address to serve legal documents (Locked Bag 10, Australia Square, PO Sydney NSW 1215), which also got what attention he was able to muster. On the second attempt he passed me on to his supervisor, Jub, who was more sensible. She said that it needed escalation, and that I would be contacted by tomorrow evening.
Work on the backup implementation continues; I now have of the code for the storage engine interface in place.
Edwin DeSouza had been nagging me about various web pages on the subject of online backup; one, on the Microsoft Developer Network, claims that it's bad to optimize your backup strategy for backup speed.
That's an interesting issue, and one that we thought about early on. We decided that, in general, it is good to optimize your backup strategy for speed. One size doesn't fit all, of course, but:
The basic issue is that a fast backup can bring a slow recovery in its wake. That's often the case, but not always.
The impact of a slow recovery depends on how often you do it. Our attitude is that a recovery is a very rare occurrence. The fact that we've got by without an online backup at all for so long reinforces this viewpoint.
Backups can't proceed without some impact on performance. A “fast” backup frequently has less impact on performance than a slow one.
There are different reasons to do backups in the first place; some of them don't involve disaster recovery.
This is only peripheral to the tenor of the article, though. The author is primarily trying to get people to think through their strategy, and the title basically refers to one example. One way to speed up backups is to do incremental or differential backups, or log backups. The example of recovering a database from a month-old backup and a month's worth of log backups makes my hair stand on end, and that's probably the intention. At least it gets people to think about the issues, and that can only be good.
To reinforce this viewpoint, the author wrote a second article about the different reasons for making backups; though he doesn't compare his statements there with the other article, he does state that archival backups don't need to be so frequent. And that frequently means faster backups.
The question is, which kind of backup is more common? You'd think the disaster recovery scenario is more important, but even there there are two kinds:
Catastrophic hardware or software failure. In practice, RAID will recover from most hardware failures; after a system crash, there are software recovery techniques, usually involving logging, to recover table data.
User error. The second of the articles refers to the case where a user accidentally issues a DROP TABLE command. This scenario usually requires a recovery from backups, and it may be the only one. Disregarding foreign key constraints, recovery in this particular case could be performed on an otherwise active database.
I don't have the overview myself, but people whom I respect tell me that the main use of backups nowadays is for archival purposes. We're using that as the basis for design decisions for the backup software, though it will be on a per-storage-engine basis: most transactional engines will do a “dirty backup” of the table (i.e. copy the table data without regard for consistency), followed by the redo log for the duration of the backup. Restore will then restore the dirty table and perform a recovery operation on it. This is slower than just restoring clean data, but it makes the backup much faster. I'd be interested in feedback on this approach.
| Friday, 23 June 2006 | Echunga | Images for 23 June 2006 |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
I don't believe it! Only last month I was complaining about firefox brain-death, but now they've made it even worse! I tried to view a PDF document with the newly installed firefox, and was presented with the following dialogue box:
|
|
||||||||
Last month I complained that firefox had broken the pathname into directory name and file name; now they've made it even worse. There is NO WAY to enter a file name. And of course, they still completely ignore the concept of search path for executables and offer me a directory containing ancient pre-Microsoft DOS as a potential candidate to view PDFs.
It didn't even give me any idea of how to find the executable. Went looking and found that my preferences are stored in a directory /home/grog/.mozilla/firefox/07vn6ri5.Default User/ (yes, with spaces in the path name!). Just getting there was difficult:
=== grog@wantadilla (/dev/ttypm) ~ 1 -> cd .mozilla/firefox/07vn6ri5.Default\ User/
=== grog@wantadilla (/dev/ttypm) ~ 2 -> command cd .mozilla/firefox/07vn6ri5.Default\ User/
=== grog@wantadilla (/dev/ttypm) ~/.mozilla/firefox/07vn6ri5.Default User 3 ->
The background here is that I have a bash macro that changes the title on the xterm to display the directory name, using the /usr/ports/x11/xtset/ program. That strips off the \ in the input above, so it ends up with two parameters and tries to change to the first. This is indicative of the stupidity of using spaces in file names; they don't buy you anything, but they limit you in what you can do. Not coincidentally, it also broke the Emacs macro that I use when writing this diary to highlight the prompt colours.
In this directory I found a file mimeTypes.rdf which included the line:
With the help of wh, another script, I established:
=== grog@wantadilla (/dev/ttypm) ~/.mozilla/firefox/07vn6ri5.Default User 5 -> wh acroread
wh is pretty much like which(1), of course; the difference, apart from the ls -l format, is that it shows all executables in the path in sequence, not just the first one; this can be useful when things are behaving strangely because you're not executing the file you think you're executing.
So the problem was that the location of the executable had changed. No problem for a shell, but fatal because of firefox's brain-dead ignorance of PATH. Tried changing the pathname, with no success. Finally, quite angry, followed a maze of twisty little menus down to /usr/X11R6/bin/acroread. That worked, of course, but why have we been taken back to the stone age? Then went and compared the “before” and “after” state of the file:
The first set of diffs are just a change in sequence; the real difference is to unset “use system default” (which for some reason is also set to a null string). Why does this have to be so difficult? Are people losing all sense of reason?
Spent the day working on the handler interface for online backup. Finally found a list of handlers (an array, in fact) in sql/handler.cc:
This is in the current version of 5.1, which hasn't been released yet; it looks as if it will change. There's also a reference to an array handlertons[], but that doesn't exist beyond the reference. Maybe I should take some time to do a tidy up here.
| Saturday, 24 June 2006 | Echunga | Images for 24 June 2006 |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
firefox continues to infuriate me. Spent some time this morning trying to work out a good recipe for coconut chutney, and printed out the draft. The print settings had apparently been set to default—in this case, 15 point characters. What kind of default is this? Investigated and discovered that firefox has no idea at all about the basics of page layout:
|
|
|||||||||
Instead of point sizes, it talks in percentages—presumably of some undefined ”page size”. It seems independent of the size of the display, which is some consolation, anyway.
While experimenting with the settings, decided to print to a file. What directory was I offered? bin!
|
|
|||||||||
Why bin? Which bin? The one to throw firefox into? It didn't say, of course. Tried printing there—I didn't have permissions—to see what the error message was, but—there wasn't one. Where did it go? I searched the entire disk for a file, and that didn't exist either. It seems that firefox just ignored the problem.
Set the directory to /var/tmp (sorry, tmp) and was able to print to a file. Sort of:
|
|||||||||||||||||||||||
For some reason, some characters were swapped. When I repeated the print (having to fill out “print to file” and the file name again), it printed correctly. I don't know what causes that, except maybe that the first time I overwrote an existing file.
I use the word “correctly” loosely. The line spacing was too small, and the margins were incorrect. Although I had selected DIN A4 and “metric” margins of 12.7mm, it printed the headers and footers on the edge of the paper, where most laser printers can't print anything. When I tried to print out my diary entry, it produced a page break after the header. On investigation, it proved to be this line:
Taking that out made the right margins ragged, of course, but the page break was gone. It didn't occur under opera, so I'd say that this is yet another bug in firefox.
Once again I'm left wondering how so much breakage can occur. We have at least:
Loss of pathnames. What's bin? What's tmp? How do I know where my file is going?
Is my file going anywhere? firefox has completely inadequate error reporting. An attempt to print to bin (wherever that may be) fails silently.
Inappropriate directory defaults. It seems that the default directory for searching for executables is the same default as for storing “print to file” files.
Complete lack of understanding of basic page layout. By default, and like most other web browsers, firefox prints in PostScript, which has very clear and well-defined information about layout. firefox doesn't even seem to understand the concept of point sizes. Leading is completely beyond it, it seems, and there are basic but significant inaccuracies in the margin calculations.
Lack of memory for things like “print to file”; at least once I forgot to set “print to file” and sent a test printout to the printer.
File names with spaces in them. Yes, lots of people practice this obscenity, but it's still a pain, and it's completely unnecessary.
This is the future of web browsers? O brave new world, That has such people in't! I've come to the conclusion that they made firefox extensible so that the team can say “If you don't like it, it's in your power to fix it”. This makes life easier for the team, but that's about the only advantage. It's certainly no excuse for writing such patently bad software.
After that, I had intended to do some work on getting MythTV to work, but decided I had had one frustration for the weekend, and did some cooking instead. That was no better: my coconut chutney was a disaster. At least, it tasted nothing like what I was aiming for. One factor is probably the way it's made: you can't grind coconut in a blender. I used our meat mincer instead, but the result left something to be desired.
| Sunday, 25 June 2006 | Echunga | |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Another day where I didn't feel like doing anything with computers. Went out riding again, once again in the Peter Creek Road part of Kuitpo forest; this time Darah was more her usual self, though she's still pretty out of condition.
| Monday, 26 June 2006 | Echunga | |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Printing with acroread failed with the message /compat/linux/usr/bin/lpr does not exist. Correct; it seems that the newer versions of Linux don't include it, probably because they use the can of worms that is CUPS. I've already established that the standard /usr/bin/lpr doesn't either; I get the same message, incorrect in this case. However, managed to work around the problem:
=== root@wantadilla (/dev/ttyp9) ~ 77 -> ln /usr/bin/lpr /compat/linux/usr/bin/
Today I had two meetings that I didn't really want to happen today: the ACS opened a new office in Adelaide, and in conjunction I was invited to lunch with the president. Then, despite our resolution at the last meeting, the ICT council had another Monday meeting.
Things didn't quite go according to plan: these are “dress up” events, where I need to wear a suit and tie, and even more difficult for me, shoes. Got a phone call just before leaving, with the result that I left in a bit of a hurry. I was already on the other side of Adelaide before I realized that I had left my shoes behind, an “accident” that prevented my attendance.
The ICT council meeting was in a bit of a hurry; presented an item about current intellectual property issues in Australian law, but we were in a hurry, and I sensed a bit of an impression that this is an “open source” (i.e. pimply teenagers and communists) initiative. People need to think out a description that is understandable to business people.
| Tuesday, 27 June 2006 | Echunga | |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Why doesn't man work on eucla, my Fedora Core 5 work machine? For some time I've been getting things like:
=== grog@eucla (/dev/pts/1) /home/MySQL/5.1-Online-Backup/mysql-test/var/log 29 -> man fsync
That looked like some kind of file corruption, so set to to look for it. strace seemed to be the obvious starting point, and fortunately I had another Linux box to read the man pages on. Found that:
To trace child processes, use the -f option (and not -i as in BSD's ktrace)
The problem is in a subshell, started with:
That's not enough output. To get all the information, you need both the -v flags to show all the parameters, and to set the string length to something more than the default 32; I chose -s 512. This gave:
So it looks as if man(1) is using system(3) to format the output. But what's that final component (highlighted above)? It's piping into an empty string. It should be piping into $PAGER. Sure enough, it turned out that PAGER was set to the null string due to some changes I had made to my .bashrc. But man should check for that.
So, this is Open Source, right? Well, sort of. With FreeBSD I would fix things, but here I don't have the sources. I could get the sources, but then I wouldn't be sure that they matched, and there doesn't seem to be an obvious way to return the fix. I suppose I should enter a bug report, but that's no better an option than closed source.
More work on the handler interface for the backup system. It's fascinating how relatively minor changes require complete restructuring.
| Wednesday, 28 June 2006 | Echunga | Images for 28 June 2006 |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
More coding today; spent most of the day writing the handler interface. Nothing of great interest in the process.
Finally got round to investigating why my secondary MX mail bounced last week. Finally found the postfix variable mynetworks_style, which appears to be new; it didn't appear at all in my old main.cf. It was set to host, which had the effect of making all other systems on the local net to appear to be remote and thus untrustworthy. This modification solved the problem:
| Thursday, 29 June 2006 | Echunga | |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
Spent most of the day walking through the parser to get an idea of how to write the parser code for backup. There are some good functions in there, but finding my way around the source tree (one of my perennial issues) has proved less good than I would like. etags seems even less useful than previously, possibly because of all the C++ code in MySQL. Two years ago I started looking at source code navigation tools, and since then nothing much seems to have changed. Downloaded GNU global, but didn't get as far as using it properly.
I've been keeping this diary in RSS form for some months now, and it's been satisfying the RSS Validator for a while, but my feed to Planet MySQL still doesn't work right. Spent some time looking at that with Arjen; it would help to have a definition of what the aggregators want to see.
| Friday, 30 June 2006 | Echunga | |
| Top of page | ||
| previous day | ||
| next day | ||
| last day |
If there's one thing that inflames the passion of programmers, it's indentation style. I've been working out my own for some years, since before the advent of the C programming language, so it's not surprising that it's at variance with other indentation styles.
I started programming in Algol 60; an early program of mine, written in 1971, included this function (sorry, procedure) declaration:
This was written for a compiler that respects the fact that Algol 60 didn't have reserved words, it had symbols. The numbers at the end of some lines are card numbers; I don't know why they don't all exist.
Converted to a more modern version of Algol 60, this might look like:
This makes liberal use of Algol "features" such as spaces in variable names and implicit comments after an end keyword. Converted to C, but without changing the indentation, it would look like this:
Isn't that ugly? At this point, we can all agree. Now let's consider how it could be changed. The FreeBSD style(9) would make this into:
Isn't that so much more legible? Yes, it needs more comments (in those days most of mine seem to have been rows of asterisks to highlight the program structure), but that's not the issue here. It's just readable. But I still don't like it:
In my original, begin and end were lined up against each other. FreeBSD (and many other styles, including Linux) use hanging braces, so you can't line up matching braces in the same column.
The indentation has grown. In the original, the indents were 6 characters, Here they're 8. style(9) still restricts the width of a line to 80 characters, in memory of the long-defunct punched card, so it's easy to get to a point where the remaining width of the line is not enough. People then typically resort to either creating functions, changing the program structure to meet the indentation requirements, or shorten variable names (the ones here are obviously too short, but we're still getting close to the right margin). Both of these solutions are attacking the wrong problem.
Spaces are in the wrong place. In all Western languages, brackets (()) are preceded by a space. Here, as in many other styles, it's prohibited after function names, but not after keywords like if. I've never understood why this is, but I don't like it.
My own layout looks more like this:
Here I've reduced the indent to only 2 characters. I've got to this value over the years, starting at the 6 characters in the first examples and reducing one by one, and I find it the best compromise. Indentation is still clearly recognizable, but you don't wander off the right margin so easily. I've also lined up the braces over each other, and put spaces in “natural” places.
I started writing adapting the Emacs c-mode macros to this style years ago—the date in my comments is 13 November 1991. In the course of time I added nice little macros like things to automatically indent the next line, indent comments and so on. In the meantime, the official c-mode changed, and I became increasingly incompatible.
It would be too much to expect the MySQL style to be the same, and of course it's not. But it's surprisingly similar. Adapted to the MySQL style, the example above would look like this:
The differences here are:
The spaces in front of the brackets are gone again. I've already said why I don't like this.
There's no space between the name of a variable and the = sign in an assignment. The reason stated for this is that it's easier to search for assignments like this. I still don't like it.
The braces are at the same level as the surrounding block. I don't like this because conceptually the braces are part of the block they enclose, so they should be at the same indentation level.
The row of * to the left of the comment are gone. I don't like this because it makes it more difficult to recognize long blocks of comments.
Still, it's pretty close, and I can live with it, especially since it has my beloved 2 character indent.
But how do I format code like this? My macros are old and crufty, and the Emacs macros suggested in the style guide don't work for me. Lars wanted me to indent the code correctly, so spent a lot of today looking at the macros, with only partial success. A lot has happened to Emacs since I learnt it, and I need to read up on things. To make matters worse, the cc-mode documentation on my system (in this horrible info format that I haven't been able to get to like in the over 15 years I've been using it) and on the web site don't agree on a number of things; the latter, in PostScript form, is nearly 100 pages long. Much to learn.
Also with Emacs, finally got my RSS feed for Planet MySQL sorted out. I've had a lot of trouble with RSS. Apart from the fact that I don't like the approach, I've had difficulty understanding how the code should look like. The final hurdle seems to be that Planet MySQL wanted a <pubDate> tag (and yes, the capitalization is significant). Ended up modifying my html-to-rssxml macro to add the link. Starting with an HTML entry like this:
It creates XML like this:
Had some discussion about the pubDate with Arjen; it's not clear that it has to match the date in the title.
My brewing fridge has failed: the compressor runs, but it didn't cool. Off in the evening, through rain and storm, to pick up another one in Balhannah.
Do you have a comment about something I have written? This is a diary, not a “blog”, and there is deliberately no provision for directly adding comments. It's also not a vehicle for third-party content. But I welcome feedback and try to reply to all messages I receive. See the diary overview for more details. If you do send me a message relating to something I have written, please indicate whether you'd prefer me not to mention your name. Otherwise I'll assume that it's OK to do so.
| Top of page | Previous month | Greg's home page | Today's diary entry | Next month | Greg's photos | Copyright information |