2007 journal

2007 December 02 23:16

Stormywindyrainy here in the Bay Area.

I’ve decided to try hosting email on Google Apps. I don’t mind if my web site is down – even for days – but not having email is a drag; and my current setup (server in the basement) means I’m at the mercy of a capricious DSL modem.

Having my DNS and email hosted off-site makes my life simpler.

2007 November 27 16:00. pkgsrc’s tenth anniversary

pkgsrc, NetBSD’s third-party software packaging system (based on, and descended from FreeBSD’s ports) turned ten in October. To mark the occasion the pgksrc people published several interviews with key people involved in NetBSD’s, OpenBSD’s, and FreeBSD’s ports & packages systems – Alistair Crooks, Hubert Feyrer, and Jordan K. Hubbard, among others.

When I ran Linux, years ago (1995 to 2000, roughly), I always thought that building everything from source was the One True Way. When I discoverend that FreeBSD does this – for the kernel, the userland (the applications bundled with the kernel in a release), and ports (where all the third-party software goes) – I was thrilled. I switched and have (mostly ;-) never looked back. I still think it’s the way to go, but in practice ports and packages on FreeBSD have some issues. It’s my least favorite part of the system, actually.

I ran NetBSD on a server for a year or so, and my experience with pkgsrc was that it worked better than ports. I’m happy to be using it now on DragonFlyBSD as well.

People have discovered that using make as the core of a packaging system – while an elegant and in some sense obvious choice – creates awkward problems. Replacing this with an entirely new model is an interesting idea.

There are projects doing just that. Two that seem quite appealing are pkgjam and Nix. pkgjam is a “radically re-engineered pkgsrc” using a real database (sqlite) and replacing make as the dependency engine; Nix is a “purely functional” package manager, and sounds like the system that I wanted to build once I began to understand Git’s object store.

2007 November 18 15:26

I seem to have managed to hose my laptop – stubb, running FreeBSD – by runnig “make installworld” and having it fail in the middle. I’m not sure what happened or why, but the make stopped with an error. Something about not being able to find a file with the name <heart> (the heart card suit character from the console font). I tried rerunning it, I tried copying things by hand. Nothing worked. The system got worse. Lots of segfaults.

I can boot – sort of. The kernel starts, but when it tries to run the services, it’s a mess.

I’ve run out of options. I’m going to boot via floppy and run a binary upgrade to re-install the binary bits. I hope that will solve the problem.

I’ve run make build/installworld lots of times, on different machines, with nary a hitch. The worst part of this experience is not knowing why the installworld failed.

2007 November 17 21:25

The wiki now redirects to canonical URIs. What this means in practice is that the browser’s address bar will always show a valid URI. It used to be – esp when visiting the front page – that you would see

  http://www.nimblemachines.com/show

since Apache redirects the first request ("/") to "/show” and then the script simply displays the home page. Now it redirects to the home page.

Also, if a request had a slash after the page name, with or without more trailing junk, the request would fail. Now those are stripped, and a redirect is again done to a canonical URI.

2007 November 17 02:49

Well, that was quite interesting. For several hours now I’ve been sending my myriad readers (and spiders) into a redirect black hole.

How could this possibly happen? (you ask). There are several pieces to the puzzle.

DNS: inside or out?

My internal DNS here at home resolves both nimblemachines.com and www.nimblemachines.com to the web server, but because of details of my cheap & nasty dynamic DNS, for everyone outside, nimblemachines.com doesn’t resolve. To anything.

Who is doing the redirecting?

The redirections that my script does (see redirecting properly after HTTP post) use “root relative” URIs like

  /show/RecentChanges

which redirect to the same host, so if the first request resolved, the second will as well. Not so with Apache’s internal redirector, which I now invoke because I am redirecting URIs with query strings to new shiny pathinfo-style URIs (via mod_rewrite), which generates true absolute URIs like

  http://nimblemachines.com/show/RecentChanges

These can fail if they redirect from a reachable host (“www.nimblemachines.com”) to an unreachable one (“nimblemachines.com”).

What is this site called again?

Apache gets its idea of the domain name – for redirects, eg – from a ServerName directive in the virtual hosts configuration. Up till now I had something like this:

  ServerName nimblemachines.com
  ServerAlias www.nimblemachines.com

This allows connections from clients asking for either one (via the Host: header in their HTTP request). Because of my local DNS, I am able to generate requests with the bare “nimblemachines.com”, but no one outside can.

What does this all add up to? Google, or a reader, requests

  http://www.nimblemachines.com/wiki?show=RecentChanges

receives a redirect 301 (permanent) to

  http://nimblemachines.com/show/RecentChanges

because Apache (mod_rewrite) is doing this redirection, and gets its idea of the server’s name from ... the ServerName directive. This sends Google or the reader into a black hole. There is no domain there... so they wait, and then time out. A reader might get lucky: if they are running Firefox (but not IE!!), the browser will usually try to prepend “www.” and this will work.

I never fell into the hole because nimblemachines.com resolves, for me, and the server accepts it, so the redirection works fine.

I’ve removed those “resolutions” from the local DNS, and changed the ServerName to “www.domain”; looking at the logs I can see that Google is successfully seeing the new URIs. Finally! Wow.

2007 November 16 21:00

I’d like to announce that this site is now running my modular wiki code. I worked on it a lot a few months ago, and, as with my other projects, after getting eighty percent done I ran out of steam. I was able to finish the job today, and I’m much relieved. A bunch of small hacks I wanted to make had been postponed until the modular code was live. Having already “forked” the code, I wasn’t thrilled with having to keep two branches in sync. Esp when changes to the original wiki file map to one of about six files now.

There are two aspects of this system that might be of broad interest (to web people anyway):

• redirecting properly after HTTP post
• redirecting URIs with query strings

The code (it’s not brilliant) is here.

2007 November 15 18:16. Getting Git, redux

After successfully using the command line PIC assembler (MPASMWIN) to build a couple of little PIC demos, I’ve decided to move things along. I’m already tired of Windows, and I think I could pretty easily use the PICkit2 with open source tools – my own or others’ – esp now that Microchip has released a command line programming tool, and folks have ported it (to Linux at least). Porting this to FreeBSD would be a good start, for me.

I’ve also, thanks in part to watching Randal Schwartz’s Git talk at Google, gotten re-excited about using Git. I was feeling a bit overwhelmed by it – afraid of it, almost. Having gone back to using Git after a three-month hiatus has been pretty sweet. Git is fun!

Another reason I wasn’t using it was fear that the versions I was running were subtly broken – a while after building and installing Git, on two machines, I ran the test suite and it failed on both machines. So I spent some time building and testing Git, and documented my findings – esp useful for people running BSD systems.

Now I’m thinking about removing (and archiving) this Windows disk, getting another PATA disk, and starting over with FreeBSD. I still like like it too!

2007 November 06 02:01. Does commercial software suck?

I built a Windows machine so I could work with Microchip’s PIC microcontrollers. I bought a PICkit2 programmer, and sampled a few parts. I started down the (long) road of creating my own software to assemble and program the parts (a Forth-based assembler, and a Lua-based programmer) and got bogged down in gory detail, and lost momentum.

Recently I wanted to get going on those projects again, so I snatched the body of my web server machine (putting an inferior one in its place) and built myself a nice Windows XP box. Installing Windows was a joke, as always, but now that the machine is running, it’s reasonably easy to use.

I can’t say the same about MPLAB, Microchip’s IDE for their chips. mplab stinks. Several times I sat gaping in awe at its stupidity.

I’m tempted either to

• use the tools provided from the command line, perhaps from MSYS; or
• chuck the whole thing, install Linux or BSD, and go back to what I was doing before, and hope I don’t get as stuck this time.

I definitely prefer BSD/Linux to Windows; and I find mouse-centric IDEs (with completely brain-damaged user interfaces) so awful that I want to throw things. And my hand hurts from mousing. And I still got nothing done.

2007 November 04 19:01

I’ve been thinking about an Arduino killer – a board based on the NXP LPC2141, an ARM7TDMI microcontroller with built-in USB that costs about what the Atmel ATmega168, which powers the Arduino, costs.

However, the Atmel chip lacks USB capability; the Arduino requires another chip, which costs as much as the microcontroller!

A fairer comparison would be between the LPC2141 and the similar, but much slower, PIC18F4550 – a “high-end” PIC device with built-in USB. However, this is an 8-bit device, running at one-fifth the speed of the ARM.

And – surprise! – it costs more than the ARM does.

When 32 bits are cheaper than 8, the time seems to have come when eight bit microcontrollers are obsolete.

2007 November 03 17:36.

A few quick notes about using MinGW and MSYS.

2007 November 03 04:43.

I’m going to attempt installing MinGW and MSYS. I probably won’t get anywhere until later.

2007 November 03 02:29. New infrastructure

I decided to switch things around. Tashtego – the Free Geek machine running DragonFlyBSD that hosts this site – has a new body. An old body, really. I dug up an old Pentium II (300 MHz!) machine and moved Tash’s brain into it. Works great, and is doubtless beefy enough to handle my not-so-overwhelming traffic.

That freed up the Free Geek machine – a gigahertz Pentium III – to use as a Windows box (I know – bleech) for hardware development. I made a mistake in this process, though: I assumed that I could install Windows on the old machine, and simply move the hard drive over. I’ve done this heaps of times with NetBSD & FreeBSD, and the drive always boots in the new machine (my kernels, though not GENERIC, only lack SCSI and oddball network drivers; I leave in the common stuff, and all the IDE drivers). Not so with Windows. Sadly, having already spent several hours getting my new XP install updated to the latest bits, I ended up having to “repair” the installation (essentially a fresh install) followed by a re-update. Happily, the repair process left my registry intact, so I didn’t have to also reinstall any apps.

I wouldn’t have had this problem if I hadn’t assumed that Windows would actually behave reasonably. I’m an incurable optimist in this regard, expecting that one day it will make sense. It never does. Why Windows can’t be as flexible and smart as BSDs is a mystery.

I had another problem, too, one that wasn’t caused by naiveté/optimism: this new machine (Flask, another character from Moby Dick) has only wired Ethernet, but the only wired network is in the basement between the DSL modem/router and Tashtego. I pressed an unused Linksys WAP11 (802.11b access point) into service as an access point client, bridging the traffic on its local LAN (connected only to Flask) to the wireless. It acted flaky – doubling pings, not routing other pings – until I moved its IP address out of the range of the wireless subnet (192.168.0/24) and into its own (192.168.1/24). Now it’s solid.

2007 October 29 19:31. Arduino!

Lots of people are talking about the Arduino, a hardware & software platform for doing fun things with electricity. It’s intended as a simple-but-powerful microcontroller that is accessible to creative people for whom this kind of technology is normally out of reach.

I haven’t played around with one yet, but wanted to share some pointers and initial thoughts. Here in Portland, the Dorkbot crew are talking about having an Arduino soldering party, which sounds like a lot of fun.

I’ve created an Arduino page that discusses the some of the variants that are available.

2007 October 29 02:38. Neat things

One-wheeled electric scooters (think of a cross between a skateboard and a Segway):

• Ben Smither’s
• Chris Coleman’s (inspired by previous)

Machine Project, an amazing non-profit gallery-cum-workshop space in Los Angeles that does all kinds of neat technology workshops for artists and has very cool gallery shows. Some highlights:

• Blubber Bots. These are happy, slow, & friendly helium balloon autonomous robots!
• Difficult music. A performance & discussion series.
• Electronics for artists
• Felt + circuits. Make your own electronic plush toy!
• Hekla Dögg Jónsdóttir show
• intro sewing workshop!

... and lots of other happenings, past, present, and future.

Is there anything like this in Portland? If so, where is it? If not, we need to make one!

2007 October 25 16:19. Exploring Vim

I’ve been using Emacs for a long time. I’ve never loved it. I don’t use most of its features. It seems like a bloated dying whale to me. However, my fingers know how to tickle it.

Vim has seemed for a long time to be a compelling option. The whole “learning a new set of keystrokes” thing made me a bit shy – though, I must admit, that from 1982 to 1988 or so I used vi intensively, so Vim’s vi-ness will be familiar. It’s learning Vim’s “improvements” that seemed daunting.

Last night I read through a lot of the manual and played around a bit with Vim. I liked it. Bram seems to have done a thorough job not only of writing great documentation but also anticipating wants, needs, and confusions. In early 2007 Bram gave a tech talk about using Vim to be more productive.

I decided to install it on tashtego. The version I was using last night was 6.4, installed on a FreeBSD machine via ports (vim-lite). Today I built and installed a “vanilla” Vim from pkgsrc (/usr/pkgsrc/editors/vim), and was irritated to find that F1 doesn’t fire up the help.

My travails with hosting on Tashtego tells the whole story (and a few others too).

I may have more to say about Vim as I continue to play around with it.

2007 October 13 04:47

It appears that Google needs sysadmin talent.

Douglas Shacteau just tried to send me email, which my server bounced – it’s picky about things to ward off spammers. Apparently the outgoing Gmail server announced itself as “hs-out-2021.google.com” – but that name doesn’t exist in the DNS!

Oops.

Got mail server administration skills? Need a job? Google sure needs you.

2007 October 08 15:17. James Howard Kunstler!

Some of his beliefs:

• We are living, now, in the time of Post Peak Oil.
• The housing bubble is collapsing, signalling an end to suburbia and a reality check on having sold out our manufacturing infrastructure and replaced it with delusionary finance and a suburbia-building juggernaut.
• We could solve some of the hassle of air travel, and save lots of fuel, by rebuilding our passenger train infrastructure. People love trains! Why did we get rid of ours?
• Modern architecture, and the Banal Suburban, have done more than miss the point: dementor-like, they are sucking the American soul.

Quotes from Clusterfuck Nation:

I’m hardly an advocate of the US giving up and committing suicide. What I advocate is a broad recognition that reality is compelling us to change our behavior. Reality is trying to tell us that we can’t run an economy based on nothing more than investment schemes without directing investment into activities that produce things of value. Reality is telling us to be very worried about living arrangements that can only function with copious imports of oil from people who are disgusted with us. Reality is telling us that we can’t divert our food crops into making motor fuels without people becoming unable to afford either fuel or food. Reality is telling us to redirect our culture more toward things-we-do-with-other-people and less toward things-we-do-with-new-things. Reality is telling us to shift from avoidance behavior and denial to engaging with reality in order to lead lives that are consistent with reality.

Are we building a society with a future? Does our culture affirm life or yearn for destruction? Are our daily ceremonies and rituals meaningful or empty? Are our hopes and dreams consistent with what reality has to offer? Can we look in the mirror and say that we are upright people?

A few tidbits from his Eyesore of the Month, a lovely collection of architectural mistakes as a symptom of ongoing cultural collapse. His skewering of Modernists – with which I heartily agree – is breathtaking.

• Hope for the future: the revitalization of small-city America (focussing here on Troy, NY, and Baltimore, MD).
• Journey by foot to catch the train: how pedestrians have been completely left out of the planning & design process.
• Rem Koolhaas’ ridiculous Seattle Public Library. Neatly skewered.

2007 September 28 03:19. Bicycle bits

I’ve been spending a lot of time reading & thinking about cycling and my bike projects – not doing much else, frankly. I wanted to share a few things here.

First, if you want to read some snarky & hilarious (& intelligent) commentary about the contemporary cycling scene – with extra special care taken to skewer urban fixie-riding hipsters – read Bike Snob NYC. Don’t be drinking anything though, unless you want it shooting out your nose and ending up all over your laptop/monitor. The guy is funny.

Second, I finished and delivered (to my dad!) the Schwinn Le Tour, set up as a singlespeed (40x18 gearing on 700c wheel), with upright cruiser bars. It’s a lot of fun to ride. I don’t have pictures yet, but hope to soon.

Lastly, a conundrum (for me anyway). Right now I’m working on two bikes: my old (1987) Bianchi Brava road bike, and my urban utility project bike (the Nishiki Olympic twelve). The Bianchi has always had its bars too low, even with the stem maxed out. So, I need a new stem. (The Nitto Technomic is nice and long.) I figure if I’m replacing the stem I could put nicer bars on it too – like 44cm Nitto Noodles – the current bars are 40cm Nitto Mod55. It would be nice to have wider bars, and the Noodles sound great.

The rear wheel is pretty trashed. And it’s “obsolete”: 126mm spacing, 6 speed, thread-on freewheel.

I could get two new wheels for it from Peter White Cycles for about $260. The rear would be a 126mm spacing cassette freehub that will take 7 speed cassettes – which, though obsolete, are still available, and pretty cheap. By buying a couple – or several even – I could buy cheap obsolescence insurance.

The big question: does it make sense to spend this time and money on an old road bike? I have ridden and loved this bike for twenty years. If I don’t fix it up I’m going to stop riding it, and selling it seems silly.

But I could instead be buying new wheels for my fixie project...

2007 July 27 14:18

Simon Peyton Jones was in Portland this week, at OSCON and at Galois, giving interesting talks. Wednesday I heard him talk (at Powell’s Technical bookstore) about his chapter on Beautiful Concurrency in the new book Beautiful Code. Yesterday I went to Galois to hear his amazing talk about Software Transactional Memory, also the subject of “Beautiful Concurrency”.

I haven’t watched it but there is also an hour-long interview in which he talks about STM.

STM is a principled way of doing lock-free concurrency on a shared-memory multiprocessor machine (which is what lots of people have now and everyone will have in the future). As each thread reads and writes memory within an “atomic” block, other threads see all or none of the thread’s writes. The details are interesting, and the Haskell implementation makes good use both of the type system and monads, to make large classes of bugs unwritable.

Seeing Simon talk is always an inspiration. He is passionate, articulate, and witty. His talks always reinvigorate my interest in Haskell.

In fact I’m building Hugs (a Haskell interpreter) right now. ;-)

In other Haskell news, Don Stewart is moving from Sydney to Portland, to work for Galois. He is also co-authoring a much-needed book, on Real-world Haskell.

Oh, and he has solved my window manager woes. He is also the co-author of xmonad, a monadic tiling X window manager (of very modest size, principled design, and QuickChecked implementation). I’m looking forward to trying it.

2007 July 10 13:00

I was worried that after the move to dynamic DNS Google had given up on me, but early this morning I noticed a bunch of traffic from Googlebot, so I’m still being indexed.

Last night I tried running the battery of tests on Git and discovered that none of the Git executables I have built pass all the tests. Not so good. I’m going to spend some time documenting building and testing Git.

2007 July 05 15:27

My server (really Randy Thelen’s) died on June 27, after that post. While the machine is back up, we are going to change the arrangement. I’m now hosting on Tashtego (my Free Geek machine), using some dynamic DNS tricks. The one thing that no longer works is that bare domains, like http://nimblemachines.com don’t have any records in the DNS any more. There used to be an A record pointing to Randy’s machine, but he’s switching to dynamic IP, which is what I have here at home ... so there is nowhere to point an A record.

However! Subdomains (like http://www.nimblemachines.com) can be CNAMEs for my dynamic host name, and in fact that is how things are currently.

If you are using a modern browser, chances are that following the first link above will take you to the right place, because it will try prepending “www.” after finding nimblemachines.com unreachable.

We’ll see how it goes with the web spiders...

2007 July 03 14:15

I decided to set up my Free Geek freebie machine as my web hosting platform! This means I’ll be experimenting some more with dynamic IP resolution (see dynamic DNS), and also with DragonFlyBSD.

2007 June 27 16:06

I’m going to try importing from Subversion to Git. We’ll see how it goes...

2007 June 23 19:47

I’m thinking about version control again. I started using CVS in 2002; in 2005 I dumped it for Subversion; now I’m thinking of dumping Subversion for Git, Linus Torvalds’s new-ish distributed version control system.

2007 June 17 20:13

The muforth source and its info word refer to the following URI:

  http://nimblemachines.com/browse?show=muforth

Because I have changed the scripts that publish this site, that URI doesn’t work any more.

I plan to fix the source code, but instead of leaving the web site broken until I do, I added the following to my Apache config:

  RedirectMatch /browse(.*)  /wiki$1

Note that the much easier to type

  Redirect  /browse  /wiki

does not work. Apache complains that the right hand side is not a fully-qualified URI (ie, doesn’t start with “http://" or another URI scheme, and doesn’t contain a fully-qualified domain name (FQDN)).

Apache can be really arbitrary and annoying.

  Redirect /browse http:///wiki

  Redirect  /browse  http://nimblemachines.com/wiki

2007 June 17 14:53

I put the “c” (cascading) in CSS finally. Now the wiki code will accept as a config setting a link to a stylesheet local to that wiki; its settings override the default style sheet.

In other news, yesterday evening I finally finished the build program at Free Geek, and came home with my free computer – the box and keyboard, anyway: I opted to forgo the monitor (have one already) and mouse (ditto). The only thing interesting about this is that I carried it home on my bicycle! This wasn’t a “full size” (ie, mid-tower) box, though; it was a Dell GPX150 “desktop” – about the size of a full-sized VCR, but much heavier.

I strapped it to the side of my pannier. No problem! I haven’t booted it again yet to see if I trashed the hard drive on the ride...

Here are the (rather modest) specs:

• 1000M P-III
• 256M SDRAM (PC133)
• 20G Western Digital Caviar hard drive
• ATI Rage128 video (the onboard video had hash in it)

It has Ubuntu Linux installed on it. I’m likely to install either Windows (so I can do hardware development with it using lame proprietary IDEs), or Fedora Core, so I can play with Planet CCRMA.

2007 June 16 03:19. Smoke test!

I just hugely reworked the Apache configuration for this site (and its sister sites). They used to each have a copy of the wiki code, and for various reasons it was messy and kludgy. Now there is one shared copy of the wiki code, the URIs are treated uniformly, and all is well in the world.

Since I haven’t tried to save any edits yet, this is the smoke test!

There was a tiny bit of smoke, but with a minor tweak, everything worked.

2007 June 15 02:57

I was just poking around on WardsWiki and stumbled across OrwellsParody, a transcription of Orwell’s rewriting of a lovely verse from Ecclesiastes in “modern English”.

His criticisms reminded me sharply of my own about “modern coding practice” – the way we accrete layer upon layer. I find it distasteful, and prefer embedded programming down to bare metal.

I especially liked this bit:

It consists in gumming together long strips of words which have already been set in order by someone else, and making the results presentable by sheer humbug.

2007 June 14 01:15

Both Ward Cunningham and Michael Pruemm agree that having both a Tweak (for small changes, like typos and misspellings), and a Save (for larger, more substantive changes) is bad. Having the choice is bad, and Tweak allows you to hide – accidentally or intentionally – substantive changes. (Save edits the recent changes file; Tweak does not.) So, Tweak is gone.

Or, as Yoda might say: “There is no Tweak; there is only Save.”

Enjoy.

(Tweak was called MinorEdit on WardsWiki.)

2007 June 13 14:56

Debugging ssh and bash to figure out which shell startup scripts get sourced when I ssh to another machine. I need to get the PATH right, so my svn commands (using the svn+ssh scheme, which runs svnserve on the remote end) will work!

It turns out to be rather obscure, but I debugged it using shell variables.

2007 June 11 18:49

I’m helping to set up a Mac mini as a hosting platform, so I’m working on

• building Apache on OSX
• building Subversion on OSX
• building Postfix on OSX
• building Erlang and Yaws on OSX
• building Git on OSX

In each case I’m building from scratch. It’s not hard, really – just giving configure the right options!

2007 June 11 16:50

Since I just spent a couple of hours figuring out to do it, I thought I’d share my thoughts on locking down Postfix against spammers. Just basic simple configuration – not even any content filtering. It’s amazing how few spammers can manage to send mail from a properly configured mailer!

2007 June 11 13:57

I’m fiddling with config files again, and re-discovered my hack for relaying Postfix thru Gmail.

2007 June 07 14:02

Last night I went to my first dorkbotpdx meeting. It was really fun talking about Forth and microcontrollers. And people had heard of Trimpin, which was cool.

I may end up giving a little talk/presentation about muforth at the upcoming dorkbotpdx event. That should kick me into gear to finish some things related to my ARM Forth target compiler, and my code to download to the Philips LPC ARM series of microcontrollers.

On an unrelated subject (bicycling), a friend sent me this link. It’s a page of photos and comments by an American who sat on a corner in Amsterdam and took photos for 73 minutes. His attitude is very US-centric, and occasionally annoying, but the pictures are fun, and reminded me of how much I liked the bike culture in the Netherlands. He has graciously posted reader responses to the page; it’s the most interesting content!

The pictures made me rethink a few things about my city bike project (my Nishiki Olympic twelve). I think I want an upright seating position, “touring” (ie, three speed style) handlebars, and perhaps simple platform pedals that I can use with normal shoes.

I’ve been riding with either cleats (had a pair of lovely wood-soled Duegis until 1996 or so) or SPD shoes for almost thirty years! It’ll be hard to go back, but I’ve been wishing recently that I could ride to the store or to go out in the evenings in something other than my Shimano mountain biking shoes. It’ll be an interesting experiment.

2007 June 03 16:12. FreeBSD installer considered harmful?

I just discovered 8 gigs of unused, unpartitioned space on my hard drive, and I blame Poul-Henning Kamp (PHK).

I’ve had issues in the past with the FreeBSD installer not using all the drive space because it was trying to align partitions with its idea of cylinder geometry, which, we all know, is a fiction on modern drives. The culprit is libdisk, the core library used by fdisk and the FreeBSD installer. libdisk was written by PHK and is covered by his “beer-ware license”, which says that if you like his code that you’ll consider buying him a beer if you meet him in person. I wrote to a friend, during my time of travail with libdisk (via the FreeBSD installer), that if I ever met PHK I’d be more likely to punch him in the nose.

I’m feeling that way again.

I’m thinking about axing the FreeBSD 6.1 install on my (old) laptop and replacing it with Linux. Maybe I haven’t figured out how to use portupgrade competently, but I find using FreeBSD as a desktop OS frustrating. There are compatibility issues (no way to run Flash inside Firefox, eg), and keeping large third-party packages, like Firefox, updated is unwieldy and error-prone. I love being able to buildworld & installworld, but outside of the base system FreeBSD is less friendly.

I’d like to “check in” again on the Linux world. I haven’t run Linux since I first installed FreeBSD in 2000. It seemed so much better that I never looked back. Well, almost. I briefly installed Debian on my laptop when I upgraded the hard drive, but was annoyed by some things about it – like that hwclock, which was broken in 1996, was still broken – so I relented and put FreeBSD back on.

I’d really love to try playing with Planet CCRMA, a huge collection of audio and computer music software, curated by Stanford University’s CCRMA computer music center – one of the foremost research centers of its kind. Using their repository requires that you run some flavor of Fedora Core. (They haven’t “upgraded” yet to Fedora 7.)

Some of you may think that Fedora is evil. Maybe you’re right. I’ve never been much of a fan of Red Hat. But Planet CCRMA seems pretty cool, and I’ve had my eye on it for several years now.

So what does all this have to do with PHK, libdisk, and the FreeBSD installer? In figuring out a way to get Fedora onto my laptop – which lacks a CDROM drive, and which can’t boot from USB sticks – I discovered to my great surprise that 8 gig hole on my hard disk – thanks, I have no doubt, to PHK and libdisk.

Or maybe I left room for something that never got installed. (Windows 98, anyone?)

In any case, it gave me a great idea: why not leave some space – 32M perhaps – at the end of the disk to put special-purpose boot code for installing operating systems? Right now I am limited to systems whose installers can boot from floppy, and they are a vanishing breed. My list currently is: Debian, FreeBSD, NetBSD. Ubuntu won’t work; neither will Fedora. Unless I can “misuse” some of the empty space on my disk.

Hmm. Maybe I owe PHK a beer after all.

2007 June 01 13:48 // Updates to Kepler

I just discovered that the Kepler project – a very lightweight web “framework” written in the Lua programming language – has a new version, which is not only much easier to install, but now sports a wiki, called Sputnik.

Pretty cool stuff!

2007 May 31 23:19

I was just reading about Joomla! and WordPress and noticed some “similarities” in the documentation. I’ve quoted two pairs of excerpts here.

First this:

Development of WordPress moves fairly quickly and day-to-day things break as often as they are fixed. This high churn is part of our development process that aims to produce the most stable releases possible. (source)

Development of Joomla! moves fairly quickly and day-to-day things break as often as they are fixed. This high churn is part of our development process that aims to produce the most stable releases possible. (source)

I’m not sure how high churn yields stable releases. Whatever.

And then this:

After the 1.5 release we’re moving our source control off of the legacy Sourceforge CVS system and on to a modern Subversion repository. When we get confused about Subversion we use this great Subversion book that’s available for free online. Here are some quick tips for checking out and working with the WordPress repository. (source)

The Joomla! source code repository is run on a Subversion (SVN) server. When we get confused about Subversion we use this great Subversion book that’s available for free online. Here are some quick tips for checking out and working with the Joomla! repository. (source)

Who “borrowed” from whom? It seems a bit tacky to use someone else’s text without attribution.

Is it a coincidence that both projects use PHP?

2007 May 30 16:14. Gearing up to use source forge

I’m going to be working on a project hosted on SourceForge; details to follow. To get started I registered as a new user, and then created a new SSH key pair for myself:

  ssh-keygen -t dsa -C <username>@shell.sourceforge.net

When asked where to put the key data I specified ~/.ssh/id_dsa_sourceforge, to keep it separate from my other identities. Normally this defaults to ~/.ssh/id_dsa. To make having more than one SSH “identity” not painful in practice, I edited my ~/.ssh/config and added this:

  Host sf
    HostName shell.sourceforge.net
    User <username>
    IdentityFile ~/.ssh/id_dsa_sourceforge

Now I can say

  ssh sf

and all the right things will happen. That is, once I’m registered as a member of a project...

2007 May 26 10:01

I hate WD-40. You know, that spray lubricant stuff. I hate it. I really do.

I think it’s because it’s more solvent than lubricant, and it smells funny, and I always hope it’s going to do something that it never does.

The last time I used it (and I mean that in every way possible!) I had just rebuilt the rear wheel on my road bike – lovely repacked new bearings, freshly greased, perfectly adjusted – and I was trying to get the freewheel to be less grindy and noisy. Note (for bike geeks): this is an old-style wheel with a thread-on freewheel/cluster. I put the wheel on the bike, spun it, and sprayed WD-40 into the freewheel. It made a mess.

Some WD-40 also ran down onto the axle and into my freshly packed bearings and dissolved a bunch of the grease, so I had to take the freewheel off, repack the drive side, and readjust it (and I was grumbling the whole time). I also let the WD-40 drain out of the freewheel and – off the bike and off the wheel – I spun it some more and squirted it full of motor oil. A bunch of black gritty stuff came out. It doesn’t sound any quieter or smoother – I’m pretty sure it’s trashed and needs replacing – but one thing is for sure:

I’m never using WD-40 again. Ever.

2007 May 25 13:24. Finding my tribe in PDX

Last night I went to a post-BarCamp meetup and talked with Audrey Eschright about DorkbotPDX. Good stuff! Too bad I missed the last meeting (two days ago), but looking at some of the participants’ interests and experience on the DorkbotPDX wiki, I realized that these may be some of My People. I’m excited to meet and talk with them.

Shared interests include: microcontrollers, kinetic sculpture, digital audio & music. (Mis)using technology to make art!

2007 May 19 01:44. Bicycle projects

I’ve (re)started two bicycle projects. I’m working on a Schwinn Le Tour (as a utility bike for my dad), and a Nishiki Olympic twelve (the new fixie/singlespeed project). I have given up on my Motobecane fixie and will be essentially junking it.

I’ve also added a bicycle parts directory so I can keep track of what I find.

In the process of trying to prepare myself for riding a fixed gear bicycle I’ve been singlespeeding for the last two years on my road bike (a 1987 Bianchi Brava) and have fallen in love. I’ll try to elucidate why single speeding thrills me.

2007 May 11 17:47. BarCamp Portland!

Having recently got back from a month (!!) in California, visiting family and friends, I stumbled over a link to BarCamp Portland, happening this weekend. I’m planning to go, and I’ll be writing about what happens there!

2007 February 27 16:56. Blogging redux!

I’m thinking again about blogging – “proper” blogging, with RSS, pingbacks, the works. Though people are finding my pages here (thru Google, mostly) I still feel a bit invisible.

I can’t remember exactly when or how I first got excited about blogging (in its modern form). I have wanted to “be online” since 1997. I was reading Jakob Nielsen then. I knew that the “real” Web was about text, the human voice, and personal expression. I didn’t have a good idea about what tools to use, and in fact I’ve been stumbling around since then.

In 1997-8 I was using a strange combination of m4 macros and automated FTP to build a small site for my Earthlink account. Around 2000 or so I learned how to run Apache myself, and helped my sister build an online presence for her company, by setting up email and web servers, but they were responsible for building their own site, which they did by hand in HTML. Yikes!

I knew that mere mortals shouldnt write HTML. It is too low-level, and at the time lacked good semantic markup. (This has since changed dramatically; however, it’s still rather tedious to type all those angle brackets.).

Somewhere in here, 2000 or 2001 probably, I discovered Jon Udell’s book Practical Internet groupware. This was another quantum shift for me. He makes a compelling case for writing documents in XML, using some kind of semantic markup. He used slightly-modified HTML at the time; now you could probably do the same in straight XHTML. He did a lot of really neat things integrating HTTP and NNTP servers to build collaborative spaces out of off-the-shelf technology. His book is mind-bendingly innovative, and full of ideas.

With a headful of XML, I went in search of tools, and got truly bogged down by the idea of running some kind of Apache toolset. The Apache group were (are?) busy building XML tools, but in Java. Yuck. I had, and have, no interest in learning Java, and I was then, and am now, running FreeBSD on my web server machine. At the time Java didn’t run on FreeBSD. (There has since been a huge effort to port it, once Sun released the source, so this is no longer an issue. I think. I haven’t tried it in years.)

The Apache XML & Java worlds are overwhelming. There are tools to do everything, but everything is hierarchic. Navigating the pyramid, just to figure out what was available, made me crazy. I probably spent several weeks, on several occasions, trying to get my head around what they were doing. I’m not sure I ever succeeded. I was going to use Cocoon, but finally realized that for what I wanted to do it was massive overkill. And I couldn’t run Java on my server.

Back into search mode. I forget the exact chronology – and don’t have a blog to refer back to! – but between 2001 and 2004 I discovered both wikis and blogging. Wiki technology seemed like a good fit for what I wanted to do. Whenever I thought about how to “organize” my interests (I was thinking hierarchically) I would get frustrated. Then I discovered how Categories work on Ward’s wiki. They are essentially an early form of tagging. A page can belong to multiple Categories, just as it can have several tags. Same idea. This completely changed my world. I knew this was the answer.

I also liked the way the page “name space” was flat, rather than hierarchical. Now I was free to write without having to worry about where to put my ideas. Clay Shirky’s paper (and talk) about ontology and folksonomy simply re-affirmed these ideas. It’s really worth reading and listening to.

Now I needed a wiki. Yikes! Another huge process. There are 50000 implementations, all different, all using different markup. At first I wanted to be a purist (which is to say, true to Ward’s ideals and markup, which are intertwingled) but I have since changed my mind about certain things. CamelCase and accidental linking are nice ideas, but ugly on the page, and they interfere with Google searches. I’m still using CamelCase on this site, and while it’s nice to be able to easily make links, it’s going to go away in favor of something more like Wikipedia’s link markup.

I looked at the source of several wiki’s (the “free” code bundled with Ward’s wiki book – free only for non-commercial use), and the code to UseMod and OddMuse. (Interestingly, OddMuse has a lot of the features I was thinking of adding to this wiki to make it more blog-like. Hmm.) Everything I looked at did more than I wanted, so I wrote my own. Of course this was a slow, fitful, and somewhat painful process (character flaw of mine) but Michael Pruemm egged me on, and in late January 2005 this site was up.

Somewhere in here, in fall 2004, I started a blog on Blogger. After posting a few times I realized that I loved the idea (of a blog) but hated Blogger and mechanics of blogging there. I’m still searching for a good blogging tool. Wordpress seems highly regarded, but every time I visit the documentation site I feel nauseated. It seems really disorganized and overly complex. Also, Wordpress is written in PHP, an ugly “language”. I want my tools to be not only convivial tools but also beautiful ones. PHP is not beautiful. The Lua programming language is.

I got really into the idea of blogging, and found out about tagging, and about flickr and del.icio.us by listening to podcasts by Dave Winer, Robert Scoble (older blog), Jon Udell, and others. (I’ve got some of this bookmarked.) Somehow hearing about this stuff, and hearing some of conference talks (BloggerCon III was especially cool) got me pretty amped about blogging. (There is also audio from BloggerCon IV which I haven’t listened to.)

After my frustration with Blogger and my inability to settle on and set up tools for myself, I contented myself with pseudo-blogging – what you see on this site. I have a macro – [now] – that, when the file is saved, is converted to the current time. I use this to make a heading, add an hrule (by hand), and voilà. (The OddMuse author has a better way: the <author> tag will troll thru the pages and find ones whose names are a date, and pull several of them together to make a page. This makes creating an RSS feed much easier than teasing apart “posts” that are separated on a page by “----”.)

In a recent podcast about Scoble and Shel Israel – who together authored the book Naked Conversations, about corporate blogging – Scoble mentioned that he trolls various blog search engines – particularly technorati – looking for memes, like any mention of “Linux”. Since I’m not pinging technorati and they aren’t scanning my feed – since I don’t have one! – I’m invisible to Scoble. That’s too bad. ;-)

So, I’m thinking about how to change that. I only need my “journal” (this front page) to have an RSS feed, so breaking these posts out OddMuse-style, and then having two ways to render them – HTML and RSS – would be a pretty easy and elegant solution.

Someday I’ll be blogging using Lua, but I don’t want everything I want to say to have to wait until then.

2007 January 21 21:02. USB HID woes. Aaaaaaagh!

I’ve been having trouble getting my PICkit2 programmer to talk to FreeBSD. There has been a range of issues:

• bugs in usbhidctl(1) made me think that the driver was misinterpreting the device’s report descriptor, when in fact usbhidctl’s separate implementation (!!) of the descriptor parsing code was broken;
• FreeBSD’s uhid(4) was “in the way”, trying to do things that I didn’t need it to do, so removing the driver from the kernel and having the PICkit2 show up as a ugen(4) (general-purpose USB) device worked much better; in fact, it wasn’t until I did this that I was able to exchange messages with the PICkit2 and discover that...
• ...the PICkit2 is running older firmware than I expected! This threw me off because I had the source for the 2.xx firmware and was basing my expectations of the communications protocol on that code; but my PICkit2 is running 1.20.0 firmware, which implements a completely different protocol!

Now that I have the basic communication path down, my next struggle is to decide whether to base my host software on the 1.xx firmware, or to write an updater (that talks to the PICkit2’s built-in bootloader) to download the newest 2.xx firmware, and plan to talk to that. The 2.xx firmware can program a greater variety of devices, and moves the complexity of the different programming algorithms into scripts which all live in a database, which I’ve already written Lua code to read. I would prefer to use the 2.xx firmware, but I’m afraid of b0rking the PICkit2 in the process of updating it.

That’s the basic story; I’m working on a longer story of the pic kit two and USB.

2007 January 17 12:57

I’m going in several directions at once. I’ve written an 8080 assembler (in Forth) and now I’m revamping my meta-compiler (again) so it’s generic. I’d like to be able to use it for the 8085, the ARM, and maybe the AVR or PIC18F chips.

I haven’t worked on it for a few days, but I was able, after writing a small C library to unpack binary data from a string, to use Lua to read Microchip’s database of programming info for all the PICs. Using this it should be pretty easy to “script” the PICkit2 programmer using Lua. I still need a Lua interface to the host’s USB capability, so that’s another small library, and extra work to make it multiplatform.

Thanks are due to Microchip for publishing the source code for both the PICkit2 firmware and their host-side programming tool. Writing a Lua-based open-source programming tool would be nearly impossible without being able to refer to their code – and to use their PIC database!

While thinking and learning about USB I’ve been looking around for cheap & fun development hardware. There are several interesting options. I’ve collected details on some USB development boards.

2007 January 14 15:27

I was at Free Geek yesterday, trying to build myself a new machine. The one I picked had a bad on-board net interface, so I’m going to try again on Wednesday. Having fried (irrevocably write-protected) the BIOS chip on my oldish Athlon box, I thought while there I might look around for some Flash chips.

I managed to “harvest” about twenty Flash BIOS chips from a huge box of motherboards that were going to be recycled. It’s basically the only part on the board that isn’t soldered down. I pried them loose and scraped off the stickers, and ended up with about half 1Mbit (128k x 8) and half 2Mbit (256k x 8) chips, some of which are sector programmable – meaning that you can use them as little silicon disk drives! These will be great for some “old hardware” projects I’m tinkering with, involving 8085s (in one case), and an original Altair 8800 (in another).

I’d like the Altair to boot into Forth. The sector Flash can be used to store a binary boot image but also the source “blocks” (1k chunks of ASCII source code). Probably a waste of time, but I’ve been meaning to resurrect that machine for ages.

I’m still working on the PIC stuff too. ;-)

Read the 2006 journal.