The only way to get something done

... is to start and then don't stop.

Note to self: default parameter values are mutable in Python

Just hit a somewhat unexpected behaviour in Python code. What would the following code snippet print, what do you say ?

def foo(x = []):
    x.append("bar")
    print x

foo()
foo()

If you think about the def statement as a declaration, the answer is obvious - it should print

[ 'bar' ]
[ 'bar' ]

but in fact it prints

[ 'bar' ]
[ 'bar', 'bar' ]

Why ? Because in Python def is an executable statement, which means that the list of arguments for the method to be created (x) and most importantly their default values ([]) are themselves nothing but arguments to def. Something like this:

foo = def(x, default_x)

and when this gets executed, default_x is bound to something that at that point evaluates to an empty list, but remains mutable. Then, whenever the created foo is executed, the append method modifies the contents of default_x - effectively the "default value" of x.

This sounds strange, but is clearly documented in the language reference, quote from http://docs.python.org/ref/function.html

Default parameter values are evaluated when the function definition is executed. This means that the expression is evaluated once, when the function is defined, and that that same "pre-computed" value is used for each call. This is especially important to understand when a default parameter is a mutable object, such as a list or a dictionary: if the function modifies the object (e.g. by appending an item to a list), the default value is in effect modified.

The spec suggests using None for all the default parameters, but you should have no problem using any immutable objects as well. For example:

def foo(x = None):
    print (x or []) + [ "bar" ]

where x is None, or

def foo(x = ()):
    # oops, no append method
    print x + ( "bar", )

Phew, I've been lucky using None's so far...

Sympathy for Mr. Software

No software does what the user is supposed to be doing, because that is not known to the user in the first place. Should that have become known, the person could have been replaced with a machine. Instead, software helps people by making small and routine parts of their jobs easier to do. Even that it does imperfectly.

The users constantly struggle to make the software do what they want, the way they want. Sometimes they find the features useful the way they are. Sometimes they adapt to their quirks. Sometimes they find ways around. And sometimes they dump the whole thing.

As such, software is a dumb servant. A very dumb one. It needs assistance by itself.

To be useful, software needs assistance from the user. There must be a way for the user to explain what he wants even though the software may not have this capacity.

But then, returning to the dumb servant metaphor, the user must be willing to help. Therefore, here is my point - the software should be appealing to the user's sympathy or even pity. The user should be empathically connected to the software. To help and not to throw it away should be its first reaction.

I don't have any recipes on how to build such software. Arousing human's sympathy can be difficult even for another human, not for a piece of software. The only answer that I have is that the user should like the software for what seems to be nothing in particular.

The way I see it now, software should be written in such way that the user likes it for no apparent reason from the first sight. If it's pretty and it behaves consistently and it doesn't jump in your face and it knows when to speak and when to shut up and it looks familiar and it looks novel and it has square buttons and it has round buttons, then perhaps the user likes it. But you never know.

Note to self: exc_info is only available after except handler

It looks wrong to me, but in Python an exception is registered only when it hits the except handler, not immediately after it's thrown. In the following code snippet the first finally block is totally unaware of the exception.

import sys
try:
    try:
        try:
            raise SystemExit()
        finally:
            print "1:", sys.exc_info()
    except:
        print "2:", sys.exc_info()
        raise
finally:
    print "3:", sys.exc_info()

prints

1: (None, None, None)
2: (<class exceptions.SystemExit ... )
3: (<class exceptions.SystemExit ... )

This is not how I would expect it to work.

Upon closer inspection of the sys module's documentation:

exc_info()
This function returns a tuple of three values that give information about the exception that is currently being handled.
Here, 'handling an exception' is defined as 'executing or having executed an except clause.'

This contradicts to my common sense, but it is the way it is, so I'll have to find another solution for the problem at hand.

Re: Backpack

Having listened to Jason Fried as he speaks about the process they use at 37signals:

http://itc.conversationsnetwork.org/shows/detail471.html

I couldn't agree more to nearly all of his points. Also while listening to it, I had this associaction of the process with nothing but Brownian motion.

See, Jason essentially suggests that you have a small team of positive people in a small and lightweight cart and then allow every single individual product user push it in whichever direction he or she feels appropriate. A single user's push may not matter much, but if many of them push in the same direction, the cart moves. Then, this ease of feedback will also encourage the users to push more.

There are many other fine points in his speech and so I would definetely recommend to listen to it. Anyhow, here is a few other thoughts.

For one, the word "architecture" doesn't come up, but I do believe in architectures. Although architecture wouldn't emerge from the process outlined above, I thought that it would still present at the end, as one of the artifacts Jason calls "embraced constraints". These are restrictions that you enforce on your project in pursuit for optimal solutions. And architecture is just that.

For two, the described process is perfect for exactly this particular niche - web-based collaboration-like projects open for mass public access. It wouldn't work in many other cases, for example, if you build a project which you target towards big companies. Or, if the audience is too few, so that there is not enough user mass to push the cart.

For three, it seems logical to me that such brownian motion wouldn't last forever, the project trajectory would converge to some ideal point. And having such ideal point in mind from day one may be beneficial.

Otherwise, there was may be one thing that I don't agree with, and even that was minor and taken out of the context, it's when Jason says "get rid of boxes and arrows". While he probably meant functional diagrams in the context of the speech, getting rid of my favourite boxes and arrows feels scary. I would never give up such powerful mental instruments as boxes and arrows.

Overall, a great speech.

Re: To mistake is human

By programming we delegate the right to failure to the computer.

But the responsibility is still with us.

The most valuable pages of the World Wide Web

The World Wide Web contains a huge amount of files. Of those, HTML pages can point to each other (and other files too) with hyperlinks. Thus created hypertext structure can be presented with a directed graph.

Technically, the World Wide Web graph can contain cycles, but this is only possible if a page has been modified after it has been referenced. Specifically a page which has never been modified after it was created, cannot participate in a cycle. Therefore there could exist leaf pages, which are only linked to and not contain links themselves.

It is also seems reasonable that if page A links to page B, then the owner of page A somehow values page B, even if in some negative kind of way. Otherwise, he wouldn't even bother to put it there.

Now, the question is - aren't the most valuable pages of the World Wide Web the ones that are only referenced to but do not reference other pages themselves ? Taking it one step further, is a value of a page a function of N/M (where N is the number of the links to this page and M is the number of links from this page) ? Then a page with no links in it will indeed have infinite value.

Re: Tono Bungay

I've recently had a chance to read some Herbert Wells. It's been my opinion before that he was one of the greatest science fiction writers, a great visionary if you like. But it turns out that most of his novels are non-fiction. The ones that I've read are deeply inspirational and more interesting to read since they still have a strong connection to the real life, no matter that a hundred years have passed.

Quote from his Tono Bungay:

Always before these times the bulk of the people did not over-eat themselves, because they couldn't, whether they wanted to do so or not, and all but a very few were kept "fit" by unavoidable exercise and personal danger. Now, if only he pitch his standard low enough and keep free from pride, almost any one can achieve a sort of excess. You can go through contemporary life fudging and evading, indulging and slacking, never really hungry nor frightened nor passionately stirred, your highest moment a mere sentimental orgasm, and your first real contact with primary and elemental necessities, the sweat of your death-bed.

And this, mind you, is written in 1909.

Mirroring system drive under FreeBSD. Simply.

When configuring a new server, one of the first thing I do is build a RAID-1 mirror across the two hard drives that present in nearly all entry-level to mid-range servers. Good if you have a hardware RAID controller, but the cheaper models don't - they just have two fast (often SATA, not SCSI) drives to live with.

The question is - how do you mirror the system drive once the system has been installed ? Oh, did I mention it is FreeBSD ?

I used to follow the instructions here:

http://people.freebsd.org/~rse/mirror/

and it is good and correct and worked for me many times. Ralf S. Engelschall has undoubtedly put a lot of effort having the script even better over time. It essentially suggests that whenever you need to build a mirror across ad0 and ad1, you copy ad0 to ad1, then build a "mirror" with a single drive ad1 and reboot from it. After a reboot you have a "mirrored" drive ad1 as system and ad0 left aside. Then you add ad0 to the mirror, wait for it to resynchronize and reboot again. You also have to modify a few configuration files, calculate sizes of partitions etc. Such hassle is necessary because you presumably cannot add the system drive ad0 to a mirror when you have booted from it.

Unfortunately something went wrong and it didn't work with the server I had to build just recently. After the first reboot the gmirror provider failed to recognize the mirror on ad1 and boot process failed. Tried a few times - no luck. I guess the problem was in that both ad0 and ad1 have previously participated in a mirror, and I didn't have the drives cleaned as well I should, and some meta information left lurking on ad0. Anyhow, after some googling I found this other article

http://www.onlamp.com/pub/a/bsd/2005/11/10/FreeBSD_Basics.html

It manages to do the same thing with no hassle whatsoever. You simply add /dev/ad0 to mirror/gm0 and that's it. Oh, the little problem about not being able to execute gmirror label on the drive the system is currently booted from ? Easy - an undocumented (I'd presume) sysctl:

# sysctl kern.geom.debugflags=16

will rid of this nuisance. You simply set this debug sysctl and the mirror can be built right away. Somewhat an unclean solution but how much simpler.

You can't write everything by yourself - the missing link

I always tend to build software myself, not just coding, but also thinking about it, architecting it, experimenting with it and so on. It is always difficult to convince anyone in this approach compared to an off-the-shelf product reuse. Whenever such an argument comes up, I often hear this - "Well, you can't write everything by yourself, you have to use product X !" Indeed, it is impossible to rewrite an OS or a database engine, or a programming language, you have to reuse an existing piece of software. Not that I believe in a developer with a golden keyboard who writes infallible code (in Redmond, Bangalore or anywhere else) but doing so is just very impractical. Moreover, the truth is - you really can reuse just about anything, and doing so often gives good results. I never knew an answer to this, until now. I suddenly understood that it is this:

Even if you haven't written it yourself, you ought to know it so deep, as though you have.

The typical approach contradicts to this, you install something and just assume that it does what you need, exactly the way you need, that it will save you from all the problems you might possibly have. This is a mistake, a very convenient illusion. More often than not, it results in products being used in the wrong way, performing terribly, breaking on you, being cursed and thrown away only to be replaced with another of the same kind, only more expensive and fashionable.

It is therefore crucial to understand the working principles of the software that you reuse, to know the internals, what kind of problems to expect, and so on. And then, will you excuse me for repeating myself, but doing a lot of development on your own helps enormously in understanding the software written by somebody else.