December 28, 2007

Karlsson is from Sweden after all

You know Karlsson who lives on the roof. Everybody knows him.

Being a soviet kid, I've always had this impression that the "true" Karlsson appears in the well known soviet short animated film "Malysh and Karlsson", created in 1968. I've watched it countless times, just like any other soviet kid at the time.

Now I've had a chance to watch a newer 2002 Swedish full length version: "Karlsson pa taket".

What was really surprising is how similar the images of Malysh (the little one, in Russian adaptation) and Karlsson himself were. And not just those two, the entire visual style is strikingly similar. See for yourself.

Russian version:

Swedish version:

Excuse me, but I don't believe that two different teams of animators working at different times in different countries can come up with something so similar. What's going on here ?

Here is what I found.

The first clue is right there in the titles to the Swedish film - it is declared to be based on the work by Ilon Wikland. As it turns out to be, she has illustrated the first edition of the book back in 1955.

The design of characters in Russian version is ultimately attributed to Anatoly Savchenko, although at least here he is said to have been inspired by illustrations to the first Swedish edition of the book. You know, judging by the look of it, I wouldn't even call it inspired, rather based upon.

This is it then, the one and only Karlsson belongs to Ilon Wikland and not to the biggest soviet animation studio Soyuzmultfilm.

What is sad though is that the Russian version doesn't indicate the borrowing. I can see that at the time the Russian film was made it would have been a suicide to refer to a western source in Russian product. Still, it is very sad and obscuring fact.

November 30, 2007

Re: The end of America

Having listened to Naomi Wolf as she speaks about "10 steps to fascism" here:

and here is a supporting story in Guardian:,,2064157,00.html

I was applying the principles she suggests to current situation in Russia one by one, and surprisingly, they hardly applied:

1. Invoke a terrifying internal and external enemy.

Pass. None of those exist in modern Russia. Chechnen terrorists looked like a mixed ex/internal threat once, but then quickly diminished. No external power is considered a threat. There are "me too" kind of reactions to the America-declared global war of terrorism, such as the absurd requirement to take off one's shoes in the airports, but that looks like a totally random acts of power rather than a iron fist lead.

2. Create a gulag.

Well, I wouldn't know. In a country whose leader has invented gulag, I'm sure as hell there are secret prisons, but then they don't have to be secret, any would do. So, I'd say no, there is no gulag in modern Russia in the meaning of the word Naomi Wolf puts in it.

3. Develop a thug caste.

No such thing. Or, multiple such things, depending on what you mean. There is no single dedicated paramilitary force and none are emerging. There is army, state police, corporate security guards, and all sorts of criminal organisations, I'd presume. They all may apply force pursuing their arbitrary goals, but I don't think they orchestrate. This is not to obscure the fact that the police or army could at any point be given any orders.

4. Set up an internal surveillance system.

There easily could be files on anyone, just as with the mentioned Stasi, but I don't think there is a global network of surveillance and the percentage of informants has hopefully decreased since KGB time.

5. Harass citizens' groups.

Citizen groups ? What citizen groups ? None of active opposition to the power, that's for sure. Groups of political hobbyists and minorities of all sorts may be present, but noone worth mentioning with real power or a threat to power. Therefore there is no reason to persuasively harrass anyone. At any rate, they don't make a show out of it.

6. Engage in arbitrary detention and release.

It's not comforting, but I'd guess, yes, it's just like that. The stories of people being detained, kept in prisons, beaten and tortured appear every now and then. And the purpose could very well be the same - to terrorize and intimidate the entire population, even if in subtle way.

7. Target key individuals.

Check. Except, there is no key individuals. There are occassional public executions of someone who is sort of in opposition, but the truth is - there is no opposition. Oh, and nevermind the reporters murders.

8. Control the press.

Check. I mean, absolutely.

9. Dissent equals treason.

I'd answer this question if anyone could tell me what the today's Russia consent is ? Anything sacred ? Take Americans, they worship their democracy and freedom (ahem, given the steps to fascism title, this sounds awkward), but in Russia - what is the true way ? The way I see it, right now Russia is happily doing nothing, basically selling oil for food.

Actually, it's funny how I can't come up with anything that would sound plausible and treasonous at the same time. Brainwashed with no access to the facts - the most likely cause.

10. Suspend the rule of law.

Well, there is no martial law in Russia right now and I hope not to ever see it. We have laws and justice, right ? Although Russians are traditionally very sceptical to the laws and justice, but you can't argue that codices exist and trials work.

So, it sounded not so bad, right ? But wait until it comes to about minute 41 of the speech, and the answer should have been obvious in the context - the closed society doesn't look like one.

To quote Naomi Wolf:

We have this, like, wrong notion of what a closed society looks like. ... A closed society, even a violent military dictatorship, [looks a lot like] civil society, there are still elections, they are just corrupted. ... There is still a judiciary in a closed society, they are just not free to adjudicate freely. ... There's still academics, they're just watching what they're saying. There is still newspapers, you just know how far to push the enquiry.

Duh ! What can I say, welcome to the closed society. We'll have to see how it works out in America. Anyway, the speech was very thought-provocative.

November 19, 2007

On software reliability

Despite the common prejudice and the name, for a system to be reliable is not the same as to have no errors or never crash (noone should ever be promising that). The way I see it, reliabilty is more of a predictability. A system is reliable if it behaves in a predictable fashion - so much that you can rely on that. Even if all it does reliably is crashing.

The systems we build, they don't exist in isolation. Again, despite a popular myth, programmers don't pull things out of a thin air. We base our work on the work of others - hardware, operating systems, servers, frameworks, libraries, compilers, you name it. Reuse is the boon and the bane of the industry. Client-server pairs, interfaces and contracts are not just about OOP, they are everywhere. Any interaction between software components is about grabbing something somebody else has.

I therefore take it for granted, that there are parts of the system not written by us or not belonging to us. This implies that they cannot be controlled in any way (actually, I mostly work on integration middleware, which makes my experience worse). Then this inability to control leads to inability to fix. More often than not, you cannot fix problems in systems you have to depend on.

What do you do when you encounter an unexpected behaviour from somebody else's system you have to depend on ? I do this: first - fetch a cookie for being lucky, second - understand the cause, third - find a workaround.

The point here is - a problem known is not a problem. Because if you know the exact circumstances under which it hits, you can work around. Figuratively, you have to walk the minefield, but every previously found mine can be sidestepped. Therefore,

For a component to be reliable, you must be able to work around any problem encountered, and for a system to be reliable, you need to know all the problems it can cause.

To conclude, here is a few examples of something which is broken and reliable at the same time, because there is a workaround:
  • Reliable is a broken CPU which always fails upon certain floating point division. The numeric libraries fall back to software emulation when encountering this particular kind of chip.
  • Reliable is the XML parsing library which crashes when you attempt to set attribute value to something with letter "Ё" in it. I replace "Ё" with "Е", which is a good enough if slightly ambiguous substitute.
  • Reliable is the compiler which chokes and dies upon too complex a template. I rearrange the angle brackets and it goes on fine.
  • Reliable is the provider's server which crashes every other Friday at 13:00. I schedule an offline gap at that time and noone complains.
  • Moreover, reliable is the provider's server which crashes sporadically, but will gracefully handle repeated access attempts. As you might guess, this last example is the warmest to my heart and is one of the bases for Pythomnic - the platform for developing network services I'm developing and using.

November 16, 2007

November 07, 2007

The only way to get something done

... is to start and then don't stop.

October 19, 2007

Note to self: default parameter values are mutable in Python

Just hit a somewhat unexpected behaviour in Python code. What would the following code snippet print, what do you say ?
def foo(x = []):
print x

If you think about the def statement as a declaration, the answer is obvious - it should print
[ 'bar' ]
[ 'bar' ]
but in fact it prints
[ 'bar' ]
[ 'bar', 'bar' ]
Why ? Because in Python def is an executable statement, which means that the list of arguments for the method to be created (x) and most importantly their default values ([]) are themselves nothing but arguments to def. Something like this:
foo = def(x, default_x)
and when this gets executed, default_x is bound to something that at that point evaluates to an empty list, but remains mutable. Then, whenever the created foo is executed, the append method modifies the contents of default_x - effectively the "default value" of x.

This sounds strange, but is clearly documented in the language reference, quote from

Default parameter values are evaluated when the function definition is executed. This means that the expression is evaluated once, when the function is defined, and that that same "pre-computed" value is used for each call. This is especially important to understand when a default parameter is a mutable object, such as a list or a dictionary: if the function modifies the object (e.g. by appending an item to a list), the default value is in effect modified.
The spec suggests using None for all the default parameters, but you should have no problem using any immutable objects as well. For example:
def foo(x = None):
print (x or []) + [ "bar" ]
where x is None, or
def foo(x = ()):
# oops, no append method
print x + ( "bar", )
Phew, I've been lucky using None's so far...

October 08, 2007

Sympathy for Mr. Software

No software does what the user is supposed to be doing, because that is not known to the user in the first place. Should that have become known, the person could have been replaced with a machine. Instead, software helps people by making small and routine parts of their jobs easier to do. Even that it does imperfectly.

The users constantly struggle to make the software do what they want, the way they want. Sometimes they find the features useful the way they are. Sometimes they adapt to their quirks. Sometimes they find ways around. And sometimes they dump the whole thing.

As such, software is a dumb servant. A very dumb one. It needs assistance by itself.

To be useful, software needs assistance from the user. There must be a way for the user to explain what he wants even though the software may not have this capacity.

But then, returning to the dumb servant metaphor, the user must be willing to help. Therefore, here is my point - the software should be appealing to the user's sympathy or even pity. The user should be empathically connected to the software. To help and not to throw it away should be its first reaction.

I don't have any recipes on how to build such software. Arousing human's sympathy can be difficult even for another human, not for a piece of software. The only answer that I have is that the user should like the software for what seems to be nothing in particular.

The way I see it now, software should be written in such way that the user likes it for no apparent reason from the first sight. If it's pretty and it behaves consistently and it doesn't jump in your face and it knows when to speak and when to shut up and it looks familiar and it looks novel and it has square buttons and it has round buttons, then perhaps the user likes it. But you never know.

September 27, 2007

Note to self: exc_info is only available after except handler

It looks wrong to me, but in Python an exception is registered only when it hits the except handler, not immediately after it's thrown. In the following code snippet the first finally block is totally unaware of the exception.
import sys
raise SystemExit()
print "1:", sys.exc_info()
print "2:", sys.exc_info()
print "3:", sys.exc_info()
1: (None, None, None)
2: (<class exceptions.SystemExit ... )
3: (<class exceptions.SystemExit ... )
This is not how I would expect it to work.

Upon closer inspection of the sys module's documentation:

This function returns a tuple of three values that give information about the exception that is currently being handled.
Here, 'handling an exception' is defined as 'executing or having executed an except clause.'
This contradicts to my common sense, but it is the way it is, so I'll have to find another solution for the problem at hand.

September 25, 2007

Re: Backpack

Having listened to Jason Fried as he speaks about the process they use at 37signals:

I couldn't agree more to nearly all of his points. Also while listening to it, I had this associaction of the process with nothing but Brownian motion.

See, Jason essentially suggests that you have a small team of positive people in a small and lightweight cart and then allow every single individual product user push it in whichever direction he or she feels appropriate. A single user's push may not matter much, but if many of them push in the same direction, the cart moves. Then, this ease of feedback will also encourage the users to push more.

There are many other fine points in his speech and so I would definetely recommend to listen to it. Anyhow, here is a few other thoughts.

For one, the word "architecture" doesn't come up, but I do believe in architectures. Although architecture wouldn't emerge from the process outlined above, I thought that it would still present at the end, as one of the artifacts Jason calls "embraced constraints". These are restrictions that you enforce on your project in pursuit for optimal solutions. And architecture is just that.

For two, the described process is perfect for exactly this particular niche - web-based collaboration-like projects open for mass public access. It wouldn't work in many other cases, for example, if you build a project which you target towards big companies. Or, if the audience is too few, so that there is not enough user mass to push the cart.

For three, it seems logical to me that such brownian motion wouldn't last forever, the project trajectory would converge to some ideal point. And having such ideal point in mind from day one may be beneficial.

Otherwise, there was may be one thing that I don't agree with, and even that was minor and taken out of the context, it's when Jason says "get rid of boxes and arrows". While he probably meant functional diagrams in the context of the speech, getting rid of my favourite boxes and arrows feels scary. I would never give up such powerful mental instruments as boxes and arrows.

Overall, a great speech.

September 17, 2007

Re: To mistake is human

By programming we delegate the right to failure to the computer.

But the responsibility is still with us.

September 14, 2007

The most valuable pages of the World Wide Web

The World Wide Web contains a huge amount of files. Of those, HTML pages can point to each other (and other files too) with hyperlinks. Thus created hypertext structure can be presented with a directed graph.

Technically, the World Wide Web graph can contain cycles, but this is only possible if a page has been modified after it has been referenced. Specifically a page which has never been modified after it was created, cannot participate in a cycle. Therefore there could exist leaf pages, which are only linked to and not contain links themselves.

It is also seems reasonable that if page A links to page B, then the owner of page A somehow values page B, even if in some negative kind of way. Otherwise, he wouldn't even bother to put it there.

Now, the question is - aren't the most valuable pages of the World Wide Web the ones that are only referenced to but do not reference other pages themselves ? Taking it one step further, is a value of a page a function of N/M (where N is the number of the links to this page and M is the number of links from this page) ? Then a page with no links in it will indeed have infinite value.

September 03, 2007

Re: Tono Bungay

I've recently had a chance to read some Herbert Wells. It's been my opinion before that he was one of the greatest science fiction writers, a great visionary if you like. But it turns out that most of his novels are non-fiction. The ones that I've read are deeply inspirational and more interesting to read since they still have a strong connection to the real life, no matter that a hundred years have passed.

Quote from his Tono Bungay:

Always before these times the bulk of the people did not over-eat themselves, because they couldn't, whether they wanted to do so or not, and all but a very few were kept "fit" by unavoidable exercise and personal danger. Now, if only he pitch his standard low enough and keep free from pride, almost any one can achieve a sort of excess. You can go through contemporary life fudging and evading, indulging and slacking, never really hungry nor frightened nor passionately stirred, your highest moment a mere sentimental orgasm, and your first real contact with primary and elemental necessities, the sweat of your death-bed.

And this, mind you, is written in 1909.

July 03, 2007

Mirroring system drive under FreeBSD. Simply.

When configuring a new server, one of the first thing I do is build a RAID-1 mirror across the two hard drives that present in nearly all entry-level to mid-range servers. Good if you have a hardware RAID controller, but the cheaper models don't - they just have two fast (often SATA, not SCSI) drives to live with.

The question is - how do you mirror the system drive once the system has been installed ? Oh, did I mention it is FreeBSD ?

I used to follow the instructions here:

and it is good and correct and worked for me many times. Ralf S. Engelschall has undoubtedly put a lot of effort having the script even better over time. It essentially suggests that whenever you need to build a mirror across ad0 and ad1, you copy ad0 to ad1, then build a "mirror" with a single drive ad1 and reboot from it. After a reboot you have a "mirrored" drive ad1 as system and ad0 left aside. Then you add ad0 to the mirror, wait for it to resynchronize and reboot again. You also have to modify a few configuration files, calculate sizes of partitions etc. Such hassle is necessary because you presumably cannot add the system drive ad0 to a mirror when you have booted from it.

Unfortunately something went wrong and it didn't work with the server I had to build just recently. After the first reboot the gmirror provider failed to recognize the mirror on ad1 and boot process failed. Tried a few times - no luck. I guess the problem was in that both ad0 and ad1 have previously participated in a mirror, and I didn't have the drives cleaned as well I should, and some meta information left lurking on ad0. Anyhow, after some googling I found this other article

It manages to do the same thing with no hassle whatsoever. You simply add /dev/ad0 to mirror/gm0 and that's it. Oh, the little problem about not being able to execute gmirror label on the drive the system is currently booted from ? Easy - an undocumented (I'd presume) sysctl:

# sysctl kern.geom.debugflags=16

will rid of this nuisance. You simply set this debug sysctl and the mirror can be built right away. Somewhat an unclean solution but how much simpler.

May 28, 2007

You can't write everything by yourself - the missing link

I always tend to build software myself, not just coding, but also thinking about it, architecting it, experimenting with it and so on. It is always difficult to convince anyone in this approach compared to an off-the-shelf product reuse. Whenever such an argument comes up, I often hear this - "Well, you can't write everything by yourself, you have to use product X !" Indeed, it is impossible to rewrite an OS or a database engine, or a programming language, you have to reuse an existing piece of software. Not that I believe in a developer with a golden keyboard who writes infallible code (in Redmond, Bangalore or anywhere else) but doing so is just very impractical. Moreover, the truth is - you really can reuse just about anything, and doing so often gives good results. I never knew an answer to this, until now. I suddenly understood that it is this:

Even if you haven't written it yourself, you ought to know it so deep, as though you have.
The typical approach contradicts to this, you install something and just assume that it does what you need, exactly the way you need, that it will save you from all the problems you might possibly have. This is a mistake, a very convenient illusion. More often than not, it results in products being used in the wrong way, performing terribly, breaking on you, being cursed and thrown away only to be replaced with another of the same kind, only more expensive and fashionable.

It is therefore crucial to understand the working principles of the software that you reuse, to know the internals, what kind of problems to expect, and so on. And then, will you excuse me for repeating myself, but doing a lot of development on your own helps enormously in understanding the software written by somebody else.

April 13, 2007

Push a button, have ten bucks paid, repeat a thousand times

Consider a situation, where a customer pays for something on the Internet. There appears to be a huge perception difference between the client who pays and the provider who collects the payments. The impression that the two sides have on the scale of the affair is completely different.

See, if you are a customer and all you want is to pay $10 for something on the Internet, to you it's a matter of efficiency - through which hoops you have to jump to have it done and how fast you get the stuff you pay for. In an online transaction like that the money themselves don't matter much to a client, for the following reasons -

1. The client thinks of the electronic payment as of the payment with real money, which cannot be mishandled or require any processing which can be delayed or refused;

2. It's actually the merchandise or the service which the client wants at the time. The necessity to pay money, even online, is the mandatory inconvenience, an obstacle to it;

3. The amount of money in question is not that large. Even in the worst case the customer's risk is nearly zero.

I'm not saying a customer will tolerate losing money in an online transaction. What I'm saying is that at the moment of such transaction the client will not worry much about what could happen. To a client, it's all in "push a button, have ten bucks paid, how complicated such a simple procedure could possibly be ?"

And it wouldn't unless there were thousands of customers. When serving a single customer, it's easy to take the money, even manually, over a counter and process the payment, but processing a stream of thousand payments is different. Then you have a different perspective at the same $10.

When you are a provider, the problems that you face all have the same root - that you are a money pipe - anyone can use your services to buy something for themselves. What are the outcomes ?

1. Responsibility before customers. You simply cannot afford to fail. When you fail to deliver, the customers will haunt you, even for the same lousy $10. Dealing with this requires certain investments in reliability of the solution.

2. Freedom to be abused. A strong incentive is presented to the entire world to hack you and profit from that. This asks for security-oriented thinking.

3. Overwhelming complexity. Unlike the customer, you understand the guts of the service, and see the great many places in which any given payment can fail. And you have to maintain it.

See, from the provider's point of view the same ten bucks payment becomes nothing short to a hand grenade.

As it is my primary job to develop such solutions, I'm obviously on the provider's side. And since I'm a software developer, I have one more problem to deal with - the deceitfully simple outside look of the solution, remember - one button, ten bucks... Should the management adopt a customer's view, then for them it's a similar question of "how difficult a development of something that simple could possibly be ?".

But then, indeed, how difficult could it be to develop something that simple ?

March 13, 2007

Re: The Illusion of Certainty

Found this article on Design Observer:

The illusion of certainty

An interesting observation on how illusionary a structure imposed by form (or anything really) can be.


... the rational side of our brains leads us to such solutions because they gesture to an odd kind of certainty. That tension — between structure and freedom, between form and its variation — is an essential characteristic of design thinking.

How new is the new Blogger ?

A quick question before I proceed to another post I was originally up to.

How new is the new Blogger compared to the old one, if to user experience they differ only in the login form ?

March 06, 2007

Do radars have screensavers ?

I was just wondering -

do radars have screensavers themselves ?

March 01, 2007

What difference does it make to know your people ?

A banner has just popped up - ", 40 000 000 resumes". Or something like that.

Imagine a database of 40 000 000 documents prepared by people who have strong incentive to lie. What would level of noise be ? How reliable would it be ? What amount of sifting through is required to find anyone ?

Compare this to how this company works - Core Search Group

A recruiter from them once contacted me, but not with a standard template offer, heck, not even with an offer at all. The guy has actually read my blog and (I'd guess) my published resume and addressed me with remarks about it that made sense ! I responded with a detailed explanation of who I am and how I work and this, beside my resume may be somewhere on their file.

I realize this is their work, but still, what difference does it make !

February 26, 2007

What did you do today ?

This one kept bugging me for a long time now.

It appears that many people hate the work they do. I can see how one can be forced to take a job he doesn't like, fate, bad luck, blah blah, but still, I totally don't understand this - how could you live if you cannot be proud of the work that you do ?

What do you say when you come home every day ? And I don't mean - to your family, but even to yourself, what do you say to yourself - I did what today ?

How comfortable it is when you can't point a finger and say "I made this" ?

When you are making crap, and everybody knows it, when people curse and spit when they encounter something that you've made, how do you feel ?

I do believe that most people still feel bad when then do their jobs wrong. Although they can find excuses and even blame somebody else, there has to be something, because if such behaviour was perfectly ok for them, they wouldn't have been looking for excuses and blaming others, and the more agression means more discomfort.

Which means - a lot of people hate their jobs, do them miserably and are at more or less permanent stress about that. Isn't that terrible ?

February 20, 2007

Never underestimate the power of randomness

I've just returned from a deep testing and debugging session and all I can say is again - wow ! never underestimate the power of randomness !

The system I was testing is a complex of network services build on top of Pythomnic platform with multiple Python processes scattered across multiple servers and intertwined together in redundant and fault-tolerant fashion. When it's live, it's going to be the billing hub service for the bank where I work. It has to deal with all sorts of payments to all sorts of providers. And so my job is to build a system to which modules for specific providers will be plugged later. It is also transferring money, so it'd better be reliable.

Someday I'm going to describe the design of that system as a case-study for Pythomnic and publish it on its web site. That will be, but for now, here is my recipe for the best testing:

Stress + failure injection + randomness

Stress: don't spare the system you are testing. The users will not. Give it as high load as it can handle and then some. It's no problem if it breaks now, but the amount of problems (not always bugs) that are revealed under unbearable load is surprising.

Failure injection: don't expect the problems to happen just because you are testing. Make them happen. Break stuff. Insert something like:

if random() < 0.01:
raise Exception("failure before provider request")

if random() < 0.001:
raise Exception("provider request hangs")

result = provider_request()

if random() < 0.01:
raise Exception("failure after provider request")

Insert it all over the place. Well, there is no point inserting failures between each two statements, it quickly gets cumbersome, but you should decorate each "external" call with such injected failure frame. It may be a database request, a specific API call etc.

Randomness: that's my favourite part, in testing, you can't beat randomness. You would never make up such a combination of failures that random() would. Make sure your random switches cover all the major code paths and let it running for a while. If it succeeds you can be pretty certain the system is working. To be sure, such random testing may not catch all of the special border cases in each of the modules, but for load testing - it's invaluable.

January 18, 2007

What eternal have we produced ?

For any system, living or artificial to span a long time it ought to change. Evolution, change over time are a sure signs of life. Rigidness, immutability and stasis are a no less sure signs of death.

Now, there is an interesting perspective to this. The strive to perfection leads us to creating something absolute, things that will exist forever. Any artist or craftsman would certainly like his creation to last for thousand years. But is eternal unchanged presence good ? By the way, have you ever seen anything eternal recently ?

Look around. Look for perfect. Indestructible. Eternal. See anything like this ?

Plastic, glass, and radiation

Pretty much anything man-made will eventually disappear. Any piece of art will die out. Any building or construction will collapse. But not those three, not in any foreseeable future.

Every single grocery bag will stay forever. Empires rise and fall but the empty milk bottle will swamp around. Radiation will remain invisible, but the half-decay principle will ensure it will only vanish in asymptotically distant future.

Isn't it ironic ? Do we need eternal plastic bags ? No. Perfect milk bottles ? No. But this is what we've got. To make it worse, nothing really complex and useful can be done out of plastic or glass alone. Anything constructed with it will decay and leave around useless pieces of eternal stuff. Somehow we have finally managed to produce something that will outlast us million times. Something perfect. Something perfectly useless and dead.