Re: The Illusion of Certainty

Found this article on Design Observer:

The illusion of certainty

An interesting observation on how illusionary a structure imposed by form (or anything really) can be.

Quote:

... the rational side of our brains leads us to such solutions because they gesture to an odd kind of certainty. That tension — between structure and freedom, between form and its variation — is an essential characteristic of design thinking.

How new is the new Blogger ?

A quick question before I proceed to another post I was originally up to.


How new is the new Blogger compared to the old one, if to user experience they differ only in the login form ?

Do radars have screensavers ?

I was just wondering -



do radars have screensavers themselves ?

What difference does it make to know your people ?

A banner has just popped up - "monster.com, 40 000 000 resumes". Or something like that.

Imagine a database of 40 000 000 documents prepared by people who have strong incentive to lie. What would level of noise be ? How reliable would it be ? What amount of sifting through is required to find anyone ?

Compare this to how this company works - Core Search Group

A recruiter from them once contacted me, but not with a standard template offer, heck, not even with an offer at all. The guy has actually read my blog and (I'd guess) my published resume and addressed me with remarks about it that made sense ! I responded with a detailed explanation of who I am and how I work and this, beside my resume may be somewhere on their file.

I realize this is their work, but still, what difference does it make !

What did you do today ?

This one kept bugging me for a long time now.

It appears that many people hate the work they do. I can see how one can be forced to take a job he doesn't like, fate, bad luck, blah blah, but still, I totally don't understand this - how could you live if you cannot be proud of the work that you do ?

What do you say when you come home every day ? And I don't mean - to your family, but even to yourself, what do you say to yourself - I did what today ?

How comfortable it is when you can't point a finger and say "I made this" ?

When you are making crap, and everybody knows it, when people curse and spit when they encounter something that you've made, how do you feel ?

I do believe that most people still feel bad when then do their jobs wrong. Although they can find excuses and even blame somebody else, there has to be something, because if such behaviour was perfectly ok for them, they wouldn't have been looking for excuses and blaming others, and the more agression means more discomfort.

Which means - a lot of people hate their jobs, do them miserably and are at more or less permanent stress about that. Isn't that terrible ?

Never underestimate the power of randomness

I've just returned from a deep testing and debugging session and all I can say is again - wow ! never underestimate the power of randomness !

The system I was testing is a complex of network services build on top of Pythomnic platform with multiple Python processes scattered across multiple servers and intertwined together in redundant and fault-tolerant fashion. When it's live, it's going to be the billing hub service for the bank where I work. It has to deal with all sorts of payments to all sorts of providers. And so my job is to build a system to which modules for specific providers will be plugged later. It is also transferring money, so it'd better be reliable.

Someday I'm going to describe the design of that system as a case-study for Pythomnic and publish it on its web site. That will be, but for now, here is my recipe for the best testing:

Stress + failure injection + randomness

Stress: don't spare the system you are testing. The users will not. Give it as high load as it can handle and then some. It's no problem if it breaks now, but the amount of problems (not always bugs) that are revealed under unbearable load is surprising.

Failure injection: don't expect the problems to happen just because you are testing. Make them happen. Break stuff. Insert something like:

if random() < 0.01:
raise Exception("failure before provider request")

if random() < 0.001:
sleep(3600)
raise Exception("provider request hangs")

result = provider_request()

if random() < 0.01:
raise Exception("failure after provider request")

Insert it all over the place. Well, there is no point inserting failures between each two statements, it quickly gets cumbersome, but you should decorate each "external" call with such injected failure frame. It may be a database request, a specific API call etc.

Randomness: that's my favourite part, in testing, you can't beat randomness. You would never make up such a combination of failures that random() would. Make sure your random switches cover all the major code paths and let it running for a while. If it succeeds you can be pretty certain the system is working. To be sure, such random testing may not catch all of the special border cases in each of the modules, but for load testing - it's invaluable.

What eternal have we produced ?

For any system, living or artificial to span a long time it ought to change. Evolution, change over time are a sure signs of life. Rigidness, immutability and stasis are a no less sure signs of death.

Now, there is an interesting perspective to this. The strive to perfection leads us to creating something absolute, things that will exist forever. Any artist or craftsman would certainly like his creation to last for thousand years. But is eternal unchanged presence good ? By the way, have you ever seen anything eternal recently ?

Look around. Look for perfect. Indestructible. Eternal. See anything like this ?

Plastic, glass, and radiation.

Pretty much anything man-made will eventually disappear. Any piece of art will die out. Any building or construction will collapse. But not those three, not in any foreseeable future.

Every single grocery bag will stay forever. Empires rise and fall but the empty milk bottle will swamp around. Radiation will remain invisible, but the half-decay principle will ensure it will only vanish in asymptotically distant future.

Isn't it ironic ? Do we need eternal plastic bags ? No. Perfect milk bottles ? No. But this is what we've got. To make it worse, nothing really complex and useful can be done out of plastic or glass alone. Anything constructed with it will decay and leave around useless pieces of eternal stuff. Somehow we have finally managed to produce something that will outlast us million times. Something perfect. Something perfectly useless and dead.

Why having processing power wasted ?

I don't know about you, but seeing unused hardware troubles me. Doesn't matter if it's an old 386 desktop or a 8 years old Compaq Proliant 7000 which I've recently acquired.

Hardware is always fun to set up, see it running and play around with. Folklore about 100s refurbished Pentium's making up a cluster in a garage fascinates me. I do realize that a single modern server will eat that hundred for breakfast, but still, wouldn't that be fun, running that garage ?

And so, my working PC and the servers that are in testing or spare, are normally loaded up high most of the time, doing what - running workload tests of course ! Right now I'm leaving it running a stress test for Pythomnic, specifically its new capabilities of distributed transactions with recovery. The test will be running for a few days now, and I hope when I return from holidays it's still up and running.

JMS: lock on Java

I have to admit, I have only a basic knowledge of Java. And I wouldn't ever touch it. For some reason, when writing in Java, I feel like swimming in tar - possible, but hardly enjoyable. There may be some hidden beauty beyond my knowledge, but I simply don't want to spend time on getting deep into it, so deep that that beauty-of-possibility is revealed. And frankly, I don't believe there is one.

But then "industrial standard" kind of arguments bite, and all of a sudden there is a significant development around in Java. It's totally beyond me why solutions have to be adapted to technologies, not the other way around, but hey, it's real world.

And so, here comes another industrial standard - JMS. But, if you ask me, JMS is no industrial standard. It's a standard way of implementing messaging in Java world. In other words, JMS is a perfect example of a platform-specific technology, created with one thing in mind - to lock the developers to that platform. Many of the J-things are such, don't you think ?

See, JMS is Java Messaging Service, which asserts right from the start that anything non-Java is out. Should its designers have cared about interoperability and ease of integration, they would have standardized an open network protocol. How much easier life would have been if there was simply generic Message Queueing Protocol, say MQP based on TCP, optionally UDP, with very simple and clear semantics for swapping plaintext (i.e. XML) messages. By now we would have had tons of implementations in different languages, of better quality too. Not the case. Not only JMS transport may be based on RMI, Java-specific thing by definition, it's also focused on transferring instances of Java classes for messages and/or using messages for Java object to Java object communications. A perfect lock on Java. Pity.

Here is one more weirdness - in order to connect to a JMS server you have to know precisely what kind of JMS server software is run on it. Imagine that you'd have to install different software for sending your e-mail via SMTP server running Sendmail vs. Exchange. Sounds awful, but not for industrial standard it's not. What it means is two things - first, the JMS specification failed to provide implementation independence (I'm not sure that it was even supposed to do so), and two, with JMS you have to lock yourself not only to Java, but to particular server implementation, which is even worse. And I mean - theoretically, you should be able to modify your classpath, client configuration file - pf-f-f-t - and switch to a different implementation transparently, but then again - you have to explicitly switch to a particular server implementation, and different servers may have slight differences in behaviour (just love it), hence your clients may break anyhow.

What kind of industry requires such standards I wonder ?

Moscow: exploding buildings

They indeed failed to explode demolish a condemned building in Moscow. I told you.