July 04, 2006

Open source, closed source, can anyone tell a good piece of software from a bad one ?

To start the discussion, let me ask you this - shall we release early and often ?

If we release early, we are likely to release crap, but have better exposure. But people are already used to software being crap. Besides, crap or not, there is no easy way for users to tell.

But if we go the long way of making the software better as we see it, and release later, the quality might be better, but it's still expected to be crap (release 1.0, anyone ?) and even though the experience might prove otherwise, the impression is still about the same. Besides, whi will care ?

Paradoxically, we are pushed towards producing crap faster.

Again, the key problem here is that we have no way of evaluating software quality. The closed source used to rely on the quality signals that, once been in favour have long since sold out - brands, advertisement, experts, reviews. The open source may rely on community judgement, popularity and zealotry, but what is it if not bandwagon effect ? Is it any better then ? I don't think so. Both sides rely on experience, but then again, experience is nothing but a way of convincing oneself.

Anyhow, my point is - open source, closed source, both parties have their reasons, politics and propaganda, both are eager to sell, but not only neither knows a way of producing high quality software, nobody can even know for sure if any given piece of software is good or not.

Think about it. We make stuff. We are proud artists (each a Picasso, no less). But when presented with somebody else's work, we cannot tell its quality. Isn't it outright ridiculous ? What kind of an industry is that ?

We thus are tackling the wrong problem. We need to build quality things. Instead we invent different ways of building things of unknown quality.

We desperately need reliable indicators of software quality.


Anonymous said...

Here's the simplest way. Pick ten features you'd like to add to your application but probably won't need for a year. Estimate how long it will take to do them. Wait six months and estimate them again. If the estimates are substantially higher, you're writing bad software.

Good software practice resists entropy. Bad practice lazily succumbs.

Dmitry Dvoinikov said...

This is indeed a good way of estimating progress of your own software, I agree. On the other hand I was talking about estimating somebody else's software quality, or even software quality in general.

Anonymous said...

One indication is the presence or absence of automated unit tests. When you have them, a lot of change is possible. When you don't, it's nearly impossible to refactor or add unexpected features without breaking stuff.

Dmitry Dvoinikov said...

True, automated unit tests are a good indication of a better software. But then again, it's a good practice for rolling out your own software.

It'd be great if more 3rd party software comes bundled with unit tests, but then we have chicken and egg problem - if you can't trust the the software, can you trust the results of the tests that say it's working fine ? They are written by the same people after all, and they do have strong incentive, well, may be not to lie per se, but at least to cut costs on testing.