June 17, 2008

This is Python, everything is executable

Dynamically typed language is by definition the one where variables don't have type, but the actual values do. This is by all means true in Python where the following code works fine
x = 10
x = "ten"
print(x) # prints ten
but limiting dynamism to untyped variables only would be missing the point.

Like with many "scripting" languages a Python program is started by passing the name of its main module to the "interpreter", such as
c:> python main.py
The transition of Python source code to an actually executed program begins with loading and parsing the module file. This step succeeds as soon as the module does not contain any syntax errors fatal for the parser. Successful parsing only guarantees that the module is not totally broken - a weak guarantee, only useful for checking for unbalanced parentheses and such.

What happens next is magic - the parsed module file is executed as though it was just a chunk of a source code. Wait a minute ! It is a chunk of a source code ! Anyway, execution of every module at its first import is the major part of Python program run. This process is identical no matter if the module being loaded is the program's main module or some other module explicitly imported by demand.

I have arrived to Python from C++, it took me a long time to change the perspective and the change is this - in Python you should look at everything as though it is an executable statement, because it really is. To illustrate this principle, consider definitions vs. declarations.

In statically typed languages, declarations exist for the sake of separate compilation - for the compiler to be able to tell whether one part of code is compatible with another without having to dig through the entire program. In Python, which is a dynamic language, there is no compilation stage, therefore declarations are useless, and what's left only looks like definitions.

For example, where in C++ you have two files (if you do it properly)
// foo.h                  // foo.cpp                         
class Foo int Foo::GetX(void) const
{ {
private: return x;
int x; }
public:
int GetX(void) const;
};
the .h file is a declaration - your promise to the compiler that you will provide the matching implementation and the .cpp file is that promise fulfilled. In Python there is no compiler so you don't have to feel obliged. Identical code in Python would be
class Foo:
def get_x(self):
return self.x
What you see in this Python code is neither a declaration nor a definition. It is a piece of executable code, which, when executed, introduces a new class to the containing module's namespace. Rewritten to its actual effect in pseudocode it would look like this:
class Foo:       temp1 = new class()

def get_x(self): temp2 = new method()
return self.x temp2.__code__ = return self.x
temp1["get_x"] = temp2

module["Foo"] = temp1
What you just saw was an illustration that a Python class definition is an executable statement, just like anything else and it executes once when the module is first imported. For example, it is possible to do something like C++'s conditional compilation:
class Foo:
if os.platform == "win32":
def do_it(self): # windows way
...
else:
def do_it(self): # unix way
...
The effect of the above code is that when the module is imported, the compiled version of class Foo will contain method do_it matching the current environment. It is not the same as the straightforward approach, where the check would have been performed upon each call to do_it:
class Foo:
def do_it(self):
if os.platform == "win32": # windows way
...
else: # unix way
...
In a similar vein, your class definition could fail to execute:
class Foo:
1 / 0 # this throws at import time
and the module will fail to import, throwing an exception to the caller.

Now it should not surprise you the least that when one module imports the other it is again not a declaration. When module foo does
import bar
the described process repeats for module bar, unless it has already been imported, in which case the import statement does nothing (from the discussed point of view). Similarly, you can import modules as you need them at runtime:
if need_time:
import time
print time.time()
Python therefore does not have any declarative semantics, only executional - ask yourself - what does it do when executed ?

To be continued...

June 16, 2008

This is Python, language installation and program structure

Installing Python is easy. If you use Windows, you have no choice at all - run setup.exe and you are done. Under Unix, Python can be preinstalled or you can install it manually. I always prefer to install from source on a clean machine, but if you have it preinstalled, you should be fine too.

Python installation is fully self-contained, and can be migrated to a different machine by copying all the files (or just the necessary ones) from c:\pythonXX or /usr/local/whatever/ to the destination. Multiple versions of Python coexist peacefully in different directories (although you should copy them around manually, because installation process registers stuff in the Windows registry and do other such things of global effect).

Python installation essentially contains the language parser+compiler and a huge and poorly structured standard library. The compiler itself along with a minimum set of libraries lives in pythonXX.dll or pythonXX.so.1, and the executable python.exe or bin/python is nothing but a simplest driver program of the (read line, execute, repeat) sort. The standard library lives in c:\pythonXX\lib + DLLs or /usr/local/lib/pythonXX/ and is just a heap of assorted utilities.

Python can be and is easily embedded into another application. It is a DLL, remember ? You take the DLL, zip the standard library and there you have it in two files - an embedded Python. In your application you create an instance of a compiler at runtime and start feeding it with stuff, that's all. Python can also be embedded into a diskless machine, it works just fine in a very restricted environment (such as the high security FreeBSD CD that I have here).

Python program consists of a set of separate modules, each module is a separate .py file containing some source code. The program is therefore available to the language in source, but Python nevertheless is not an interpreter. As each module is about to be used at runtime, it is loaded, parsed and compiled to an intermediate byte code for some virtual machine. The compiled byte code is saved alongside the original source file in an identically named .pyc file for future reuse. The outcome is the same as with Java or C# or any other language that translates source into byte code, and the difference is that in Python there is no separate compilation stage as such - the translation is performed at runtime and is in fact an important part of program execution.

To be continued...

June 11, 2008

This is Python, intro

Writing in Python for a few years now, I still get a kick of it. Wonderful and very powerful language, if used in the right way (aren't they all like that ?). No matter if code base is in megabytes, I still occasionally sit back in silence, admiring the beauty of a little code fragment or the way an idea is expressed in code.

If there is one single snippet of Python code to introduce its power of simplicity, it would be swapping of two variables. When I found it long ago in Python cookbook, it hit me like thunder. Never was my understanding of Python the same as before.

Here goes. To swap two variables in Python you need to

a, b = b, a

This utilizes Python feature called automatic tuple packing/unpacking. What's actually going on is more like

a, b <<< unpacked <<< (b, a) <<< packed <<< b, a

where (b, a) is a Python notion for an immutable sequence called tuple.

To be continued...

June 06, 2008

Slow pace of software development

If you need it fast, make sure it *looks* good, because it won't be. Pay more attention to high quality advertisement than to high quality development, but note that once you start selling promises, you will unlikely deliver a product.

May 22, 2008

Delphi is a very useful tool (with case study)

I've been working with Borland Delphi since 1996, when 2.0 was shiny new. Before that time I've been more or less formally taught Pascal and Delphi seemed a very welcome tool. And it still does, after 12 years. Strange thing is, whenever I mention using Delphi now, eyes are often rolled and "but it's not .NET" sounds. Know what ? I don't care about .NET. Pragmatically speaking, what matters is the product.

The very reason for this post is that the most recent GUI application that I've written was officially released a few days ago, and it is in Delphi. What's special about this particular application is that it comes really close to the ever sought perfect "one button" program. Ease of use was the major goal and what our group managed to produce you can see for yourself here:

Internetbank installer (in Russian!)

To give you the idea what it is for, it is a program to initiate access to remote banking web site for the bank of Severnaya kazna, which I happen to be working for right now. A client clicks the button, browser pops up and the client is brought to http://www.internetbank.ru/ from where he can manage his cards, accounts, transfers, payments etc.


Looks simple, except that before you can enter, you need to have started a cryptographically protected HTTP proxy server and provided with it a key, which is used for putting legally valid digital signatures under all your payments.

And that encryption key needs to be have been generated before. And the key expires and needs to be extended remotely. And CA certificates (the bank runs its own certification authority) also expire and need to be replaced remotely. And the user may have multiple keys to choose from.

And there might be problems with Internet access and the user needs a clear and concise yet technical enough diagnostics.

And it should run without installation from a removable drive with very restricted rights. And it should not require any runtime installation, such as .NET, but it must run on Windows 98.

And it must be remotely updatable.

And it must be simple to be usable by the most average Joe.

The result is the program works and really has just one button, which is also big and pretty. There is no confusion as to what to do next, because most of the time the user is simply not given any choice at all, clicking the only button starts the only process (of entering the protected web-site).

In a rare case when a user has to initiate some other process, such as generating a new key, the reasons for that are clearly explained right there on the program's face. Hints and careful label wording help at least those users who can and actually are reading.

Looking back I realize that the only reason why such a simple solution appeared is three years of refining the procedure. During last three years our customers had to use different software, clumsy and inconvenient (despite being also written by me). But in three years we learned all the moves and most of the problems. And then a strategic decision was made and we trashed the old software and rolled out the new one. And I like what we did.

Anyhow, returning to Delphi.

I'm using it for what it does best - pretty GUI with access to OS services or 3rd party libraries. Skin support (as well as good taste) makes programs really good looking. Being pretty is a major advantage for end-user software. Positive emotional reaction would actually help the user to deal with it.

Aside from that, Delphi programs don't require any *cough* runtime platform but run on Windows 98. And Delphi produces reasonably lightweight executables. And you don't need administrative rights to run them. And I like it.

Great stuff.

May 18, 2008

Just in: the benefits of compiling into platform-independent byte code


After the existence of Pascal became known (in 1974), several people asked us for assistance in implementing Pascal on various other machines [...] Thereupon we decided to provide a compiler version that would generate code for a machine of our own design. This code later became known as P-code. [...] Had we possessed the wisdom to foresee the dimension of this movement, we would have put more effort and care into designing and documenting P-code.

-- Niklaus Wirth, 1985

May 13, 2008

Just in: the only way to reliability is through simplicity and that can't be bought

Almost anything in software can be implemented, sold, and even used given enough determination. There is nothing a mere scientist can say that will stand against the flood of a hundred million dollars. But there is one quality that cannot be purchased in this way - and that is reliability. The price of reliability is the pursuit of the utmost simplicity. It is a price which the very rich find most hard to pay.

-- C.A.R. Hoare, 1981

О самописных программах

There is a Russian word "самописный" [səmopisni], literally meaning - "written by oneself". The word has a strong disparaging tone in it and is typically used to describe the low quality software written by one's colleagues (or even oneself), as opposed to the high quality truly industrial commercial software written by people somewhere else, even better if purchased for a huge chunk of money.

The mere existence of such word and the attitude of programmers who use it in derogative way is totally beyond me. The programmers who don't appreciate and value the work they do, how good a software they really write ? Moreover, how can they evaluate the quality of somebody else's work if they despise their own ? Besides, the absurdity of the situation is in that all the programs are really of this sort as someone has ultimately written them.

A concluding quote from a nobleman:

Clearly IBM and MIT must be possessed of some secret of successful software design and implementation whose nature I could not even begin to guess at. It was only later that they realized they could not either.

-- C.A.R. Hoare, 1981

May 11, 2008

Just in: choose your language carefully

I have regarded it as the highest goal of programming language design to enable good ideas to be elegantly expressed.

-- C.A.R. Hoare, 1981

Language designers also have an obligation to provide languages that encourage good style, since we all know that style is strongly influenced by the language in which it is expressed.

-- Donald Knuth, 1974

I have the feeling that one of the most important aspects of any computing tool is its influence on the thinking habits of those who try to use it.

-- E.W.Dijkstra, 1972

Just in: permanently low quality of software

I feel that all too often we have been satisfied with such a low level of quality that we have done ourselves harm in the process. We seem not to be able to use the machine, which we all believe is a very powerful tool for manipulating and transforming information, to do our own tasks in this very field.

-- R. W. Hamming, 1969