Python considered Pretty Good

One of the things I’m spending spare minutes on while looking for a job is filling in gaps in my resume with skills that employers are asking for that I’m currently lacking. The main things I’m looking at are Puppet – a widely used Configuration Management system – Python, a scripting language that’s been around for some years, and Powershell, Microsoft’s recent foray into decent scripting languages. The fact that all 3 start with “P” is an interesting but meaningless coincidence.

Puppet looks very straightforward offhand; it uses a configuration model very similar to Nagios, which I’ve been using for years. Powershell¬† I’ve used somewhat previously, but don’t know enough to code extensive projects. However, I’ve mainly been looking at Linux based positions so Powershell is the last of the three I’ll be concentrating on.

That leaves Python.

I’ve had the first edition of O’Reilly’s “Learning Python” decorating my bookshelf for some years without really cracking the spine. Recently I picked up the 4th edition as an ebook and have been working my way through it. The book takes a LONG time to get moving. It spends at least fifty pages telling you good reasons to learn the language and how to fire up the scripting engine. However, I’m starting to get into the meat of it now.

I have to say I’m impressed.

For some time my default scripting language has been a bastard mishmash of Bourne shell and awk. Awk can be remarkably powerful due to its associative arrays; the feature was inherited into Perl and also (as it turns out) as “dictionaries”, Python.

Python has a remarkably powerful set of basic types. In addition to the standard integer, floating point, string, boolean and so forth, it has arrays (natch), dictionaries (associative arrays, indexed by object), sets (collections of objects with no duplicates) and tuples, which are basically constant arrays that can be used as indices in dictionaries and representatives of sets. There are also a bunch of standard types in the shipped libraries to handle fractions, complex maths and fixed-point arithmetic. And others (such as files and function references).

There’s also some very nice syntactic sugar constructs that can be applied to make code powerful while still remaining fairly readable.

The very powerful core idea, however, is the extent to which Python combines the idea of types associated with objects rather than variables, and operator polymorphism.

This means that a basic function can be redefined to operate with a different class as long as the class has operations corresponding to the operators used by the class. In a language like C or Perl, if you want to pick a maximum value, your function has to be defined for each class:

int max(int var1, int var2)
{
    return (var1>var2) ? var1 : var2;
}

This needs to be defined separately for each type, so you have the same basic code for integers, floating point, and any other relevant types. Generally speaking, it also needs to be defined for each combination of types (depending on how strongly the language is typed).

In Python the type is associated with the object, so as long as you have your operators overloaded with the right types, you can simply call the function with the relevant arguments. The function looks at the object types, picks out the function or code that performs the correct comparisons, and does the job. It’s an extremely powerful approach.

In any case, I’m still working my way through the book so it will be a while before I’m fully familiar with the language. Liking it so far, however…

(The title of this article is a very indirect reference to¬†Dijkstra’s classic letter “goto considered harmful.” It’s been quite a while since I saw an actual goto in code, although modern languages have mechanisms that do similar things in a less horrible way.)

Update to the maze generator

I’ve updated the maze generator so you can now generate and navigate a 3D maze. The actual maze navigation part is written in dynamically generated javascript; you can save the generated HTML which will also save the maze itself, then play it back offline if you feel so inclined.

Just as the original program was my first really major PHP project, the javascript now incorporated into the script (and comprising about half the code) is my first significant javascript program. I will concede it’s not actually all that well written.

Still, given I knocked the javascript code and associated changes to the PHP code together in around 8-10 hours I think it’s adequate.

Source code is here. Maze page is there.

Mazes

Ever since I was a kid I’d had an odd fascination with mazes. Used to buy books of the things. After a while I started getting more interested in generating rather than solving them.

Waaaaay back in high school I wrote a simple maze generating program on my Z80-based VZ-200 (in BASIC). Later I revamped this on my Amiga and recoded it to show a 3D viewpoint.

Since I’m at loose ends and need to keep my coding skills sharp, I’ve revisited that old project and written a modern variant in PHP. The new version also has several options for hex-based (rather than rectangular grid) mazes; the programs written back in my school/university days lacked that option.

I’ll probably be adding a few additional features over the next week or two:

  • Add a 3D view, as well as an ability to navigate it.
  • Allow the 3D view to navigate a maze with three actual dimensions (i.e. three spatial coordinates). Also probably an option to have the 3D mazes avoid vertical connections where possible, as my old Amiga work demonstrated that people aren’t well equipped to navigate 3D spaces.
  • Eventually do the same for a 4D space. Navigating a hypercube strikes me as irredeemably cool.

I might also rejig the code into perl and/or python as an exercise in language familiarity.

This is actually the first major piece of PHP code I’ve written. In the past I’ve modified and hacked other peoples’ PHP code for various things, but this is my first project written from scratch. Fortunately PHP is a part of the same language family as C, perl and awk, all of which I know passably well.

It took about ten hours to code, which isn’t bad for a 400+ line debugged program in an unfamiliar language.

If you want to poke around the link is in the link bar, or else you can find it here.

A brief comparison of Virtualisation Solutions

If you’ve looked at virtualising a set of servers, you’ll be aware that there are a number of different solutions out on the market at the moment. I’ll break this up by “free” (no cash outlay) vs “commercial” solutions.

This is NOT a comprehensive review, just a quick overview of the particular platforms I’ve used.

Free:

  1. VMware – There’s a free solution available for a single host, but vSphere only allows a single hypervisor, so live migration is not possible. Hardware support is also not particularly broad. Performance is solid, however.
  2. Xen (Open Source version) – This was previously the default virtualisation platform built into RedHat and CentOS. Works well, but my tests showed that system performance was better on KVM unless the system was using a paravirtualised kernel (i.e. the kernel was talking to the hardware with minimal intervention by the hypervisor.) Live migration works with Xen under KVM.
  3. KVM – This is the “Linux native” virtualisation layer. Getting clustering working properly is a pain in the neck – in particular sharing a SAN is difficult to set up properly – but it is possible. However, as far as I am aware the free version of KVM does not have a good way to ensure that only a single instance of a VM is running when sharing resources across multiple hypervisor (host) systems. This may have changed since I last had to set up a KVM cluster a few years ago.

Overall, if picking a “free” solution, I would pick VMware if only using a single hypervisor host or KVM if clustering multiple hosts. My home lab uses VMware. This is partly because it allows nesting of VMs and I don’t want to run multiple hypervisors to reduce power requirements.

In a commercial environment, you should almost always run multiple hypervisors. This allows you to migrate guests between systems for server upgrades and patching, without requiring downtime for your guest systems.

Commercial:

  1. VMware – Solid and flexible, with decent performance. As the industry leaders, there are a LOT of tools around to help. Probably the easiest system for P2V (Physical to Virtual) conversions as well.
  2. Hyper-V – This is the Microsoft solution and as such lies on top of Windows 2008 or 2012. Windows 2012 includes substantial improvements. P2V tools are limited to recent releases of Windows. It is possible to build NEW non-Windows VMs, however. Seems to work well, but I can’t claim extensive experience.
  3. Xen – Citrix Xen is still available as a commercial product but I can’t comment on it beyond saying that hardware support seems a little limited.

While Hyper-V looks cheaper at first blush the VMWare Essentials package covers three hypervisors for a similar cost to three Windows 2012 licences. On the other hand, hardware support under Windows is hard to beat.

In general I would suggest VMWare but check hardware compatibility BEFORE you buy your hardware, and confirm with VMware directly if possible. If you don’t have control over your hardware, or you want your hypervisor to perform other tasks outside of a virtual environment, Hyper-V is a better solution.