“Learning Python”: A quick review

My voyages into the Python programming language have largely been courtesy of a text called “Learning Python”, written by Mark Lutz and published by O’Reilly and Associates.

It has to be said: as a text for an experienced programmer learning Python, it’s severely lacking. For a beginning programmer, it’s probably worse.

Its main problem is that it seems to have trouble deciding what it is: a reference or a tutorial. The text is written largely in a tutorial style, going over the details of the language. However, the chapters are arranged largely as a reference, with each component of the language given a chapter or two to itself. Actual programming exercises are left to the end of the section, where each “Section” consists of the set of chapters covering a particular component of the language.

Control constructs, for example, are not introduced until well after page 300 of the eText. They’re introduced after a hundred or so pages introducing you to how Python runs programs (summary: a bytecode) and another couple of hundred introducing the fundamental types.

The net result is that you read hundreds of pages of text without writing a single line of code beyond retyping the chapter examples. I have an above-average memory, but remembering a hundred pages of text in detail without using its contents is not something I would regard as a good pattern for teaching. You learn programming by writing code, and there is precious little of that in this text.

Another minor point is that it uses as its reference version of python the 3.x stream, whereas most of the Python code and libraries available today are from the 2.x stream. As Python 3.x is not entirely compatible (in either direction!) with Python 2.x, you spend most of your time focusing on a version of Python that is just not used very much. It does actually lay down the differences between versions whenever they crop up, but the reference version is 3.x.

Altogether I can’t recommend it. As soon as I find a better text I’ll do a similar review. For now your best bet is probably www.learnpython.org.

Python considered Pretty Good

One of the things I’m spending spare minutes on while looking for a job is filling in gaps in my resume with skills that employers are asking for that I’m currently lacking. The main things I’m looking at are Puppet – a widely used Configuration Management system – Python, a scripting language that’s been around for some years, and Powershell, Microsoft’s recent foray into decent scripting languages. The fact that all 3 start with “P” is an interesting but meaningless coincidence.

Puppet looks very straightforward offhand; it uses a configuration model very similar to Nagios, which I’ve been using for years. Powershell¬† I’ve used somewhat previously, but don’t know enough to code extensive projects. However, I’ve mainly been looking at Linux based positions so Powershell is the last of the three I’ll be concentrating on.

That leaves Python.

I’ve had the first edition of O’Reilly’s “Learning Python” decorating my bookshelf for some years without really cracking the spine. Recently I picked up the 4th edition as an ebook and have been working my way through it. The book takes a LONG time to get moving. It spends at least fifty pages telling you good reasons to learn the language and how to fire up the scripting engine. However, I’m starting to get into the meat of it now.

I have to say I’m impressed.

For some time my default scripting language has been a bastard mishmash of Bourne shell and awk. Awk can be remarkably powerful due to its associative arrays; the feature was inherited into Perl and also (as it turns out) as “dictionaries”, Python.

Python has a remarkably powerful set of basic types. In addition to the standard integer, floating point, string, boolean and so forth, it has arrays (natch), dictionaries (associative arrays, indexed by object), sets (collections of objects with no duplicates) and tuples, which are basically constant arrays that can be used as indices in dictionaries and representatives of sets. There are also a bunch of standard types in the shipped libraries to handle fractions, complex maths and fixed-point arithmetic. And others (such as files and function references).

There’s also some very nice syntactic sugar constructs that can be applied to make code powerful while still remaining fairly readable.

The very powerful core idea, however, is the extent to which Python combines the idea of types associated with objects rather than variables, and operator polymorphism.

This means that a basic function can be redefined to operate with a different class as long as the class has operations corresponding to the operators used by the class. In a language like C or Perl, if you want to pick a maximum value, your function has to be defined for each class:

int max(int var1, int var2)
{
    return (var1>var2) ? var1 : var2;
}

This needs to be defined separately for each type, so you have the same basic code for integers, floating point, and any other relevant types. Generally speaking, it also needs to be defined for each combination of types (depending on how strongly the language is typed).

In Python the type is associated with the object, so as long as you have your operators overloaded with the right types, you can simply call the function with the relevant arguments. The function looks at the object types, picks out the function or code that performs the correct comparisons, and does the job. It’s an extremely powerful approach.

In any case, I’m still working my way through the book so it will be a while before I’m fully familiar with the language. Liking it so far, however…

(The title of this article is a very indirect reference to¬†Dijkstra’s classic letter “goto considered harmful.” It’s been quite a while since I saw an actual goto in code, although modern languages have mechanisms that do similar things in a less horrible way.)

Update to the maze generator

I’ve updated the maze generator so you can now generate and navigate a 3D maze. The actual maze navigation part is written in dynamically generated javascript; you can save the generated HTML which will also save the maze itself, then play it back offline if you feel so inclined.

Just as the original program was my first really major PHP project, the javascript now incorporated into the script (and comprising about half the code) is my first significant javascript program. I will concede it’s not actually all that well written.

Still, given I knocked the javascript code and associated changes to the PHP code together in around 8-10 hours I think it’s adequate.

Source code is here. Maze page is there.

Mazes

Ever since I was a kid I’d had an odd fascination with mazes. Used to buy books of the things. After a while I started getting more interested in generating rather than solving them.

Waaaaay back in high school I wrote a simple maze generating program on my Z80-based VZ-200 (in BASIC). Later I revamped this on my Amiga and recoded it to show a 3D viewpoint.

Since I’m at loose ends and need to keep my coding skills sharp, I’ve revisited that old project and written a modern variant in PHP. The new version also has several options for hex-based (rather than rectangular grid) mazes; the programs written back in my school/university days lacked that option.

I’ll probably be adding a few additional features over the next week or two:

  • Add a 3D view, as well as an ability to navigate it.
  • Allow the 3D view to navigate a maze with three actual dimensions (i.e. three spatial coordinates). Also probably an option to have the 3D mazes avoid vertical connections where possible, as my old Amiga work demonstrated that people aren’t well equipped to navigate 3D spaces.
  • Eventually do the same for a 4D space. Navigating a hypercube strikes me as irredeemably cool.

I might also rejig the code into perl and/or python as an exercise in language familiarity.

This is actually the first major piece of PHP code I’ve written. In the past I’ve modified and hacked other peoples’ PHP code for various things, but this is my first project written from scratch. Fortunately PHP is a part of the same language family as C, perl and awk, all of which I know passably well.

It took about ten hours to code, which isn’t bad for a 400+ line debugged program in an unfamiliar language.

If you want to poke around the link is in the link bar, or else you can find it here.

A brief comparison of Virtualisation Solutions

If you’ve looked at virtualising a set of servers, you’ll be aware that there are a number of different solutions out on the market at the moment. I’ll break this up by “free” (no cash outlay) vs “commercial” solutions.

This is NOT a comprehensive review, just a quick overview of the particular platforms I’ve used.

Free:

  1. VMware – There’s a free solution available for a single host, but vSphere only allows a single hypervisor, so live migration is not possible. Hardware support is also not particularly broad. Performance is solid, however.
  2. Xen (Open Source version) – This was previously the default virtualisation platform built into RedHat and CentOS. Works well, but my tests showed that system performance was better on KVM unless the system was using a paravirtualised kernel (i.e. the kernel was talking to the hardware with minimal intervention by the hypervisor.) Live migration works with Xen under KVM.
  3. KVM – This is the “Linux native” virtualisation layer. Getting clustering working properly is a pain in the neck – in particular sharing a SAN is difficult to set up properly – but it is possible. However, as far as I am aware the free version of KVM does not have a good way to ensure that only a single instance of a VM is running when sharing resources across multiple hypervisor (host) systems. This may have changed since I last had to set up a KVM cluster a few years ago.

Overall, if picking a “free” solution, I would pick VMware if only using a single hypervisor host or KVM if clustering multiple hosts. My home lab uses VMware. This is partly because it allows nesting of VMs and I don’t want to run multiple hypervisors to reduce power requirements.

In a commercial environment, you should almost always run multiple hypervisors. This allows you to migrate guests between systems for server upgrades and patching, without requiring downtime for your guest systems.

Commercial:

  1. VMware – Solid and flexible, with decent performance. As the industry leaders, there are a LOT of tools around to help. Probably the easiest system for P2V (Physical to Virtual) conversions as well.
  2. Hyper-V – This is the Microsoft solution and as such lies on top of Windows 2008 or 2012. Windows 2012 includes substantial improvements. P2V tools are limited to recent releases of Windows. It is possible to build NEW non-Windows VMs, however. Seems to work well, but I can’t claim extensive experience.
  3. Xen – Citrix Xen is still available as a commercial product but I can’t comment on it beyond saying that hardware support seems a little limited.

While Hyper-V looks cheaper at first blush the VMWare Essentials package covers three hypervisors for a similar cost to three Windows 2012 licences. On the other hand, hardware support under Windows is hard to beat.

In general I would suggest VMWare but check hardware compatibility BEFORE you buy your hardware, and confirm with VMware directly if possible. If you don’t have control over your hardware, or you want your hypervisor to perform other tasks outside of a virtual environment, Hyper-V is a better solution.

Wireless insecurity

I’m not generally a fan of wireless networking. If you have the option of using a cabled network, you should probably take the option. This is partly because the radio spectrum available is shared between users, and so will never be as fast as an equivalent wired network; and partly because a wireless network can be cracked by somebody nearby and you have no easy way of knowing it’s happening.

Lifehacker has an article on cracking WPA protected networks which is worth a read. The motto: Turn off WPS if your router supports it. Ideally, permit access to systems by MAC address (most wireless routers support this). (If you’re using WEP… don’t. Switch to WPA2.)

I should add that wireless networking does have its place, but you should work on the assumption that if you’re using wireless then your network may not be secure.

Philosophy of tech support

A few observations on working in technical support.

  • Make sure you understand the user’s underlying need. Sometimes you’ll get a request asking to do one thing, when the user has actually taken the first few steps in thinking through solutions themselves, and is asking you to solve that problem rather than their real problem (which may be much simpler.) There may be a simpler, more elegant solution to their problem if you know what they’re trying to accomplish.
  • Make it easy for users to describe their problem. For example, keep network drive assignments consistent across the board (The T: drive should always point to the same share). Colour-code printer names; it’s a lot easier to say there is a fault with “the black printer” than with “the Kyocera 2100D near where Jacqui sits.”
  • Provide frequent feedback. If you are dealing with a fault, let the person requesting the fix know that it’s still being worked on, even if little progress is being made. Communication is key.
  • Underpromise and Overdeliver. When asked what benefits will be provided, be conservative about the benefits and timetable, but try to deliver more than was promised and ahead of time. If you promise a feature in one week and deliver in two, your users will be unhappy. If you promise delivery in three weeks and deliver in two, they will typically be happy. This leads to a related point…
  • Don’t overestimate yourself. I’ve found most technical staff (including myself) will underestimate the time taken for any given task. Most commonly this is because they haven’t taken the time needed for testing & debugging into account. When making an estimate, remember your limitations. For nontrivial tasks, your first impulse should be to double your initial estimate.
  • Complaints are golden. Any complaint is a vote of confidence in your ability to solve a problem. If your users are very quiet it may not be a good sign – your users may have given up on you as a source of help.
  • Don’t be afraid to inform users of the scope of a request. As technical staff, when asked to do something or if something is possible, our initial response is initially to say “yes, but…” People don’t hear the “but”; they only hear the “yes”. Couch your answer in terms of costs, or answer “effectively no”. If your answer is “That will take eight hours’ work; are you sure you don’t want me working on (some other important problem) instead?” most people will decline gracefully.
  • Don’t always turn down the hard requests. This is the flipside of ensuring that users know how hard a request is. It can sometimes be tempting to tell somebody that a job that is merely difficult is impossible.Taking those jobs will stretch your skills and, if done right, substantially benefit your user base.