Intercepting class method invocations using metaclass programming in Python

In Ruby, objects have a handy method called method_missing which allows one to handle method calls for methods that have not been defined. Most examples out there explain how to implement this in Python using __getattr__, however, none of them (honestly, none) explain how to intercept class method (@classmethod) invocations using __metaclass__.

And that’s why I wrote this post.

The function type is the built-in metaclass Python uses, it not only lets you know the type of an object, but also create classes on the fly. When you write, for example: class Example(object) the class object Example is not created in memory straight away. Python looks for the __metaclass__ attribute in the class definition and if it finds it, it uses it to create the object class Example. If it doesn’t, it uses type to create the class. The main purpose of a metaclass is to change the class automatically, when it’s created.

Here’s an example of how to use metaclass programming to intercept class method calls similar to the method_missing technique in Ruby:

class ClassMethodInterceptor(type):

    def __getattr__(cls, name):
        return lambda *args, **kwargs: \
                   cls.static_method_missing(name, *args, **kwargs)

    def static_method_missing(cls, method_name, *args, **kwargs):
        e = "type object 'static.%s' has no attribute '%s'" \
            % (cls.__name__, method_name)
        raise AttributeError(e)

class Example(object):

    __metaclass__ = ClassMethodInterceptor

    def __getattr__(self, name):
        return lambda *args, **kwargs: \
                   self.method_missing(name, *args, **kwargs)

    def method_missing(self, method_name, *args, **kwargs):
        e = "type object '%s' has no attribute '%s'" \
            % (self.__class__.__name__, method_name)
        raise AttributeError(e)

    @classmethod
    def static(cls):
        print 'static.%s' % cls.__name__

    def instance(self):
        print self.__class__.__name__

Console:

>>> Example.static()
static.Example
>>> Example.foo()
Traceback (most recent call last):
...
  File "example.py", line 12, in static_method_missing
    raise AttributeError(e)
AttributeError: type object 'static.Example' has no attribute 'foo'
>>> e = Example()
>>> e.instance()
Example
>>> e.foo()
Traceback (most recent call last):
...
  File "example.py", line 26, in method_missing
    raise AttributeError(e)

AttributeError: type object 'Example' has no attribute 'foo'

If you ever implement something like this, remember that Python doesn’t distinguish between methods and attributes the way Ruby does. There is no difference in Python between properties and methods. A method is just a property whose type is instancemethod.

Java, C, Python and nested loops

Java has no goto statement, to break or continue multiple-nested loop or switch constructs, Java programmers place labels on loop and switch constructs, and then break out of or continue to the block named by the label. The following example shows how to use java break statement to terminate the labeled loop:

public class BreakLabel
{
    public static void main(String[] args)
    {
        int[][] array = new int[][]{{1,2,3,4},{10,20,30,40}};
        boolean found = false;
        System.out.println("Searching 30 in two dimensional int array");

        Outer:
        for (int intOuter = 0; intOuter < array.length ; intOuter++) {
            Inner:
            for (int intInner = 0; intInner < array[intOuter].length; intInner++) {
                if (array[intOuter][intInner] == 30) {
                    found = true;
                    break Outer;
                }
            }
        }

        if (found == true) {
            System.out.println("30 found in the array");
        } else {
            System.out.println("30 not found in the array");
        }
    }
}

Use of labeled blocks in Java leads to considerable simplification in programming effort and a major reduction in maintenance.

On the other hand, the C continue statement can only continue the immediately enclosing block; to continue or exit outer blocks, programmers have traditionally either used auxiliary Boolean variables whose only purpose is to determine if the outer block is to be continued or exited; alternatively, programmers have misused the goto statement to exit out of nested blocks.

What’s interesting is that Python rejected the labeled break and continue proposal a while ago. And here’s why:

Guido van Rossum wrote:

I’m rejecting it on the basis that code so complicated to require this feature is very rare. While I’m sure there are some (rare) real cases where clarity of the code would suffer from a refactoring that makes it possible to use return, this is offset by two issues:

1. The complexity added to the language, permanently.

2. My expectation that the feature will be abused more than it will be used right, leading to a net decrease in code clarity (measured across all Python code written henceforth). Lazy programmers are everywhere, and before you know it you have an incredible mess on your hands of unintelligible code.

But what’s more interesting is that the idea of adding a goto statement was never mentioned. Common sense perhaps?

Redis: The Lamborghini of Databases

Antonio Cangiano wrote:

Redis is a key-value database written in C. It can be used like memcached, in front of a traditional database, or on its own thanks to the fact that the in-memory datasets are not volatile but instead persisted on disk. Redis provides you with the ability to define keys that are more than mere strings, as well as being able to handle multiple databases, lists, sets and even basic master-slave replication. It’s amazingly fast. On an entry level Linux box, Redis has been benchmarked performing 110,000 SET operations, and 81,000 GETs, per second.

Despite being a very young project, it already has client libraries for several languages: Python and PHP, Erlang, and Ruby. Salvatore Sanfilippo, the creator of Redis, has implemented a Twitter clone known as Retwis to showcase how you can use Redis and PHP to build applications without the need for a database like MySQL or any SQL query.

Salvatore will be publishing a beginner’s article based on the PHP Twitter clone he wrote, soon.

Full article: Introducing Redis: a fast key-value database

Fabric: Simple Pythonic Deployment Tool

Here’s the thing: you’re developing a server deployed application, it could be a web application but it doesn’t have to be, and you’re probably deploying to more than one server. Even if you just have one server to deploy to, it still get tiresome in the long run to build your project, fire up your favorite SFTP utility, upload your build, log in to the server with SSH, possibly stop the server, deploy the build, and finally start the server again.

What we’d like to do, is to build, upload and deploy our application with a single command line. Fabric is a tool that, at its core, logs into a number of hosts with SSH, and executes a set of commands, and possibly uploads or downloads files.

Fabric Project Site

Geo Proximity Search: The Haversine Equation

I’m working on a project that requires Geo proximity search. Basically, what I’m doing is plotting a radius around a point on a map, which is defined by the distance between two points on the map given their latitudes and longitudes. To achieve this I’m using the Haversine formula (spherical trigonometry). This equation is important in navigation, it gives great-circle distances between two points on a sphere from their longitudes and latitudes. You can see it in action here: Radius From UK Postcode.

This has already been covered in some blogs, however, I found most of the information to be inaccurate and, in some cases, incorrect. The Haversine equation is very straight forward, so there’s no need to complicate things.

I’ve implemented the solution in SQL, Python and PHP. Use the one that suits you best.

SQL implementation

set @latitude=53.754842;
set @longitude=-2.708077;
set @radius=20;

set @lng_min = @longitude - @radius/abs(cos(radians(@latitude))*69);
set @lng_max = @longitude + @radius/abs(cos(radians(@latitude))*69);
set @lat_min = @latitude - (@radius/69);
set @lat_max = @latitude + (@radius/69);

SELECT * FROM postcode
WHERE (longitude BETWEEN @lng_min AND @lng_max)
AND (latitude BETWEEN @lat_min and @lat_max);

Python implementation

from __future__ import division
import math

longitude = float(-2.708077)
latitude = float(53.754842)
radius = 20

lng_min = longitude - radius / abs(math.cos(math.radians(latitude)) * 69)
lng_max = longitude + radius / abs(math.cos(math.radians(latitude)) * 69)
lat_min = latitude - (radius / 69)
lat_max = latitude + (radius / 69)

print 'lng (min/max): %f %f' % (lng_min, lng_max)
print 'lat (min/max): %f %f' % (lat_min, lat_max)

PHP implementation

$longitude = (float) -2.708077;
$latitude = (float) 53.754842;
$radius = 20; // in miles

$lng_min = $longitude - $radius / abs(cos(deg2rad($latitude)) * 69);
$lng_max = $longitude + $radius / abs(cos(deg2rad($latitude)) * 69);
$lat_min = $latitude - ($radius / 69);
$lat_max = $latitude + ($radius / 69);

echo 'lng (min/max): ' . $lng_min . '/' . $lng_max . PHP_EOL;
echo 'lat (min/max): ' . $lat_min . '/' . $lat_max;

It outputs the same result:

lng (min/max): -3.1983251898421/-2.2178288101579
lat (min/max): 53.464986927536/54.044697072464

That’s it. Happy Geolocating!

Snake Wrangling for Kids

There’s a wide misconception that today’s technology and software have grown so complex that it’s impossible to teach children how to program them. Nonsense. “Snake Wrangling for Kids” is a printable electronic book, for children 8 years and older, who would like to learn computer programming. Written by Jason Briggs, the book covers the very basics of programming, and uses the Python 3 programming language to teach the concepts. There are 3 different versions of the book, one for Mac, one for Linux and one for Windows.

Jason Briggs is a writer with over 15 years experience in the IT industry. He has written for ONJava, ONLamp, Java Developers Journal, JavaWorld and Linux Devices, and worked on projects for clients such as HSBC Investment Bank UK.

Snake Wrangling for Kids

NautilusSVN: Linux TortoiseSVN Equivalent

Based on Stuart Langridge’s original script, Jason Field and Bruce van der Kooij created a set of Python scripts which integrate a load of Subversion functionality into the Gnome Nautilus browser. It’s basically a clone of the TortoiseSVN project on Windows.

NautilusSVN currently supports the following functionality:

  • Checkout
  • Commit
  • Revert
  • Diff (using Meld or gvimdiff)
  • Log Viewer
  • Revision and SVN User as columns in Nautilus views
  • Emblems to show file status (though buggy)
  • SSL authentication (buggy)
  • Username and password entry dialog
  • Editing Properties

NautilusSVN Project

No need for set/get methods in Python

Python code doesn’t typically use the get and set methods so common in PHP. Normally, when writing PHP code, you carefully protect your instance variables by making them private, so callers can only interact with them via getter and setter methods. For example:

class Book {
  private $title;

  public function setTitle($title) {
    $this->title = $title;
  }

  public function getTitle() {
    return $this->title;
  }
}

$book = new Book;
$book->title = 'Code Complete';

However, in PHP, the code above returns the following error:

Fatal error: Cannot access private property Book::$title

Python’s solution to this problem is more readable, it has a construct called a “property”. Properties are a neat way to implement attributes whose usage resembles attribute access, but whose implementation uses method calls. These are sometimes known as “managed attributes”. Basically, a property is a way to make a function call look like an instance variable reference. For example:

class Book {
  private $title = property(getTitle, setTitle);

  public function setTitle($title) {
    $this->title = $title;
  }

  public function getTitle() {
    return $this->title;
  }
}

$book = new Book;
$book->title = 'Code Complete';

Whenever someone writes $book->title, they’re really calling $book->getTitle(). Although the class has getters and setters, the callers of the class still get to use the original, simpler, easier-to-read syntax for accessing the value.

This way you get the best of both worlds: clean and simple client access to your class, and protection and flexibility within the class.

Examples

Python (run code)

class Book(object):

    def get_title(self):
        return self._title

    def set_title(self, title):
        self._title = title

    title = property(get_title, set_title)

Book.title = 'Code Complete'

PHP (run code)

class Book {
  private $title;

  public function __get($property) {
    $method = 'get' . ucfirst($property);
    if (!method_exists($this, $method)) {
      throw new Exception('No such method: ' . $method);
    }
    return $this->$method();
  }

  public function __set($property, $value) {
    $method = 'set' . ucfirst($property);
    if (!method_exists($this, $method)) {
      throw new Exception('No such method: ' . $method);
    }
    $this->$method($value);
  }

  public function setTitle($title) {
    $this->title = $title;
  }

  public function getTitle() {
    return $this->title;
  }
}

$book = new Book;
$book->title = 'Code Complete';

As you can see in the example above, PHP allows you to do something similar using the __get and __set methods, but can be a bit tricky to get right. Python’s “property” construct lets you do this painlessly.

Django 1.0 will be released in early September

Django has not been updated for a long time. The most current release, 0.96, was released in March 2007. This is a very long time when you look at the roadmap of other frameworks. But guess what, all that is about to change, because Django 1.0 is only two months away! Over the next six weeks sprints will take place all over the world to help get everything done by the deadline. This document details the schedule and roadmap towards Django 1.0. Although “Release early, release often” is not as easy as it seems, I hope it becomes part of Django’s development model.

The framework is now owned by a new, nonprofit, organization. The foundation, headed by Jacob Kaplan-Moss, will now be able to accept donations to help it grow. Finally we can say that Django is in the hands of the people who care most about its long-term success.

Django 1.0 Roadmap and Schedule