Metaclasses in Five Minutes

http://www.voidspace.org.uk/python/articles/five-minutes.shtml

 

Metaclasses in Five Minutes

• Everything is an Object
• Honestly, it's True
• The Class of a Class is...
• So when you create a class...
• Easy to Demonstrate
• The Magic of __metaclass__
• In Action...
• What can we do with this?
• But...

Originally a lightning talk first given at PyCon UK 2008.

Metaclasses have a reputation for being 'deep-black-magic' in Python. The cases where you need them are genuinely rare (unless you program with Zope...), but the basic principles are surprisingly easy to understand.

Everything is an Object

• Everything is an object
• Everything has a type
• No real difference between 'class' and 'type'
• Classes are objects
• Their type is type

Typically the term type is used for the built-in types and the term class for user-defined classes. Since Python 2.2 there has been no real difference and 'class' and 'type' are synonyms.

For classic (old-style) classes, their type is types.ClassType.

Honestly, it's True

Python  2.5 .1  ( r251 : 54869 ,  Apr  18  2007 ,  22 : 0 8 : 04 )
>> >  class  Something ( object ) :
. . .      pass
. . .
>> >  Something
< class  '__main__.Something' >
>> >  type ( Something )
< type  'type' >

Here we can see that a class created at the interactive interpreter is a first class object.

The Class of a Class is...

Its metaclass...

Just as an object is an instance of its class; a class is an instance of its metaclass.

The metaclass is called to create the class.

In exactly the same way as any other object in Python.

So when you create a class...

The interpreter calls the metaclass to create it...

For a normal class that inherits from object this means that type is called to create the class:

>>> help(type)
Help on class type in module __builtin__:

class type(object)
 |  type(object) -> the object's type
 |  type(name, bases, dict) -> a new type

It is this second usage of type that is important. When the Python interpreter executes a class statement (like in the example with the interactive interpreter from a couple of sections back), it calls type with the following arguments:

• The name of the class as a string
• A tuple of base classes - for our example this is the 'one-pl' [1] (object,)
• A dictionary containing members of the class (class attributes, methods, etc) mapped by their names

Easy to Demonstrate

>> >  def  __init__ ( self ) :
. . .      self . message  =  'Hello World'
. . .
>> >  def  say_hello ( self ) :
. . .      print  self . message
. . .
>> >  attrs  =  { '__init__' :  __init__ ,  'say_hello' :  say_hello }
>> >  bases  =  ( object , )
>> >  Hello  =  type ( 'Hello' ,  bases ,  attrs )
>> >  Hello
< class  '__main__.Hello' >
>> >  h  =  Hello ( )
>> >  h . say_hello ( )
Hello  World

This code creates a dictionary of class attributes, and then calls type to create a class called Hello.

The Magic of __metaclass__

We can provide a custom metaclass by setting __metaclass__ in a class definition to any callable that takes the same arguments as type.

The normal way to do this is to inherit from type:

class  PointlessMetaclass ( type ) :
     def  __new__ ( meta ,  name ,  bases ,  attrs ) :
         # do stuff...
         return  type . __new__ ( meta ,  name ,  bases ,  attrs )

The important thing is that inside the body of the __new__ method we have access to the arguments passed to create the new class. We can introspect the dictionary of attributes and modify, add or remove members.

It is important to override __new__ rather than __init__. When you instantiate a class both __init__ and __new__ are called. __init__ initialises an instance - but __new__ is responsible for creating it. So if our metaclass is going to customise class creation we need to override __new__ on type.

The reason to use a new type rather than just a factory function is that if you use a factory function (that just calls type) then the metaclass won't be inherited.

In Action...

>> >  class  WhizzBang ( object ) :
. . .      __metaclass__  =  PointlessMetaclass
. . .
>> >  WhizzBang
< class  '__main__.WhizzBang' >
>> >  type ( WhizzBang )
< class  '__main__.PointlessMetaClass' >

WhizzBang is a class, but instead of being an instance of type the class object is now an instance of our custom metaclass...

What can we do with this?

Well (I'm glad you asked)... our metaclass will be called whenever a new class is created that uses it. Here are some ideas:

• Decorate all methods in a class for logging, or profiling.
• Automatically mix-in new methods.
• Register classes as they are created. (Auto-register plugins or create a db schema from class members for example.)
• Provide interface registration, auto-discovery of features and interface adaptation.
• Class verification: prevent subclassing, verify all methods have docstrings.

The important things is that the class is only actually created by the final call to type in the metaclass - so you are free to modify the dictionary of attributes as you see fit (and the name plus the tuple of base classes of course).

Several of the popular Python ORM (Object Relational Mappers for working with databases) use metaclasses in these ways.

Oh, and because metaclasses are inherited so you can provide a base-class that uses your metaclass and sub-classes inherit it without explicitly having to declare it.

But...

I've ever needed to use one in production code... (I have used them for profiling and we make extensive use of them in Ironclad - but I didn't write these.)

Of course all this only applies to Python 2.X. The mechanism changes in Python 3.

type(type) is type

With Python 2.6 you can now use class decorators to achieve a lot of the things that previously you might have used metaclasses for.

For a truly awful example (with a slightly more in depth but still easy-to-digest look at metaclasses) see The Selfless Metaclass. It does bytecode and method signature rewriting to avoid the need to explicitly declare self. 

[1]	A 'one-pl' is tuple with only one element...