Metaprogramming in
python & javascript

Programs manipulating programs

What is metaprogramming?

• Programs writing another programs
• Programs executing another programs
• Programs understanding their own semantics
• Programs changing themselves

Write a program, in just one line and using no recursion, nor loops, nor "go-to"s to print 100 times: I do not talk in class

exec("for n in range(100):\n  print 'I do not talk in class'")

eval("for(var i=0; i++<100;) console.log('I do not talk in class');");

First-class objects:
Behaviour as data

Javascript

function get_projects(id) {
  console.log('GET http://mycompany.com/projects/' + (id || ''));
}

Python

def get_projects(id=''):
  print 'GET http://mycompany.com/projects/%s' % str(id)

Functions are objects

• Functions are the best way to define behaviour
• They have properties and methods
• A special method call() executes the function

>>> get_projects instanceof Object
true
>>> get_projects.call
call()
>>> get_projects.call(null, 8)
GET http://mycompany.com/projects/8
          

>>> isinstance(get_projects, object)
True
>>> get_projects.__call__
<method-wrapper '__call__' of function object at 0xd3b7d0>
>>> get_projects.__call__(8)
GET http://mycompany.com/projects/8

Functions in expressions

• Functions are values (R-values), they can be anywhere a value could be
• They can be used as parameters...
• ...or as return values

function count(f) {           // Accepts a function as parameter
  function wrapped_f() {      // Here is a totally new function!
    wrapped_f.count++;
    return f.apply(null, arguments);
  }

  wrapped_f.count = 0; // Used as an object
  return wrapped_f;    // Function as the return expression
}

// Reuse the name (L-value) to get an augmented version of the function
get_projects = count(get_projects); // Function as a parameter
get_projects(8);
get_projects(10);
console.log(get_projects.count)

def count(f):                     # Accepts a function as parameter
  def wrapped_f(*args, **kwargs): # Here is a totally new function!
    wrapped_f.count += 1
    return f(*args, **kwargs)

  wrapped_f.count = 0  # Used as an object
  return wrapped_f     # Function as the return expression

# Reuse the name (L-value) to get an augmented version of the function
get_projects = count(get_projects) # Function as a parameter
get_projects(8);
get_projects(10);
print get_projects.count

Extra!
decorators in Python

# A decorator only requires to accept a function and return a function
def count(f):
  def wrapped_f(*args, **kwargs):
    wrapped_f.count += 1
    return f(*args, **kwargs)
  wrapped_f.count = 0
  return wrapped_f

# This is only syntactic sugar
@count
def get_projects(id=''):
  print 'GET http://mycompany.com/projects/%s' % str(id)

Used with @ before a function definition, count() applies itself to the function and updates the name to point the returned value.

Don't Repeat Yourself!

class MyCompany(object):
  def get_projects(self, id=''):
    print 'GET http://mycompany.com/projects/%s' % str(id)

  def get_employees(self, id=''):
    print 'GET http://mycompany.com/employees/%s' % str(id)

  def get_profiles(self, id=''):
    print 'GET http://mycompany.com/profiles/%s' % str(id)

  def get_partners(self, id=''):
    print 'GET http://mycompany.com/partners/%s' % str(id)

Do you see the repetition?

def get_<concept>(self, id=''):
  print 'GET http://mycompany.com/<concept>/%s' % str(id)

function MyCompany() { }
MyCompany.prototype.get_projects = function(id) {
  console.log('GET http://mycompany.com/projects/' + (id || ''));
}
MyCompany.prototype.get_employees = function(id) {
  console.log('GET http://mycompany.com/employees/' + (id || ''));
}
MyCompany.prototype.get_profiles = function(id) {
  console.log('GET http://mycompany.com/profiles/' + (id || ''));
}
MyCompany.prototype.get_partners = function(id) {
  console.log('GET http://mycompany.com/partners/' + (id || ''));
}

Do you see the repetition?

MyCompany.prototype.get_<concept> = function(id) {
  console.log('GET http://mycompany.com/<concept>/' + (id || ''));
}

Solution?
Function factories

• Please, not to be confused with factory method pattern
• They act as templates for other functions
• Return another customized function

def build_get(concept):
  # A template for get_<concept>() methods
  def get_concept(self, id=''):
    print 'GET http://mycompany.com/%s/%s' % (concept, str(id))
  return get_concept

function build_get(concept) {
  // A template for get_<concept>() methods
  function get_concept(id) {
    console.log('GET http://mycompany.com/' + concept + '/' + (id || ''));
  }
  return get_concept;
}

def build_get(concept):
  def get_concept(self, id):
    print 'GET http://mycompany.com/%s/%s' % (concept, str(id))
  return get_concept

class MyCompany(object):
  get_projects  = build_get('projects')
  get_employees = build_get('employees')
  get_profiles  = build_get('profiles')
  get_partners  = build_get('partners')

function build_get(concept) {
  function get_concept() {
    console.log('GET http://mycompany.com/' + concept + '/' + (id || ''));
  }
  return get_concept;
}

function MyCompany() { }
MyCompany.prototype.get_projects = build_get('projects');
MyCompany.prototype.get_employees = build_get('employees');
MyCompany.prototype.get_profiles = build_get('profiles');
MyCompany.prototype.get_partners = build_get('partners');

More improvements?

More D.R.Y?

It depends on language capabilites:

• Dynamic languages
• Extensible APIs

Live APIs

It's alive!

>>> class MyClass(object):
...   '''A test class'''
>>> a = MyClass()
>>> a.me()
AttributeError: 'MyClass' object has no attribute 'me'
>>> def here_I_am(self):
...   print 'Here I am'
>>> MyClass.me = here_I_am
>>> a.me()
Here I am

>>> function MyClass() { /* A test class */ }
>>> var a = new MyClass();
>>> a.me()
TypeError: a.me is not a function
>>> MyClass.prototype.me = function() { console.log('Here I am'); }
>>> a.me()
Here I am

From DRY to DRY-est

class MyCompany(object):
  '''My company website'''

for concept in ['projects', 'employees', 'profiles', 'partners']:
  setattr(MyCompany, 'get_%s' % concept, build_get(concept))

function MyCompany() { /*My company website*/ }

['projects', 'employees', 'profiles', 'partners'].forEach(
function(concept) {
  MyCompany.prototype['get_' + concept] = build_get(concept);
});

Beware:

• Metaprogramming is an advanced and versatile feature
• Use only when it make sense
• You can end writing only-write code

Dynamic dispatching

The art of building interfaces on the fly

Beautiful API

• Natural
• Predictible
• Easy to extend
• Easy to maintain
• Usability in JavaScript APIs

How will it fail?

>>> function MyCompany() {}
>>> var mc = new MyCompany();
>>> mc.get_reports();

>>> class MyCompany(object):
...   '''My company website'''
>>> mc = MyCompany();
>>> mc.get_reports();

The method does not even exist!

But it sounds reasonable to guess it exists...

So let's generate it on the fly!

Retrieving process I

>>> class MyCompany(object):
...   '''My company website'''
>>> mc = MyCompany();
>>> mc.get_reports();
AttributeError: 'MyCompany' object has no attribute 'get_reports'

Before calling, Python tries first to get the name get_reports looking in...

1. The object itself
2. The object class
3. The object superclass and so on...

Retrieving process II

1. If finally not found, __getattr__() is called
2. Method __getattr__() is a member of class object
3. As any other member, it can be overriden

Let's combine with metaprogramming!

class MyCompany(object):

  @staticmethod
  def __build_get(concept):
    def get_concept(self, id=''):
      print 'GET http://mycompany.com/%s/%s' % (concept, str(id))
    return get_concept

  def __getattr__(self, name):
    # If the name is recognized as a get method, creates a new one
    if name[:4] == 'get_':
      setattr(MyCompany, name, self.__build_get(name[4:]))

    # Retrieve the name normally
    return getattr(self, name)

And what about JavaScript?

• No way for intervene dispatching process
• Wait for proxy objects in Harmony or...
• Hack a little bit!

Retrieving process I

>>> function MyCompany() {}
>>> var mc = new MyCompany();
>>> mc.get_reports();
TypeError: mc.get_reports is not a function

JavaScript see the calling as a whole.

JavaScript tries to look for the method following the prototype chain

1. The object itself
2. Prototype of the object
3. Prototype of the prototype of the object and so on...

Retrieving process II

1. Let's send messages with a custom method send()
2. Borrow method_missing() from Ruby
3. And follow a special (but a little bit artificial) pattern

try {
  // Try to call the method in a normal way
  mc.get_reports(9);

// Catch if there is no such method
} catch (err) {

  // Use send to itervene the message sending
  mc.send('get_reports', 9);
}

// Now the method exists, use normally
mc.get_reports(10);

method_missing

When Ruby does not find a method, calls method_missing() with the name of the original function and its parameters

// Default implementation just raises an error
Object.prototype.method_missing = function() {
  throw new Error('NoMethodError');
}

Object.prototype.send = function(name) {
  try {
    // Arguments for the function are after the name of the message
    var args = [].slice.call(arguments, 1);
    this[name].apply(this, args);

  // Does not exists? Call method missing on the object
  } catch (err) {
    this.method_missing.apply(this, arguments);
  }
}

// Override method missing
MyCompany.prototype.method_missing = function (name) {
  function build_get(concept) {
    function get_concept(id) {
      console.log('GET http://mycompany.com/' + concept + '/' + (id || ''))
    }
    return get_concept;
  }

  // Identify method, add to the class and invoke it
  if (name.substr(0, 4) === 'get_') {
    MyCompany.prototype[name] = build_get(name.substr(4));
    var args = [].slice.call(arguments, 1);
    return MyCompany.prototype[name].apply(this, args);
  }
  // fallback to default implementation
  throw new Error('NoMethodError');
}

Excuse me...

What the hell are proxies?

Forecasting!
JavaScript proxies

Proxies are the way to define semantics for an object

>>> var target = {};
>>> var semantics = {
  get: function (target, name, receiver) { return 42; }
}
>>> var p = new Proxy(target, semantics);
>>> p.answer_to_life_the_universe_and_everything_else
/*
Same as:
semantics.get(target, 'answer_to_life_the_universe_and_everything_else', p)
*/
42