November 19, 2016

Be Polite With Strangers

If I say something unconventional, uncommon, or obviously wrong in conversation with you it likely means we’re already good friends or I find you interesting and want to see what you might say in return. I’m always polite to strangers.

The standard advice for maintaining relationships as I understand it is to avoid topics of contention. If politics, religion, sex, money, or some other uncomfortable topic was raised at a party in which there isn’t clear uniformity of belief it’s too common that someone will inject that these topics aren’t good to talk about. It’s not polite. And that’s likely good advice for weak-tie relationships that you’d like to keep that way.

The reason it’s good advice is that most political or moral conversations lead toward a predictable exchange of flag waving. Neither side is particularly interested in learning anything. Responses are spoken more toward the audience than the ostensible converser. This is bad for your dinner party. It’s also boring, despite how irresistible joining the battle often is.

I more enjoy talking to people that are less constrained. The most interesting people to talk to love to explore things you can’t say. To be sure, I don’t mean the most provocative things you can think of. Some provocative topics are interesting. Many are similarly boring. Controversial is not the same as interesting.

The conversations that are most fun are the ones you haven’t had before. A conversion of this sort should be an exploration, not a performance. If you start to get a feeling that you’re trying to win, you’re doing it wrong. If you end up updating your beliefs, you did it right. Beliefs that you never have to defend from yourself or anyone else, are likely to be poorly supported.

It’s hard to picture a close friendship that doesn’t contain shared secrets. Most often these shared secrets are personal. But if you’re weird like me, you should consider supplementing those with secrets about the world. You might end up with both a friend and a little more wisdom.

June 12, 2016

The Qualifications Game

It’s common for someone just starting out in their career to think they’re hopelessly unqualified. You look at all of blank spaces in your resume, and compare them against the list of features asked of you in job ads. Perhaps worse, you might look at too many of the LinkedIn resumes of people already working in the industry, or already inside the companies you’d like to join.

How did you end up so far behind your peers? Where did all of the people that meet the list of required skills on those job ads develop all of these skills?

The not-so-secret secret is they didn’t. After gaining a few years of experience working and interviewing in industry, you’ll start to get a more accurate picture of the job market.

Job Ads Draw an Ideal

The first thing you’ll notice is that very few people already working the job you’re thinking of applying for meet all of the listed requirements. Perhaps they do in a hand-wavy, well-they-could-learn-it-if-needed, sort of way. By extension, this means that people in that job are not actively using all of the skills described. At least, not regularly, or without help.

Go look through a bunch of job listings looking for software engineers. You’ll start to think that to be qualified for a good job you need to be able to architect and implement the data model, craft the third party API, build out the website or app, manage the continuous integration process, as well as design and maintain a scalable logging system. Oh, and it sure would be nice if you have experience with machine learning and data visualization.

Maybe that’s a bit of an exaggeration, but it resembles reality. Job qualification lists seem to describe either that organization’s strongest employee, or worse, an ideal they’re hoping might exist. In practice, most people tend to focus on 1 or 2 of the desired functions in the ad. It’s not uncommon to need to accomplish something outside of that realm of knowledge, but that will most often mean you’ll be getting help from someone else who knows how to do that thing already. Good companies and managers don’t throw you into a dark forest unless it’s clear you want to go exploring and already know how to survive.

Resumes Are Only Accurate to the Degree They Can Be Proven False

It isn’t news that people embellish on their resume. We’re all told we need to “sell ourselves” on our resumes. Perhaps to the surprise of some, many people took that advice to heart. In general, people do not lie about companies, dates, and titles. There are simple ways for hiring managers to falsify claims regarding those. But you can be confident that when lurking through LinkedIn, many of the resumes you see will exhibit a distortion between their descriptions of past roles and what the writer actually did day-to-day in those roles.

It’s not that people flat out lie. That’s rare. It’s that they describe their contributions in ways that lead readers to believe their responsibilities, level of leadership, and general impact were larger than they really were.

It’s hard to tell what a person really did based on their resume. Or at least, I certainly don’t know how to do it. This is probably why interviews for software positions are famously unpleasant. An applicant typically has to solve several analytic, design, and general algorithm problems with an audience watching their every misstep in order to prove they know how to solve problems and communicate their thought processes to co-workers. This is because the person on the other end of the table has little idea if you really were the primary contributor to and designer of the recommendation system at your last job, or if you were encouraged to look for other opportunities. And they certainly don’t know how to judge the ability level of fresh graduates or new-to-industry candidates. The only information on those resumes they can go on is how hard it was to get into the school the candidate went to. But this is a noisy signal as well.

Don’t Worry So Much About Filling Every Skill Desired

Don’t be afraid to apply for jobs that feel like a stretch. You may find out that the real bar was set lower than expected. The superhuman peers we assume get all of the best jobs mostly don’t exist. A few do, and you should hope to meet them and learn all you can! Applying to stretch jobs isn’t a bad way to try.

Obviously, this is not to say that learning and real qualifications don’t matter. Learning is what allows someone to level up and tackle bigger, more interesting problems. The value of constant learning and reading is hard to overstate.

Anyone that’s read a dozen resumes or given a few interviews learns that resumes can’t be taken at face value. It probably doesn’t give a candidate much of a boost to exaggerate past contributions. If it does, the company that candidate landed at may not be filled with the sort of co-workers they had hoped it would be.

Looking for your first or second job can be a nerve-racking and self-doubting experience. You will get rejected. And send quite a few resumes into the void. But there isn’t an army of better qualified candidates out there. Learn from interviews that go badly and keep trying. I went from not knowing what a for-loop was to writing software for Google 3.5 years later. So I suppose I have experience being highly unqualified. I’ve never applied for a job I was confident I would get.

May 26, 2016

Toys Aren't Only About Fun

Children’s toys are generally assumed to be bought and played with because they’re fun. Many of them are. But we tend to underestimate how similar children’s social behaviors are to our own. On the surface they’re doing completely different things, but similar patterns still occur. We don’t become an adult and suddenly start behaving completely differently.

Just as adults buy things for reasons that aren’t obvious outside of the social context, so do kids.

Action Figures

When I was between the ages of 4 and 7 my favorite toys were action figures. In order: Ghostbusters, Teenage Mutant Ninja Turtles, and Aliens.

My friends and I did play with these in the way one would imagine — acting out battles and such. But what I remember about it more was the collecting. As far as I can remember, the main thing kids did with action figures was compare their collections. Having the rarest set of figures was the goal.

The most rare Ninja Turtle figure was Shredder, the villain. None of my friends had that figure. We weren’t sure if he actually existed. He was never in any of the stores. I remember begging my parents to go to Toys R Us so we could check one more time. Then out of nowhere my uncle happened to find that toy, and knowing the importance of the toy from my mother, bought it. A hero was made!

Aliens had a very similar story. The alien everyone wanted, but no one had, was the Queen.

Playing Cards

In fourth grade the new big thing was Magic: The Gathering. Magic was a set of playing cards that represented various creatures, magical spells, and resources. Using the best deck of 60-100 cards you could construct, you would play against another player. It was a skill-based game, but the quality of your deck was also hugely important. Cards were bought in small packs of perhaps 20. You couldn’t be sure what cards would be inside a given pack. The best you could do was buy as many packs as your parents granted and hope you got lucky.

All of a sudden everyone in my class was forming a deck and playing before class. Almost as important as winning, was the quality of your deck and ability to make good trades. During a big trade discussion there would be extensive evaluation from the sidelines. If someone who didn’t have a good handle on the relative value of the cards found himself with a strong card, you could be certain a battle to see who would successfully trade for that card would occur. A fool and and his cards will soon be parted, as they say. As I remember it, in that classroom the biggest signal of popularity was the quality of your Magic card deck. The playing of games almost seemed like a ceremony performed to make sure that deck quality wasn’t meaningless. That Christmas I asked for nothing but Magic card packs.

At some point I had accumulated a pretty strong deck. I only had one real rival for the title of best deck in the class — that of my best friend at the time. One day I couldn’t find my baggy of Magic cards. I had had it inside my coat pocket in the back of the class that morning, but by the end of the day it was gone (the teacher had forced Magic cards to be put away during class). I was devastated.

A few days later one of the other boys in class, one not known to have a strong set of cards, showed up to school with a suspiciously similar deck to the one I had lost. An exact match. The two of us and a couple card-witnesses had to go down to the guidance counselor’s office to settle the dispute. My emotional week turned to relief when he judged in my favor. This was the first time any of us had seen someone get in trouble for an act that we knew to be against the real rules. I don’t mean rule breaking in general. Kids got in trouble all the time. But for things that we all sort of knew were only against kid-rules: talking in class, being mean, running in the halls, not doing homework, etc. Now we had someone stealing. And stealing the things most important within our little economy.

I remember being shocked by it at the time. What would cause someone to steal? And then if you did steal that deck of cards, how could you not know you’d get caught by bringing them back to the scene of the crime? I suspect the thief might not have realized until after he’d taken them that he’d need to bring them back to the scene either. Outside of our classroom, the cards didn’t hold much value. He had to try and show them off where they were important.

Pogs

The other really popular elementary school game I remember was Pogs. I wasn’t part of that movement for some reason. Maybe it started in a different classroom than the one I was in that year. I forget. But I do remember that I never joined in. It was too late to catch up to the collections of those that played, so it was best to abstain altogether. Better to remain neutral than participate and lose the game.

I don’t know if I explicitly thought of it like that at the time, but looking back that seems like the reason.

Status in a Simulated World

In school the official rewards and punishments we get are based on what adults decide. It’s a system that rewards boring things like doing your homework. And the rewards are fake. A scratch-and-sniff sticker and a smiley face drawn on your paper only go so far. The often repeated scare stories about how “you won’t get away with that in the X+1th grade!” ring false after the first time it proves untrue.

I think kids often sense the lack of real payoff and turn to inventing a kids-only social hierarchy. One where the teachers and parents aren’t the sole source of power. Collectable toys seem like an evergreen source for these social systems.

I think we can safely say that toy companies understand this. There’s a reason that action figures in a given set aren’t supplied in an equal distribution. It’s not that they produce more of the most popular characters, but the opposite. They drive demand by creating an uneven playing field.

There’s a reason that the best toys in a set must be very rare. And I don’t mean in order to drive extra sales. That’s a convenient bonus. Collectables without rare items are not collected.

January 14, 2013

The Pursuit of Magic

In my better moments I pursue magic. I find myself in awe of a mystery and feel drawn to chase its shadow. Not so much to catch the rabbit, but more to simply chase it down the hole and see where I end up.

Magic is the excitement you feel when experiencing that which you don’t understand. It is stopping to consider a small bit of all that we don’t have a good grasp upon. It is the rush of shining a light into the dark and finding that you’ve revealed an even wider range of darkness to explore.

Confusion on the road to Wonderland

Magic can be uncomfortable. To pursue it, you must admit that you don’t know.

The world is pretty simple when you’ve got everything under control. When I was younger (and sometimes still) I liked to act as if I had things all figured out. It’s a tempting way to define yourself. When you’re doing it, you might not even realize you’re pretending most of the time.

I think many people find it preferable to have a false belief than to leave an issue undetermined. For example, consider the various conspiracy theories populating the internet about how group XYZ is secretly controlling the world. They each promise to enlighten us with the true nature of the world and only ask that we spread the word!

At first glance, these stories might seem like they make the world a more complex, dangerous, and confusing place to live in for their adherents. But perhaps it’s the opposite. Maybe these storylines are popular because they reduce complexity. They allow the believer to condense innumerable considerations and unknowns into a single story. Paradoxically, they offer the feeling of control. No longer is the believer lost amongst the confusion of world economics, politics, and sociology. Now they are an insider that holds the secret. The ills of society are not generated from our own nature, but by a certain small group of other people, and we know who they are! All we have to do is prevent their continued interference and then all will be well again.

The first step toward exploring the unknown must be admitting ignorance. Not only ignorance of the phenomena you are drawn to, but more importantly, ignorance of the outcome of your pursuit. Perhaps you’ll discover you aren’t capable of understanding or accomplishing what you set out to. Maybe you will find rejection. After your journey down the rabbit hole you might not even know if you found what you were searching for!

Ordinary magic

The funny thing about magic is that it’s hard to see. It disguises itself as the mundane.

Real magic does not hide behind a curtain. It does not have a price of admission. And you don’t need someone else to show it to you.

In technical circles someone might call a thing or process that they don’t understand the internal workings of a black box. We summon it and it conjures up a result for us. We all encounter countless things every day that from our point of view are black boxes. What is electricity? How does a computer’s CPU really work? How do human languages form? Why do we go to sleep every night?

I don’t have anything more than a cursory answer for any of those, but I use or do each of them every day. These are all ordinary things in my world, yet I bet hardly anyone I know could give a good, thorough answer to even one of them.

As a web developer, I deal with an endless series of black boxes daily. Maybe I understand how all of the code I’ve personally written works, but I take most of the functions in my libraries and frameworks for granted. My sites run in web browsers, but I only have a general understanding of how browsers are built. And I certainly don’t know how my operating system’s kernel was implemented.

My point isn’t that we should try to understand everything in our lives. That’s well beyond possible and likely not desirable even if it were. But I think that if and when you find yourself intrigued by a mystery, it can be a great experience to explore it. If something excites or scares you, there might be something there for you.

It can be a mystery on any level. It could be technical, personal, or even ontological. You don’t need to find the key to your black box necessarily — maybe there isn’t one. Simply giving it a good ponder might be fun. Sometimes you’ll gain a better understanding of something, and sometimes you’ll find out you don’t understand something.

Pursue at will

Black boxes are often useful as mental shortcuts. They let us take something for granted and bypass it so that we can think about whatever is most necessary at the moment. But they also prevent deeper, more interesting understandings and experiences.

To pursue a small bit of magic, identify one potentially interesting mental shortcut you often take and see what lies beneath it. Slow down, pay attention, and learn some magic spells!

January 2, 2013

Decorators and Functional Python

Decorators are one of Python’s great features. In addition to their intrinsic usefulness in the language, they also help us to think in an interesting way — a functional way.

I intend to explain how decorators work from the ground up. We’ll start by covering a few topics you’ll need in order to understand decorators. After that, we’ll dive in and explore a few simple decorators and how they work. Finally, we’ll talk about some more advanced ways to use decorators, such as passing them optional arguments or chaining them together.

First, let’s define what a Python function is in the simplest way I can think of. From that simple definition, we can then define decorators in a similarly simple way.

A function is a block of reusable code that performs a specific task.

Okay, so then what is a decorator?

A decorator is a function that modifies other functions.

Now let’s start to expand on that definition of decorators, starting with a couple prerequisite explanations.

Functions are first class objects

In Python, everything is an object. What this means is that functions can be referred to by name and passed around like any other object. For example:

def traveling_function():
    print "Here I am!"

function_dict = {
    "func": traveling_function
}

trav_func = function_dict['func']
trav_func()
# >> Here I am!

traveling_function was assigned as the value of the func key in the function_dict dictionary and can still be called like normal.

First class functions allow for higher order functions

We can pass functions around like any other object. We can pass them as values to dictionaries, put them in lists, or assign them as object properies. So couldn’t we pass them as arguments to another function? We can! A function that accepts another function as a parameter or returns another function is called a higher order function.

def self_absorbed_function():
    return "I'm an amazing function!"


def printer(func):
    print "The function passed to me says: " + func()

# Call `printer` and give it `self_absorbed_function` as an argument
printer(self_absorbed_function)
# >> The function passed to me says: I'm an amazing function!

So here you can see that a function can be passed to another function as an argument, and that function can then invoke the passed in function. This allows us to create some interesting functions, like decorators!

The basics of a decorator

At heart, a decorator is just a function that takes another function as an argument. In most cases they return a function that is a modified version of the function they are wrapping. Let’s look at the simplest decorator we can that might help us understand how all of this works — the identity decorator.

def identity_decorator(func):
    def wrapper():
        func()
    return wrapper


def a_function():
    print "I'm a normal function."

# `decorated_function` is the function that `identity_decorator` returns, which
# is the nested function, `wrapper`
decorated_function = identity_decorator(a_function)

# This calls the function that `identity_decorator` returned
decorated_function()
# >> I'm a normal function

Here, identity_decorator does not modify the function it wraps at all. It simply returns a function (wrapper) that when called, will invoke the original function identity_decorator received as an argument. This is a useless decorator!

What’s interesting about identity_decorator is that wrapper has access to the func variable even though func was not passed in as one of its arguments. This is due to closures.

Closures

Closure is a fancy term meaning that when a function is declared, it maintains a reference to the lexical environment in which it was declared.

When wrapper was defined in the previous example, it had access to the func variable in its local scope. This means that throughout the life of wrapper (which got returned and assigned to the name decorated_function), it will have access to the func variable. Once identity_decorator returns, the only way to access func is through decorated_function. func does not exist as a variable anywhere other than inside decorated_function’s closure environment.

A simple decorator

Now let’s create a decorator that will actually be of a little use. All this decorator will do is log how many times the function it modifies gets called.

def logging_decorator(func):
    def wrapper():
        wrapper.count += 1
        print "The function I modify has been called {0} times(s).".format(
              wrapper.count)
        func()
    wrapper.count = 0
    return wrapper


def a_function():
    print "I'm a normal function."

modified_function = logging_decorator(a_function)

modified_function()
# >> The function I modify has been called 1 time(s).
# >> I'm a normal function.

modified_function()
# >> The function I modify has been called 2 time(s).
# >> I'm a normal function.

We said a decorator modifies a function, and it can be useful to think of it like that. But as you can see in our example, what logging_decorator does is return a new function that is similar to a_function, but with the addition of a logging feature.

In this example, logging_decorator not only accepts a function as a parameter, it also returns a function, wrapper. Each time the function that logging_decorator returns gets called, it increments wrapper.count, prints it, and then calls the function that logging_decorator is wrapping.

You might be wondering why our counter is a property of wrapper instead of a regular variable. Wouldn’t wrapper’s closure environment give us access to any variable declared in its local scope? Yes, but there’s a catch. In Python, a closure provides full read access to any variable in the function’s scope chain, but only provides write access to mutable objects (lists, dictionaries, etc.). An integer is an immutable object in Python, so we wouldn’t be able to increment its value inside of wrapper. Instead, we made our counter a property of wrapper, a mutable object, and thus we can increment it all we like!

Decorator syntax

In the last example, we saw that a decorator can be used by passing it a function as an argument, thus ‘wrapping’ that function with the decorator function. However, Python also has a syntax pattern that makes this more intuitive and easier to read once you are comfortable with decorators.

# In the previous example, we used our decorator function by passing the
# function we wanted to modify to it, and assigning the result to a variable

def some_function():
    print "I'm happiest when decorated."

# Here we will make the assigned variable the same name as the wrapped function
some_function = logging_decorator(some_function)
# We can achieve the exact same thing with this syntax:

@logging_decorator
def some_function():
    print "I'm happiest when decorated."

Using the decorator syntax, this is the bird’s eye view of what happens:

  1. The interpreter reaches the decorated function, compiles some_function, and gives it the name ‘some_function’.

  2. That function is then passed to the decorator function that is named in the decoration line (logging_decorator).

  3. The return value of the decorator function (usually another function that wraps the original) is substituted for the original function (some_function). It is now bound to the name ‘some_function’.

With these steps in mind, let’s annotate the identity_decorator a little for clarification.

def identity_decorator(func):
    # Everything here happens when the decorator LOADS and is passed
    # the function as described in step 2 above
    def wrapper():
        # Things here happen each time the final wrapped function gets CALLED
        func()
    return wrapper

Hopefully those comments are instructive. Only commands that are inside of the function that the decorator returns get called each time the wrapped function is invoked. Commands written outside of that return function will only happen once — when the decorator first gets passed its wrapped function in step 2 described above.

There’s one more thing I’d like to explain a little before we start looking at some more interesting decorators.

*args and **kwargs

You may have seen these sometimes confusing brothers before. Let’s talk about them one at a time.

  • A python function can accept a variable number of positional arguments by using the *args syntax in its parameter list. *args will combine all non-keyword arguments into a single tuple of arguments that can be accessed within the function. Conversely, when *args is used in the argument list of a function invocation, it will expand a tuple of arguments out into a series of positional arguments.

    def function_with_many_arguments(*args):
        print args
    
    # `args` within the function will be a tuple of any arguments we pass
    # which can be used within the function like any other tuple
    function_with_many_arguments('hello', 123, True)
    # >> ('hello', 123, True)
    
    def function_with_3_parameters(num, boolean, string):
        print "num is " + str(num)
        print "boolean is " + str(boolean)
        print "string is " + string
    
    arg_list = [1, False, 'decorators']
    
    # arg_list will be expanded into 3 positional arguments by the `*` symbol
    function_with_3_parameters(*arg_list)
    # >> num is 1
    # >> boolean is False
    # >> string is decorators
    

    To reiterate: in a parameter list, *args will condense a series of arguments into one tuple named ‘args’, in an argument list, *args will expand an iterable of arguments into a series of positional arguments it applies to the function.

    As you saw in the argument expansion example, the * symbol can be used with names other than ‘args’. It is just convention to use the form *args when condensing/expanding generic argument lists.

  • **kwargs behaves similarly to its brother, *args, but it works with keyword arguments instead of positional. If **kwargs is used in a function parameter list, it will collect as many extra keyword arguments the function has received and place them into a dictionary. If used in a function argument list, it will expand a dictionary into a series of keyword arguments.

    def function_with_many_keyword_args(**kwargs):
        print kwargs
    
    function_with_many_keyword_args(a='apples', b='bananas', c='cantalopes')
    # >> {'a': 'apples', 'b': 'bananas', 'c': 'cantalopes'}
    
    def multiply_name(count=0, name=''):
        print name * count
    
    arg_dict = {'count': 3, 'name': 'Brian'}
    
    multiply_name(**arg_dict)
    # >> BrianBrianBrian
    

Now that you understand how *args and **kwargs work their magic, let’s move on to studying a decorator you just might find useful.

Memoization

Memoization is a way to avoid repeating potentially expensive calculations. You do this by caching the result of a function each time it runs. This way, the next time the function runs with the same arguments, it will return the result from the cache, not having to calculate the result an additional time.

from functools import wraps


def memoize(func):
    cache = {}

    @wraps(func)
    def wrapper(*args):
        if args not in cache:
            cache[args] = func(*args)
        return cache[args]
    return wrapper

@memoize
def an_expensive_function(arg1, arg2, arg3):
    ...

You probably noticed a strange @wraps decorator in this code sample. I’ll explain that briefly before we talk about memoize as a whole a little more.

  • A side effect of using decorators is that the function that gets wrapped loses it’s natural __name__, __doc__, and __module__ attributes. The wraps function is used as a decorator that wraps the function that a decorator returns, restoring those three attributes to the values they would have if the wrapped function was not decorated. For instance: an_expensive_function’s name (as seen by an_expensive_function.__name__) would have been ‘wrapper’ if we did not use the wraps decorator.

I think memoize represents a good use case for decorators. It serves a purpose that will be desired in a lot of functions, and by creating it as a generic decorator, we can add its functionality to any function that can benefit from it. This avoids the need to reproduce this functionality in many different places. By not repeating ourselves it makes our code easier to maintain, and also easier to read and understand. You instantly understand that the function is memoized by reading a single word.

I should note that memoization is only appropriate to use on pure functions. That is a function that is guaranteed to always produce the same result given a certain set of arguments. If it is dependent upon global variables not passed as arguments, I/O, or anything else that might affect the return value, memoization will produce confusing results! Also, a pure function does not have any side effects. So if your function increments a counter, or calls a method on another object, or anything else that isn’t represented in the return value the function produces, that side effect will not be performed when the result is returned by the cache.

Class decorators

Originally, we said a decorator is a function that modifies another function, but they can also be used to modify classes or methods. It is not as common to decorate classes, but it can be useful tool in certain instances as an alternative to metaclasses.

foo = ['important', 'foo', 'stuff']


def add_foo(klass):
    klass.foo = foo
    return klass


@add_foo
class Person(object):
    pass

brian = Person()

print brian.foo
# >> ['important', 'foo', 'stuff']

Now any objects of class Person will have the super-important foo attribute! Notice that since we’re decorating a class, our decorator doesn’t return a function, but naturally, a class. Let’s update our decorator definition:

A decorator is a function that modifies functions, methods or classes.

Decorators as classes

It turns out I withheld something else from you earlier. Not only can a decorator decorate a class, a decorator can be a class! The only requirement of a decorator is that its return value must be callable. This means that it must implement the __call__ magic method, which gets called behind the scenes when you call an object. Functions set this method implicitly of course. Let’s recreate the identity_decorator as a class to see how this works.

class IdentityDecorator(object):
    def __init__(self, func):
        self.func = func

    def __call__(self):
        self.func()


@IdentityDecorator
def a_function():
    print "I'm a normal function."

a_function()
# >> I'm a normal function

Here is what happens in this example:

  • When IdentityDecorator decorates a_function, it behaves just like a decorator that is a function. This snippit would be equivalent to the example’s decoration syntax: a_function = IdentityDecorator(a_function). The class decorator gets called (thus instantiated) with the function it decorates passed to it as an argument.

  • When IdentityDecorator is instantiated its initialization function, __init__, gets called with the decorated function passed as an argument. In this case all it does is assign that function to an attribute so it can be accessed later by other methods.

  • Finally, when a_function (which is really the returned IdentityDecorator object wrapping a_function) gets called, the object’s __call__ method gets invoked. Since this is just an identity decorator, it simply calls the function it decorates.

Let’s update our definition of a decorator once more!

A decorator is a callable that modifies functions, methods or classes.

Decorators with arguments

Sometimes you need to change your decorator’s behavior on a case by case basis. You can do that by passing arguments.

from functools import wraps


def argumentative_decorator(gift):
    def func_wrapper(func):
        @wraps(func)
        def returned_wrapper(*args, **kwargs):
            print "I don't like this " + gift + " you gave me!"
            return func(gift, *args, **kwargs)
        return returned_wrapper
    return func_wrapper


@argumentative_decorator("sweater")
def grateful_function(gift):
    print "I love the " + gift + "! Thank you!"

grateful_function()
# >> I don't like this sweater you gave me!
# >> I love the sweater! Thank you!

Let’s take a look at how this decorator function would work if we didn’t use decorator syntax:

# If we tried to invoke without an argument:
grateful_function = argumentative_function(grateful_function)

# But when given an argument, the pattern changes to:
grateful_function = argumentative_decorator("sweater")(grateful_function)

The main thing to notice is that when given arguments, a decorator is first invoked with only those arguments — the wrapped function is not one of them like normal. After that function call returns, the function the decorator is wrapping is passed to the function that was returned by the initial invocation of the decorator with its arguments (in this case, the return value of: argumentative_decorator("sweater")).

Step by step:

  1. The interpreter reaches the decorated function, compiles grateful_function, and binds it to the name ‘grateful_function’.

  2. argumentative_decorator is called, and passed the argument "sweater". It returns func_wrapper.

  3. func_wrapper is invoked with grateful_function as an argument. func_wrapper returns returned_wrapper.

  4. Finally, returned_wrapper is substituted for the original function, grateful_function, and is thus bound to the name ‘grateful_function’.

I think this line of events is a little harder to follow than when there are no decorator arguments, but if you take some time to think it through, hopefully it will make sense.

Decorators with optional arguments

There are many ways to accept optional arguments with decorators. Depending on if you want to use positional arguments, keyword arguments, or both, you will have to use a slightly different pattern. I’ll show one way to accept an optional keyword argument:

from functools import wraps

GLOBAL_NAME = "Brian"


def print_name(function=None, name=GLOBAL_NAME):
    def actual_decorator(function):
        @wraps(function)
        def returned_func(*args, **kwargs):
            print "My name is " + name
            return function(*args, **kwargs)
        return returned_func

    if not function:  # User passed in a name argument
        def waiting_for_func(function):
            return actual_decorator(function)
        return waiting_for_func

    else:
        return actual_decorator(function)


@print_name
def a_function():
    print "I like that name!"


@print_name(name='Matt')
def another_function():
    print "Hey, that's new!"

a_function()
# >> My name is Brian
# >> I like that name!

another_function()
# >> My name is Matt
# >> Hey, that's new!

If we pass the keyword argument name to print_name it will behave similarly to argumentative_decorator in the previous example. That is, first print_name will be called with name as its argument. Then the function that first invocation returned will be passed the function it is wrapping.

If we don’t provide a name argument, print_name will behave like the argument-less decorators we’ve seen in the past. It will just get invoked with the function its wrapping as the sole argument.

print_name accounts for both possibilities. It checks to see if it received the wrapped function as an argument. If not, it returns a function waiting_for_func that will get invoked with the wrapped function as its argument. If it did receive the function as an argument, it skips that intermediary step and just invokes the actual_decorator immediately.

Chaining decorators

Let’s explore one last feature of decorators today: chaining. You can stack more than one decorator on any given function. This can be used to construct functions in a way that is similar to how multiple inheritance can be used to construct classes. It is probably best to avoid going crazy with this though.

@print_name('Sam')
@logging_decorator
def some_function():
    print "I'm the wrapped function!"

some_function()
# >> My name is Sam
# >> The function I modify has been called 1 time(s).
# >> I'm the wrapped function!

When you chain decorators, the order in which they are stacked is bottom to top. The function that is being wrapped, some_function, is compiled and passed to the first decorator above it (logging_decorator). Then the return value of that first decorator is passed to the second. And so it will continue for each decorator in the chain.

Since both of the decorators we used here print a value and then run the function they were passed, this means the last decorator in the chain, print_name, will print the first line of output when the wrapped function is called.

Conclusion

I think one of the largest benefits of decorators is that they allow you to think at a slightly higher level of abstraction. If you begin to read a function definition and see that it has a memoize decorator, you will instantly understand that you’re looking at a memoized function. If the memoization code were included inside of the function body it would require extra mental parsing, and introduce possible misunderstanding. Using decorators also allows for code reuse, which can save time, ease debugging, and make refactoring easier.

Playing with decorators is also a great way to learn about functional concepts like higher order functions and closures.

I hope this has been an enjoyable and informative read!

← Older Posts