A guide to Python Namespaces

This post is part of the Powerful Python series where I talk about features of the Python language that make the programmer’s job easier. The Powerful Python page contains links to more articles as well as a list of future articles.

Namespaces are a fundamental idea in Python and can be very helpful in structuring and organizing your code (especially if you have a large enough project). However, namespaces might be a somewhat difficult concept to grasp and get used to if you’re new to programming or even coming from another programming language (in my case, Java). Here’s my attempt to make namespaces just a little easier to understand.

What’s in a name?

Before starting off with namespaces, you have to understand what Python means by a name. A name in Python is roughly analogous to a variable in just about any other language, but with a few extras. First of, because of Python’s dynamic nature, you can apply a name to just about anything. You can of course give names to values.

a = 12
b = 'B'
c = [1, 2, 3, 4]

But you can also give names to things like functions:

def func():
    print 'This is a function'

f = func

Now whenever you want to use func(), you can use f() instead. You can also take a name and reuse it. For example, the following code is perfectly legal in Python:

var = 12
var = "This is a string now"
var = [2, 4, 6, 8]

If you accessed the name var in between assignments, you’d get a number, a string and a list at different times. Names go hand in hand with Python’s object system, ie. everything in Python is an object. Numbers, strings, functions, classes are all object. The way to get to the objects is often through a name.

Modules and Namespaces go hand in hand

So much for names. A namespace, is obviously enough, a space that holds a bunch of names. The Python tutorial says that they are a mapping from names to objects. Think of it as a big list of all the names that you’ve defined, either explicitly or my importing from modules. It’s not something than you have to create, it’s created whenever necessary.

To understand namespaces, you also have to have some understanding of modules in Python. A module is simply a file containing Python code. This code can be in the form of Python classes, functions, or just a list of names. Each module gets it’s own global namespaces. So you can’t have two classes or two functions in the same module with the same name as they share the namespace of the module (unless they are nested, which we’ll come to later).

However each namespace is also completely isolated. So two modules can have the same names within them. You can have a module called Integer and a module called FloatingPoint and both could have a function named add(). Once you import the module into your script, you can access the names by prefixing them with the module name: FloatingPoint.add() and Integer.add().

Whenever you run a simple Python script, the interpreter treats it as module called __main__, which gets its own namespace. The builtin functions that you would use also live in a module called __builtin__ and have their own namespace.

Importing pitfalls

Of course, modules are useless unless you import them into your program. There are a number of ways to do imports, and each has a different effect on the namespace.

1. import SomeModule

This is the simplest way to do imports and generally recommended. You get access to the module’s namespace provided you use the module’s name as a prefix. This means that you can have names in your program which are the same as those in the module, but you’ll be able to use both of them. It’s also helpful when you’re importing a large number of modules as you see which module a particular name belongs to.

2. from SomeModule import SomeName

This imports a name (or a few, separated by commas) from a module’s namespace directly into the program’s. To use the name you imported, you no longer have to use a prefix, just the name directly. This can be useful if you know for certain you’ll only need to use a few names. The downside is that you can’t use the name you imported for something else in your own program. For example, you could use add() instead of Integer.add(), but if your program has an add() function, you’ll lose access to the Integer’s add() function.

3. from SomeModule import *

This imports all the names from SomeModule directly into the module’s namespace. Generally not a good idea as it leads to ‘namespace pollution’. If you find yourself writing this in your code, you should be better off with the first type of import.

These imports apply to classes and other data just as much as functions. Imports can be confusing for the effect they have on the namespace, but exercising a little care can make things much cleaner.

Scoping

Even though modules have their own global namespaces, this doesn’t mean that all names can be used from everywhere in the module. A scope refers to a region of a program from where a namespace can be accessed without a prefix. Scopes are important for the isolation they provide within a module. At any time there are a number of scopes in operation: the scope of the current function you’re in, the scope of the module and then the scope of the Python builtins. This nesting of scopes means that one function can’t access names inside another function.

Namespaces are also searched for names inside out. This means that if there is a certain name declared in the module’s global namespace, you can reuse the name inside a function while being certain that any other function will get the global name. Of course, you can force the function to use the global name by prefixing the name with the ‘global’ keyword. But if you need to use this, then you might be better off using classes and objects.

Classes

Classes and namespaces have special interactions. The only way for a class’ methods to access it’s own variables or functions (as names) is to use a reference to itself. This means that the first argument of a method must be a ‘self’ parameter, if it to access other class attributes. You need to do this because that while the module has a global namespace, the class itself does not. You can define multiple classes in the same module (and hence the same namespace) and have them share some global data. While this is different from other object-oriented languages, you’ll quickly get used to it.

Hopefully this guide will help you avoid some of the pitfalls that can arise if you don’t understand namespaces. They can lead to unusual results if you don’t use them properly, but they can help you write clean, properly separated code if you use them well. A further source of information on namespaces and classes is the excellent Python tutorial.

About these ads

27 thoughts on “A guide to Python Namespaces

  1. Thank you for this post! I am a newbie and got so lost on my professor’s explanation of modules and name space (more like a terminology burp). You explained it really well.

  2. Thank you, this is really newbie friendly! Didn’t bump into here anytime before, though.

    I’d suggest not to use CamelCase in SomeModule since that would be against PEP8. :)

  3. I’m studying a introductionary programming course on Python, it is reaching its end now and we’ve encountered most of the stuff you are discussing in one way or another, this post clears some stuff up and adds to understand. Code examples might be a way to improve good writing even more, thanks for sharing your knowledge!

  4. Awesome post. Thank you for this succinct and clear explanation…which was hard to come by via any number of other resources.

  5. Great post. I have a question:
    How can developers make modules with the same name? Since to use a module you use ‘import modulename’, isn’t possible for developers to make modules with the same name by coincidence? I’m thinking of common names like ‘web’ or ‘Image’.
    Thanks mate!

    1. Hi Toby, it’s definitely possible for modules to conflict with each other. Python a list of places in the filesystem to look for module in sys.path. I think it will pick the first one it finds. You should look more into sys.path and module loading.

  6. Could you maybe distinguish a bit better between classes (which are objects in their own right with their own namespace) and class instances (which are also objects in their own right obviously, and with their own namespace) under the section headed “Classes”, as you’re saying “class” there when what you really mean is “class instance”.

  7. I think this helped me to not only understand name spaces, but “self” in classes too.
    Correct me if I am wrong, but essentially you need to use self in classes because of the name spaces. Am I correct, or am I misunderstanding something ?

  8. I have run into a situation sometimes in python where multiple modules have something like
    from my_super.py import MySuperclass
    and each defines some subclass. Under some circumstances I haven’t been able to quite pin down some of the modules have a different MySuperclass (as visible using id(MySuperclass)) than others. When this occurs an instantiation of the subclass fails on its super call to __init__ because it things self is of the wrong type. How exactly does that work and how may one never ever fall into this confusing trap? And why isn’t there a global namespace global to the entire process that one can force things like class names into? At the very least why doesn’t importing from the same module guarantee that the class imported from said module is in fact the same across the project? This seems like breakage in python namespace to me.

  9. Just wondering ok but, is anyone else seeing like little white dots coming down the page, especially at the top? I don’t see this any other page, and I just wanted to know to make sure I don’t need to see a specialist lol

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s