Using zc.buildout in a Twisted project
I’ve been learning how to use zc.buildout in my Python projects. It seems to be used mostly by the Zope and the Plone communities, but I feel that it might be a useful tool to be used in any Python project. I do lots of work with Twisted and I wanted to try zc.buildout with a Twisted project. It wasn’t trivial and I wasn’t able to find any good examples, so I thought I might as well document my experiences here.
Let’s start this exercise by setting a new virtual Python environment using virtualenv. It gives you a nice constrained Python environment that does not mess up your system. Easiest way to get virtualenv on your system is by using setuptools. When you have setuptools installed you can get virtualenv by typing:
Now that you have virtualenv installed it’s time to create the virtual
environment. In this example I will create the environment in a
directory called twistedenv
. You can place it anywhere you want in
your system.
Now that the virtual environment is setup you can activate it by typing:
This will activate the new Python interpreter that was installed in
twistedenv/bin/python
.
Ok, now we can get to the actual topic, that is how to use zc.buildout
with a Twisted project. What you will need is a bootstrap script that
will install zc.buildout inside your environment and a buildout
configuration file. Let’s start with the configuration file, since we
need that before running the bootstrap script. In zc.buildout the
configuration file is called buildout.cfg
. It will tell the buildout
system, what packages are needed and setup your development system in
a blink of an eye. I’ll start with an example that will install
Twisted and some dependencies and a sample Twisted application that is
under development. Here’s the buildout.cfg
that will do this.
I’ll briefly explain what it does. Section buildout
is the first
thing to put in the config file. It lists what parts of the
configuration file are used and the location of the code that is being
developed. The section called depends
installs the dependencies for
Twisted using recipe minitage.recipe
. Section twisted
naturally
installs Twisted, but it does not install the scripts that are
installed to the bin
directory. That is what the section twisteds
is for. By the way, you are free to choose the names of the sections
or parts except for the buildout
section. In the twisteds
section
we also define the variable interpreter
to create us a new Python
interpreter that will be used when running these scripts. For this
interpreter we define which eggs are included in the interpreter path
and using extra-paths
variable we can include our development code
there also. It is useful that we can keep our own code separate from
3rd party code while developing it. Section versions
can be used to
specify which versions of 3rd party libraries we want to use. If you
don’t specify a version, a newest version available is used.
Now that we have the buildout.cfg
file ready, we can bootsrap the
buildout. Download the bootstrap script from this
URL
and put it in the twistedenv
directory. Before running the bootstrap
script you can make sure that you are using the Python interpreter
from the virtual environment to run it instead of the syste Python. Do
that by typing:
As you can see, for me it shows that I’m using the Python interpreter
from inside the twistedenv
where we created the virtual
environment. That is good. Naturally, for you the complete path is
something else. Then we can run the bootstrap script by typing:
As you can see from the output the script created couple of
directories and generated a script called buildout
to the bin
directory. Now our buildout environment is almost ready. The only
thing is missing is that our ./src
directory does not contain any
code. You will see an error if you now try to run bin/buildout
.
Let’s create a quick sample Twisted application to demonstrate a more complete buildout environment. This is what we need. Create the required directories for our code:
Then we need the following files:
src/sample/tap.py
:
src/twisted/plugins/sample.py
:
And last, but not least, src/setup.py
:
These files will create a Twisted application called sample
that
runs a webserver on port 8080
that responds “It works!” when the
root URL is accessed.
Now, run:
I’ve not put the output to the example since there is so much of
it. But after the buildout is ready and if everything worked fine you
can test that your environment is ready by running the bin/twistd
script like this:
The output should contain our sample
application and you can run it
by typing:
Now you can start developing your Twisted application and test it in your safe and restricted environment using virtualenv and buildout. By the way, if you want to get out of your virtual Python environment you can do that simply by typing:
It would be probably good to include some kind of a test runner in the buildout script to make sure that your buildout is not broken. How that can be done is probably a blog post topic of it’s own. That’s all for now. Have fun with buildout and Twisted. Feel free to ask in the comments if this didn’t work for you for some reason.