Farm Development

The Python Make tool

A while back there was a lot of talk about needing a Make tool for Python, one like Ruby Rake. There are a few packages now:

... and more recently:

Personally, I liked Ian Bicking's suggestion to just use distutils but apparently his examples came off convoluted. I think this is because distutils is somewhat convoluted (to be fair, it was created a long time ago in software years).

At the time of that article I took Ian's advice and tried to do Make-like work in distutils commands. It was a pain. Mostly, I kept forgetting how everything needed to be set up. Since distutils silently ignores errors it doesn't tell you what's wrong, your command just doesn't work. But I still think the core concept is a good one. You probably already have a setup.py file anyway, so why not add some commands to it?

disthelper

So ... I made a helper to reduce the pain of creating setup.py commands: disthelper

It's the simplest possible approach: it just automates the creation of standard setup.py commands. You need setuptools and Paste to run disthelper but you don't need anything beyond the stdlib distutils module to run your custom command. I think more pain can be abstracted out of creating distutils commands. First off, I've been experimenting with an optparse-based wrapper around distutils.core.Command. This isn't ready for release but so far I am convinced it is possible to make a nicer Command class that's fully backwards compatible with distutils.core.Command. This could be taken even further and grow a decorator interface or something simpler that didn't require a class (for simple commands). If anyone wants to help dream up an interface, I've laid down the ground work to make this distutils-compatible and am willing to help implement it (my time permitting, of course). As an aside, I think such a wrapper would make a nice addition to the stdlib as long as backwards-compatibility is maintained.

Somewhat related is the buildutils project. Buildutils defines many custom tasks you typically want to perform on your project. At some point I may also make disthelper a repository for typical commands but right now I'd like to focus on making it a facilitator of creating setup.py commands.

  • Re: The Python Make tool

    I've done a lot of Java back-in-the-day... Java is practically unusable without an IDE, because there's so much boilerplate for things used everywhere like getters and setters.

    That's part of the reason that I took the approach of building a different front end rather than going with distutils/setuptools extensions. I've written my own commands for those before, and the process to do so is not what I want to do when I just have a couple of lines of code to add.

    That said, I think it's great to see so much activity around this problem now. Your work on an optparse-based wrapper might be really handy to me, too, depending on the directions you take it in. I have a desire for something like that too... I'd to make the familiar "python setup.py install" actually call an install paver target with the distutils options managed properly. This will make it really easy to override behavior.

    BTW, it's worth noting that Ian uses something other than distutils commands in managing his projects at Open Planning.

  • Re: The Python Make tool

    Yeah, I'm still not sure if I'll always want to create "setup.py the_command" ... I'll have to see what sticks after using it more.

    After reading about Paver today, it definitely attacks way more problems than disthelper does. Especially the dependency problem. I can see both of us have been talking to Zed ;)

    I plan to try out both vellum and paver for some real life Make-like problems that would benefit from dependency resolution. I too prefer having a Python solution rather than a new mini-language solution.

  • Re: The Python Make tool

    Chris McD's buildit is an option too

  • Re: The Python Make tool

    Quick correction, Vellum doesn't use YAML anymore. It uses a simple lists/dicts syntax with a parser I wrote. It's loads of fun, and if you need Python then you can write commands that are accessible from the build specification.

    Enjoy.

  • Re: The Python Make tool

    ah, this I did not know. I updated the description, thanks.

  • Re: The Python Make tool

    You missed waf, http://code.google.com/p/waf/ .

  • Re: The Python Make tool

    You forgot aap [1]

    I use it for a lot of things like:

    * building Delphi application

    * building pdf from latex

    [1] http://www.a-a-p.org

  • Re: The Python Make tool

    I also released a python based build tool this month. You might want to take a look http://python-doit.sourceforge.net/

Note: HTML tags will be stripped. Hit enter twice for a new paragraph.

Recent Projects

  • JSTestNet

    Like botnet but for JS tests in CI.

  • Nose Nicedots

    Nose plugin that prints nicer dots.

  • Fudge

    Mock objects for testing.

  • Fixture

    Loading and referencing test data.

  • NoseJS

    Nose plugin that runs JavaScript tests for a Python project.

  • Wikir

    converts reST to various Wiki formats.