Farm Development

Thoughts on Python

back to all thoughts

What's Happening at Mozilla?

Most people at Mozilla are remote so each quarter we sync up face to face as a group for an all-hands meeting. There are over 600 employees! We of course sync up in smaller groups more frequently but this is a chance to see what's going on across the entire Mozilla horizon.

So what's happening at Mozilla? We're on the cusp of a huge shift towards an open web platform. That is, something more than a web browser -- something you can run "native" apps on. There's a lot of work left to do, of course. Here is a random dump of interesting projects in the works...

Fudge Goes 1.0

Fudge, the python mock tool, goes 1.0! You can grab it with pip install -U fudge or directly from PyPI. This marks the end of a long incubation period where the community and I used Fudge in real world scenarios to see what worked and what didn't. I'm sure there are many more improvements to make but as of 1.0 I'm very satisfied with what we've accomplished. This is thanks to its small but vocal community of users, to all contributors and to everyone who pointed out flaws...

The Promise of the Cloud

As web developers we are faced with this problem: how do we scale up our code to handle high traffic? A lot of time and engineering goes into this problem -- time to simulate the traffic we expect and add servers to our cluster, cache heavy database access, etc, in anticipation of the load. Time is precious. This time could be spent optimizing the usefulness of our web product and creating interesting content. No one really congratulates you when a website works, they expect it to work.

When Google App Engine was released their pitch was...

I've Joined the Web Dev Group at Mozilla

Whenever I'd hear about someone from the Python community getting hired by Mozilla I'd get really excited because I knew they'd continue to share and collaborate in the open source world that I was a part of. So here I am about a month into joining Mozilla myself to work with the WebDev team. Everything Mozilla does is right out in the open: ideas are posted on blogs, code is committed to public repositories--free to use, free to fork, etc. They take a firm stance that everything you do on the web should be free and open even to the point where the new Firefox 4 audio API (which is amazing) doesn't even support the patented, closed MP3 format despite its ubiquity.

This transparent approach to technology is really powerful...

Compiling Python 2.5 on Mac OS X 10.6 (Snow Leopard)

The Mac dev community has outgrown Python 2.5 it seems but you might still need it if, say, you work on a Google App Engine project. I recently set up a fresh new Snow Leopard machine and really didn't want to eff it up with Macports. I like Macports but it leaks dynamic libraries here and there and this screwed me on my last machine. Instead I've had good luck with Homebrew (so far) but I couldn't find a Python 2.5 recipe anywhere. Snow Leopard ships with a version of Python 2.5 but I ran into some trouble with it as soon as I tried using virtualenv. Apple usually messes up Python in some way so I figured I'd just compile one. Since I couldn't find anything succinct while searching online here are my notes...

Python Package Index (PyPI) Mirrors

Most deployment systems depend on PyPI, the Python Package Index, for fetching and installing dependencies. Although performance and reliability has greatly improved, there are still days when you may find PyPI down and thus are unable to deploy through the normal fashion.

Finally, there is now a concerted effort to create official mirrors of PyPI that everyone can use (see PEP 381). The mirrors are pretty simple--they're not full blown package indexes--and you can already start using them for experimentation. Also, thanks to Richard Jones, Martin Löwis, Guido, and others at EuroPython, my half-baked idea of using Google App Engine as a PyPI mirror has been resurrected! I don't know if it's fully implemented yet but you can try it out at App Engine seems like a logical place for a mirror due to its scalability. However, I am skeptical of how well it will perform. App Engine still has a long way to go with regards to stability.

To get involved with the mirroring project you can follow the discussions on the Python Catalog SIG mailing list. The next step will be modifying clients to discover and fallback on available mirrors.

PyCon 2010 - Get Your Proposals In

Proposals for talks at PyCon 2010 are due October 1st. That's in two days! PyCon is really a little more like an unconference and is an excellent place to share your ideas or showcase your work to the Python developer community. If you're having fun with something Python related then chances are it will make a great talk. Also, don't be afraid to submit a proposal that you feel is unfinished. As long as you have a basic idea for a talk and can provide a decent outline then now is the time to sneak it in under the deadline.

I have given some talks at PyCon that I thought went pretty well over the past couple years: A talk about the Fixture module, Unicode In Python Completely Demystified (these slides still get a lot of traffic), and Strategies For Testing Ajax. All these talks were really fun. I considered proposing another talk but I've come to the realization that I have way too many spare-time projects on the go and I need every single minute of the next 12+ months to jam. I will of course be attending PyCon and hopefully can cook up a lightning talk on Fudge or one of my other pet projects in time.

Fixture 1.3, Now With That Tangy Django Flavor

The fixture Python module is a utility for managing data needed for automated tests. Its new version, 1.3, adds support for the Django backend. This was a champion effort by Ben Ford who wanted an alternative to Django's own JSON / YAML based data loading mechanism. Thanks, Ben! Here is the complete changelog. As usual, you can run easy_install fixture or pip install fixture to get it. Or you can download it from PyPI.

Why you should NOT license your code as GPL

Zed Shaw recently wrote a clear and concise defense for why he used the GPL on Lamson. I've seen a few mentions on twitter that alarmed me because people seemed to think now is the time to release all software as GPL. Here's what you need to ask yourself before you license your code as GPL. First, do you have a business plan that involves selling your software? Most people do not, most business plans have to do with actually using custom software. If that's you then your software and your business become better as more developers work on your software. I.E. when your software is applied to more real world situations, more bugs are fixed and more patches for features are received. GPL does not help you gain users because it shuts out most commercial enterprises ...

Nose 0.11 released (nifty new features)

Nose 0.11 has just been released. Woo! This has been a long time in the making and got a nice boost from many devs sprinting during PyCon. Here are its nifty new features:

Also, for forward thinking types, there is branch to support Python 3k but it's not ready for production use and is not recommended for daily use.

Degradable Ajax by Sharing Mako Templates With Dojo

For a Pylons site I have been working on I wanted to provide Ajax functionality for the users but also allow the content to be crawled by search engines. Let me point out that not all of the Ajax content needs to degrade to static HTML, only the content that a user might search for on a search engine. Some people might decide that none of their site needs to be crawled by search engines, say, if it was Gmail or something similar.

Since the site is using Pylons it also uses Mako for templating and on the JavaScript side it uses the Dojo toolkit. The easiest way to render content dynamically (via altering the DOM in JavaScript) and render static HTML seemed to be by sharing my templates between Pylons and Dojo. While probably not the most elegant solution, here's how I did it ...

The Python Packaging Problem

At PyCon 2009 the fact that Python needs to solve the "packaging problem" came up a few times. This is not a new discussion. However, the problem is still not completely solved so here I'll point out the details of the problem, the unsolved parts, the solved parts, and how the solved parts could be solved better ...

PyCon Happenings

I really like PyCon. It's been said many times but is worth repeating: the hallway track is what makes PyCon such a fun unconference. With that said, here are some happenings:

See you there.

Oh! I almost forgot. If you are coming in from out of town, check the wiki for cool restaurants and bars in Chicago. I love this town.

Fudge 0.9.2 Released

Just a quick note that there is a new version of Fudge, a mock and stub library for Python. This fixes a lot of bugs in the old release and adds some nice new features:

Thanks for all the feedback thus far. Special thanks to June Kim for testing this release early and providing feedback on the new interfaces.

I keep getting asked why I created yet another Python mock framework. I really didn't want to and explain my motivation here. I am a huge fan of PyPI and would be lost without all the hard work from the open source community but there is always room for more packages. It provides more options to developers and oftentimes rewriting software can be largely rewarding at a small cost. For example, since I wrote Fudge from the ground up I was able to focus on small things like ensuring that all object representations are sane and that exception messages are as informative as possible. Little things like that can be hard to retrofit into an existing library if they were not written right the first time.

A new version of Fudge, mock object library for Python

I just released 0.9.1 of the Fudge module which is a tool for working with fake objects while testing Python code. Some call these mocks, stubs, or actors, but I just call them all fakes because that way you don't have to change the names in code if you update your tests. You can get Fudge from PyPI or by running easy_install -U fudge. This release contains some nice new features and several contributions by Cristian Esquivias. It has more documentation and some bug fixes but note that some functions have been deprecated.

See the changelog for all new features and details on the deprecations. Big thanks to Cristian for his contributions. Also, thanks goes to Marius Gedminas whose comments on my original Fudge announcement led to better names for some commonly used functions.

There is also an experimental, partially-implemented JavaScript port if you feel adventurous.

What Makes Pylons Stand Out As a Web Framework

Googlebot's Fatal Flaw And How You Can Fix It (or Get Rich Trying)

Fudge: Another Python Mock Framework

Debugging doctests interactively

Chicago JavaScript Meetup: JS.Chi()

Python 3.0 On Mac OS X (alongside 2.6, 2.5, etc)

Are you hiring web developers?

Try out the Mercurial Subversion extension (hg svn) on Mac OS X

Automated Model Based Testing of Web Applications (GTAC 2008)

Taming The Beast: How To Test an AJAX Application (GTAC 2008)

The Future of Testing (GTAC 2008)

Presenting a Solid Tutorial at PyCon

T'is be'a Fixture 1.1.1 fer ya!

Web Frameworks Do Not Make DBAs Happy

Real Test Engineers Love Dots

It's Time to USE The Web : Mozilla Labs Releases Ubiquity A portable record player for the Internet

Chicago's Google App Engine Hack-A-Thon Recap

Fixture Goes 1.0 (Testing With Data In Python)

An In-Process, Headless Web Browser for Python?

After the release of Google App Engine, the Rubyists ...

Making Erlang indentation-sensitive

The Python Make tool

Testing Google App Engine sites

PyPi (Cheeseshop) on Google App Engine

Unicode In Python, Completely Demystified (slides available)

Data mining in Python and beyond?

The Monty Hall Problem (win a goat or a car)

Building Flash/ActionScript sites entirely in code and using FireBug for debugging

Software is written by hand

Converting ReStructuredText to Wiki syntax

Leapfrog Online is looking for some Django developers (Chicago area)

WSGI Intercept Has A New Home

importing modules from (chicken vs. egg!)

How To Get Started Writing Open Social Applications

Pycon in Chicago, Excited Yet?

GTAC Highlights Part 1 - Selenium is Alive and Well, Model Based Testing Is Smart, And...

Python on TextMate demo (Chicago area)

Hacking python frames

fileinput : nice module for file processing

context_tools, bridging the gap between test methods and test classes?

Going to the GTAC (Google Test Automation Conference)

What does the def-star-variable (or def-asterisk-parameter) syntax do in Python?

What I Thought I Knew About Unicode in Python Amounted To Nothing

documentation for fixture module

multiple inheritance woes

testing just got easier (a few nose plugins)

unicode and unicorns

Live doctest in TextMate (IPython + Twisted?)

PyCon: A Star Schema in pure python code? Is this guy INSANE?

You vs. The Real World: Writing Tests With Fixtures (Sunday at Pycon!)

Why People Don't Use Hand Dryers

2 stupid things I coded this week

Coffee! ... and python

Housecall from the pydoctor (finally, a doc generator that works!)

Generating python with python

Python gets true closures in 3000 - do I care?

You vs. The Real World: Testing With Fixtures (Coming Soon)

Creating a subversion checkout/ dev target for easy_install

Recent Projects

  • JSTestNet

    Like botnet but for JS tests in CI.

  • Nose Nicedots

    Nose plugin that prints nicer dots.

  • Fudge

    Mock objects for testing.

  • Fixture

    Loading and referencing test data.

  • NoseJS

    Nose plugin that runs JavaScript tests for a Python project.

  • Wikir

    converts reST to various Wiki formats.