Thoughts

Python Package Index (PyPI) Mirrors

posted in Google App Engine, Python, The Future by kumar on Tuesday Jul 27th, 2010 at 1:05p.m.

Most deployment systems depend on PyPI, the Python Package Index, for fetching and installing dependencies. Although performance and reliability has greatly improved, there are still days when you may find PyPI down and thus are unable to deploy through the normal fashion.

Finally, there is now a concerted effort to create official mirrors of PyPI that everyone can use (see PEP 381). The mirrors are pretty simple--they're not full blown package indexes--and you can already start using them for experimentation. Also, thanks to Richard Jones, Martin Löwis, Guido, and others at EuroPython, my half-baked idea of using Google App Engine as a PyPI mirror has been resurrected! I don't know if it's fully implemented yet but you can try it out at pypi.appspot.com. App Engine seems like a logical place for a mirror due to its scalability. However, I am skeptical of how well it will perform. App Engine still has a long way to go with regards to stability.

To get involved with the mirroring project you can follow the discussions on the Python Catalog SIG mailing list. The next step will be modifying clients to discover and fallback on available mirrors.

PyCon 2010 - Get Your Proposals In

posted in PyCon 2010, Python by kumar on Tuesday Sep 29th, 2009 at 2:16p.m.

Proposals for talks at PyCon 2010 are due October 1st. That's in two days! PyCon is really a little more like an unconference and is an excellent place to share your ideas or showcase your work to the Python developer community. If you're having fun with something Python related then chances are it will make a great talk. Also, don't be afraid to submit a proposal that you feel is unfinished. As long as you have a basic idea for a talk and can provide a decent outline then now is the time to sneak it in under the deadline.

I have given some talks at PyCon that I thought went pretty well over the past couple years: A talk about the Fixture module, Unicode In Python Completely Demystified (these slides still get a lot of traffic), and Strategies For Testing Ajax. All these talks were really fun. I considered proposing another talk but I've come to the realization that I have way too many spare-time projects on the go and I need every single minute of the next 12+ months to jam. I will of course be attending PyCon and hopefully can cook up a lightning talk on Fudge or one of my other pet projects in time.

Fixture 1.3, Now With That Tangy Django Flavor

posted in Python, Testing by kumar on Thursday Sep 24th, 2009 at 9:16a.m.

The fixture Python module is a utility for managing data needed for automated tests. Its new version, 1.3, adds support for the Django backend. This was a champion effort by Ben Ford who wanted an alternative to Django's own JSON / YAML based data loading mechanism. Thanks, Ben! Here is the complete changelog. As usual, you can run easy_install fixture or pip install fixture to get it. Or you can download it from PyPI.

Using Dropbox As A Worm Hole To All My Computer Worlds

posted in Music, Whatever by kumar on Thursday Aug 20th, 2009 at 9:32a.m.

Dropbox has nailed a use case that I've struggled with for a long time. I have a computer at work. I have a computer at home. Generally I keep files in sync via version control (Subversion, Mercurial, etc) but this is cumbersome for large files, specifically mp3 files. I consume a lot of music, digital and otherwise. How do I keep my music in sync between computers? ...

Why you should NOT license your code as GPL

posted in Django, Python, The Future by kumar on Thursday Jul 16th, 2009 at 3:57p.m.

Zed Shaw recently wrote a clear and concise defense for why he used the GPL on Lamson. I've seen a few mentions on twitter that alarmed me because people seemed to think now is the time to release all software as GPL. Here's what you need to ask yourself before you license your code as GPL. First, do you have a business plan that involves selling your software? Most people do not, most business plans have to do with actually using custom software. If that's you then your software and your business become better as more developers work on your software. I.E. when your software is applied to more real world situations, more bugs are fixed and more patches for features are received. GPL does not help you gain users because it shuts out most commercial enterprises ...

Unit Testing JavaScript With JsTestDriver

posted in Java, JavaScript, Testing by kumar on Monday Jun 8th, 2009 at 2:38p.m.

I've been experimenting with a new tool that was released open source recently called JsTestDriver

It's a tool that makes unit testing JavaScript simultaneously on many browsers from the command line very easy for the developer. Actually, there aren't many other tools like it that I know of. It also provides hooks for continuous integration (e.g. Xunit output) and is designed to help you run all unit tests when you click the save button in your code editor.

Here are some features it provides that I thought were nice ...

Nose 0.11 released (nifty new features)

posted in Python, Testing by kumar on Thursday May 7th, 2009 at 10:58a.m.

Nose 0.11 has just been released. Woo! This has been a long time in the making and got a nice boost from many devs sprinting during PyCon. Here are its nifty new features:

Vastly more readable documentation with a ton of new documentation added.
Parallelize your test runs with the --processes switch (note: except on windows, support for which is in the works)
Output test results in the popular Xunit xml style using --with-xunit; this was designed for Hudson or other CI tools that display or collect stats on test runs.
Hate dealing with the logging module? So do I! :) Nose now captures logging messages logged during failing tests.
Re-run only the last batch of failing tests with the --failed switch until you get them to pass. This one was inspired by TestNG.
Collect tests in all modules without running them.
Better support for IronPython. (Note that Nose has supported Jython for several versions now.)

Also, for forward thinking types, there is branch to support Python 3k but it's not ready for production use and is not recommended for daily use.

Degradable Ajax by Sharing Mako Templates With Dojo

posted in JavaScript, Python by kumar on Tuesday May 5th, 2009 at 12:54p.m.

For a Pylons site I have been working on I wanted to provide Ajax functionality for the users but also allow the content to be crawled by search engines. Let me point out that not all of the Ajax content needs to degrade to static HTML, only the content that a user might search for on a search engine. Some people might decide that none of their site needs to be crawled by search engines, say, if it was Gmail or something similar.

Since the site is using Pylons it also uses Mako for templating and on the JavaScript side it uses the Dojo toolkit. The easiest way to render content dynamically (via altering the DOM in JavaScript) and render static HTML seemed to be by sharing my templates between Pylons and Dojo. While probably not the most elegant solution, here's how I did it ...

The Python Packaging Problem

posted in Python, The Future by kumar on Monday Mar 30th, 2009 at 2:51p.m.

At PyCon 2009 the fact that Python needs to solve the "packaging problem" came up a few times. This is not a new discussion. However, the problem is still not completely solved so here I'll point out the details of the problem, the unsolved parts, the solved parts, and how the solved parts could be solved better ...

PyCon Happenings

posted in Chicago, Pycon 2009, Python, Testing by kumar on Thursday Mar 26th, 2009 at 1:23p.m.

I really like PyCon. It's been said many times but is worth repeating: the hallway track is what makes PyCon such a fun unconference. With that said, here are some happenings:

I'll be talking Friday after lunch about some fun I've been having trying to test Ajax web applications: Strategies For Testing Ajax. I'm pretty excited about it because there are still a lot of unsolved problems so I'm interested to hear about how other people are testing Ajax.
- slides are here
- and a recent video of my talk is here
There will be a Testing In Python BoF (birds of a feather). Not really sure what we'll do but a lot of people seem interested in it. There have been murmurs of a mock library shootout. Hmm ... I better bring my gun :)
I'll be on a panel Sunday to discuss Functional Testing Tools in Python. I'll be offering the Nose perspective.
Ian Bicking will be talking about Topics of Interest which sounds mysterious. As it happened I got a hot lead that there may be ... shall we say ... refreshments to aid in conversation (or instigate heckling?). Don't miss this!
One of my colleagues Kevin Boers is giving an ambitious talk called Building a Simple Configuration-driven Web Testing Framework With Twill. It's pretty neat.
Another one of my colleagues, Terry Peppers, is giving a very entertaining talk called A Configuration Comparison in Python
Too many great talks to mention! I'm psyched about the Windmill talk, Jesse Noller's talk on Multiprocessing, and pretty much everything tagged with testing.

See you there.

Oh! I almost forgot. If you are coming in from out of town, check the wiki for cool restaurants and bars in Chicago. I love this town.

Fudge 0.9.2 Released

posted in Python, Testing by kumar on Friday Mar 20th, 2009 at 1:21p.m.

Just a quick note that there is a new version of Fudge, a mock and stub library for Python. This fixes a lot of bugs in the old release and adds some nice new features:

You can now declare method expectations more expressively using argument inspectors.
Declaring an expected method order is now possible with fudge.Fake.remember_order().
Added fudge.Fake.raises() for simulating exceptions.
Declaring and modifying call chains is easier and more readable now.
Lots of bugs fixed, all listed in the Changelog.

Thanks for all the feedback thus far. Special thanks to June Kim for testing this release early and providing feedback on the new interfaces.

I keep getting asked why I created yet another Python mock framework. I really didn't want to and explain my motivation here. I am a huge fan of PyPI and would be lost without all the hard work from the open source community but there is always room for more packages. It provides more options to developers and oftentimes rewriting software can be largely rewarding at a small cost. For example, since I wrote Fudge from the ground up I was able to focus on small things like ensuring that all object representations are sane and that exception messages are as informative as possible. Little things like that can be hard to retrofit into an existing library if they were not written right the first time.

A new version of Fudge, mock object library for Python

posted in JavaScript, Projects, Python, Testing by kumar on Tuesday Mar 10th, 2009 at 11:21p.m.

I just released 0.9.1 of the Fudge module which is a tool for working with fake objects while testing Python code. Some call these mocks, stubs, or actors, but I just call them all fakes because that way you don't have to change the names in code if you update your tests. You can get Fudge from PyPI or by running easy_install -U fudge. This release contains some nice new features and several contributions by Cristian Esquivias. It has more documentation and some bug fixes but note that some functions have been deprecated.

See the changelog for all new features and details on the deprecations. Big thanks to Cristian for his contributions. Also, thanks goes to Marius Gedminas whose comments on my original Fudge announcement led to better names for some commonly used functions.

There is also an experimental, partially-implemented JavaScript port if you feel adventurous.

What Makes Pylons Stand Out As a Web Framework

posted in Python by kumar on Monday Feb 23rd, 2009 at 2:49p.m.

After 6 release candidates, Pylons 0.9.7 final has been released. Big congrats to all developers. Although it's just a point release, I think this is a huge milestone for Pylons for one reason: It has proper documentation! I've been using both Pylons and Django for several web applications and Pylons' old docs were sparse and hard to navigate. This didn't stop me from digging in and playing around with the framework but for a project to really reach a wide audience it needs well-organized, readable documentation. I can think of many well designed, solid programming tools that have failed because, simply, their docs were not readable. There is a difference between having comprehensive documentation and having documentation that is readable. Django got this right from day one.

As a Pylons user, I'd like to point out a few reasons why I like it and some reasons I don't. It's not meant as a fair comparison to Django, but Django is the only other web framework I know pretty well so it is my point of reference ...

Googlebot's Fatal Flaw And How You Can Fix It (or Get Rich Trying)

posted in JavaScript, Python, The Future by kumar on Thursday Feb 12th, 2009 at 2:48p.m.

I came across this article today on Coding Horror about how Google has a monopoly on search engines and how something must be done about it. I'm not one who falls into the "Google Is Evil" camp; I actually think they are a benevolent force in the world :) However, as with any monopoly, the lack of competition stifles progress. And when I think about the state of today's technology, I can't help but wonder why Google has not fixed the most fatal flaw in their Googlebot :

It does not behave like a web browser.

Search engines are made for people and the majority of people browse the Internet with a web browser. The first comment on the article is a cry for help: "What can we do?" I have an answer to that question. And you can take my answer and turn it into a business plan and climb the golden staircase to success. Any smart investor would be begging you to take their money. Google generated $5.37 billion dollars in Q2 of 2008 and their flagship product doesn't even work! In fact, I'm going to give this to you all for free; all I ask is that you visit me one day and say thanks. Are you ready?

Fudge: Another Python Mock Framework

posted in JavaScript, Python, Ruby, Testing by kumar on Tuesday Feb 3rd, 2009 at 11:34a.m.

I'm excited to announce the release of Fudge, a Python module for replacing real objects with fakes (mocks, stubs, etc) while testing.

Fudge started when a co-worker introduced me to Mocha, a mocking framework for Ruby and a simpler version of jMock (Java). Up to that point I had been building mocks "by hand" in Python with a post mortem approach; I'd set up some fakes then inspect their call stacks at the end of the test. I like the jMock approach better—you declare a fake object that expects methods and arguments, you replace the real object with the fake one, run your code, and you're done. If your code doesn't live up to the expectations then your test fails.

Debugging doctests interactively

posted in Python, Testing by kumar on Monday Feb 2nd, 2009 at 4:14p.m.

Jens W. Klein has just released a pretty cool doctest debugger tool called interlude. It's designed for a situation where you are writing doctests (perhaps in the comments of your code) and you think to yourself, hmm, what happens when I run this test? Instead of the back and forth run-test-edit cycle, well, why not just drop into a doctest session from your test suite, interact with the shell until you got it right, then copy / paste the session back into your comments? This little tool is genius. And surprisingly simple: 11 lines long (3 of those are for the shell startup message).

Jens describes an installation process that involves invoking a custom doctest runner. This can introduce a bootstrapping problem, especially if you are using a doctest runner like Nose because it's hard to customize the doctest runner. Well, actually, this bootstrapping step isn't even necessary. Here's an example ...

Chicago JavaScript Meetup: JS.Chi()

posted in Chicago, JavaScript, Python, Testing by kumar on Saturday Jan 24th, 2009 at 3:53p.m.

I went to my first Chicago JavaScript meetup on Thursday (the 4th meetup they've had so far) and it looks like a promising group. The topics were pretty basic but the speakers seemed very deep into JavaScript so I'm looking forward to future meetings. By raise of hands we decided to split into two groups, an advanced and a beginner, which was an excellent idea. The turnout was incredible — about 60 (or more?) people showed up. The original venue that got canceled only held 40 so some of those people were actually on a waiting list ...

Glögg, Swedish mulled wine, the Chicago recipe

posted in Chicago, Whatever by kumar on Sunday Dec 21st, 2008 at 12:51p.m.

Now that it's -5 F in Chicago with a windchill of -25 F I thought it was an appropriate time to share one of the secrets to surviving a Chicago winter: Glögg! If you live in the old Swedish neighborhood (Andersonville) like I do then you can sip Glögg at most local bars but with weather like this, why even leave your house? Here's my recipe...

Python 3.0 On Mac OS X (alongside 2.6, 2.5, etc)

posted in Python, The Future by kumar on Friday Dec 5th, 2008 at 4:41p.m.

Python 3.0 is out. Woo! Some people have been saying that they'd like to be able to run it alongside 2.6, 2.5, etc to test it out. Well, Python is actually designed for this. All you have to do is build it with make altinstall and that will give you a python3.0 binary without touching your default python binary. But on Mac OS X it's a little different ...

Are you hiring web developers?

posted in Chicago, JavaScript, Python, Ruby, Testing, The Future by kumar on Wednesday Dec 3rd, 2008 at 10:58a.m.

Try out the Mercurial Subversion extension (hg svn) on Mac OS X

posted in Python by kumar on Monday Nov 3rd, 2008 at 4:16p.m.

As a Subversion user, I am hesitant to get excited about distributed version control systems (DVCS). What I like about Subversion is that there is a central line of development. This is great for teams because it forces them to integrate their code often. This is great for open source projects because the users have one place to go to get the source; there is one version of the truth.

However, I usually hack on code during my train commute to work and one thing Subversion does not [yet] do is let you commit offline. This is all I want and this is what a DVCS can provide. Everything else is nice to have. It's nice to have an experimental work space where I can create private branches all day and all night but you can do this in Subversion too. Experimental work branches are dangerous if they do not get integrated with mainline development often. Using a DVCS makes it easier to be lazy about integration.

DCVS lets you maintain mirrors of projects, potentially with your own patches to the project. While this is convenient, I also put this in the dangerous category because the last thing I want is for someone to say, yeah my Django patch wasn't accepted and probably never will be, so just use my clone of the repository! This is an integration nightmare.

Anyway, Subversion is a long way from supporting offline commits so I've been experimenting with Mercurial ...

Automated Model Based Testing of Web Applications (GTAC 2008)

posted in GTAC, GTAC 2008, Java, Python, Ruby, Testing, The Future by kumar on Monday Oct 27th, 2008 at 9:36a.m.

Last year at the Google Test Automation Conference (GTAC) 2007, the talk by Atif was one of my favorites. He had been working on a system of model based testing for desktop GUI applications (affectionately called GUITAR) and hinted that he will be applying the work to web applications next. Now, at GTAC 2008, here is a taste of what his dept. has been up to. The talk is by Oluwaseun Akinmade and Prof. Atif M. Memon. Both are at University of Maryland.

The idea of automated model based testing hints at a future where software can be used to figure out how to test itself. That is, when software is modeled in a way that exposes inputs, outputs, event handlers, and end-points, then introspection can be done to find all possible interactions within an application and test them. Yes, it is one step away from artificial intelligence. This is fascinating to me but I think it still needs a lot of work. Atif is asking for as much feedback as possible from industry professionals to find out how this can best be used in the real world.

Here are my notes from the talk ...

Taming The Beast: How To Test an AJAX Application (GTAC 2008)

posted in GTAC, GTAC 2008, JavaScript, Python, Testing, The Future by kumar on Saturday Oct 25th, 2008 at 9:06p.m.

This was one of the talks at GTAC 2008 that I was most looking forward to before the conference. It was excellent, I was not let down. The talk was given by Markus Clermont and John Thomas who work at Google. Since the talk was right after lunch they decided to take a Q & A approach. It sort of went off in tangents at points but overall the format seemed to work.

In my own work I've been struggling at maintaining a now bloated test suite for an AJAX website but their approach made something click in my head. I'm already working on a refactoring plan.

Here is my abbreviated interpretation of the talk ...

The Future of Testing (GTAC 2008)

posted in GTAC, GTAC 2008, Java, JavaScript, Python, Ruby, Testing, The Future by kumar on Friday Oct 24th, 2008 at 3:58a.m.

Google Test Automation Conference (GTAC) is my all-time favorite conference. It's free. It's on a single track — this means you don't miss any talks and everyone experiences the same journey of thought. Also, since you have to apply for admittance with a short essay, everyone who attends is really passionate about testing. It's still sort of "underground" which keeps it small and very social.

Last year, I made some kind of attempt to live blog summaries of the GTAC talks but I never made it past part 1. We'll see how far I get this year, stay tuned.

The videos for 2008 aren't online yet but check youtube often because last year they were up in less than a day.

The Future of Testing was the first talk of the GTAC 2008 conference on Thursday Oct 23rd given by James A. Whittaker, a very entertaining speaker who works for Microsoft. His talk was excellent and I highly recommend keeping a lookout for the video. Here are my notes...

When Online Advertising Actually Works

posted in The Future, Whatever by kumar on Monday Oct 13th, 2008 at 11:04a.m.

Telemarketing is one of the most ineffective forms of advertising. Hello? Yes? I'm cooking dinner, why would I want to buy something? How do you know I'm even remotely interested in your product? Online advertisements like text ads and banner ads are slightly different. I usually buy things online out of convenience so the venue is good for advertising. There is also a lot of information about me online ...

Presenting a Solid Tutorial at PyCon

posted in Presentations, Pycon 2009, Python by kumar on Friday Sep 26th, 2008 at 12:14p.m.

Proposals for tutorials at PyCon 2009 are now being accepted through October 31st.

Due to many projects I have going on right now I probably won't be able to propose a tutorial myself but I thought I'd share some thoughts from last year...

Adrenallin For The Brain

posted in The Future, Whatever by kumar on Saturday Sep 20th, 2008 at 12:56p.m.

Someone passed me a link to this really nice article, Annals of Science: The Eureka Hunt, which talks about how neuroscientists have been studying what goes on in the brain when we get those amazing ideas that seem to come from nowhere. In my own life I can think of several times where I've experienced a sudden "burst" of thought like this. Sometimes it seems like...

Thoughts

Recent Projects

Fudge

Fixture

NoseJS

Wikir