Thoughts on Testing

Fixture 1.3, Now With That Tangy Django Flavor

posted in Python, Testing by kumar on Thursday Sep 24th, 2009 at 9:16a.m.

The fixture Python module is a utility for managing data needed for automated tests. Its new version, 1.3, adds support for the Django backend. This was a champion effort by Ben Ford who wanted an alternative to Django's own JSON / YAML based data loading mechanism. Thanks, Ben! Here is the complete changelog. As usual, you can run easy_install fixture or pip install fixture to get it. Or you can download it from PyPI.

Unit Testing JavaScript With JsTestDriver

posted in Java, JavaScript, Testing by kumar on Monday Jun 8th, 2009 at 2:38p.m.

I've been experimenting with a new tool that was released open source recently called JsTestDriver

It's a tool that makes unit testing JavaScript simultaneously on many browsers from the command line very easy for the developer. Actually, there aren't many other tools like it that I know of. It also provides hooks for continuous integration (e.g. Xunit output) and is designed to help you run all unit tests when you click the save button in your code editor.

Here are some features it provides that I thought were nice ...

Nose 0.11 released (nifty new features)

posted in Python, Testing by kumar on Thursday May 7th, 2009 at 10:58a.m.

Nose 0.11 has just been released. Woo! This has been a long time in the making and got a nice boost from many devs sprinting during PyCon. Here are its nifty new features:

Vastly more readable documentation with a ton of new documentation added.
Parallelize your test runs with the --processes switch (note: except on windows, support for which is in the works)
Output test results in the popular Xunit xml style using --with-xunit; this was designed for Hudson or other CI tools that display or collect stats on test runs.
Hate dealing with the logging module? So do I! :) Nose now captures logging messages logged during failing tests.
Re-run only the last batch of failing tests with the --failed switch until you get them to pass. This one was inspired by TestNG.
Collect tests in all modules without running them.
Better support for IronPython. (Note that Nose has supported Jython for several versions now.)

Also, for forward thinking types, there is branch to support Python 3k but it's not ready for production use and is not recommended for daily use.

PyCon Happenings

posted in Chicago, Pycon 2009, Python, Testing by kumar on Thursday Mar 26th, 2009 at 1:23p.m.

I really like PyCon. It's been said many times but is worth repeating: the hallway track is what makes PyCon such a fun unconference. With that said, here are some happenings:

I'll be talking Friday after lunch about some fun I've been having trying to test Ajax web applications: Strategies For Testing Ajax. I'm pretty excited about it because there are still a lot of unsolved problems so I'm interested to hear about how other people are testing Ajax.
- slides are here
- and a recent video of my talk is here
There will be a Testing In Python BoF (birds of a feather). Not really sure what we'll do but a lot of people seem interested in it. There have been murmurs of a mock library shootout. Hmm ... I better bring my gun :)
I'll be on a panel Sunday to discuss Functional Testing Tools in Python. I'll be offering the Nose perspective.
Ian Bicking will be talking about Topics of Interest which sounds mysterious. As it happened I got a hot lead that there may be ... shall we say ... refreshments to aid in conversation (or instigate heckling?). Don't miss this!
One of my colleagues Kevin Boers is giving an ambitious talk called Building a Simple Configuration-driven Web Testing Framework With Twill. It's pretty neat.
Another one of my colleagues, Terry Peppers, is giving a very entertaining talk called A Configuration Comparison in Python
Too many great talks to mention! I'm psyched about the Windmill talk, Jesse Noller's talk on Multiprocessing, and pretty much everything tagged with testing.

See you there.

Oh! I almost forgot. If you are coming in from out of town, check the wiki for cool restaurants and bars in Chicago. I love this town.

Fudge 0.9.2 Released

posted in Python, Testing by kumar on Friday Mar 20th, 2009 at 1:21p.m.

Just a quick note that there is a new version of Fudge, a mock and stub library for Python. This fixes a lot of bugs in the old release and adds some nice new features:

You can now declare method expectations more expressively using argument inspectors.
Declaring an expected method order is now possible with fudge.Fake.remember_order().
Added fudge.Fake.raises() for simulating exceptions.
Declaring and modifying call chains is easier and more readable now.
Lots of bugs fixed, all listed in the Changelog.

Thanks for all the feedback thus far. Special thanks to June Kim for testing this release early and providing feedback on the new interfaces.

I keep getting asked why I created yet another Python mock framework. I really didn't want to and explain my motivation here. I am a huge fan of PyPI and would be lost without all the hard work from the open source community but there is always room for more packages. It provides more options to developers and oftentimes rewriting software can be largely rewarding at a small cost. For example, since I wrote Fudge from the ground up I was able to focus on small things like ensuring that all object representations are sane and that exception messages are as informative as possible. Little things like that can be hard to retrofit into an existing library if they were not written right the first time.

A new version of Fudge, mock object library for Python

posted in JavaScript, Projects, Python, Testing by kumar on Tuesday Mar 10th, 2009 at 11:21p.m.

I just released 0.9.1 of the Fudge module which is a tool for working with fake objects while testing Python code. Some call these mocks, stubs, or actors, but I just call them all fakes because that way you don't have to change the names in code if you update your tests. You can get Fudge from PyPI or by running easy_install -U fudge. This release contains some nice new features and several contributions by Cristian Esquivias. It has more documentation and some bug fixes but note that some functions have been deprecated.

See the changelog for all new features and details on the deprecations. Big thanks to Cristian for his contributions. Also, thanks goes to Marius Gedminas whose comments on my original Fudge announcement led to better names for some commonly used functions.

There is also an experimental, partially-implemented JavaScript port if you feel adventurous.

Fudge: Another Python Mock Framework

posted in JavaScript, Python, Ruby, Testing by kumar on Tuesday Feb 3rd, 2009 at 11:34a.m.

I'm excited to announce the release of Fudge, a Python module for replacing real objects with fakes (mocks, stubs, etc) while testing.

Fudge started when a co-worker introduced me to Mocha, a mocking framework for Ruby and a simpler version of jMock (Java). Up to that point I had been building mocks "by hand" in Python with a post mortem approach; I'd set up some fakes then inspect their call stacks at the end of the test. I like the jMock approach better—you declare a fake object that expects methods and arguments, you replace the real object with the fake one, run your code, and you're done. If your code doesn't live up to the expectations then your test fails.

Debugging doctests interactively

posted in Python, Testing by kumar on Monday Feb 2nd, 2009 at 4:14p.m.

Jens W. Klein has just released a pretty cool doctest debugger tool called interlude. It's designed for a situation where you are writing doctests (perhaps in the comments of your code) and you think to yourself, hmm, what happens when I run this test? Instead of the back and forth run-test-edit cycle, well, why not just drop into a doctest session from your test suite, interact with the shell until you got it right, then copy / paste the session back into your comments? This little tool is genius. And surprisingly simple: 11 lines long (3 of those are for the shell startup message).

Jens describes an installation process that involves invoking a custom doctest runner. This can introduce a bootstrapping problem, especially if you are using a doctest runner like Nose because it's hard to customize the doctest runner. Well, actually, this bootstrapping step isn't even necessary. Here's an example ...

Chicago JavaScript Meetup: JS.Chi()

posted in Chicago, JavaScript, Python, Testing by kumar on Saturday Jan 24th, 2009 at 3:53p.m.

I went to my first Chicago JavaScript meetup on Thursday (the 4th meetup they've had so far) and it looks like a promising group. The topics were pretty basic but the speakers seemed very deep into JavaScript so I'm looking forward to future meetings. By raise of hands we decided to split into two groups, an advanced and a beginner, which was an excellent idea. The turnout was incredible — about 60 (or more?) people showed up. The original venue that got canceled only held 40 so some of those people were actually on a waiting list ...

Are you hiring web developers?

posted in Chicago, JavaScript, Python, Ruby, Testing, The Future by kumar on Wednesday Dec 3rd, 2008 at 10:58a.m.

As the US economy spins into a death spiral I unfortunately now know some highly skilled web developers (Python, Ruby, JavaScript, etc) in the Chicago area who are looking for work. And I mean highly skilled people whose ninja like qualities will be sorely missed. Most hiring budgets are probably frozen — at least until next year — but they will soon thaw. Those that thaw sooner will make space for big prizes of talent. I can make introductions over email or via Linked In.

Automated Model Based Testing of Web Applications (GTAC 2008)

posted in GTAC, GTAC 2008, Java, Python, Ruby, Testing, The Future by kumar on Monday Oct 27th, 2008 at 9:36a.m.

Last year at the Google Test Automation Conference (GTAC) 2007, the talk by Atif was one of my favorites. He had been working on a system of model based testing for desktop GUI applications (affectionately called GUITAR) and hinted that he will be applying the work to web applications next. Now, at GTAC 2008, here is a taste of what his dept. has been up to. The talk is by Oluwaseun Akinmade and Prof. Atif M. Memon. Both are at University of Maryland.

The idea of automated model based testing hints at a future where software can be used to figure out how to test itself. That is, when software is modeled in a way that exposes inputs, outputs, event handlers, and end-points, then introspection can be done to find all possible interactions within an application and test them. Yes, it is one step away from artificial intelligence. This is fascinating to me but I think it still needs a lot of work. Atif is asking for as much feedback as possible from industry professionals to find out how this can best be used in the real world.

Here are my notes from the talk ...

Taming The Beast: How To Test an AJAX Application (GTAC 2008)

posted in GTAC, GTAC 2008, JavaScript, Python, Testing, The Future by kumar on Saturday Oct 25th, 2008 at 9:06p.m.

This was one of the talks at GTAC 2008 that I was most looking forward to before the conference. It was excellent, I was not let down. The talk was given by Markus Clermont and John Thomas who work at Google. Since the talk was right after lunch they decided to take a Q & A approach. It sort of went off in tangents at points but overall the format seemed to work.

In my own work I've been struggling at maintaining a now bloated test suite for an AJAX website but their approach made something click in my head. I'm already working on a refactoring plan.

Here is my abbreviated interpretation of the talk ...

The Future of Testing (GTAC 2008)

posted in GTAC, GTAC 2008, Java, JavaScript, Python, Ruby, Testing, The Future by kumar on Friday Oct 24th, 2008 at 3:58a.m.

Google Test Automation Conference (GTAC) is my all-time favorite conference. It's free. It's on a single track — this means you don't miss any talks and everyone experiences the same journey of thought. Also, since you have to apply for admittance with a short essay, everyone who attends is really passionate about testing. It's still sort of "underground" which keeps it small and very social.

Last year, I made some kind of attempt to live blog summaries of the GTAC talks but I never made it past part 1. We'll see how far I get this year, stay tuned.

The videos for 2008 aren't online yet but check youtube often because last year they were up in less than a day.

The Future of Testing was the first talk of the GTAC 2008 conference on Thursday Oct 23rd given by James A. Whittaker, a very entertaining speaker who works for Microsoft. His talk was excellent and I highly recommend keeping a lookout for the video. Here are my notes...

T'is be'a Fixture 1.1.1 fer ya!

posted in Google App Engine, Projects, Python, Testing by kumar on Friday Sep 19th, 2008 at 10:44a.m.

Y'aharrr me seabound mateys! Thar be'a fine gully of'a wind shakin' ye jigger today as m'announce a new release of Fixture, a python module fer loadin' and referencing test data. O'er yonder ye find a cap'n's Changelog fer ye royal subjects.

Riches abound! Booty abaft! Me could'a n'er dunnit not be'a the help o'a few fine pirates amidst ye Python vagabonds. Me best salute go t'a Tomas Holas, Alex Marandon, and bslesinsky fer'a ya bug reports and patches. Been'a some quiet waters thus far but much treasure huntin' lies o'er th'horizon.

Plunder Fixture 1.1.1!

(And it be'a fine day fer pirates, aye)

Real Test Engineers Love Dots

posted in Python, Testing by kumar on Wednesday Sep 10th, 2008 at 3:05p.m.

"The world's largest particle collider passed its first major tests by firing two beams of protons in opposite directions around a 17-mile underground ring ... After a series of trial runs, two white dots flashed on a computer screen at 10:26 a.m. indicating that the protons had traveled clockwise along the full length of the 4 billion Swiss franc (US$3.8 billion) Large Hadron Collider."

— from Massive particle collider passes first key tests

What better way to indicate a passing test than a single dot?! Simple, effective. Mmmm, dots.

Thoughts on Testing

Fixture 1.3, Now With That Tangy Django Flavor

Unit Testing JavaScript With JsTestDriver

Nose 0.11 released (nifty new features)

PyCon Happenings

Fudge 0.9.2 Released

A new version of Fudge, mock object library for Python

Fudge: Another Python Mock Framework

Debugging doctests interactively

Chicago JavaScript Meetup: JS.Chi()

Are you hiring web developers?

Automated Model Based Testing of Web Applications (GTAC 2008)

Taming The Beast: How To Test an AJAX Application (GTAC 2008)

The Future of Testing (GTAC 2008)

T'is be'a Fixture 1.1.1 fer ya!

Real Test Engineers Love Dots

Chicago's Google App Engine Hack-A-Thon Recap

Fixture Goes 1.0 (Testing With Data In Python)

Testing Google App Engine sites

Building Flash/ActionScript sites entirely in code and using FireBug for debugging

Leapfrog Online is looking for some Django developers (Chicago area)

WSGI Intercept Has A New Home

GTAC Highlights Part 1 - Selenium is Alive and Well, Model Based Testing Is Smart, And...

context_tools, bridging the gap between test methods and test classes?

Going to the GTAC (Google Test Automation Conference)

documentation for fixture module

testing just got easier (a few nose plugins)

You vs. The Real World: Writing Tests With Fixtures (Sunday at Pycon!)

You vs. The Real World: Testing With Fixtures (Coming Soon)

Recent Projects

Fudge

Fixture

NoseJS

Wikir