NoseGAE: Test support for Google Application Engine

Contents

Overview

NoseGAE is a nose plugin that makes it easier to write functional and unit tests for Google App Engine applications.

When the plugin is installed, you can activate it by using the --with-gae command line option. The plugin also includes an option for setting the path to the Google App Engine python library, if it is not in the standard location of /usr/local/google_appengine.

What does it do?

Functional tests

The plugin sets up the GAE development environment before your test run. This means that you can easily write functional tests for your application without having to actually start the dev server and test over http. Note however that this kind of testing requires that your application be a wsgi application.

Consider a simple hello world wsgi application:

import wsgiref.handlers
from google.appengine.ext import webapp

class Hello(webapp.RequestHandler):
    def get(self):
        self.response.headers['Content-Type'] = 'text/plain'
        self.response.out.write('Hello world!')

def application():
    return webapp.WSGIApplication([('/', Hello)], debug=True)
        
def main():
    wsgiref.handlers.CGIHandler().run(application())

if __name__ == '__main__':
    main()

And a simple functional test suite for the application:

from webtest import TestApp
from helloworld import application

app = TestApp(application())

def test_index():
    response = app.get('/')
    assert 'Hello world!' in str(response)

The important part to note is the application() function that returns the application. That function provides a way get the application under test and call it directly, without having to pass through the dev app server. And that's all you need to do for basic functional testing.

nosetests -v --with-gae
test.test_index ... ok
<BLANKLINE>
----------------------------------------------------------------------
Ran 1 test in ...s
<BLANKLINE>
OK

Unit tests

Functional tests are only one kind of test, of course. What if you want to write unit tests for your data models? Normally, you can't use your models at all outside of the dev environment, because the Google App Engine datastore isn't available. However, since the NoseGAE plugin sets up the development environment around your test run, you can use models directly in your tests.

Consider a simple models file that includes some doctests:

from google.appengine.ext import db

class Pet(db.Model):
    """
    The Pet class provides storage for pets. You can create a pet:

    >>> muffy = Pet(name=u'muffy', type=u'dog', breed=u"Shi'Tzu")
    >>> muffy # doctest: +ELLIPSIS
    Pet(name=u'muffy', type=u'dog', breed=u"Shi'Tzu", ...)
    >>> muffy_key = muffy.put()

    Once created, you can load a pet by its key:

    >>> Pet.get(muffy_key) # doctest: +ELLIPSIS
    Pet(name=u'muffy', type=u'dog', breed=u"Shi'Tzu", ...)

    Or by a query that selects the pet:

    >>> list(Pet.all().filter('type = ', 'dog')) # doctest: +ELLIPSIS
    [Pet(name=u'muffy', ...)]

    To modify a pet, change one of its properties and ``put()`` it again.

    >>> muffy_2 = _[0]
    >>> muffy_2.age = 10
    >>> muffy_key_2 = muffy_2.put()

    The pet's key doesn't change when it is updated.

    >>> bool(muffy_key == muffy_key_2)
    True
    """
    name = db.StringProperty(required=True)
    type = db.StringProperty(required=True,
                             choices=set(["cat", "dog", "bird",
                                          "fish", "monkey"]))
    breed = db.StringProperty()
    age = db.IntegerProperty()
    comments = db.TextProperty()
    created = db.DateTimeProperty(auto_now_add=True, required=True)

    def __repr__(self):
        return ("Pet(name=%r, type=%r, breed=%r, age=%r, "
                "comments=%r, created=%r)" %
                (self.name, self.type, self.breed, self.age,
                 self.comments, self.created))

Without NoseGAE, the doctests fail.

nosetests -v --with-doctest
Failure: ImportError (No module named google.appengine.ext) ... ERROR
<BLANKLINE>
======================================================================
ERROR: Failure: ImportError (No module named google.appengine.ext)
----------------------------------------------------------------------
Traceback (most recent call last):
...
ImportError: No module named google.appengine.ext
<BLANKLINE>
----------------------------------------------------------------------
Ran 1 test in ...s
<BLANKLINE>
FAILED (errors=1)

With NoseGAE, they pass.

nosetests -v --with-doctest --with-gae
Doctest: models.Pet ... ok
<BLANKLINE>
----------------------------------------------------------------------
Ran 1 test in ...s
<BLANKLINE>
OK

Realism in testing

Besides the dev appserver and the datastore, the main sticking point for testing Google App Engine applications is the highly restrictive runtime environment. When you test without NoseGAE, tests that should fail (because the tested code will fail when run inside the Google App Engine) may pass.

For instance, consider an app that uses the socket module, like this one:

import socket
import wsgiref.handlers

class App:
    def __call__(self, environ, start_response):
        # This won't work under GAE, since this is app code
        here = socket.gethostbyname('localhost')
        start_response('200 OK', [('Content-type', 'text/plain')])
        return ['Hello %s' % here]

def application():
    return App()
        
def main():
    wsgiref.handlers.CGIHandler().run(application())

if __name__ == '__main__':
    main()

With a simple functional test:

from webtest import TestApp
from bad_app import application
import socket

app = TestApp(application())

def test_index_calls_gethostbyname():
    # this works, because in test code GAE sandbox is not active
    host = socket.gethostbyname('localhost')
    response = app.get('/')
    assert 'Hello' in str(response)
    assert host in str(response)

This test will pass when run outside of the Google App Engine environment.

nosetests -v
test.test_index_calls_gethostbyname ... ok
<BLANKLINE>
----------------------------------------------------------------------
Ran 1 test in ...s
<BLANKLINE>
OK

When run with NoseGAE, it will fail, as it should.

nosetests -v --with-gae
test.test_index_calls_gethostbyname ... ERROR
<BLANKLINE>
======================================================================
ERROR: test.test_index_calls_gethostbyname
----------------------------------------------------------------------
Traceback (most recent call last):
...
AttributeError: 'module' object has no attribute 'gethostbyname'
<BLANKLINE>
----------------------------------------------------------------------
Ran 1 test in ...s
<BLANKLINE>
FAILED (errors=1)

It is important to note that only application code is sandboxed by NoseGAE. Test code imports outside of the sandbox, so your test code has full access to the system and available python libraries, including the Google App Engine datastore and other Google App Engine libraries.

For this reason, file access is not restricted in the same way as it is under GAE, because it is impossible to differentiate application code file access from test code file access.