Supporting All Versions of Python All The Time With Tox

We Are Python Developers

• We build packages
• We share packages
  • github
  • PyPI
• Fabric of our community
• We are like hunter gatherers

• Humans have been on earth for 2.5 million years
• For 90% of that time we lived in egalitarian hunter gatherer societies where all resources were shared. There was joint ownership of everything.

The Python Package Index

http://pypi.python.org/pypi

pip install numpy

• 13,560 packages on PyPI
• Ready for Python 3?
• 374

• less than 4%

Python 2

PYTHON 2

IS

DEPRECATED

• 2.7 was the last release, maintenance mode
• 3.2 was just released!

How do we get on Python 3?

Developing in tandem with tox: http://codespeak.net/tox/

Developing a package

$ virtualenv ~/tmp/fudge
$ source ~/tmp/fudge/bin/activate
(fudge)$ python setup.py develop
(fudge)$ pip install Nose
(fudge)$ nosetests
......................................................................................................................................................................................
----------------------------------------------------------------------
Ran 182 tests in 0.589s

OK

Now ... with tox

tox lets you develop in multiple versions of Python

• virtualenv per interpreter
• installs your package
• pip / easy_install requirements
• runs tests

$ cd fudge
$ tox -e py26,py32

[demo]

tox.ini config

[tox]
envlist=py26,py32

[testenv]
deps=nose
commands=
  nosetests []

Test runner args

[tox]
envlist=py26,py32

[testenv]
deps=nose
commands=
  # tox -- --with-nicedots --stop
  nosetests []

Any test command

[tox]
envlist=py26,py32

[testenv]
deps=pytest
commands=
  py.test []

Python 3 Syntax

python3
>>> import fudge
Traceback (most recent call last):
  ...
  File "fudge/patcher.py", line 83
    raise etype, val, tb
               ^
SyntaxError: invalid syntax

Python 3 Syntax

Python 2:

print u'Hello %s' % name

Python 3:

print('Hello {0}'.format(name))

• 2to3 (stdlib)
• converts most syntax automatically

Dual platform setup.py

kw = {}
if sys.version_info >= (3,):
    kw['use_2to3'] = True

setup(name='YourModule',
      version='0.1',
      **kw)

$ python setup.py build

(requires distribute)

setup.py classifiers for PyPI

setup(
  name='YourModule',
  version='0.1',
  classifiers=[
    'Programming Language :: Python :: 3'
  ],
  **kw
)

Single Source for 2 and 3

• requires more work and some hacks
• Example: mock
  • http://www.voidspace.org.uk/python/articles/porting-mock-to-python-3.shtml
  • supports 2.4 - 3.2

Python 3 + tox: single source

2to3: distributed tests

2to3 w/ distributed tests

fudge/__init__.py
fudge/tests/__init__.py

[testenv]
deps=nose
commands=nosetests []

[testenv:py32]
commands=
  nosetests fudge.tests.all_tests

2to3: non-distributed tests

2to3 w/ non-distributed tests

nose/__init__.py
unit_tests/
functional_tests/

[testenv:py32]
commands=
  {envpython} setup.py build_tests
  cd build/tests
  {envpython} runtests.py []

https://bitbucket.org/jpellerin/nose

Sphinx doctests

[testenv:docs]
changedir=docs
deps=sphinx
     nose
commands=
    make clean
    make doctest
    make html

[demo]

Test against Django trunk

[testenv]
deps=django
commands=python manage.py test

[testenv:trunk]
basepython=python2.6
deps=
  svn+http://.../svn/django/trunk/#egg=django

Built-in tox environments

• py24 ... py32
• jython
• pypy
• docs (Sphinx)

(interpreters must be installed and in $PATH)

All Environments

LET'S

TEST

EVERYTHING

[demo]

Using tox in CI

Jenkins Matrix Build:

Jenkins Matrix Build

$TOXENV=py26

tox -e py26

Jenkins Matrix Build

Bootstrapping tox

• Must be installed locally (virtualenv?)
• Zero-configuration Jenkins slave:

# Python code for builder to execute:
url = "https://pytox.googlecode.com/hg/toxbootstrap.py"
d = dict(__file__='toxbootstrap.py')
exec urllib.urlopen(url).read() in d
d['cmdline'](['--recreate'])

Variables

• global
  • {toxinidir}
  • {toxworkdir}
  • {homedir}
  • ...
• test environment
  • {envname}
  • {envpython}
  • {envdir}
  • ...

Custom PyPI

[tox]
indexserver =
    DEV = http://mypypi.org

[testenv]
deps =
    docutils
    :DEV:yourmodule

Now you know...

• how to support more Pythons!
• how to toxify with tox.ini

How to support Python 3:

• single source
• convert with 2to3
  • run tests by import path
  • build tests separately

Questions?

tox: http://codespeak.net/tox/

Thank You

• slides: http://farmdev.com/talks/tox/
• me: @kumar303 http://farmdev.com/
• thanks to Holger Krekel for tox!
• Mozilla is hiring
  • we love open source :)