As a python dev who deploys a lot of software, I found this article to be wonderfully helpful and informative, and a good reflection of current best practices.
Summary of the deployment tools mentioned:
- Manage remote daemons with supervisord
http://supervisord.org/
- Manage python packages with pip (and use `pip freeze`)
http://pypi.python.org/pypi/pip
http://www.pip-installer.org/en/latest/requirements.html
- Manage production environments with virtualenv
http://www.virtualenv.org/
- Manage Configuration with puppet and/or chef
http://puppetlabs.com/
http://www.opscode.com/chef/
- Automate local and remote sys admin tasks with Fabric
http://fabfile.org
Other tips:
- Don't restrict yourself to old Python versions to appease your tools / libs.
- Strongly consider rolling your own DEB/RPMs for your Python application.
Author also touted:
- Celery for task management
http://celeryproject.org/
- Twisted for event-based python.
http://twistedmatrix.com/trac/
- nginx / gunicorn for your python web server stack
http://www.nginx.com/
http://gunicorn.org/
Thanks for that! We gave up on chef when one of their version updates failed to work with a prior version, both of which were the OS package defaults. Chef silently failed, no error message, nothing in the docs, nothing even in the source code. Had to do a fair bit of searching to find out why.
When open source projects like chef have nobody interested in even documenting much less testing backwards incompatibilities we move them to the bottom of our to-eval list.
This also illustrates a problem in article's blind enthusiasm for the latest revisions and libraries i.e., it dismisses the headaches this causes end-users, who often don't have staff or budget to fix whatever breaks during an upgrade. That said we are at least talking about python, which has had better release QA and backwards compatibility than perl, ruby or, gasp, php.
That said we are at least talking about python, which has had better release QA and backwards compatibility than perl....
I'm curious as to your experience here. I've found that Perl has by far the best backwards compatibility and release QA of the major dynamic languages. What did you encounter?
We don't use as much perl as we used to but the last upgrade issue was amavisd-new (a Spamassassin wrapper). Spamassassin has perl version issues every so often as well. NetDNS used to introduce new bugs about every 4th revision but seems to have been stable for the past couple of years. GNUmp3d and many audio libraries have non-perl revision-related, backwards-compatibility issues with some regularity.
Cool, I'm also investigating the python-based salt stack: http://saltstack.org/ for this purpose but it seems a bit heavy just starting out. Gonna try ansible next.
Likewise +1 to Nginx + uwsgi. In the various performance benchmarks* uwsgi beats gunicorn, and uwsgi comes bundled with Nginx now, so there's really no reason to bother with setting up gunicorn any more.
I use gevent instead of twisted for event-based python. Gevent is a lot nicer to work with and doesn't make your code less readable in the way twisted does with its callbacks and errbacks. It's also a lot easier to do unit testing with gevent.
Summary of the deployment tools mentioned:
Other tips: Author also touted: