michael schurter

At the CherryPy BOF[1] at Pycon in Chicago we discussed how people are using CherryPy in lots of different ways. We’re afraid people aren’t grokking how flexible CherryPy is just by briefly glancing at the web site and thought having some interesting case studies would be useful.

The wiki has a good start. However, its a bit out of date, and it’d be nice to have some great Case Studies as well as a separate list of Projects using CherryPy.

If your company deploys CherryPy or you’ve used CherryPy for a client, I’d love to hear the story. Here’s a basic format you could follow:

1. Who is using CherryPy: (your name)
2. Where are you using Cherrypy?
   (Company or client name. You can of course remain anonymous)
3. What do you do with CherryPy?
   (What’s your project: a traditional public web site? internal web services?)
4. How is CherryPy deployed?
   (Behind modproxy/modpython/modwsgi, on a cluster behind a load balancer, or is it your frontend HTTP server?)
5. How long have you used CherryPy?
   (The “when” question)
6. Why did you choose CherryPy?
   (While “because its the best” is obviously the answer, a little more detail would be nice. )

Feel free to include screenshots, links, code snippets, diagrams, etc, and e-mail me!

With 3.1 nearing release, it’d be nice if we could launch an updated version of the web site at the same time, so your help is greatly appreciated!

I also posted this over at the cherrypy-user mailing list.

[1]

if excuse not in ['at funeral', 'giving birth', 'saving planet']:
	raise Exception('%s is no reason to miss the CherryPy BOF' % excuse)

The little recipe I posted for a dedicated Linux firewall with a CherryPy powered administrative interface finally has a permanent home over at Google Code.

Many thanks to Kyle Waremburg for creating the project page and helping me develop firewall-admin! I hope other people find it useful.

Today I deployed my first Django application for a client. Its yet-another-blog, so I’ll refrain from posting the code and cluttering up the django-*blog* namespace on Google Code. Before you roll your eyes and complain about why I didn’t use an existing solution, I think I have 2 somewhat valid reasons:

1. The client actually needed a subset of the features most blogs offer, so I wouldn’t really have anything to contribute back to an existing project.
2. Blogs are one of the simplest content driven web applications in existence. Wikis are just a bit simpler perhaps. At any rate, creating a blog app is an excellent way to learn a framework.

Python Deployment Decisions

In the past I’ve used CherryPy as my framework and a simple mod_proxy configuration to run the applications behind Apache. Django considers its built-in web server a development tool only, so I figured it was time to explore the myriad of Python web app deployment alternatives: mod_python, FastCGI, modwsgi. I’m sure there are many more (SCGI?), but I’d say those are the big 3.

I had tried to deploy Python web applications on DreamHost using FastCGI before and entered the hell that is deploying Python web apps on shared hosts. So FastCGI wasn’t my first choice this time.

I had also tried mod_python for deploying CherryPy apps on my Linode before and for whatever reason just found mod_proxy to be much easier to setup and manage.

I was kind of eager to try out modwsgi because its been getting a lot of attention lately, so I downloaded the source and compiled it on my Debian Etch server.

Deploying a Django App via modwsgi

modwsgi was quite easy to setup as long as you follow the instructions in their wiki for Django integration. I was hit by bug #3762, but the modwsgi documentation got me through it. (For what its worth the attached wsgi.patch also worked, but I don’t really want to run a patched version of Django.)

One big problem I ran into was sqlite3 gave me OperationalError: unable to open database file whenever I did anything that would write to the database. My database file was owned by www-data (the Apache process owner) and had the permissions 664.

I switched to PostgreSQL, ran syncdb, and everything worked beautifully.

My wsgi script file /srv/spam/eggs/eggs.wsgi:

import os, sys
sys.path.append('/srv/spam')
sys.path.append('/srv/spam/eggs')
os.environ['DJANGO_SETTINGS_MODULE'] = 'eggs.wsgi_settings'
 
import django.core.handlers.wsgi
 
_application = django.core.handlers.wsgi.WSGIHandler()
 
def application(environ, start_response):
    environ['PATH_INFO'] = environ['SCRIPT_NAME'] + environ['PATH_INFO']
    return _application(environ, start_response)

Note I use wsgi_settings instead of my usual settings file. wsgi_settings just imports my main settings file and changes some to their production values.

My Django application actually drops into the /blog/ and /accounts/ folders under a VirtualHost otherwise occupied by static files and some PHP scripts. modwsgi made this easy by putting this in my existing VirtualHost:

WSGIScriptAliasMatch /(blog|accounts)/.* /srv/spam/eggs/eggs.wsgi
 
# A simple Alias directive handles my static files
Alias /static/ /srv/spam/eggs/static/

Bottom Line

I highly recommend using modwsgi for deploying Python web applications. sqlite3 may work for you. In my case its probably best I use PostgreSQL for a number of reasons.

I’m working on a small CherryPy web service that among other things saves timestamps to a database. The timestamp is in RFC 3339 format (like 2007-07-31T16:05:00.000-05:00), and I needed to store the timezone.

Luckily mxDateTime and SQLObject’s DateTimeCol both support full dates with times and time zone. Unfortunately its not immediately obvious from SQLObject’s lackluster documentation how to use mxDateTime instead of Python’s built-in datetime.

A little searching brought me to a mailing list post about how to use mxDateTime by default in SQLObject. (I don’t know why the sample code includes the conditional as I would think you’d want your code to outright fail if you’re unable to use the datetime library you expect.)

So my model’s code looks something like this:

from sqlobject import *
from sqlobject import col
 
col.default_datetime_implementation = MXDATETIME_IMPLEMENTATION
 
class Foo(SQLObject):
    timestamp = DateTimeCol(default=DateTimeCol.now)

Then my parsing code looks something like this:

import model
from mx import DateTime
 
timestamp = '2007-07-31T16:05:00.000-05:00'
bar = model.Foo(timestamp=DateTime.DateTimeFrom(timestamp))
print 'UTC Timestamp:', bar.timestamp
print 'Local Timestamp:', bar.timestamp.localtime()

Basically once you use the magic line col.default_datetime_implementation = MXDATETIME_IMPLEMENTATION, everything Just Works.

At my previous job I built a transparent firewall with the help of a student. He hacked up some iptables scripts, and I hacked up a CherryPy application to control the firewall. It turned out to be pretty handy, so I’m finally getting around to posting it somewhere…

Recipe: A transparent firewall to block certain IP addresses and a nice web based control panel to edit the blacklist.

Ingredients:

• Old computer (preferably a PIII) with 3 NICs
• Debian Etch (or your Linux flavor of choice)
• Packages: iptables, bridge-utils, Python, setuptools, CherryPy 3 (use easy_install), Genshi (use easy_install), Python-ADNS, SQLObject
• firewall-admin.tar.gz - my exceedingly creatively named firewall administration CherryPy web app

Directions:

1. Extract firewall-admin.tar.gz and change to the base directory. By default its setup to be in /srv/firewall-admin
2. If you didn’t extract to /srv/firewall-admin, edit etc/rc.local and basedir in firewalladmin.config to reflect the current directory.
3. By default firewalladmin/lib/bridge.py bridges eth1 and eth2, and eth0 should be attached to your LAN to access SSH and the web control panel.
4. Edit firewalladmin.config to run on the IP address assigned to your administrative NIC and remember what port its set to run on.
5. Add the commands from etc/rc.local to your system’s existing /etc/rc.local script. This will start the transparent firewall and web control panel on boot.
6. Next you’ll need to setup the database. Edit line 28 in firewalladmin/model.py to set a default password and then run createdb.py
7. You’re now ready to start the firewall and control panel simply by running sudo etc/rc.local (see Caveats below). You can always test out just the web interface by running start-firewalladmin.py
8. Browse to the web interface using the IP and Port setup in step 4, login using the username and password setup in step 6, and start configuring your transparent firewall!

The firewall allows creating multiple blacklists (aka Categories) which can be edited/paused/deleted individually. It has has allow lists (aka Whitelists) which can be used to allow specific internal IPs access to specific sites that might otherwise be blocked by a blacklist.

Important: When a user visits a blocked site they are redirected to the IP and Port specified on line 10 of firewalladmin/lib/iptables.py. We setup Apache to listen on that port and serve up a generic You’ve been blocked page, but you could be even more clever. You’ll need a .htaccess file like the following to properly map all blocked traffic to your block page:

RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.html [L,QSA]

Caveats

• All scripts as well as the web control panel are executed as root. This setup should only be run on dedicated hardware and not on a server with other services.
• No test suite. Mea culpa.
• Little to no error handling. You’ve been warned.
• Basically this is a quick hack and should not be used in the same way you use tested and maintained software. YMMV

This little setup has proved very useful at the school for augmenting their existing content filtering system, and all web traffic passes through it without trouble. An old PIII can run a 3,000 domain blacklist at wirespeed on a 10 Mbps link while using less than 10% of the CPU.

Recently I built a small CherryPy application to allow users to upload photos to a gallery. The application is pretty trivial and just uses the Python Imaging Library to resize the photos and Genshi to output some XML which is used by a Flash gallery.

However, after deployment a user complained every 3rd upload would fail. She was using Safari on Mac OSX, and I couldn’t duplicate the error in Firefox. Luckily she didn’t mind switching to Firefox and it fixed her problem:

Working with Firefox has helped tremendously! I’ll only use it from now on, and we should be fine with uploads.

Has anyone else seen this? I deployed CherryPy behind Apache 2.2 using mod_proxy, and Apache logs the following error when an upload fails (IPs changed):

[error] proxy: client 000.000.000.000 given Content-Length did not match number of body bytes read
[error] (70014)End of file found: proxy: pass request body failed to 127.0.0.1:9595 (127.0.0.1) from 00.000.000.001 ()

So it appears Safari just messes up the Content-Length. All CherryPy would receive is a POST with no data.