Blog

I'm just on the way home now from Europython 2009 along with some of the rest of the Netsight team. The conference has been massively inspiring, with nearly 100 talks over the three main conference days. A massive thanks to the organising team, who did a great job of both the logistics and social side of the conference, and of course to all the speakers.

I did two talks this year, one a case study on a project we are currently working on using WSGI and Deliverance to skin a legacy .NET portal entitled Lipstick on a Pig. The second was an attempt to try and show how you can use some of the technology used by Zope outside of Zope: in this case a beginners' talk on Zope Page Templates. When I submitted the talk for the Zope page templates, the response from the talks team was 'Great! Finally a Zope talk!' as they hadn't had any yet. Ironically my talk was actually about using Zope stuff *outside* of Zope. I put a call out on twitter urging some more people to submit Zope/Plone talks, but alas it seems not many were forthcoming.

When I first attended Europython back in 2004, before any of the other frameworks existed, there was actually a dedicated Zope track at the conference, and there were a load of Zope and Plone talks there. It was actually a bit of an odd feeling, as you had a very distinct split in the conference attendees: those (mainly academics) that did hardcore stuff writing python compilers and simulated particle physics; and those people out in 'the commercial world' developing web apps with python in Zope and developing Plone, Silva, etc.

There was certainly a feeling that those doing Zope work were 'outside' the rest of the python community to a certain degree. This was mainly due to Zope being a trailblazer in terms of what it was doing and hence having to develop quite a lot of its own libraries and practises. Examples of this are libraries such as the DateTime library that Zope had before python had anything similar. I guess in just 'getting things done' some of Zope was maybe not quite as pure as python academics might have wanted, and Zope was a fairly monolithic system with little practical chance for its code to be used outside of Zope.

As a side note: in one of the keynotes this year Sir Tony Hoare talked about the differences between Scientists and Engineers. The former chasing absolute perfection, validation and proof in an ideal world; the latter concerned with an imperfect world and doing only exactly what is necessary to achieve the specification. This ties in with my feelings above, and it could be said that at that time the Zope people were the engineers and the rest of the python academic community the scientists. But times have moved on.

A year or so later the 'Zope track' became the 'web framework track' and Django, turbogears, pylons, etc joined in. This year the talks were completely mixed up together with commercial and scientific talks interspersed. This gave the event a much more coherent feel, and has to me been the best, most friendly, most inclusive Europython I've been to. Steve Holden, Chairman of the PSF, said that in his after dinner speech: Python really is about the people. Bruce Eckel had similar feelings in his keynote when he said after a stressful flight and journey to get here he walked into the conference and immediately relaxed with a sigh saying 'Ahhh... python people'.

I really agree with them and I think that python really is a very friendly environment to work in, both the language itself and the amazing community around it.

That said, we have a problem...

Looking at the talk abstracts for Europython there are 97 talks listed. How talks have the word Plone in the abstract? Zero. How about Grok? Zero. Repoze? Zero. Zope? One. That's my talk I did on using Zope Page Templates outside Zope. Silva? Two.

C'mon people, this is shocking! Zope and related projects and technologies have nearly completely dropped off the radar at this conference.

How many talk abstracts mention Django? Thirteen. Turbogears? Two. Pylons? Three.

Today Zope is a very different thing to what it was back then, with the entire Zope 2 application server being eggified and easy_install'able. The Zope Toolkit (previously known as Zope 3) also a collection of independently usable eggs. Technologies such as the ZODB, Page Templates, and Component Architecture are all usable outside of Zope and can be used in general python work. Projects such as Repoze are splitting things up further and allowing Zope to be used in a WSGI stack and re-using parts of the Zope Toolkit to produce repoze.bfg a lighter weight framework. We have zc.buildout which is an amazing tool for deployment of not just Zope projects, not just python projects, but pretty much anything. Grok, a layer on top of the Zope Toolkit provides a very rapid 'convention rather than configuration' approach to MVC web development, much like Rails does for Ruby.

But... I don't think the rest of the python community have quite got this yet. Maybe they still see Zope as 'that strange beast from years back', maybe the Zope community concentrates its resources on speaking at other events, e.g. Plone has not only its annual conference this year in Budapest (which has the same order of magnitude of attendees as Europython, but exclusively focussed on Plone) but additionally both a European Symposium and and US Symposium. That is a lot of time people will be spending traveling and attending and talking at events, but I think we really do need to get some more visible presence at wider python community events. We need to make sure the rest of the Python community see all the fantastic code and products that have come out of the Zope world.

There was a great talk by Martijn Faassen on 'Things I Helped Create' which was a breakneck speed journey through his experience in creativity in general from a small kid to where he is now. Unfortunately he ran out of time before he got really stuck in to all the Zope stuff he has done. It was still a massively enlightening talk. Christian Theune did a tutorial (alas I didn't make the tutorials) on using the ZODB for persisting objects, which would have also made a great talk (or at least lightning talk).

So this is a call to action. Next year Europython will be back again in the UK, and run to the same fantastic standard it was this year. And I want to make sure that there are more Zope/Plone/Grok/etc talks. Specifically I will be banging the drum come next year and really pushing people to do talks.

I'm even going to go out on a limb here and propose a starter list of talks:

• Using the ZODB to Persist Objects
• Using buildout to deploy stuff
• The state of Plone
• Introduction to Zope Component Architecture
• Building a Grok app in 15 minutes.

If you want to find out more about what has been going on at the conference, Reinout van Rees has been doing an excellent job liveblogging the conference.

We have a requirement for an experienced Python/Zope developer to come work on a client project with us here in the UK. The contract will be for approx 10 weeks starting as soon as possible, and will be based in our offices here in Bristol with some time on client site. Unfortunately telecommuting in this instance is not possible -- we need someone physically on site here.

You will be working alongside a couple of other great guys, working on a system that combines Zope 2 and SQL (not Plone) adding additional functionality to an existing system. It is hoped that the new code will be written in a much more modern style using Zope 3 libraries and Five, but a good portion of it is likely to be old-skool Zope 2 development via the ZMI. The role will also involve some HTML/CSS work and writing ZPTs.

So, if you are interested and able to come over to the UK to start work very soon (we can arrange accommodation, etc) then drop me an email (matth@netsight.co.uk) with some details of yourself and your experience.

If you use BeautifulSoup to parse and manipulate HTML, be aware that the prettify() method adds extra whitespace around items which is bad, use str(soup) instead:

eg.

>>> from BeautifulSoup import BeautifulSoup
>>> html = "<em>foo</em>"
>>> soup = BeautifulSoup(html)
>>> soup.prettify()
'<em>\n foo\n</em>'
>>> str(soup)
'<em>foo</em>'

Here's what to do:

wget http://www.ijg.org/files/jpegsrc.v6b.tar.gz
tar zxvf jpegsrc.v6b.tar.gz
cd jpeg-6b/
./configure
make

sudo mkdir /usr/local/include

sudo make install-lib

wget http://effbot.org/media/downloads/Imaging-1.1.6.tar.gz
tar zxvf Imaging-1.1.6.tar.gz
cd Imaging-1.1.6

python setup.py build --force

python selftest.py

sudo python setup.py install

We've now got a PyPI mirror setup for use with easy_install and buildout. The host is pypi.netsight.co.uk. To use it add the following line to the [buildout] section of your buildout config file:

index = http://pypi.netsight.co.uk/

You can also use it with easy_install as well:

easy_install -i http://pypi.netsight.co.uk/ foo.bar

Google's system for serialising data to network/disk. Much simpler than XML to process and much quicker as well:

http://code.google.com/apis/protocolbuffers/docs/overview.html

In a real world scenario I had to get a Plone 3 site to query a different Plone 2 site for certain data to be returned and processed. In the end I used python's pickle.dumps and base64 encoded the result to send, but this would have been a nicer way to handle it.

Say you have a bunch of URLs you want to query at once (aggregating RSS feeds, searching multiple external sites, etc.) normally you would do them one at a time in a loop... well using asyncore in python you can do this in parallel:

http://effbot.org/zone/effnews-1.htm

Here is how you create international characters in email headers. Useful if people have umlauts and accents in their name:

>>> from email.message import Message
>>> from email.header import Header
>>> msg = Message()
>>> h = Header('p\xf6stal', 'iso-8859-1')
>>> msg['Subject'] = h
>>> print msg.as_string()
Subject: =?iso-8859-1?q?p=F6stal?=

More info here:

http://docs.python.org/lib/module-email.header.html

Just a quick read of various strategies for improving python performance. Might be a good read for all developers for things to bear in mind:

http://wiki.python.org/moin/PythonSpeed/PerformanceTips