[The code below has now been update to reflect the advice from Martin in the comments below]

Plone 4

A few people have asked recently if Plone 4 is ready for production use yet, and so I thought I'd share the experiences of developing the Plone Conference 2010 website, which is running Plone 4.0b3.

The short answer seems to be 'yes'. The longer answer probably depends on if you are starting from scratch on a new project or upgrading an existing Plone 3 site. There are a number of 3rd party Plone products that still need further testing on Plone 4, but I'm please to see that the ones I tried all seemed to work.

Not only are we using Plone 4, but we are also using Dexterity (1.0b1), the new content types system for Plone; PloneGetPaid (0.9.1), the e-Commerce system for Plone; and plone.app.caching (1.0a1), the replacement for CacheSetup which enables Plone to work nicely with HTTP caches such as Varnish (which we are using here). All of these are the released versions from pypi, so no SVN versions needed, and work very well together. So we really are 'eating our own dogfood' here and whilst it might seem a bit scary to be on the cutting edge for a production site, the whole experience has been very good.

Plone 4 is lightning fast, and all the hard work put in by the community really makes it a joy to work with. Restarting Plone during development is much quicker than with previous versions of Plone.

If fact we are using a combination of both Dexterity and Archetypes on this site. All the regular content on the site (folders, news items, pages, images, etc) are Archetypes based, but the content types for the registrations system are Dexterity based. Why? Because the site was developed by two people, and one wanted to stick to Archetypes as that was what they were familiar with, and the other person wanted to give Dexterity a try. Both run side by side just fine.

The Registration System

My goal for the registration system was to make it as simple and user-friendly as possible. So many times previously I have registered for conferences in which the registration system is so convoluted. I especially hate it when I need to register four or five people from Netsight and I have to register each person separately and enter my credit card details over and over again for each registration. One particular European python conference can't handle the credit card owner and the attendee being separate people and so you have to abort the registration process half way through for each attendee, wait for an email to be sent (to the attendee, which they then have to forward to you) with the registration reference number in, you have to phone them read out each reference number and manually process the order!

So I was determined to make this better. I hope I've succeeded. And if you are a person that has to register multiple people for the conference, I hope you appreciate this attention to detail and you can buy me a beer ;)

The reason we tried Dexterity for the registration content types was so that we could easily create a custom 'add' form using z3c.form. By using plone.directives.form we were able to really easily create a custom form which would allow us to tailor the form layout better ie we could easily change the usual 'Save' button which we would have when adding Archetypes based content to Plone to a 'Register' button. We are also able to use Fieldsets from plone.supermodel.model to group our fields together into specific fieldsets to improve the readability of the form.

Using a z3c.form also makes it easier to allow anonymous users to create 'Attendee' objects on the site -- which is how our registration system works:

When someone clicks the 'register' link on the site, it takes them to a custom z3c.form generated form for the Attendee content type. The user fills in their details and hits 'register' this causes an Attendee item to be created in a specific folder in the site, and the item to be then added to the GetPaid shopping cart and takes the user to the cart view page. The user can then register additional people, if they want, before being taken to Paypal to pay for their registration (using getpaid.paypal). Once the payment is successful, Paypal makes a callback to the site which causes GetPaid to transition the order from 'Charging' to 'Charged' state and in turn transition all Attendee objects in that order to the 'Paid' state. The code is really quite simple. First we define our interface for out Attendee object, this is where we define the schema of the object:

# attendee.py
from five import grok
from zope.schema import TextLine, Choice, Bool
from plone.directives import form, dexterity
from netsight.ploneconf2010_registration import _

class IAttendee(form.Schema):
   """A conference attendee"""

   ### Contact details

   firstname = TextLine(
       title=_(u"First Name"),
       )

   lastname = TextLine(
       title=_(u"Last Name"),
       )

   city = TextLine(
       title=u'City',
       required=True,
       )

  shirt = Choice(
       title=u'Plone Conference 2010 T-Shirt size',
       required=True,
       vocabulary="netsight.ploneconf2010_registration.shirts"
       )


#..... further fields omitted for brevity

Then in browser/registration.py we create our custom registration form:

from five import grok
from plone.directives import form

from z3c.form import field, group

from netsight.ploneconf2010_registration.interfaces import IRegistrationFolder
from netsight.ploneconf2010_registration.attendee import IAttendee
from netsight.ploneconf2010_registration import _

class ContactGroup(group.Group):
    label=u"Contact Details"
    description=u"Enter your contact details below"
    fields=field.Fields(IAttendee).select(
        'firstname', 'lastname', 'city', 'postcode', 'country', 'email',
        )

class PreferencesGroup(group.Group):
    label=u"Preferences"
    description=u"Let us know of any dietry requirements you have, t-shirt size and if you will be attending the sprints"
    fields=field.Fields(IAttendee).select(
        'food', 'food_other', 'shirt', 'sprints',
        )


class RegistrationForm(group.GroupForm, form.Form):
   grok.name('registration')
   grok.require('zope2.View')
   grok.context(IRegistrationFolder)

   schema = IAttendee
   ignoreContext = True

   label = _(u"Register for Plone Conference 2010")
   # we want to further customise the form...
   template = ViewPageTemplateFile('templates/registration_form.pt')

   # Group the fields into fieldsets, only included a few for brevity
   groups = (ContactGroup, PreferencesGroup)

   # We want the fieldsets presented one below the other, not in tabs
   enable_form_tabbing = False

   def update(self):
         # disable Plone's editable border
         self.request.set('disable_border', True)

         # call the base class version - this is very important!
         super(RegistrationForm, self).update()

   @button.buttonAndHandler(_(u'Register'))
   def handleApply(self, action):
         data, errors = self.extractData()
         if errors:
             self.status = self.formErrorsMessage
             return

         # Create the attendee object in the attendees folder
         # setting checkContraints to false means we skip the security checking and allow anon to add
         attendees = self.context.attendees
         attendee = createContentInContainer(attendees, 'netsight.ploneconf2010_registration.attendee', 
                                           checkConstraints=False, **data)

       # Add attendee to shopping cart
       self.addToCart(attendee)

       portal_url = getToolByName( self.context, 'portal_url').getPortalObject().absolute_url()
       return self.request.response.redirect('%s/register/@@getpaid-cart' % portal_url)    

I wanted to add a chunk of HTML at the start of the form, with a table of prices and instructions on payment options, so I created a custom template referred to in the form with the HTML in templates/registration_form.pt

<html xmlns="http://www.w3.org/1999/xhtml"
     xmlns:metal="http://xml.zope.org/namespaces/metal"
     xmlns:tal="http://xml.zope.org/namespaces/tal"
     xmlns:i18n="http://xml.zope.org/namespaces/i18n"
     i18n:domain="plone.z3cform"
     metal:use-macro="context/main_template/macros/master">

   <metal:block fill-slot="main">

       <h1 class="documentFirstHeading" tal:content="view/label | nothing" />

       <img src="http://farm4.static.flickr.com/3505/4074083883_797e6c371f_m.jpg"
            align="right" width="240" height="169" alt="Plone Conference 2009 delegates" />

       <p>To register for Plone Conference 2010, fill in the registration form below.
         If you are registering a number of people then you will have a chance to add
         multiple registrations to your cart before checking out to pay.</p>

       <!-- more HTML -->

      <div id="content-core">
           <metal:block use-macro="context/@@ploneform-macros/titlelessform" />
       </div>

   </metal:block>

</html>

This give me the flexibility to add extra stuff to the form, but to retain all of z3c.forms machinery for generating the form from the IAttendee interface and to do all the validation and checking of mandatory fields. You may notice that I set the view to be registered to IRegistrationsFolder, read my previous blog post for an easy way to set the default view on an arbitrary folder.

A further refinement to the Attendee object was to get it to be able to return a title from the first and last names entered. Due to Plone using the catalog for folder view listings, and for legacy reasons it looks for an attribute or method on an object called 'Title' we needed to add this method to our class. So the extended version of the attendee class is:

# attendee.py
from five import grok
from zope.schema import TextLine, Choice, Bool
from plone.directives import form, dexterity
from netsight.ploneconf2010_registration import _

from plone.app.content.interfaces import INameFromTitle
from plone.dexterity.content import Item

class Attendee(Item):
   """ An attendee """

   def Title(self):
       """ return the title """
       return INameFromTitle(self).title


class IAttendee(form.Schema):
   """A conference attendee"""

   ### Contact details

   firstname = TextLine(
       title=_(u"First Name"),
       )

   lastname = TextLine(
       title=_(u"Last Name"),
       )

   city = TextLine(
       title=u'City',
       required=True,
       )

  shirt = Choice(
       title=u'Plone Conference 2010 T-Shirt size',
       required=True,
       vocabulary="netsight.ploneconf2010_registration.shirts"
       )


#..... further fields omitted for brevity

class TitleAdapter(grok.Adapter):
   grok.provides(INameFromTitle)
   grok.context(IAttendee)

   def __init__(self, context):
       self.context = context

   @property
   def title(self):
       return '%s, %s' % (self.context.lastname, self.context.firstname)

By registering an adapter for INameFromTitle is means that the object creation mechanism can look up what the title of the object is, and then convert that to a more meaningful id for the object created.

GetPaid

This was the first time I'd looked at GetPaid, and credit to all the people who have worked on it, it really is a fantastic product. Previously we have always ended up coding our own eCommerce products for Plone from scratch as every customer's requirements have been different enough to make using any of the existing shopping cart systems for Plone difficult. They always seemed to have made assumptions about a particular country's tax or shipping system that didn't work for us. And making small changes always ended up in us subclassing the entire system, which was a pain. GetPaid uses the Zope Component Architecture to the full and the ability to simply mark any regular piece of content as buyable makes building stores with Plone a breeze.

Imagine you wanted to create a shop online that sold jewellery. With just stock Plone and PloneGetPaid installed you could simply upload photos of each jewellery piece using the standard Plone image content type, and set the folder to thumbnail view. Then mark each image as 'buyable' and give them a price and stock code and you are done. Add getpaid.paypal and configure a Paypal account in a few minutes and you can now accept payment. No custom coding needed whatsoever. No merchant account needed. Combine that with collective.xdv and a free theme from one of the free template sites, and you are set.

Conclusion

This was the first time I've actually used z3c.form from within Plone, and combined with Dexterity it provides a very powerful system which makes it very easy to quickly develop content types and really nice flexible forms for them. I'm looking forward to using this combination in future Plone projects. The only area where Archetypes still leads is the number (and polish) of some of the widgets, but I think that z3c.form is catching up quickly. Porting over some of the more complex Widgets from Archetypes to z3c.form would be a fantastic sprint topic, and if no-one gets there beforehand, I'm going to raise it as a topic for the sprint after the Plone Conference.

Plone 4 really is amazing, and the best version of Plone yet.

See you at the Plone Conference 2010 in Bristol! Registration is Open!

-Matt

This is mainly a note for others looking to do the same. I'm sure this is documented somewhere, but I thought I'd put it here (mainly for my own record!).

I needed to set a specific view for a folder in Plone. I had created a view called 'registration' that was configured via a grok directive:

class RegistrationForm(form.SchemaForm):
    grok.name('registration')
    grok.require('zope2.View')
    grok.context(IRegistrationFolder)

    schema = IAttendee
    ignoreContext = True

    label = _(u"Register for Plone Conference 2010")
    description = _(u"A description about the registration")
    ....

I then created a marker interface in interfaces.py:

from zope.interface import Interface

class IRegistrationFolder( Interface ):
    """ marker interface for the registration folder """

I then created a regular Folder in Plone called 'Registration' and published it. I then went into the Zope Management Inferface and found the folder and then clicked on the 'Interfaces' tab and then checked the checkbox by 'netsight.ploneconf2010_registration.interfaces.IRegistrationFolder' and hit 'Add'.

This then allows me to go to /registration/@@registration and get my view on that folder.

However I wanted to go one step further and make @@registration the default view for the folder so that I can just go to /registration and show my new view. Geir Baekholt pointed out to me on #plone on IRC that the default view name is stored in a property on the folder called 'layout'.

So I went back to the ZMI to my folder and clicked on the 'Properties' tab and added a string property called 'layout' and set it to '@@registration'.

Now when I go to /registration (or click on 'Registration' in the navigation on the site) I get my custom view.

Nice and easy, and not need to subclass Folder just to add a default view, or create a RegistrationsFolder content type.

Having been part of the team exhibiting on the Plone stand for two previous technology shows, IMS and TFM&A, it was a nice change to be on the other side when I visited Internet World this week. Seemed a busy show, particularly the talks which resembled a popular Boxing Day sale.

However, for me, the single biggest surprise from the show was the sales methods I encountered from some CMS (Content Management System) vendors. Particularly when they realised I was from a digital agency. Even as someone who's involved in business development, the number of times that kickbacks and mark-ups (linked directly to the licence fee) were promised to me seemed to be, dare I say it, a little bit underhand and desperate. Next time someone recommends a particular piece of licensed software to meet your needs, certainly worth double-checking why.

I was also amazed at the details of some so-called 'transparent pricing' schemes - where some vendors will charge you additional licence costs for any of their (sometimes as many as forty) plug-ins, covering requirements as common as a contact form. And then want to charge you the whole licence figure again for using the same CMS on a second website.

Admittedly, I'm probably dwelling on the worst examples, rather than the many people who were going about their business professionally in the very crowded proprietary CMS market.

But the whole experience made me realise the extent of the positive message behind Open Source software. Not being a developer, I take it for granted that there's a huge library of plug-ins that can be offered to our clients at no licence cost. And that when someone likes an Open Source CMS so much they recommend it to their colleagues for something similar on another site, it's nice that there's not a large licence cost to bear.

At the end of the day, obviously all companies need to make a living - and Open Source providers are no different. But even if you never need access to a program's Open Source code, there's also something to be said about the value for money and culture of common sense and fairness that also comes with choosing Open Source.

One of the sites I'm currently working on has a content type called a "Report". The main purpose of the site is to display these reports, which are each contained within a folder-ish content type called a "Report Section". There are a number of Report Sections within the site.

This morning I was told that our client wanted each "Report" to be able to live in multiple sections. Aaaargh!

Initially we considered hard-coding the navigation menus so that the (limited number of) reports they wanted in multiple places would appear in the relevant report listings. However, this seemed like too much of a hack and didn't feel like the right solution. What we really wanted was the ability to create an Alias in one place that would point to a piece of content somewhere else, and in-turn behave as though it was the other piece of content.

I looked at the following two products but they weren't quite what I wanted:

Simple Alias collective.alias

Simple Alias was too simple for my needs, and Martin Aspeli's collective.alias did too much.

The Problem

I put together the following requirements:

• It should be possible for the same piece of content to appear in multiple locations throughout the site
• To a user, the content should appear to be in each distinct location
• It should be very easy for content editors to add an Alias to existing content and have everything just work
• It should only be possible to edit the original content. It shouldn't be possible to edit the same content in multiple locations

The use case is a relatively simple one.

The Solution

The solution I came up with was as follows:

New content type: AliasReport

Reference field: aliasedReport --> Report

Allow AliasReport to be added wherever it was possible to add Report.

Wherever we search the catalog for a Report, search for AliasReport too:

[meta_type=('Report', 'AliasReport')]

Point to a subclass of each View in configure.zcml:

<browser:page
     for="Products.ContentTypes.interfaces.IReport"
     class=".views.Report"
     name="report"
     template="templates/report_view.pt"
     permission="zope2.View"
     />

<browser:page
     for="Products.ContentTypes.interfaces.IAliasReport"
     class=".views.AliasReport"
     name="report"
     template="templates/report_view.pt"
     permission="zope2.View"
     />

Add the view, and define self.report as the actual Report content:

class Report(BrowserView):
   def __init__(self, context, request):
       self.context = context
       self.request = request
       self.report = context

class AliasReport(Report):
   def __init__(self, context, request):
       self.context = context
       self.request = request
       self.report = context.getAliasedReport()

Whenever we refer to an attribute or method of the context in the templates or views, instead refer to view/report. However leave context/absolute_url (and similar calls) so that the content appears in the Aliased place.

<h1 tal:content="context/document_title" />
<a tal:attributes="href string:${context/absolute_url}/buy">Buy this report</a> 

[http://site/section-one/fancy-report/buy]

<h1 tal:content="view/report/document_title" />
<a tal:attributes="href string:${context/absolute_url}/buy">Buy this report</a> 

[http://site/section-eighteen/fancy-report/buy]

And that's it. Content editors can add the Report content type into any Report Sections throughout the site. They can then add any number of Aliased Reports that point to Report content, and everything else will just work.

Extra Bit

In the first version I got working, instead of using view.report, I just redefined the context in the Aliased view class:

def __init__(self, context, request):
   self.context = context.getAliasedReport()

However it didn't feel "right" to be editing the content in the wrong place, so I decided to go with the above solution.

Today I read a great article by Jeff Potts of Optaros: A CMIS API library for Python, Part 1: Introducing cmislib. It talks about how to use the CMIS python library to mess about with CMIS and access CMIS repositories.

I've been mulling over the significance of CMIS for a while now, and still a little undecided. Yes I think its a great idea, but in reality will it actually be useful? As it is a lowest-common-denominator approach to interoperability, I'm betting that much of the value will be lost when using it for widescale migration from one CMS to another. There are already plenty or protocols for getting data in and out of systems, why add another?

I do however think it might be interesting to use it for Plone to allow Plone to access other systems as a content repository. In this case, I'm not talking deep-down integration (ie, I'm not talking about replacing the ZODB with a CMIS backend or even an Archetypes storage layer). At the moment I'm thinking quite lightweight integration at specific points. Ie imagine a portlet on a Plone-based intranet which lists the 10 most recent policy documents stored in an Alfresco/Sharepoint/Filenet system.

So I grabbed Jeff's library and had a quick play with it this morning to see what sort of thing could be done. I've kept my scope pretty small, and of course there is way more that could be done here, but I just wanted to hack together something quick to just see how it worked.

Results

The results worked pretty nicely. The version of cmislib on pypi (0.3dev) has some issues with Alfresco's latest server, meaning I couldn't do much, other than do a search and list the titles, but Jeff says that the svn trunk version fixes these.

I add my 'CMIS View' object to a Plone site, and give it the repository url, username and password:

And hit Save, and then I get a listing of the items on the CMIS repository. Here we have a listing of all items in the Alfresco public repository which contain the word 'test'.

How I did it

It was actually pretty simple, in fact most of my time was spent working out how to re-use the existing plone folder_contents template for my listing and how to actually use CMIS.

So firstly I added the cmislib package and my package I was going to develop to a fresh Plone 4.0b1 buildout:

[buildout]
...
eggs =
     cmislib
     collective.cmisview

# Reference any eggs you are developing here, one per line
# e.g.: develop = src/my.package
develop =
     src/collective.cmisview

[instance]
...
zcml =
    collective.cmisview

I then created a simple package:

matt-hamiltons-macbook-air-3:src matth$ ../../bin/paster create -t archetype
matt-hamiltons-macbook-air-3:src matth$ ../../bin/paster create -t archetype
Selected and implied templates:
  ZopeSkel#basic_namespace  A basic Python project with a namespace package
  ZopeSkel#plone            A project for Plone products
  ZopeSkel#archetype        A Plone project that uses Archetypes content types

Enter project name: collective.cmisview
Variables:
  egg:      collective.cmisview
  package:  collectivecmisview
  project:  collective.cmisview
Expert Mode? (What question mode would you like? (easy/expert/all)?) ['easy']:
Project Title (Title of the project) ['Example Name']: CMIS View
Version (Version number for project) ['1.0']: 0.1
Description (One-line description of the project) ['']: A simple content type for Plone that allows viewing a \
CMIS repository
Creating template basic_namespace
Creating directory ./collective.cmisview
...

And added a simple content type:

matt-hamiltons-macbook-air-3:collective.cmisview matth$ ../../../bin/paster addcontent contenttype
Enter contenttype_name (Content type name ) ['Example Type']: CMIS View
Enter contenttype_description (Content type description ) ['Description of the Example Type']: View of a remot\
e CMIS repository
Enter folderish (True/False: Content type is Folderish ) [False]:
Enter global_allow (True/False: Globally addable ) [True]:
Enter allow_discussion (True/False: Allow discussion ) [False]:
  Inserting from README.txt_insert into /Development/py26nsp2/cmis/src/collective.cmisview/collective/cmisview\
/README.txt
...

I then added the schema to the generated cmisview.py file:

CMISViewSchema = schemata.ATContentTypeSchema.copy() + atapi.Schema((

        atapi.StringField(
            name='cmisrepourl',
            widget=atapi.StringWidget(
                label='CMIS Repository URL',
                ),
            ),

        atapi.StringField(
            name='cmisrepouser',
            widget=atapi.StringWidget(
                label='CMIS Repository Username',
                ),
            ),

        atapi.StringField(
            name='cmisrepopass',
            widget=atapi.PasswordWidget(
                label='CMIS Repository Password',
                ),
            ),

))

and added a view class in browser/cmisbrowserview.py with a method to do the actual CMIS search:

     def getdocuments(self):
	client = CmisClient(self.context.getCmisrepourl(),
                            self.context.getCmisrepouser(),
                            self.context.getCmisrepopass()
                            )
	repo = client.defaultRepository
	res = repo.query("select * from cmis:document where contains('test')")
	return res

I added a method to iterate over the results and build up something that the standard folder_view template would like. Obviously if I was doing this in ernest I would create a custom template specifically suited to the CMIS results view.

    @property
    def items(self):
	results = []
	i = 0
        for doc in self.docs:
            if (i + 1) % 2 == 0:
                table_row_class = "draggable even"
            else:
                table_row_class = "draggable odd"

            props = doc.getProperties()
            results.append(dict(
                url = '',
	        url_href_title = '',
	        id  = doc.id,
	        quoted_id = urllib.quote_plus(doc.id),
		path = '',
                title_or_id = safe_unicode(doc.title),
	        obj_type = '',
	        size = props['cmis:contentStreamLength'],
		modified = props['cmis:lastModificationDate'],
		icon = '',
		type_class = '',
                wf_state = '',
                state_title = '',
                state_class = '',
                is_browser_default = False,
                folderish = False,
                relative_url = '',
		view_url = '',
                table_row_class = table_row_class,
	        is_expired = False,
	    ))

            i += 1
        return results

And that is pretty much it. Clearly there is a lot more you could do here, such as tidying up the info coming back (formatting the size in the listing) and registering a traversal adapter which would enable you to actually download the document listed. Maybe I'll do this once the next version of cmislib hits pypi.

We are in the process of building a new website for Netsight, and one of the items on the wish-list was a 'related items' portlet for the blog. With Plone you can manually related items, but there isn't really a way to display related items automatically. Many years ago we wrote something that did this on a client's site in Plone 1, but after a while the client asked it to be removed as it was bringing up some slightly embarrassing and controversial supposed related items.

I now wanted to resurrect the idea and create a Plone 3 portlet that did something along those lines. I'd seen topia.termextract a while back and was thinking of a fun use to put it to, and also saw collective.classification which sounded like it did something similar, but needs various external dependancies to be build (incl. C libraries I think, and download a large training corpus for it).

My final year thesis at university in 2000 was a full text indexer written in C, and so I had a fairly good grasp of relevance ranking algorithms and the likes. Indeed, back in 2002 at the first non-US Zope 3 sprint, I introduced Jim Fulton to 'Managing Gigabytes' a seminal book on indexing and information retrieval... this then lead to the creation of ZCTextIndex, the text index used in Zope today.

So I knew from the ZCTextIndex code that we should already have much of the information needed for determining similar content already calculated in the text index data structures -- which means it should be pretty fast.

The basic idea is this:

1. Find the most 'important words' in the document you are looking at
2. Search for other 'similar' documents based on those words

So how do we find the most 'important' words? Well in text indexing there is a common metric called TF*IDF. This is the Term Frequency by Inverse Document Frequency. Basically, an important word is a word that appears in this document at a higher frequency compared to other documents.

As an example the word 'the' appears in a specific document roughly the same number of times as all documents in our entire site. It also appears in virtually every document. So it is not that special. Whereas, the word 'conference' might appear in our specific document a number of times, yet doesn't appear that often in general, and doesn't appear in that many documents overall -- hence it is pretty 'important'.

The SearchableText ZCTextIndex already has a way to find all words in a document efficiently (ie. without having to parse the document again) as this information is stored in the indexes. It also stores various frequency and weight information -- ie everything we need for our calculation.

This means we can iterate over every word in our document, score each one as to how 'important' it is, and then return the top 20 words.

Once we have those words, we are onto the second part of the process and we search the catalog for all documents that match these terms. To do this efficiently, again we have to delve into the internals of ZCTextIndex and call some private methods. I know this is bad form, but is really needed for efficiency. If we used the public API to do the search then the catalog would treat the 20 terms in our query with an implicit 'and' which means that if any one of the terms doesn't appear in a candidate document then it will be excluded. This is not what we want, we want that document to be included, but if a specific term doesn't appear then just don't rank it as high.

The end result is we get a list of documents related to the one we are looking at:

I later did a bit of an experiment to see how topia.termextract would fare in comparison to my TF*IDF approach to determining which words at the most important in a document. The results were actually surprisingly close:

topia.termextract:
['2010', 'venue', 'people', 'idea', 'year', '/', 
'university', 'uk', 'bristol', 'number', 
'community', 'conference', 'city', 'lot', 'room', 
'work', 'conference', 'talk', 'plone']

tf.idf:
['bristol', 'conference', 'venue', 'ploneconf2010', 
'suits', 'rooms', 'vote', 'silicon', 'city', 
'media', 'talks', 'university', 'bid', 'delegates', 
'lots', 'bbc', 'west']

So topia.termextract is returning a list of all nouns it knows of in the document, and tf.idf is returning all words in this document that occur more frequently than average. You can probably guess the content of the document they were both looking at ;)

So for now, I'll keep with my tf.idf approach, but maybe in the future it might be interesting to see how topia.termextact can be integrated, especially if you include the phrases that topia.termextract can find:

['plone projects', 'south west', '600,000 people', 
'plone conference', 'plastercine characters', 
'computer science', 'advocacy work', 
'silicon valley', 'case studies', 
'plone conferences', 'plone community', 
'cycle city', 'industry analysts', 
'technology evaluators', ...]

The product is available to download from plone.org or via buildout and PyPi if you want to install it, just add collective.portlet.similarcontent to your eggs and zcml lines in your buildout config file.

Well, we have about seven months until the Plone Conference rocks up in Bristol in October, and we've been busy securing the venue.

I am now proud to announce the Plone Conference 2010 will be held at the 4* Thistle Grand Hotel in central Bristol.

The hotel is located in the central old part of Bristol City, with its cobbled streets and old buildings. We have four rooms booked in the hotel for the main conference, the largest holding 400 people, the others holding around 90 people each. The hotel also can offer us around 60 rooms for delegates to stay in during the conference, and there is a wide number of other hotels of various costs, serviced apartments and youth hostels in walking distance.

There is also plenty of social life around with a large number of restaurants, bars and clubs within a few minutes walk. All delegates will also have passes to the hotel's leisure centre as well.

But beyond all that, this hotel actually has quite a nice historical connection with what we are doing -- a connection I only discovered when searching for photos for this blog post.

Our main aim with the work we do with Plone and on the internet in general mostly revolves around communication. We allow others to communicate information better. Whether that be by designing public facing websites, private corporate intranets, or group collaboration spaces... we enable communication.

The Thistle Grand stands on the site of a previous important link in the world of communication -- the mail. Before the Grand Hotel there stood the White Hart and White Lion Inns. Looking at the lithograph above and the modern photo at the top you can see the same church tower behind the site of the venue. The White Lion Inn can be traced back to 1606 as an Inn and went on to become a very important part of the Kings Post:

"The White Lion, Bristol, was one of the most 
famous coaching houses in England, east, west, 
north, or south. It stood in Broad Street, a 
thoroughfare which belied its name as regards 
breadth, and could only be considered broad by 
comparison with the even narrower Small Street, 
which ran parallel with it. Yet at one time 
there were as many coaches passing in and out 
of Broad Street as any street in Bristol, or 
even in London!"

This excerpt it taken from a book originally published in 1905 entitled "The King's Post -- Being a volume of historical facts relating to the Posts, Mail Coaches, Coach Roads, and Railway Mail Services of and connected with the Ancient City of Bristol from 1580 to the present time." published online by Project Gutenberg.

Back then, with no telegraph or telephone (or even internet!) Bristol used to even have its own time, being 2 degrees 30 minutes west of London it was approx 10 minutes behind London on time, evidence of which can still be seen on the clock above the Corn Exchange in the centre which has two minute hands, one for London and one for Bristol:

With the advent of the railway, and Bristol's famous son, Isambard Kingdom Brunel who build the Great Western Railway, Bristol Time was subsequently abolished and Railway Time adopted in 1852.

Anyway, that is the history lesson over. The main thing to be excited about is we have a venue :)

We are now currently selecting a venue for the conference dinner/party and our shortlist includes a former railway terminus (the world's earliest surviving purpose built railway terminus), an art gallery, a victorian bank, and an arts and media centre. We just need to work out which one will work out best for us!

As for talks, training and all that... We'll be putting a call out for training proposals and talk submissions once Plone Symposium East and European Plone Symposium are out of the way. We will be opening booking for the event in June.

-Matt

We are packing up the van again today and heading up to the big smoke to man a Plone booth at the Technology for Marketing and Advertising expo in Earls Court, London.

If there are any fellow Plone users or developers going along, please drop by and say hi. Alas, I won't be going myself this time, but have a chat with my colleagues Chris Green and Chris Edge.

The expo is running today and tomorrow. You can register online at their site and get a free entry badge.

-Matt

At EuroPython last year, Matt Hamilton gave a talk on using Zope Page Templates (ZPT) outside of Zope. ZPT uses the Template Attribute Language (TAL) to create dynamic templates that you can use in your own web applications, reporting frameworks, documentation systems or any other project.

Why TAL?

The point of this post isn't to go into detail about why you should use ZPT/TAL. Suffice to say that TAL:

• Makes well-formed XHTML easy
• Ensures that you close all elements and quote attributes
• Escapes all ampersands by default & -> &

The Django Templating Language:

<ul>
  {% for name in row %} 
    <li>{{name}}</li>
  {% endfor %}
</ul>

TAL:

<ul>
  <li tal:repeat="name row"
      tal:content="name">
    Dummy data
  </li>
</ul>

Using ZPT in your own project

There are three steps to using ZPT in your own project.

• Install ZPT (via the zope.pagetemplate package and its dependencies)
• Create a template file
• Render the template file using the data from your application

Install zope.pagetemplate

I recommend using virtualenv for each new application.

# virtualenv zptdemo
# cd zptdemo
# bin/easy_install zope.pagetemplate

Create a template

mytemplate.pt

<html> 
  <body> 
    <h1>Hello World</h1>
    <div tal:condition="python:foo == 'bar'">
      <ul>
        <li tal:repeat="item rows" tal:content="item" /> 
      </ul> 
    </div>
  </body>
</html>

Render the template

mycode.py

from zope.pagetemplate.pagetemplatefile \
    import PageTemplateFile

my_pt = PageTemplateFile('mytemplate.pt')
context = {'rows': ['apple', 'banana', 'carrot'], 
           'foo':'bar'}
print my_pt.pt_render(namespace=context)

And that's it. This will generate the following:

Hello World

• apple
• banana
• carrot

Google App Engine

This is all very well if you're able to install your own packages, but Google App Engine (GAE) doesn't allow you to do this. You can, however, include packages with your application, which is what we'll do here. When installing zope.pagetemplate into your virtual environment, you may have noticed that the following packages were installed:

• zope.pagetemplate
• zope.i18nmessageid
• zope.interface
• zope.tal
• zope.tales

We can extract all these files and put them into our GAE application.

I've already done this for you, and you can grab a copy of the 'zope' folder on GitHub here. Put this folder into the top level of your application.

Create a template as described above, and then place the Python code that renders the template into your application's RequestHandler. It should end up looking something like this:

class MainHandler(webapp.RequestHandler):

  def get(self):
    my_pt = PageTemplateFile('main.pt') 
    context = {'rows': ['apple', 'banana', 'carrot'], 
               'foo':'bar'}
    self.response.out.write(my_pt.pt_render(namespace=context))

At this point you have ZPT working in GAE and you can take advantage of all the features that ZPT and TAL provide.

The application on GitHub can be download, tested and even deployed straight to Google App Engine without modification. You can see an example of it running here.

update: it's also possible to use Chameleon on Google App Engine, which as Martin says may be easier and faster.

Well, we've finally done it! For a number of years now at Plone conferences around the world, I've said "Wouldn't it be great to hold the Plone Conference in the UK?' and each successive year I seem to say it after fewer and fewer beers. This year, we've finally done something about it, and put a bid in to host the Plone Conference 2010 in the UK. Not only that, but the bid was accepted :)

So now comes the hard work, in terms of planning the whole event. I think if we get things started nice and early then we should be OK. So far I've been in touch with a number of previous Plone Conference organisers and got quite a good deal of feedback from them on things that worked or didn't work from their point of view with the management and running of the conference. Having been to every conference except the first one myself, I've got a pretty good idea of what makes them work from a delegates point of view (decent wifi and lots of coffee! ;) ).

Bristol's Biggest Bike Ride. Bristol is the UK's first 'Cycle City' and is known for its work on environmental issues. This photo is taken in front of @Bristol, a science museum. The big silver dome is the planetarium. Those that came to the Performance Sprint in Bristol might recognise it, as was just around the corner from the sprint venue.

Bristol is a fantastic city, located in the South West of the UK with a population of around 600,000 people. Despite its modest size, it has a very high regard with technology and media. The BBC Natural History unit is here, and a lot of BBC Dramas filmed in the city, and around this spawned a whole ecosystem of related media businesses such as post-production, graphics, audio, costume, etc. When the term 'New Media' started to surface in the early 90's Bristol was hot on the scene and as a result a large number of digital / interactive agencies formed. Bristol is also home to Aardman Animations, the creators of Morph, Wallace and Gromit, Shaun the Sheep and other well known plastercine characters. Bristol is also referred to as the 'Silicon Valley of the UK' with a whole host of silicon chip manufacturers and related businesses here.

Bristol Cathedral in the foreground and the Wills Memorial Building, part of the University of Bristol in the Background. The University received its charter in 1909, and is now one of two universities in the city -- the other being the University of West England. This building was built between 1912-1925 and considered one of the last great Gothic buildings to be built in the UK.

I came to Bristol in 1996 to attend the University of Bristol to read Computer Science, and graduated four years later to form Netsight, based just over the road. in the next 10 years, Netsight has grown to 13 staff and become well known within the Plone community for our advocacy work, and promotion of Plone.

Each year, the Plone Conference gets better and better, so I'm under no doubt that I have some pretty high expectations to meet. Last year I was leading the programme committee for the Budapest conference and we had an amazing number of talk submissions, way more than we had room to fill and we had a tough choice picking the talks. This year I want to get direct feedback from the community and get the talks organised earlier and get them all up online so the whole community can see them and vote on the talks they would most like to see. That way we can be as transparent as possible, but also make sure that we give a chance to those potential delegates that are not yet so involved with the Plone community to have a vote in which talks they want to see. It is always hard to make sure we get a good balance of talks for those new to Plone and those with lots of experience. It will also give us a chance to gauge the talk popularity beforehand and schedule the talks in the appropriately sized rooms.

Speaking of rooms, yes we do have a venue already in mind. We are talking to more than one and just negotiating the final details with them before we make it concrete and announce the venue.

One item you might have seen on the preliminary conference page is the one-day 'Analysts / Customers / Suits' mini-conference. The goal of this day is to attract those people whom we might not normally reach so easily as a (generally developer-focused) community. For this I'm talking about industry analysts and pundits, journalists, technology evaluators, customers, etc. I don't mean to be glib when I say 'suits' but hopefully most people will get what I mean. The main conference has always historically been very inward focussed as a community. Events like World Plone Day have shown the value of reaching out to a wider audience and I see the conference as an ideal time to organise something. This was if someone wants to they can come to the one-day conference and stay for the main conference as well. For the one-day event I'm hoping to invite a number of speakers to come and present case studies of Plone projects they have been involved in.

I'm open to ideas as well for all aspects of the conference. If you have any particular ideas you'd like to see, or want to get involved in the conference then by all means drop me an email on ploneconf2010@netsight.co.uk. I'm going to me asking around the community for volunteers (esp. from the other UK Plone companies) as well closer to the time.

-Matt