Open Source Content Management Systems - A Tale of Two Reports

by Matt Hamilton on Dec 15, 2010
Filed Under:

Last week two very different reports into Open Source Content Management Systems were published:

The two reports look at very different aspects of Open Source Content Management Systems, and are intended to be used in very different ways.

Idealware

The report from Idealware looks at Joomla!, Drupal, Plone and Wordpress and goes into quite significant details on each of them. There is a fully open and disclosed methodology, with details on the funding and contributors to the report. It is 81 pages long -- although a number of pages are adverts for various companies, and a directory of consultants is included. Idealware primarily work with non-profit organisations and the report starts of clarifying what a CMS is and why an organisation would need one. It gives a good rundown of the process of implementing a CMS and gives good sound advice on the work required in terms of tailoring any kind of CMS to your organisation. And goes into detail on each system in 13 key areas:

  • Ease of Hosting and Installation
  • Ease of Setting Up: Simple Site
  • Ease of Setup: Complex Site
  • Ease of Use: Content Administrator
  • Ease of Use: Site Administrator
  • Graphical Flexibility
  • Accessibility and Search Engine Optimization
  • Structural Flexibility
  • User Roles and Workflow
  • Community/Web 2.0 Functionality
  • Extending and Integrating
  • Security
  • Support/Community Strength

One of the very clear things that comes from the report is really how good all these systems have become, yet they all have strengths (and weaknesses) in different areas. As is often said, there is no one 'better' CMS only 'better for my purpose' and of course that is dependant on your intended uses.

A table of results comparing Plone, Drupal, Joomla!, and Wordpress

I hope Idealware don't mind me revealing the punchline here, but this graphic really illustrates this variation, just look at the dark blue circles which represent 'excellent' scores in that area. As you can see Plone and Wordpress are almost complete opposites.

Water & Stone

The Water & Stone report looks predominantly at the brand awareness and market share of 20 Open Source Content Management Systems. Whilst it covers the same four CMSs as the Idealware report it also covers a number of others, some of which I've hardly heard of.

I have to say, I'm not very comfortable with this report. Whilst it does openly say it is looking at the market share of the systems, it does so without any consideration for the use and suitability of those systems to those tasks. For example, is it really any surprise that Wordpress shows up as the highest ranked site by Alexa? Wordpress is a blogging tool with a hosted blogging service, more people setup blogs than, say, setup Alfresco for Enterprise Content Management. Does that make Alfresco a less 'good' tool for the job? No of course not, it is just a different tool for a different job.

Similarly they compare number of mentions of the systems on Twitter. What does this really tell us? I follow what goes on in the Plone world on Twitter and there are more than enough tweets each day for me to keep up with. Drupal looks like it has around 50 times the mentions. That doesn't really help me though if I am interested in Drupal, I'd probably just drown in the volume.

Does it help me choose a CMS? No, not at all. That would be like saying I should plough my field with a Honda Civic rather than a Massey Ferguson just because they are both vehicles and Honda Civic is mentioned most on Twitter. Conversely it doesn't matter how many books are published on Massey Ferguson if one of your requirements is to fit a child seat to your prospective vehicle. As an aside, did you know that Lamborghini used to make tractors?

One aspect that Water & Stone didn't touch on if they are measuring the health and size of the communities for these projects is the number of conferences, developer sprints and training days they have around the world and the number and diversity of attendees.

So whilst this report does offer some interesting insights into various brand metrics of Open Source Content Management Systems, I don't think it offers much value to those looking to choose or evaluate a CMS for their needs.

Filed under: , , , , ,
Jan Ulrich Hasecke
Jan Ulrich Hasecke says:
Dec 16, 2010 06:46 AM

Being "popular" on social media and the web in general can mean that you are doing a bad job and many people are complaining about this. Google might address this problem by adjusting their search algorithm. http://googleblog.blogspot.com/2010/12/being-bad-to-your-customers-is-bad-for.html

Matt Hamilton
Matt Hamilton says:
Dec 16, 2010 10:37 AM

That is a very interesting conundrum for Google. I guess it is the digital equivalent of the saying 'There is no such thing as bad publicity'.

I think it is great to have a report which highlights some of the 'intangibles' of a CMS and its community. The Plone Community is one of the great strengths of Plone, yet it doesn't appear as a tick-box on an RFP. That said, if the Water and Stone report was to truly look at the value of these intangibles then I think it needs to look at things like developer meetups, number of committers, code commit numbers, conference attendees and outputs etc.

-Matt

Ric Shreves
Ric Shreves says:
Dec 16, 2010 01:39 PM

Thanks for the coverage of the report (water&stone). I would point out a couple of things however: First, the report states very clearly on page 1 that it makes no judgements about suitability of systems -- the report is solely about market share and brand strength. This is the third year we've done the survey and the focus has never waivered. Second, your point about WordPress seems to clearly miss that we deal with WordPress.org, not WordPress.com. This is the subject of a discussion early in the paper and where there is likelihood for confusion we highlight it.

Best,
ric shreves
analyst
water&stone

Commenting has now closed on this post.