z3c.autoinclude does not automatically include a plone package

Something I stumbled over just recently and is a PEBKAC again. What happens if you create a plone package and you always need to explicitly specify the zcml slot for that package? Check the namespace declaration in the setup.py. That means, what I had configured:

 namespace_packages=['mooball'],
 entry_points="""
 # -*- Entry points: -*-
 [z3c.autoinclude.plugin]
 target = plone
 """,

But the packages layout was “mooball.portlets.latestcontent”. After digging, I found out that simply the namespace_package is incorrectly declared. Changing it to:

 namespace_packages=['mooball', 'mooball.portlets'],
 entry_points="""
 # -*- Entry points: -*-
 [z3c.autoinclude.plugin]
 target = plone
 """,

… got it correctly included via z3c.autoinclude.

No such file or directory FakePlugin.egg-info

In case you run into a similar issue which ends up in a traceback like this:

ZopeXMLConfigurationError: File "/opt/works/projects/plone4/nuw.types/nuw/types/configure.zcml", line 12.2-12.37
ZopeXMLConfigurationError: File "/home/roman/.buildout/eggs/mooball.plone.activecampaign-0.2-py2.7.egg/mooball/plone/activecampaign/configure.zcml", line 10.2-10.37
OSError: [Errno 2] No such file or directory: '/home/roman/tools/python2.7/lib/python2.7/site-packages/tests/fake_packages/FakePlugin.egg/FakePlugin.egg-info'

you might have made the same mistake then I did: install ZopeSkel in your (system-) python installation. Never do this. Mikko explains on his blog why.

So, check your python installation and uninstall/remove the PasteScript.

handleUidAnnotationEvent from Products.CMFUID throws AttributeError

I stumbled over a weird error during a package setup, which turned out a PEBCAC error. The full traceback:


[...]
File "/home/roman/.buildout/eggs/zope.interface-3.6.3-py2.6-linux-x86_64.egg/zope/interface/adapter.py", line 583, in subscribers
    subscription(*objects)
File "/home/roman/.buildout/eggs/Products.CMFUid-2.2.1-py2.6.egg/Products/CMFUid/UniqueIdAnnotationTool.py", line 86, in handleUidAnnotationEvent
    uid_handler.unregister(ob)
AttributeError: 'NoneType' object has no attribute 'unregister'

I went throught my whole code and type setup unable to find the mistake I’ve made unless I saw the problem. I imported a configuration module for an ArcheTypes content type declaration from a separate package which was not declared in the setup.py of my new package, e.g.:


$ cat my.package.event.py
from separate.package.config import PROJECTNAME
from plone.directives.form import Schema

class IMyType(Schema): ...

If you stumble over this error, make sure your imports are sane.

Broken (Arche-)Type Installation in Tests

I’ve recently run into this issue, migrating some of the Plone 3 product code to Plone 4. Installing some of the old types in my tests ended in a


File "/projects/plone3/my.product/my/product/tests/test_users.py", line 19, in test_mycontent
self.portal.invokeFactory('GalleryFolder', 'spam', title='eggs')
[...]
File "/home/roman/.buildout/eggs/Products.CMFCore-2.2.4-py2.6.egg/Products/CMFCore/TypesTool.py", line 311, in constructInstance
raise AccessControl_Unauthorized('Cannot create %s' % self.getId())
Unauthorized: Cannot create GalleryFolder

Digging deeper I found out, that all of the product factories were gone. Any debugging finding the cause of this ended up in a nowhere, until I stumbled over some odd behavior. All my imports in the testing.py are sorted, e.g.:


from Products.CMFPlone.tests.utils import MockMailHost
from Products.MailHost.interfaces import IMailHost
from Products.SiteErrorLog.SiteErrorLog import SiteErrorLog
[...]
import zope.component

What I now found out by accident is, if you move the import of the MockMailHost to the end of the import block, your Archetypes types are installable again:


from Products.MailHost.interfaces import IMailHost
from Products.SiteErrorLog.SiteErrorLog import SiteErrorLog
[...]
import zope.component
from Products.CMFPlone.tests.utils import MockMailHost

I currently don’t know why, but if anyone had the time to check, I’d be happy to know.

AttributeError when Installing Custom Theme Package

If you’re creating a new Plone theme package based on plone.app.theming and run into the following error:

Traceback (innermost last):
  Module ZPublisher.Publish, line 126, in publish
  Module ZPublisher.mapply, line 77, in mapply
  Module ZPublisher.Publish, line 46, in call_object
  Module Products.CMFQuickInstallerTool.QuickInstallerTool, line 575, in installProducts
  Module Products.CMFQuickInstallerTool.QuickInstallerTool, line 512, in installProduct
   - __traceback_info__: ('my.skin',)
  Module Products.GenericSetup.tool, line 323, in runAllImportStepsFromProfile
   - __traceback_info__: profile-my.skin:default
  Module Products.GenericSetup.tool, line 1080, in _runImportStepsFromContext
  Module Products.GenericSetup.tool, line 994, in _doRunImportStep
   - __traceback_info__: plone.app.theming
  Module plone.app.theming.exportimport.handler, line 40, in importTheme
  Module plone.app.theming.utils, line 427, in applyTheme
  Module plone.registry.recordsproxy, line 43, in __setattr__
AttributeError: currentTheme

This will probably have nothing to do with a wrong setup in your theme, but simply a dependency issue to diazo.
Just make sure you install “Diazo theme support” before your theme is installed.

Reference: http://plone.293351.n2.nabble.com/plone-app-theming-1-0b5-and-Plone-4-1rc2-Plone-4106-td6418670.html

Plone ConfigurationError hides missing configuration files

This one is a quicky. If you run into the following error:


    raise ConfigurationError("Unknown directive", ns, n)
zope.configuration.xmlconfig.ZopeXMLConfigurationError: File "/opt/works/projects/plone4/apf.buildout/parts/instance/etc/site.zcml", line 16.2-16.23
ZopeXMLConfigurationError: File "/home/roman/.buildout/eggs/Products.CMFPlone-4.1-py2.6.egg/Products/CMFPlone/configure.zcml", line 99.4-103.10
ZopeXMLConfigurationError: File "/home/roman/.buildout/eggs/plone.app.dexterity-1.0.3-py2.6.egg/plone/app/dexterity/configure.zcml", line 16.4-16.52
ZopeXMLConfigurationError: File "/home/roman/.buildout/eggs/plone.directives.dexterity-1.0.2-py2.6.egg/plone/directives/dexterity/configure.zcml", line 3.4-3.35
ZopeXMLConfigurationError: File "/home/roman/.buildout/eggs/five.grok-1.2.0-py2.6.egg/five/grok/configure.zcml", line 15.2-15.40
ZopeXMLConfigurationError: File "/home/roman/.buildout/eggs/grokcore.viewlet-1.8-py2.6.egg/grokcore/viewlet/configure.zcml", line 7.2-7.44
ZopeXMLConfigurationError: File "/home/roman/.buildout/eggs/zope.contentprovider-3.7.2-py2.6.egg/zope/contentprovider/configure.zcml", line 12.4
ConfigurationError: ('Unknown directive', u'http://namespaces.zope.org/tales', u'expressiontype')

… and you are using plone.app.theming and dexterity you will want to
check your buildout configuration file.
Make sure you have the dependent configuration files included:


extends =
    http://dist.plone.org/release/4.1/versions.cfg
    http://good-py.appspot.com/release/dexterity/1.0
    http://good-py.appspot.com/release/plone.app.theming/1.0b8

The above configurations can change with future plone releases.

Buildout Entry ‘default’ missing?

I recently ran into the following issue with a Plone 3.3.4 buildout:

While:
  Installing.
  Getting section instance.
  Initializing section instance.
  Loading zc.buildout recipe entry plone.recipe.zope2instance:default.

An internal error occured due to a bug in
either zc.buildout or in a recipe being used:
Traceback (most recent call last):
  [...]
  File "/home/roman/.buildout/eggs/
        zc.buildout-1.4.3-py2.4.egg/zc/buildout/buildout.py",
        line 1009, in _install_and_load
    return pkg_resources.load_entry_point(
  File "build/bdist.linux-x86_64/egg/pkg_resources.py",
        line 305, in load_entry_point
  File "build/bdist.linux-x86_64/egg/pkg_resources.py",
        line 2243, in load_entry_point
ImportError: Entry point ('zc.buildout', 'default') not found

I tried to recompile my python 2.4 (because the OS was just newly installed and my lack on a few dev libraries), pin to different buildout versions but couldn’t figure out what was wrong in the finish.

What I ended up doing in the finish, was:

  1. remove my shared buildout directory (e.g. $HOME/.buildout)
  2. rerun a buildout, which created the eggs directory in the buildout directory which seem to work fine
  3. canceled the local buildout run,
  4. recreated the shared buildout directory with my default configuration (e.g. $HOME/.buildout, $HOME/.buildout/default.cfg and $HOME/.buildout/eggs)
  5. rerun the buildout now using the global eggs directory (e.g. $HOME/.buildout/eggs)

Another thing which I’ve noticed, is that zc.buildout does not have a ‘default’ entry point. I checked releases of zc.buildout back to 1.0. I wonder why it seems to be so arbitrarily being loaded that it works sometimes and sometimes not.

Ah … and btw. always a good one: Sometimes removing the .installed.cfg (it’s hidden with a dot in front!) helps as well.

Running Selenium tests without X11

Update: Most likely my scripts for download will lead into a dead end. Please check the github repository posted by one of my commentators: https://github.com/amenk/SelfScripts/blob/master/selenium-headless

Because writing tests with zope.testbrowser can sometimes be a pain, I recently got a hint from Christian Zagrodnick to have a look back on Selenium again. Selenium plays a pre-recorded, or scripted session to test your web-application in a browser: click here, verify if ‘hello world’ is present, click there and so forth. It’s bit like driving with hands off the steering wheel.

Eleanor Schonell Bridge

Now, I remember having used Selenium before, but

  • it was a nuissance to setup, esp to work nicely with a Plone TestCase
  • getting it setup in a continous integration environment was quite some work too.

Motivation

With Plone 4, a new testcase component is implemented which made it much, much, much easier to write tests for Plone packages. The gocept.selenium packages wraps Selenium in a nice compatible package and provides test classes for Zope2, Plone and Zope3 tests. The result is, that the setup and maintainance of tests with selenium became much, much easier as well.

The only missing piece was the continious integration (CI) environment. I’m running all tests of the packages I develop at Mooball in Hudson continously. Being also able to run all Selenium tests would be a big, big plus. But without an X server, you can’t run a browser, which is essential to run the tests.

I didn’t like the idea of installing a complete X environment on my CI server, so I searched around and found these articles to run Selenium tests without requiring a complete X server installed:

There are more around, but they helped me getting started.

Problem

My main questions were:

  1. How to run the selenium server? Start the server for each build or leave it running as a daemon in the background?
  2. Run an X server (e.g. Xvfb) or an X server with a VNC server, because Hudson already provides plug-ins for the VNC server?
  3. Run the X server per build or leave it running in the background?

Yeh - at night the wheel spins faster ;)

My Solution

1. Selenium Server

I configured the selenium server to run as a daemon. Starting/Stopping the server per test would be too resource intensive and perhaps unreliable. So I came up with a small init script. It allows me to start the selenium server at boot time and keep it running.

2. + 3. The X Server

I opted for installing Xvfb and thought first about starting/stopping it per build, but here again, I think it becomes a nuisance and too unreliable if the server for whatever reason doesn’t start up or crashes. I created another init script to keep the Xvfb running. Note though, I don’t disable the access permissions to connect to Xvfb as most of the other blog articles do.

Installation

The installation is actually pretty much straight forward:

  1. Install java and selenium-server (I preferred /opt)
  2. Configure the init script variables (port, paths, browser, etc). I for one prefer to run firefoxchrome, which works fine for me.
  3. Install Xvfb and adjust the init script variables.
  4. Start both servers. If you’re running your tests with gocept.selenium, follow the installation instructions on the packages pypi page.

That’s it. Any suggestions and updates are welcome 🙂

If acquisition comes in the way

I’ve witnessed a very strange error today.

The problem

I had a custom content-type which was partly indexed. When viewing the content type, parts of the edit form showed raw HTML code instead of widgets:

What was going on?

The solution

I hunted around for a while but checked the contents of the index. Strangely, it had a whole page indexed. After investigating further: it’s another object in the portal with the same id as the attribute on the content-type. The error happens only, if the attribute is missing on the content-type and an object in the hirarchy above has the name id.

So – if you encounter a problem like this, check if the portal_catalog may grab a different object for indexing with the same name of your attribute.

Using z3c.traverser

One of the things which looked so hard, but reveiled using so easy was z3c.traverser. I’m sometimes a bit retarded reading doctests and applying them to my needs. For those who have the same problem, here a small example on how to use the traverser.

The Muffin Traverser

Motivation

Our web application catalogues muffin recipes. Every bloody muffin shop in town has a login to our muffin-application and publishes recipes. To view such a recipe the URLs look like this:

http://muffinsinthehouse.com/shop/recipes/bluemuffin

Now, the customer who operates the site comes to us with a wish: please make the URLs shorter. Each recipe is created with a unique ID and I like to use this id to look up recipes like this:

http://muffinsinthehouse.com/bluemuffin

Implementation

“No worries mate”, you say and use z3c.traverser and implement a custom traverser. The code below shows your custom traverser plugin:

import z3c.traverser.interfaces
import zope.interface
import zope.component
import zope.publisher.interfaces
import zope.app.catalog.interfaces

class MuffinTraverserPlugin(object):
    """Traverser which tries to lookup muffins in the database."""

    zope.interface.implements(
        z3c.traverser.interfaces.ITraverserPlugin)

    def __init__(self, context, request):
        self.context = context
        self.request = request

    def publishTraverse(self, request, name):
        catalog = zope.component.getUtility(
            zope.app.catalog.interfaces.ICatalog)
        result = catalog.searchResults(objectname=name, metatype='Recipe')
        if not result:
            raise zope.publisher.interfaces.NotFound(
                self.context, name, request)
        return result[-1].getObject()

There is nothing really special here, if you know how traversal works in Zope. The plugin is a Multiadapter (a view), which implements z3c.traverser.interfaces.ITraverserPlugin. The object lookup for your muffins is happening in the publishTraverse method. This method either returns the object or raises a NotFound exception if it couldn’t lookup the object. Easy as pie. The catalog is utilised to lookup the object and if its not found a NotFound error is raised.

You register the plug-in with the following ZCML directive:

<adapter
 factory="z3c.traverser.traverser.PluggableTraverser"
 for="zope.traversing.interfaces.IContainmentRoot
 zope.publisher.interfaces.IPublisherRequest"
 />

 <subscriber
 factory=".traversing.MuffinTraverserPlugin"
 for="zope.traversing.interfaces.IContainmentRoot
 zope.publisher.interfaces.IPublisherRequest"
 provides="z3c.traverser.interfaces.ITraverserPlugin"
 />

The first directive “enables” z3c.traverser, the second registers your plugin. After that, try out the custom URL which should work. But wait, if you try to look up other objects in the database, you’ll notice that your traverser deals with them too. That was not the plan, was it?

z3c.traverser provides other plug-ins to deal with folders attributes etc. Don’t lump all those object look-ups into your plug-in! You need to register them additionally:

<subscriber
 factory="z3c.traverser.traverser.ContainerTraverserPlugin"
 for="zope.traversing.interfaces.IContainmentRoot
 zope.publisher.interfaces.IPublisherRequest"
 provides="z3c.traverser.interfaces.ITraverserPlugin"
 />

Try again. This should satisfy the customer.

Comments welcome 😉