This document describes a method for quickly importing a large number of documents into Plone. The method involves: 1) Creating lightweight brain objects with minimal metadata for each document and cataloging them in the portal catalog. 2) Using a custom traverser to allow users to access the brain objects. 3) Displaying a view when users access brains to inform them the document needs migration and providing an option to import it. This spreads the import process over many users and makes it much faster than importing all documents at once.

1. fast content import
in Plone
how to import 120k documents
in 8.43 minutes?
http://www.flickr.com/photos/luciano_meirelles/3461046001/sizes/o/

2. The problem
✓ we have 36.000 documents to migrate (which gives 120.000
with all their children)
✓ we don’t want to wait few hours to do that
✓ users shouldn’t be aware of the migration process

3. The Idea
✓ Why not just create brains
and let users decide which
document fully migrate?

4. http://www.flickr.com/photos/oscarmota/1155257083
It’s quite simple

5. First: Metadata
✓ dummy_object is a simple python object
with minimal metadata
>>> pp dummy_object.__dict__
{'title': u'dummy title',
'id': u'simple_id',
'review_state': 'private',
'path': '/plone/importfolder/ToBeMigrated_simple_id'}

6. Second: import script
✓ import script which creates brains in
portal_catalog from our dummy object
pc = getToolByName(self.context, 'portal_catalog')
pc.catalog_object(dummy_object, dummy_object.path)

7. Third: Traverser
✓ Now we need to allow users to see them.
✓ So we created traverser in our import folder
def __bobo_traverse__(self, REQUEST, name):
if name.startswith('ToBeMigrated'):
view = getMultiAdapter(
(self, self.REQUEST),
name='to_be_migrated')
view.setBrainId(name)
return view

8. At the End
✓ The view informs end-
user that this document
need to be migrated
✓ When user decides to
import it - we will start
the import process for
selected brain.uid

9. At the End
✓ The rest is pure Plone
folder_contents will lists
'to_be_migrated' brains in
import folder for you
✓ Simple and much faster

10. Thanks!
Andrew Mleczko
Plone archaeologist
andrzej.mleczko@redturtle.net
http://blog.redturtle.it