Migrating from drupal to plone with transmogrifier

Migrating From Drupal
to Plone with
Transmogriﬁer
Clayton Parker, Senior Developer
PLONE SYMPOSIUM EAST 2011

Who Am I? PLONE SYMPOSIUM EAST 2011

• claytron
• Python dev since 2003
• Plone Core Committer
• Foundation Member

What will we learn? PLONE SYMPOSIUM EAST 2011

• Transmogriﬁer basics
• Migration Planning Process
• Creating a pipeline

Migrations PLONE SYMPOSIUM EAST 2011

• One off scripts
• In multiple places
• Little to no re-usability

Transmogriﬁer PLONE SYMPOSIUM EAST 2011

• A framework for migrations
• Re-usable parts

Basics PLONE SYMPOSIUM EAST 2011

• Pipeline
• Blueprints
• Sources
• Section

Sources PLONE SYMPOSIUM EAST 2011

• A blueprint
• First item in your pipeline

Example Pipeline PLONE SYMPOSIUM EAST 2011

[transmogrifier]
pipeline =
csv_file
constructor
schemaupdater
<html>
<body>
[csv_file]
<h3>Code Sample</h3>
blueprint = collective.transmogrifier.sections.csvsource
<p>Replace this text!</p>
filename = my.migration.import:my_items.csv
</body>
</html>
[constructor]
blueprint = collective.transmogrifier.sections.constructor

[schemaupdater]
blueprint = plone.app.transmogrifier.atschemaupdater

my_items.csv PLONE SYMPOSIUM EAST 2011

_path , _type , title , description
/folder1 , Folder , First Folder , This is folder One
/folder2 , Folder , Second Folder , This is folder Two
/folder1/foo , Document , One Foo , A document named foo
/folder2/foo , Document , Two Foo , Another doc named foo

The Result PLONE SYMPOSIUM EAST 2011

Items PLONE SYMPOSIUM EAST 2011

• Each item is a mapping
• Keys are ﬁelds
• Keys with a leading underscore are controllers

Example Item PLONE SYMPOSIUM EAST 2011

{'_id': 'a-stronger-connection-to-out-customers',
'_path': 'content/stronger-connection-out-customers',
'_status': 1L,
'_text_mimetype': 'text/html',
'_transitions': 'publish',
'_type': 'Document',
'allowDiscussion': False,
'creation_date': '2011/05/14 9:20:50',
'effectiveDate': '2011/05/14 9:20:50',
'modification_date': '2011/05/18 9:22:22',
'subject': 'betterninterestingnstronger',
'text': '<p>this is some text</p>rn',
'title': 'A stronger connection to out customers'}

Migration Strategy PLONE SYMPOSIUM EAST 2011

• Investigate the source
• Prepare the destination
• Find Transmogriﬁer blueprints
• Write the pipeline

Write your own PLONE SYMPOSIUM EAST 2011

• Missing blueprint
• Write one
• Contribute it back

GenericSetup PLONE SYMPOSIUM EAST 2011

• Make migration part of your release
• Ability to package migrations

My Migration PLONE SYMPOSIUM EAST 2011

• Drupal backed by MySQL
• transmogrify.sqlalchemy
• Plone 4.0.5
• collective.blog.star
• plone.app.discussion

Package Layout PLONE SYMPOSIUM EAST 2011

my.migration
!"" my
#   !"" __init__.py
#   %"" migration
#   !"" __init__.py
#   !"" config
#   #   !"" articles.cfg
#   #   !"" base.cfg
#   #   !"" blogs.cfg
#   #   !"" comments.cfg
#   #   %"" pages.cfg
#   !"" configure.zcml
#   %"" profiles
#      %"" default
#      !"" metadata.xml
#      %"" transmogrifier.txt
!"" setup.cfg
%"" setup.py

Registering Conﬁgs PLONE SYMPOSIUM EAST 2011

<transmogrifier:registerConfig
name="my.migration.base"
title="My migration base config"
description="Base settings for all transmogrifier imports"
configuration="config/base.cfg"
/>


name="my.migration.pages"
title="Drupal pages"
description="Import pages from Drupal into Plone"
configuration="config/pages.cfg"
/>


name="my.migration.articles"
title="Drupal articles"
description="Import articles from Drupal into Plone"
configuration="config/articles.cfg"
/>


name="my.migration.blogs"
title="Drupal blog entries"
description="Import blog entries from Drupal into Plone"
configuration="config/blogs.cfg"
/>


name="my.migration.comments"
title="Drupal comments"
description="Import comments from Drupal into Plone"
configuration="config/comments.cfg"
/>

transmogriﬁer.txt PLONE SYMPOSIUM EAST 2011

my.migration.pages
my.migration.articles
my.migration.blogs
my.migration.comments

base.cfg pipeline PLONE SYMPOSIUM EAST 2011

[transmogrifier]
pipeline =
drupal
portal_type
url_normalizer
path
publication_state
text_mimetype
mimetype_encapsulator
folders
constructor
commenting
comments
schema_update
workflow
reindex_object

[settings]
# Path to use if there isn’t one given
base_path = other-content
# Have to escape python string formatting for when
# this gets passed into sqlalchemy
date_format = %%Y/%%m/%%d %%k:%%i:%%s

drupal section PLONE SYMPOSIUM EAST 2011

[drupal]
blueprint = transmogrify.sqlalchemy
dsn = mysql://user:password@localhost/drupal-transmog

articles.cfg PLONE SYMPOSIUM EAST 2011

[transmogrifier]
include = my.migration.base

[drupal]
query =
SELECT
node.title,
node.status AS status,
GROUP_CONCAT(tag_data.name SEPARATOR 'n') AS subject,
FROM_UNIXTIME(node.created, "${settings:date_format}") AS creation_date,
FROM_UNIXTIME(node.created, "${settings:date_format}") AS effectiveDate,
FROM_UNIXTIME(node.changed, "${settings:date_format}") AS modification_date,
body_data.body_value AS text
url_alias.alias AS _path
FROM node
INNER JOIN field_data_field_tags AS tag_field ON tag_field.entity_id = node.nid
INNER JOIN taxonomy_term_data AS tag_data ON tag_data.tid = tag_field.field_tags_tid
INNER JOIN field_data_body AS body_data ON body_data.entity_id = node.nid
INNER JOIN url_alias ON url_alias.source = CONCAT("node/", node.nid)
GROUP BY
node.title,
node.status,
node.created,
node.changed,
body_data.body_value,
url_alias.alias;

[portal_type]
value = string:Document

pages.cfg PLONE SYMPOSIUM EAST 2011

[transmogrifier]

[drupal]
query =
my.migration.base
SELECT
node.title,
node.status AS _status,
FROM_UNIXTIME(node.changed, "${settings:date_format}") AS modification_date,
body_data.body_value AS text,
FROM node
WHERE node.type = "page"
GROUP BY
node.title,
node.status,
node.created,
node.changed,
url_alias.alias;

[portal_type]
value = string:Document

blogs.cfg PLONE SYMPOSIUM EAST 2011

[transmogrifier]

[drupal]
query =
SELECT
node.title,
node.satus AS _status,
FROM_UNIXTIME(node.created, "${settings:date_format}") AS modification_date,
body_data.body_value AS text,
FROM node
WHERE node.type = "blog"
GROUP BY
node.title,
node.status,
node.created,
node.changed,
url_alias.alias;

[portal_type]
value = string:BlogEntry

[commenting]
value = python:True

comments.cfg PLONE SYMPOSIUM EAST 2011

[transmogrifier]

[drupal]
query =
my.migration.base
SELECT
comment.subject AS title,
FROM_UNIXTIME(comment.created, "${settings:date_format}") AS published,
FROM_UNIXTIME(comment.changed, "${settings:date_format}") AS updated,
comment.name AS author_name,
body_data.comment_body_value AS text,
url_alias.alias AS _parent_path
FROM comment
INNER JOIN field_data_comment_body AS body_data ON body_data.entity_id =
comment.cid
INNER JOIN node ON node.nid = comment.nid
GROUP BY
comment.subject,
comment.created,
comment.changed,
comment.name,
body_data.comment_body_value,
url_alias.alias;

[portal_type]
# Override the portal type to use the "comment_type"
key = string:_comment_type
value = string:plone.app.discussion

base.cfg overrides PLONE SYMPOSIUM EAST 2011

[path]
blueprint = collective.transmogrifier.sections.inserter
# only add a path if one does not exist
condition = python:'_path' not in item and not '_parent_path' in item
key = string:_path
# Add the value in the extended configuration
value = string:${settings:base_path}/${item/_id}

[portal_type]
key = string:_type
# We will add the value in the extended config, but we need a
# default set here
value = string:

[commenting]
key = string:allowDiscussion
# default to false
value = python:False

Content Creation PLONE SYMPOSIUM EAST 2011

[comments]
blueprint = transmogrify.comments

[folders]
blueprint = collective.transmogrifier.sections.folders

[constructor]
blueprint = collective.transmogrifier.sections.constructor

[schema_update]
blueprint = plone.app.transmogrifier.atschemaupdater

[workflow]
blueprint = plone.app.transmogrifier.workflowupdater

[reindex_object]
blueprint = plone.app.transmogrifier.reindexobject

Item Modiﬁcation PLONE SYMPOSIUM EAST 2011

[publication_state]
condition = python:'_status' in item and item['_status'] == 1
key = string:_transitions
value = string:publish

[text_mimetype]
# This could probably be taken from the database as well
key = string:_text_mimetype
value = string:text/html

[mimetype_encapsulator]
blueprint = plone.app.transmogrifier.mimeencapsulator
key = text
mimetype = python:item.get('_%s_mimetype', key)
field = key
condition = mimetype

PLONE SYMPOSIUM EAST 2011

DEMO

Links PLONE SYMPOSIUM EAST 2011

• collective.transmogrifier
http://pypi.python.org/pypi/collective.transmogrifier/

• plone.app.transmogrifier
http://pypi.python.org/pypi/plone.app.transmogrifier/

Useful Sources and PLONE SYMPOSIUM EAST 2011

Blueprints
• plone.app.transmogrifier
• transmogrify.filesystem
• transmogrify.sqlalchemy
• transmogrify.webcrawler
• quintagroup.transmogrifier
• transmogrify.comments

Check out
.co m/d emos
s ixfeetup

Migrating from drupal to plone with transmogrifier

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Migrating from drupal to plone with transmogrifier

Semelhante a Migrating from drupal to plone with transmogrifier (20)

Mais de Clayton Parker

Mais de Clayton Parker (20)

Último

Último (20)

Migrating from drupal to plone with transmogrifier