This presentation was delivered at the 2012 Blackboard Developer's Conference in New Orleans. It details lessons learned when creating custom content types that survive course copy.
A common Building Block task is to create a new custom content type in Blackboard. The new OpenDatabase allows you to easily create content with much greater functionality and awareness of other parts of Blackboard. You can even tidy up properly afterwards. Surely copying this content wouldn't prove too tricky? I was so wrong. This session explains how Blackboard expect this to happen, names the (undocumented) APIs and walks you through the lifecycle of a custom content item. The open source SignUp List building block is used as a case study.
USPS® Forced Meter Migration - How to Know if Your Postage Meter Will Soon be...
Copying Custom Content Breaks Links
1. Copy is a
four letter word!
Dr Malcolm Murray
9th July 2012 10.00 am – 10.45 am
2. The “Xerox” problem…
Shiny new Custom Content in Year 1…
…breaks when copied in Year 2
2
Image source: http://i.ebayimg.com/07/!B9lsCzQEGk~$(KGrHqMOKjsEzIgY)E(wBM6TBcmD!!~~0_35.JPG
4. How do we get there?
Copied Mangled
Understand what works and why Identify the limitations
Use these patterns where we can Avoidance Strategies
Work arounds
Documentation
Enhancement requests
4
8. Links from the Title
ViewServlet
?course_id=_490_1&content_id=_1424_1
course_id content_id record_id material_type
8
9. Accessing the Content File Area
blackboard.platform.filesystem.manager.CourseContentFileManager
CourseContentFileManager fManager = new CourseContentFileManager();
File contentDir = null;
try { course_id content_id
contentDir = fManager.getRootDirectory(course_id, content_id);
} catch (blackboard.platform.filesystem.FileSystemException fE){
// deal with this
} catch (java.lang.NullPointerException nE){
// deal with this
}
File propertiesFile = new File(contentDir, PROPERTIES_FILENAME);
// extract the record_id and material_type from the file
9
10. Links in the Static Body Text
ViewServlet
?course_id=_490_1&content_id=_1424_1&record_id=b26936641&material_type=c
course_id content_id
?course_id=@X@course.pk_string@X@&content_id=@X@content.pk_string@X@
record_id material_type
&record_id=b26936641&material_type=c
10
11. Successful Strategies
Copying Custom Content works if we keep things very simple
Use Template variables where possible:
?course_id=@X@course.pk_string@X@&content_id=@X@content.pk_string@X@
Store extra data in the FileSystem:
contentDir = fManager.getRootDirectory(course_id, content_id);
11
27. Link to Users and CourseMemberships
SignUp List Member
SIGNUP_PK1 (CONTENT_PK1)
USER_PK1
CU_PK1
<foreign-key name="oslt_signup_member_fk2"
reference-table="users" on-delete="cascade”>
<columnref name="user_pk1" />
</foreign-key> 27
28. So this will all copy over too, right?
Image source: http://copy-machines.95billiards.com/2012/02/08/nice-copier-copier-photos/
28
29. [ insert four-letter word here ]
Case Summary: Exception thrown when attempting to copy a course
Note Detail: {Hi Malcolm,
The <application value="Bb-wiki"/> tag was added to the manifest to avoid the entries appearing multiple times in the 'More Tools' menu. (LRN-46682-Wikis link
displayed twice under tools contextual menu while trying to click on 'More Tools' link)
Now, I have had long discussions on this issue so i may wish to call you to get your take on it.
So there is a fundamental design difference between the content-type(Content item) and application(tool) concept.
Content items (content-type) be simplistic links to either course_content table fields (information held to either
Content items (content-type) are really designed to
are really designed to be simplistic links against the content_id and anything
course_content tableinfields… …orlinks to theto the more(application-defs).tools (application-defs).
stored within the [content_id] folder the file system) or links more complex tools complex
So when a Content item (content-type) is copied it does a pure copy of the course_contents record and the [content_id] folder and asumes that all refrences are
So whencontent_id and any item (content-type) is copied it does a pure copy of the
relative to the a Content links include template variables e.g. @X@course.id@X@ @X@content.id@X@
course_contents recordthat can then be[content_id] folder anddesigned to be athat all references are what
Tools (application-defs) create tool items and the linked to from content items. These are asumes more complex container for data and thats
relativeblogs, wikis etc are tools rather than just basic content items.way of copying/exporting/importing all the data needed for that tool to work in a course.
the blackboard.platform.cxComponent works on, allowing a customised
This is why,
to the content_id and any links include template variables e.g. @X@course.id@X@
@X@content.id@X@
cxComponent is designed to copy/export/import all the tools items (application-defs) in the operation there isn't a way of copying a single item.
Copying the Content items link (content-type) doesnt touch the cxComponent.
Copying Content item links (content-type) without the associated tools result in broken links in the new course, this is true of any tool including, Wiki, Blog, and
Tools (application-defs) create tool items that can then be linked to from content items.
collab.
These areyour tool does andto idea of SignUp Listscomplex container for adata content-type, especially as it thecustom database
Looking at what designed the be a more i would consider this a tool rather than simple and thats what has a
schema.
blackboard.platform.cxComponent works on, allowing a customised way of
copying/exporting/importing easylike totool by creatingbut iapplicationyour SignUptool 'hook' to the control and course. Thislisted by
I have seen some cheats to make a content-type behave the data needed for that tool to work in panel just links to a page that
shows instructions on how to use the tool. This is an
all fix a your situation an still think where the Lists are squarely a tool, a should be able to be is
why, blogs, wikis etc are tools rather groups. just basic content items.
the instructor in one place like blogs, wikis, Assignments or even than
I am available to chat about this if you wish.
29
34. Defining the Usage
@Override
public String getName() {
// might want to localise this
return "SignUp Lists";
}
@Override
public Usage getUsage() {
// either Usage. CONFIGURABLE
// or Usage.ALWAYS
return Usage.ALWAYS;
}
34
35. Processing content
@Override
public void doCopy(CopyControl copyControl) {
// does the copy back-end magic
}
@Override
public void doImport(ImportControl importControl) {
// does the import/restore back-end magic
}
@Override
public void doExport(ExportControl exportControl) {
// does the export/archive back-end magic
}
35
37. Controls – spanning old & new
blackboard.platform.cx.component.CopyControl
getSourceCourseId() : Course Id - source
getDestinationCourseId() : Course Id - target
lookupIdMapping(Id sourceContentId) : targetContentId
copyVTBEText(String ?, Id ?, String ?) : String
getLogger() : CxComponentLogger
isExact() : boolean – true if Exact Copy (with Users)
37
38. doCopy() logic
1. Establish the CRS_MAIN_PK1 for the old & new courses
CopyControl.getDestinationCourseId()
CopyControl.getSourceCourseId()
2. Load all the old SignUpList objects from my custom
table using the old CRS_MAIN_PK1
SingleSignUpListLoader.getInstance()
.loadByCourseId(CopyControl.getSourceCourseId());
3. Iterate through this Collection
Establish the new CONTENT_ID for each SignUpList
lookupIdMapping(sourceContentId)
Copy the old table row, swap the CRS_MAIN_PK1 and CONTENT_ID
values and persist it
SingleSignUpListPersister.persist(newSignUpList);
38
39. Paper Jam
SignUp List
CONTENT_PK1 ✔
GROUP_PK1
✗
CAL_PK1
✗
CAL2_PK1
✗
TASK_PK1
✗
SignUp List Member
SIGNUP_PK1 (CONTENT_PK1)
USER_PK1
CU_PK1
39
40. Still room for improvement
Image source: http://www.cartridgesave.co.uk/news/the-evolution-of-printer-technology-then-and-now/
40
43. Slides available on dropbox at http://db.tt/nD2tWqwi
We value your feedback!
Please fill out a session evaluation.
We value documentation!
Bb please fill the gaps in the javadocs.
43
Notas do Editor
Lessons Learned Creating Custom Content Types that Survive Course CopyA common Building Block task is to create a new custom content type in Blackboard. The new OpenDatabase allows you to easily create content with much greater functionality and awareness of other parts of Blackboard. You can even tidy up properly afterwards. Surely copying this content wouldn't prove too tricky? I was so wrong. This session explains how Blackboard expect this to happen, names the (undocumented) APIs and walks you through the lifecycle of a custom content item. The open source SignUp List building block is used as a case study.Starts shallow, but gets deep at the end. If you want to find out how to intercept and shape the Course Copy action, stay tuned…
Many building blocks written to extend the functionality of blackboard require defining a customContent type. These fall into two – simple and complex. Simple ones usually survive the course copy process to work again the next year. Complex ones often don’t, complaining of broken links, throwing null pointer exceptions, reporting objects not found or access denied. All in all a very poor show and likely to trigger lots of support calls. Content mangled in the copy process is what I’m terming the Xerox problem.
Let’s try and avoid the mangled content now and in the future.The question is how?
This presentation will take examine two facets. Why do some content items copy without a problem?We will need to look at the copy process and understand what happens and why.Can we then design these approaches into our building blocks?Equally, can we identify examples where content gets mangles after a copy and establish why?Are there ways we can code a solution, or code round the issue?Are there limitations in the current API? If so can we identify these and submit enhancement request?Are there areas of the documentation that need improved (oh yes!)
Start with what works – I’m going to use an example of a custom content item I have been developing with colleagues in the Library. It survives course copy. Accident or design? I’ll leave that for you to decide!
An instructor in a Blackboard course can request a Reading List item using a custom building block. This passes the course code (e.g. ANTH1234) to the Library Database which returns the reading list in XML format. The building block parses the XML, displays a list to the user and they then select the item(s) they want to add. Each item is added to the course as a discrete custom content item. The content item contains a link that the user can click to serve them the appropriate library content, e.g. a PDF.Not shown here is the workflow required for edit options – that’s may require a trip back to the database to check the citation is up to date!
If a user clicks the content item title, the resulting action is defined in the bb-manifest file. Each content-handler has a view entry – in the simplest case this would take you to a JSP, in this example it uses a servlet. Without you doing anything the servlet is passed two parameters, both derived when the page is rendered: course_id and content_id. As we’ll see in a minute this isn’t quite enough to get to the Resource…In passing note that you can set a can-copy flag in the manifest. If false, this stops instructors copying individual content items to another location in the course (duplication) or to another course. The value you choose here matters if you want to use the CxComponent route to better copying (discussed later).
If a user clicks the title link, e.g. “Old man and wolves” in this example, they are taken to the servlet (ViewServlet) and passed two parameters. These are added automatically by the Context - course_id and content_id. We need to obtain two more values to uniquely identify the library resource and so know where to redirect the user. As we have no way of adding other variables to the URL, we need to use the servlet to obtain them. As this is a Content item, a sensible place to store extra data is in the folder associated with this content item in the Course folder on the server (N.B. NOT the Content System). To do this we need to use a bit of code…
ViewServlet can use the two variables: course_id and content_id together with a CourseContentFileManager to access the folder we need. Inside this folder we write a small properties file when the content item is created. This stores any data we can’t persist via the Content object itself. The advantage of this location is that itt stays with the content item when it is moved or copied, and is removed when the content is deleted.Tip: don’t use the course_id or content_id as the name of this file!Once we have all four variables we can log access and then using a custom LinkResolver object (not shown here) to redirect the user to the current URL for this item – be it a publisher’s website, a PDF in the eReserve folder, etc. Thus we are only coding what we need, and avoid any volatile data such as Publisher URLs.By using this folder, we can work around the fact that the link by default only passes us the content_id and the course_id.This approach survives move and copying.
If instead of clicking on the title, a user clicks the “View Resource” link in the body text (remember body text is static), they are redirected to a custom servlet. To keep things simple for me, it goes to the same servlet that is the link for the title link discussed earlier, but it doesn’t have to be.. This time, as we have more control over the link, the servlet (ViewServlet) is passed all four parameters. Two are derived when the page is rendered (course_id and content_id) using Blackboard’s template variables, two are hard coded and relate to the library resource (and so shouldn’t change).ViewServlet logs access and then redirects the user to the current URL for this item – be it a publisher’s website, a PDF in the eReserve folder, etc. using a custom LinkResolver object not shown here. If this content is copied or moved, the course_id and content_id will be updated to reflect the new values when the page is rendered Warning– Blackboard’s template variables only work in Course Content Items (and portal modules).
This approach allows us to copy simple content items and is (I think) how Blackboard expects us to use this.At this point we have approached the edge of the cliff that is “functioning as designed”. We will be jumping off this cliff shortly…
Often this approach is not enough. The functional spec requires a more complicated extension, possibly with multiple custom database tables. That’s where the fun really begins.First though we are going to address a question that is often raised when copies go wrong…
Copying complicated building blocks can easily cause errors, but understanding why moving items often doesn’t, may provide some useful insight into the reasons for paper-mangling.
Let’s begin by creating a simple Content item in a course. What really happens?We specify some information – title, title colour, possibly some text, set availability, etc. – all the usual stuff.
Upon persistence, these values are all stored in a table – COURSE_CONTENTSThe primary key to this entry (PK1) is an integer that is passed from page to page in a String representation as the content_id value.We also create a folder on the file system of the server (not in the Content System). This is usually empty. Shown here are two variants for the path on a linux install, which differ depending what you call the database.Note that the file path points to the location we obtained earlier using the blackboard.platform.filesystem.manager.CourseContentFileManagerobject, the folder _1969676_1 is where we stored the extra data (record_id and material_type) in a properties file.
If we consider the simplest possibly copy – simply duplicating the item immediately below the original – i.e. same parent location in the same course. The screenshot shows the result.Basically we copy the entire table row, and only update the PK1 (new unique content_id needed) and increment the POSITION value by 1 to show it sits below the original on the page.Note that the DTCREATED and DTMODIFIED fields remain the same! You might want to ask whether they should…Copying the file to another location would change the PARENT_ID (this is essentially the content_id of another place – be it a folder or course menu item). It might also change the value of POSITION.Copying it to another course would mean we’d change the CRSMAIN_PK1 value to match the new course_id (and it would also have a new PARENT_ID) . It might also change the value of POSITION.
If we move an item instead of copying, we do not need to create a new table row, we just amend the exiting one.The key point about a move is that the content_id DOES NOT CHANGE – the PK1 value shown here is the same before and after the move.If we move the item to a new location in the same course the PARENT_ID and possibly POSITION will change, but the file system entry will be the same – the URL to the custom folder remains unchanged.If we move the item to another course then the CRSMAIN_PK1 will change, as will the the PARENT_ID and possibly POSITION. In this case the path to the custom folder will change, but the only change will be that the COURSEID (code) value in the path will be updated to reflect that of the new course.Moving often works, because the PK1 (content_id) remains the same, preserving any lookup that rely on this value
OK, so lets look at more complicated custom content. Why would they fail and what can we do to help fix that?Many such examples take advantage of the ability to create custom tables in the Blackboard database.
I’m going to be using this example for the rest of the presentation. The source code is available for inspection at OSCELOT – go take a look.
It exposes a custom content item. Clicking the title link offers a range of extra functions, depending on your role in the course. If you want to know more it is all documented on OSCELOT.
One thing that is different about this tool is that the Content item is linked to other objects – CalendarEntries, Tasks, Groups and custom membership lists. These screensots of the create process show this in more detail.Step 1 simply shows content persisted into the COURSE_CONTENT table (though we do manipulate the body text before persisting it to add details of the list size and free places).
Everything seen on this page is stored in a custom table – OSLT_SIGNUP_LISTIt uses the content_id (PK1) as the primary key for the entry as it is guaranteed unique and identifies the relationship.If calendarEntry and Task objects are created, we save the corresponding PK1 in the OSLT_SIGNUP_LIST table in case wen need to update them later.
Similarly, if the user wants to link the list to a Group (new or existing) then we store the Group’s PK1 in the OSLT_SIGNUP_LIST table
The full database structure is defined in the Building Block’s schema object (an XML file). This includes details of primary keys, foreign keys, indexes and any actions to perform if a foreign key is deleted.
Here is the structure of the table OSLT_SIGNUP_LISTYou can see the corresponding entries for the Group, two CalendarEntries and a Task.
In the first version of the tool, all the data was stored in XML files, this was fragile and frustrating.Now we can store the list of members in a separate table – OSLT_SIGNUP_MEMBERThis has it’s own PK1 but uses the SIGNUP_PK1 (i.e. the content_id) as a foreign keyDelete the parent content item and the entries in OSLT_SIGNUP_LIST and OSLT_SIGNUP_MEMBERare automatically deleted too – nice eh?
You can see we also use the USER_PK1 to link to a specific user signed up to the list, and their CU_PK1 to identify the CourseMembership object.Why both – well CourseMemberships are more fragile than users. If someone leaves, it is useful to be sure you know who they are!This gives you a lot of power – you can react when students signed up to existing lists are removed from a course, or indeed if their account is deleted from the server.Note that this may still require some user action – do they want to fill in any gaps? At least now we can recognise these changes and act on them.Deceptively simple looking tasks like checking to see if the lists are full can now utilize the database functionality designed for a multiple transactional model, rather than relying on some horrible lock on a local XML file solutionOK, we can see the benefits of using this sort of approach, but back to the subject of this session: copying…
Given that the custom tables use the content_id as their key, you might have hoped that new database table entries would be created when SignUp Lists are copied.Sadly they aren’t. Blackboard copies across the simple content item, but none of the corresponding custom database table entries. As such without further action (code) on our part, the copied SignUp Lists are useless. It gets worse, any linked Tasks or CalendarEntries are also pointing at the wrong thing or just don’t work at all. At this point I decided to put in a support ticket.
This was a very carefully worded, supportive and informative reply to my query, it just didn’t help get me out of this hole I’d dug myself into. It seems that what I was doing with content items, they only expected people to do with course tools.It seems that Blackboard have added some listener type objects that intercept attempts to copy course tools and so provide the developer a chance to code their way to a successful copy. Not so the humble custom content item. It’s only a minor simplification of the process to describe it as “Title, colour, body text, goodbye”. Looks like I was stuck and would have to add a dummy course tool that did nothing except allow copy to proceed
We have to add an application-def entry to the manifest.We need to set is-course-tool and is-org-tool to true, so the CxComponent (listener we define in a minute) is triggered on copy (and import and export).Note the handle value used is important – it needs to match that declared at the top of the manifest.Also the required application does nothing via the user interface, we can leave it with no links
Even with no link, the tool can appear in the list available in courses.Happily this can be resolved with a quick trip to the Sys Admin panel…
To intercept copy, import and export we need to define a custom class that IMPLEMENTS blackboard.platform.cx.component.CxComponent in the manifest.
The class that implements CxComponent must override certain methods:getComponentHandle() allows it to identify the relevant content items in the COURSE_CONTENT tablegetApplicationHandle() helps it build ythe URL to any servlets etc. needed
We need to specify the name of the items that this class will attempt to copy/import/exportIt would appear to the user before actions such as Exporting a Course, depending on the Usage valueThe Usage value should be one of two types - CONFIGURABLE or ALWAYS.I have selected ALWAYS so it isnever shown in the list to the user and always runs tointercept a copy/import/export even if the user didn’t tick the box in Step 3 – and remember most of us think these are really Content items, not Tools!As an aside, note that the CxComponent is NOT invoked if a user clicks the Copy contextmenu for a content item, this is why the manifest has can-copyset to false.
Essentially three use cases – key is the CopyControl/ImportControl/ExportControl object which we look at now:
The puzzle we are trying to solve now is to map known data (source course) against unknown – the target course.Specifically we are asking How do we match up the custom table rows used by our extension and identify the new CONTENT_PK1 values to insert as the primary keys?
The CopyControl object contains information to help you map items from the old course to the new and provide access to the log.None of these are documented. They are used by Blackboard extensions such as blogs and wikis.Most are fairly self-evident from their name.The isExact() method can be used to differentiate between normal copies (e.g. a copy into another course) and complete copies with usersNever used the copyVTBEText() method so don’t know what it does, or the arguments it takes!Similar methods for export and import (not shown here)ImportControl has a isRestore() method – allows you to differentiate between Imports and Restores – i.e. will it have user?Similarly ExportControl has a isArchive() method – export or archive action?
This slide shows the basic logic used to process a copy. Nothing clever – load all the SignUp List objects and then process then one at a time, trying to match them to an entry in the new course and then copy the data.Note this model is a simplification of the actual code used, as the latter has to differentiate between different types of SignUp Lists – a complication we are spared here.
The mapping has limits…. It doesn’t work for Groups, Tasks and CalendarEntry objects for example – time for an enhancement request.Other matching currently has to use some horrible best-efforts text-based match (e.g. Group Name).Still a risk of jam…
The CxComponent object allows us to intercept copy, import and export, which helps copy custom table entries that use the content_id as a PK1 value.Unfortunately if you link to other Blackboard classes e.g. Tasks, Groups, CalendarEntries. Then you are still stuck rolling your own solution.That still leaves plenty of scope for paper jams.
A parting thought…
Feel free to contact me using any of the above details.