Instead of having to know XML, XSL, XSD, SQL, and command-line tricks because form data is stored in XML blobs -- why not just store form fields in SQL fields so you can point any off-the-shelf reporting tool at your data? Works with Crystal, Cognos, Pentaho and more...
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Reporting, the easy way
1. Reporting on your OSP Form Data...
the Easy Way
Will.Trillich@serensoft.com, Sean.Keesler@threeCanoes.com
ePortfolio Wonks
Wednesday, June 16, 2010
Thursday, June 17, 2010
2. Reporting on your OSP Form Data...
Making
it
Suck
Less
the Easy Way
Will.Trillich@serensoft.com, Sean.Keesler@threeCanoes.com
ePortfolio Wonks
Wednesday, June 16, 2010
Thursday, June 17, 2010
3. Let’s talk about Reporting
11th
Sakai
Conference
-‐
June
15-‐17,
2010 3
Thursday, June 17, 2010
4. Why Bother With Reporting?
• What’s the big deal?
• Why collect all this data?
11th
Sakai
Conference
-‐
June
15-‐17,
2010 4
Thursday, June 17, 2010
5. Why Bother With Reporting?
• What’s the big deal?
• Why collect all this data?
• We can make decisions based on patterns in the
information
• But if you can’t get information out, then putting
data in is a waste of everyone’s time and effort!
11th
Sakai
Conference
-‐
June
15-‐17,
2010 5
Thursday, June 17, 2010
6. A Sample Evaluation Form
• Imagine using an Evaluation form as shown here:
• For this to be useful you
must be able to aggregate
all these evaluations...
• per course
• per academic year
• per student
• per evaluator
• etc
11th
Sakai
Conference
-‐
June
15-‐17,
2010 6
Thursday, June 17, 2010
7. So Create Some Reports Then
• Let’s take a look at what’s going on, rst
11th
Sakai
Conference
-‐
June
15-‐17,
2010 7
Thursday, June 17, 2010
8. Oh Look...
Sakai Already Has a “Reports” Tool
Of Its Own
11th
Sakai
Conference
-‐
June
15-‐17,
2010 8
Thursday, June 17, 2010
9. Sakai’s Built-In “Reports” Tool
• Why not just use that, then?
• Once it’s set up you can pull lots of useful
patterns from your data
11th
Sakai
Conference
-‐
June
15-‐17,
2010 9
Thursday, June 17, 2010
10. Sakai’s Built-In “Reports” Tool
• Why not just use that, then?
• Once it’s set up you can pull lots of useful
patterns from your data
• “Once it’s set up”!
• Enormous investment, attention, resources to get it
working
• You don’t have to be IU—but it helps :)
11th
Sakai
Conference
-‐
June
15-‐17,
2010 10
Thursday, June 17, 2010
11. The OSP “Reports” Tool
• Here’s what it looks like after you install it in your
worksite...
11th
Sakai
Conference
-‐
June
15-‐17,
2010 11
Thursday, June 17, 2010
12. So... Now What?
• Can you see how to create a report?
• What will you need to know to get it working?
• Can you hook it up to Crystal Reports
or Cognos tools or Pentaho and get straight to
your Evaluation-form elds? Nope.
11th
Sakai
Conference
-‐
June
15-‐17,
2010 12
Thursday, June 17, 2010
13. How do you Use it?
• To use the OSP “Reports” tool here’s all you need
to know in order to create a new report...
• XML data structures and operations
• XSL to transform the XML
• XSD to see where the data comes from
• SQL to issue queries to the database
• Be Command-line Savvy, and have backstage
access to the Sakai server
• ...for EVERY NEW REPORT!
11th
Sakai
Conference
-‐
June
15-‐17,
2010 13
Thursday, June 17, 2010
14. That’s a PAINFUL learning curve!
Yikes!
11th
Sakai
Conference
-‐
June
15-‐17,
2010 14
Thursday, June 17, 2010
15. Why Is OSP Reporting So Challenging?
• When you build a new XSD form from scratch it
could have any (ANY!) structure
• The programmers couldn’t predict in advance
what data structures would be needed
• So OSP Collects form data in big, amorphous,
black-box XML chunks as a universal “solution”
• For example:
11th
Sakai
Conference
-‐
June
15-‐17,
2010 15
Thursday, June 17, 2010
16. One of Sean’s Awesome AAC&U VALUE Rubric Forms
Get
it
from
h+p://www.openEdPrac7ces.org
11th
Sakai
Conference
-‐
June
15-‐17,
2010 16
Thursday, June 17, 2010
17. Each Row in that Grid Represents a “Field”
• The actual data collected for the ve “axes” in this
form are shown below the grid:
11th
Sakai
Conference
-‐
June
15-‐17,
2010 17
Thursday, June 17, 2010
18. This is Why the Programmers Opted to Store XML Blobs
• So the Sakai Programmers let us create any form
structure we want
• The result gets stored as an XML blob, such as:
<eval-‐form>
<organiza7on>
3
</organiza7on>
<language>
4
</language>
<delivery>
4
</delivery>
<support>
3
</support>
<message>
4
</message>
<commentary>
Great
improvement...
</commentary>
<ra7ng>
3.6
</ra7ng>
</eval-‐form>
11th
Sakai
Conference
-‐
June
15-‐17,
2010 18
Thursday, June 17, 2010
19. A Contact Info Form Will Look Completely Different
<contact-‐info>
<name>
<first>Bob</first>
<last>Smith</last> All
these
different
forms
</name> are
too
amorphous
for
the
programmers
to
predict!
<emails>
<work>bob@work.net</work>
<personal>kahuna@mongo.org</work>
</emails>
</contact-‐info>
• Difficult to conjure a database arrangement to
accommodate all these different structures
11th
Sakai
Conference
-‐
June
15-‐17,
2010 19
Thursday, June 17, 2010
20. Forms can Vary Widely
• Other forms are totally different
• Résumé/CV, Contact Info Form, Re ection Form...
• The programmers knew we’d conjure up all kinds
of different XSD form structures
• So the programmers chose to just store the XML
• But XML is for transport, not storage...
11th
Sakai
Conference
-‐
June
15-‐17,
2010 20
Thursday, June 17, 2010
21. PROBLEM: How can you Reach the Field-Level Data?
• If you want an average or total of a particular eld
for all students...
• It’s buried inside all this text!
<eval-‐form>
<organiza7on>
3
</organiza7on>
<language>
4
</language>
<delivery>
4
</delivery>
<support>
3
</support>
<message>
4
</message>
<commentary>
Great
improvement...
</commentary>
<average>
3.6
</average>
</eval-‐form>
11th
Sakai
Conference
-‐
June
15-‐17,
2010 21
Thursday, June 17, 2010
22. You Could use XSL to parse it...
• XSL is designed to transform XML to something
else (often HTML)
• But there’s a STEEP learning curve there
• Requires IT mojo and lots of caffeine
• Sure would be nice if we could just use a point-
and-click reporting tool like Crystal Reports or
Cognos or Pentaho...
11th
Sakai
Conference
-‐
June
15-‐17,
2010 22
Thursday, June 17, 2010
23. Pentaho, Crystal, Cognos and more!
The Report-Enabling Gizmo to the Rescue!
11th
Sakai
Conference
-‐
June
15-‐17,
2010 23
Thursday, June 17, 2010
24. It makes your Form Data Reachable
• It makes everything SQL-selectable!
• Don’t need to learn XML, XSL, XSD, backstage
command-line hooey
• Cognos? Pentaho? Crystal Reports?
No problemo
• We came up with some sample reports
to illustrate:
11th
Sakai
Conference
-‐
June
15-‐17,
2010 24
Thursday, June 17, 2010
25. A Sample “Cognos Impromptu” Report
• Who’s getting what scores?
• Marie is squeaking by Blaise
• And who’s giving what scores?
• Hmm, let’s see this another way...
11th
Sakai
Conference
-‐
June
15-‐17,
2010 25
Thursday, June 17, 2010
26. Another Example Cognos Report
• Chip is giving
everyone a
perfect score!
• Time to have
a talk with
him...
11th
Sakai
Conference
-‐
June
15-‐17,
2010 26
Thursday, June 17, 2010
27. Reports like this Help you Make Decisions and Take Action
• Here we can see one evaluator is likely not taking
his task seriously
• Reporting enables you to see stuff like this and
then take action to nd out more, or to work it
out
• Sure is easy to come up with reports like this if
you can use Cognos or Crystal or Pentaho
• ...which you can’t do when the data is buried
inside XML blobs
11th
Sakai
Conference
-‐
June
15-‐17,
2010 27
Thursday, June 17, 2010
28. Let’s See the OSPTOOL Gizmo in its Element
• Once the tool is installed in your !admin
workspace:
• Open the OSPTOOL there
• Specify which forms to process
• Click “Parse Selected Forms”
• Kick back, be awesome, bask in the accolades
11th
Sakai
Conference
-‐
June
15-‐17,
2010 28
Thursday, June 17, 2010
29. How to use the “!admin” Interface
• Select which forms
to parse
• Then “Parse” them!
• Let’s follow an
example...
11th
Sakai
Conference
-‐
June
15-‐17,
2010 29
Thursday, June 17, 2010
30. Create a New Filled-Out Form Instance
• Evaluator responds to a Student Submission:
Evaluator
fills
out
Evalua7on
form
Normal
workflow:
Student’s
cell
is
now
LOCKED
11th
Sakai
Conference
-‐
June
15-‐17,
2010 30
Thursday, June 17, 2010
31. Now “Parse” the Chosen Forms
• Visit !admin worksite
• Make sure your chosen form is selected for
Parsing
• Then... Parse!
11th
Sakai
Conference
-‐
June
15-‐17,
2010 31
Thursday, June 17, 2010
32. Now you can just REFRESH your Reports
• Evaluator added new Evaluations for 2 Students!
«
BEFORE
REFRESH
...Then
a
quick
PARSE... AFTER
REFRESH
»
11th
Sakai
Conference
-‐
June
15-‐17,
2010 32
Thursday, June 17, 2010
33. Magic!
How Does This Work?
11th
Sakai
Conference
-‐
June
15-‐17,
2010 33
Thursday, June 17, 2010
34. Reporting Tools Require Access to SQL Fields
• If the data is available at the SQL- eld level then
off-the-shelf reporting tools like Cognos or
Pentaho or Crystal can reveal patterns
• The OSPtool Gizmo makes all the form elds
available directly via SQL
• (You do still need to know SQL, or have access to
someone who does, but now you can forget the XSD
and XSL and command-line voodoo)
11th
Sakai
Conference
-‐
June
15-‐17,
2010 34
Thursday, June 17, 2010
35. Common Patterns in All Form Structures
• Sure, some forms have text elds for email, others
have numeric elds for scores...
<eval-‐form>
<organiza7on>
3
</organiza7on>
<contact-‐info>
<language>
4
</language>
<name>
<delivery>
4
</delivery>
<first>Bob</first>
<support>
3
</support>
<last>Smith</last>
<message>
4
</message>
</name>
<commentary>
Great
improvement...
</comme
<emails>
<average>
3.6
</average>
<work>bob@work.net</work> </eval-‐form>
<personal>kahuna@mongo.org</work>
</emails>
</contact-‐info>
11th
Sakai
Conference
-‐
June
15-‐17,
2010 35
Thursday, June 17, 2010
36. Common Patterns in All Form Structures
• Sure, some forms have text elds for email, others
have numeric elds for scores...
contact-‐info
<contact-‐info>
<name>
name emails
<first>Bob</first>
<last>Smith</last>
</name>
first last work personal
<emails>
<work>bob@work.net</work>
<personal>kahuna@mongo.org</personal> But
each
field
element
</emails> has
a
“parent”
element
</contact-‐info> and
belongs
to
a
form
11th
Sakai
Conference
-‐
June
15-‐17,
2010 36
Thursday, June 17, 2010
37. Common Patterns in All Form Structures
• Sure, some forms have text elds for email, others
have numeric elds for scores...
• But all eld elements have a position inside the
form data
• All eld elements have a “parent” element
• All eld elements have a root-level “form” element
• Why not leverage that?
11th
Sakai
Conference
-‐
June
15-‐17,
2010 37
Thursday, June 17, 2010
38. That’s Exactly what the OSPtool Gizmo Does
• Each eld knows what its “parent” is
• Easy to determine the “root” form for any eld
• That’s it!
11th
Sakai
Conference
-‐
June
15-‐17,
2010 38
Thursday, June 17, 2010
39. So here’s the upshot
The Good and the Bad
11th
Sakai
Conference
-‐
June
15-‐17,
2010 39
Thursday, June 17, 2010
40. The Good Parts
• It’s freely available as Source Code
• https://source.sakaiproject.org/contrib/serensoft/
• Your report-writers will only need to know SQL
• Any off-the-shelf SQL reporting package will work
• Pentaho
• Cognos
• Crystal Reports
• etc
11th
Sakai
Conference
-‐
June
15-‐17,
2010 40
Thursday, June 17, 2010
41. Even Better
• Wouldn’t it be great if your OSP form data were
stored this way by default instead of having to go
through a separate “parsing” process?
• But I digress... :)
11th
Sakai
Conference
-‐
June
15-‐17,
2010 41
Thursday, June 17, 2010
42. The Bad Parts
• Does it scale?
• Absolutely no idea, please test it!
• Is the !admin interface elegant and clean?
• Um, no
• It’s not “live”
• Updated (and new) forms require re-parsing before
the data is visible in SQL
• Does it auto-run (via Quartz jobs)?
• Not at the moment, “Parse” is manual-only
11th
Sakai
Conference
-‐
June
15-‐17,
2010 42
Thursday, June 17, 2010
43. Please Use it and Get to the Patterns in your Data!
• Please install it and try it out
• You won’t need an army of XSL/XML/XSD
engineers, only someone with some SQL skills
• Have fun Unravelling your XML!
11th
Sakai
Conference
-‐
June
15-‐17,
2010 43
Thursday, June 17, 2010
44. http://www.surveymonkey.com/s/sakai10
https://source.sakaiproject.org/contrib/serensoft/
will.trillich @ serensoft.com
11th
Sakai
Conference
-‐
June
15-‐17,
2010 44
Thursday, June 17, 2010