UCISA Learning Anaytics Pre-Conference Workshop
•Transferir como PPTX, PDF•
1 gostou•1,181 visualizações
UCISA Learning Analytics Pre-Conference Workshop Mike Moore - Sr. Advisory Consultant - Analytics Desire2Learn, Inc. UCISA Conference 2014, Brighton, UK Presented Mar 26, 2014
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (19)
Semelhante a UCISA Learning Anaytics Pre-Conference Workshop
Semelhante a UCISA Learning Anaytics Pre-Conference Workshop (20)
Último
Último (20)
UCISA Learning Anaytics Pre-Conference Workshop
- 1. Learning Analytics The Science Behind Success UCISA 2014 Conference #D2L, #UCISA14
- 2. Michael Moore, MSCIS Sr. Advisory Consultant – Analytics Desire2Learn, Inc.
- 3. Please introduce yourself – • Name & institution • Your role • What is your experience with learning analytics • What is your hope/expectation for this workshop Introductions
- 4. Society for Learning Analytics Research: Learning analytics is the collection and analysis of data generated during the learning process in order to improve the quality of both learning and teaching. 2013, Siemens, Dawson & Lynch What are Learning Analytics?
- 5. EDUCAUSE – Next Generation Learning Initiative The use of data and models to predict student progress and performance, and the ability to act on that information. 2010, Siemens What is Learning Analytics?
- 6. The Horizon Report, 2011 The interpretation of a wide range of data produced by and gathered on behalf of students in order to assess academic progress, predict future performance, and spot potential issues. Johnson et. al., The Horizon Report 2011 What is Learning Analytics?
- 7. Data Gathering Analysis of Data Decisions Based on Data Learning Analytics Process More than just collecting data More than just analyzing data Goal is: Provide deeper insights to make smarter decisions based on facts! Michael Ticknor, July 2012, Teacher’s College – Columbia University https://www.youtube.com/watch?v=SEFmvaBTZ3I
- 8. InsightandInformationValue D2L Integrated Learning and Advanced Analytics Platform Risk Forecasting Predictive Modeling What will happen? Stage Two Reporting Data Access What has happened? What is happening? Stage One Optimization Strategic What do I want to happen? Stage Three Advanced Predictive Advanced Adaptive What do you want to happen for you?? Stage Four ILP - Analytics Capability and Maturity Model
- 9. Horizon Report 2014 “As learners participate in online activities, they leave an increasingly clear trail of analytics data that can be mined for insights.”
- 10. Any data contained in the VLE can be considered “learner data” Just having the data does not bring improvement, What you do with the data brings improvement Learner Data
- 11. Learner Data •Valuable insight about: •Content consumption •Learning behaviors •Student interactions •Digital breadcrumbs •Personalization •Data has Value • Mine (use/extract) that data Source: HSBC
- 12. Source: Matthew Aslett, The 451 Group Updated database landscape graphic, Nov 2, 2012 http://blogs.the451group.com/information_management/2012/11/02/updated-database-landscape-graphic/
- 13. Don’t Under-estimate: • The VASTNESS of the data available • The VALUE of existing data Don’t Over-estimate: • The ACCURACY of the existing data Learner Data
- 14. Educational Data Mining Developing methods to explore the unique types of data that come from an educational context and, using these methods, to better understand students and the settings in which they learn. Romero et. al., The Data Mining Handbook How do we get the data?
- 15. How do we Analyze the Data? Chatti, A Reference Paper for Learning Analytics
- 16. Do you know what data your organization tracks, monitors or measures? What types of data matter to you? Do you know the types of data that would matter (provide value) for a progression, retention, or attainment initiative at your institution? Open Discussion
- 17. Learning Analytics • Learning moment • Data right where it matters most • Mid-course correction
- 18. Decisions can only be as good as the data they are based upon. Why Bother? • Improved decisions • Improved student learning • Personalized student learning • Course improvement • Improved learning outcomes • Etc. Primary Purpose
- 19. Objectives Chatti, A Reference Paper for Learning Analytics
- 20. Data Policies Data Governance Scope of the data • Who needs it • Who sees it • Who uses it Data Consistency Data Policies • Enforcement • Validation • Who owns it • Who maintains it Source: http://commons.wikimedia.org/wiki/File:DARPA_Big_Data.jpg Used under public domain – Defense Advanced Research Projects Agency (DARPA)
- 21. Data Policies Data Retention How long do you keep it? Where do you keep it? Who needs access to it? What access is needed? Data Privacy Who can see what data? What can students see about themselves? Source: http://commons.wikimedia.org/wiki/File:DARPA_Big_Data.jpg Used under public domain – Defense Advanced Research Projects Agency (DARPA)
- 22. Institutional Data Governance Program/Department Standards Course Practices Data Policies More Tactical More Strategic Course Offering Instances
- 23. • Communication • Data Standards/Policies • Data Strategy • Data Processes Elements of Data Governance Just because data exists does not automatically mean you can get to it or report on it.
- 24. Potential Issues •Plan for resources • Hardware, software, systems • Staff!! (plan for people/capacity) •Education • Focus on “self-serve” • Enable end-users • Train staff Source: http://www.flickr.com/photos/dellphotos/11354480054/in/photostream/ Used under Creative Commons Attribution license https://creativecommons.org/licenses/by/2.0/
- 25. Morning Break Group 1 Discussion - Plan for resources • Hardware, software, systems • Staff!! (plan for people/capacity) Group 2 Discussion – Education • Focus on “self-serve” • Enable end-users • Train staff Source: http://www.flickr.com/photos/dellphotos/11354480054/in/photostream/ Used under Creative Commons Attribution license https://creativecommons.org/licenses/by/2.0/
- 27. Group 1 Discussion - Plan for resources • Hardware, software, systems • Staff!! (plan for people/capacity) Group 2 Discussion – Education • Focus on “self-serve” • Enable end-users • Train staff Discussion Questions
- 28. How to Use the Data? Belinda Tynan & Simon Buckingham Shum (2013). Designing Systemic Learning Analytics at the Open University
- 29. User Needs •Understand use cases •Clearly define requirements •Identify stakeholders •Understand nuances of the data •Provide user-friendly views and reports Source: http://www.flickr.com/photos/dellphotos/11354379865/in/photostream/ Used under Creative Commons Attribution license https://creativecommons.org/licenses/by/2.0/
- 30. No Report is Perfect • Understand the goal, the use, the decisions • Use appropriate visualizations where possible • Identify relationships, trends, patterns • Focus on “Key Question” to be answered Reporting
- 34. Q#1 – what are some examples in your personal life of a rich visualization done right? Q#2 – how would you apply that visualization to learning analytics? Q#3 – do instructors have specific visualization needs? Q#4 – what about accessibility needs re: visualizations? Open Discussion
- 35. Personalized
- 36. Predictive
- 39. Global Graduation Data #1-Switzerland #2-United Kingdom #5-EU-27 #10-United States Pct 34% 29% 23% 15% 0% 10% 20% 30% 40% Source: http://epp.eurostat.ec.europa.eu/statistics_explained/index.php/Tertiary_education_statistics Ranked by graduation percentage rate.
- 40. US Higher Ed Problem Student Retention •First year attrition rates exceed 25% •Some states reach 40% •Only 1 in 2 students ever complete a degree Degree Completion •75% students are non- traditional •40% students are not academically prepared •40% are part time Time to Degree •60% FT students complete 4 yr Bachelor’s within 8 yrs •24% PT students complete 4 yr Bachelor’s within 8 yrs •20% take more courses than needed Efficacy of Higher Ed Source: Complete College America Time Is the Enemy - Summary http://completecollege.org/docs/Time_Is_the_Enemy_Summary.pdf
- 41. Adaptive Learning •Knowillage LeaP •Adaptive learning engine •Personalized learning experience What if textbooks could learn? Source: http://www.flickr.com/photos/m00by/2538526391/Used under Creative Commons Attribution license https://creativecommons.org/licenses/by/2.0/
- 42. Desire2Learn LeaP – Adaptive Learning Adaptive analytics and semantic learning engine for personalized learning
- 43. Semantic learning Data mining analysis Examples: Survey responses Customer feedback Journals and publications Discussion forums Email threads Text Analytics
- 44. Invited to attend and deliver opening night address Bridging the gap between research and vendor worlds Based on D2L’s Our support of the analytics research community Our delivery of research-backed solutions to market Practical application of our current toolkits Product-based research Brand new ideas Leverage scientific research in the field Vetted proofs of concept that can help solve the challenges educators are facing today LAK 2014
- 45. Partnership Research Examples of learning analytics research • Predictive analytics algorithm and data modeling techniques for Insights S3 and Degree Compass (past research works) • Semantic Content Analysis and Big Data (current research works) • Learning Objectives and Unstructured Text (future research works)
- 47. Questions? Michael Moore Sr. Advisory Consultant – Analytics Desire2Learn, Inc. Direct 888.772.0325 x6604 Twitter: @MikeMooreD2L Michael.Moore@Desire2Learn.com Slides available on SlideShare www.slideshare.net/MikeMoore14 Thank You Let the dataset change your mindset
- 48. Desire2Learn, Campus Life, CaptureCast, Desire2Learn Binder, myDesire2Learn, Insert Stuff, Insert Stuff Framework, Instructional Design Wizard, and the molecule logo are trademarks of Desire2Learn Incorporated. The Desire2Learn family of companies includes Desire2Learn Incorporated, D2L Ltd., Desire2Learn Australia Pty Ltd, Desire2Learn UK Ltd, Desire2Learn Singapore Pte. Ltd. and D2L Brasil Soluções de Tecnologia para Educação Ltda. Let’s transform teaching and learning, together.
Notas do Editor
- Can cite the latest George Siemens SoLAR paper that has Insights S3 and UWS cited.
- HIPPOBig DataData AnalysisPredictive AnalysisPrescriptive Analysis
- How did we do that? Where did we start?First generation LMS was first step way back in 1999 – Stage OneFirst generation analytics technology added - Stage TwoSecond generation analytics technology with deeper/richer learning data curation (optimization of Stage Two tools and technology) – Stage ThreeSecond generation predictive and personalized/adaptive learning analytics with the Learner in absolute control of their destiny. Institutions are beacons for learners and drive focus and guidance with predictive and adaptive tools and technology. Institutions who employ these types of tools and technology will attract the largest student population and deliver the most skilled and capable graduates into the field. – Stage FourStage Four – I made this blue like Stage One as this will become the new normal or new baseline for learning in the 21st century. It will be the benchmark/baseline for all learning tools and technology moving forward. See next slides for more details on what is possible for learning as we move towards the end of the second decade of the 21st century. D2L is building the foundation (ie APIs, analytics/predictive analytics, adaptive, gaming, etc.) for Stage 4 entrance and expansion.Data access – just dataReporting/OLAP - what happenedForecasting – why did it happenPredictive modeling – what will happenOptimization (real-time predictive analysis)– what is the best that could happenSense and respond. Predict and Act.Marketing blurb:Already using the learning environment data available to report on key learning outcomes, student engagement and enrolment metrics as well as student grades data, Desire2Learn’s vision for big data in education was to move the institution from traditional activity reporting functions to a big data-driven framework with learning and academic analytics functionality at its core. By developing partnerships with key industry leaders in enterprise analytics applications, the Desire2Learn Analytics portfolio for education would be transformational.Understanding that big data concepts were new to education, Desire2Learn Analytics was specifically packaged into bundled offerings not only to suit different institutional reporting needs but also to address different institutional strategies around big data. Further, by developing a strategic roadmap to include predictive modules in their offering, Desire2Learn’s Analytics portfolio would offer an unprecedented suite of products with the capability to tap into the vast amounts of big data available in education today. Desire2Learn Analytics Portfolio delivers a multi-tiered analytics solution that offers customers a path forward to manage the analytics initiatives that are critical to their institutional effectiveness. Whether those analytics initiatives focus onincreasing operational efficiency, optimizing learning outcomes or creating the conditions for learner success, Desire2Learn’s Analytics Portfolio delivers two analytics solutions that are integral to your institution's strategic process improvement efforts.
- Can cite the latest NMC Horizon report that has Insights S3 and UWS cited.
- Big Data is volume, variety & velocity per Gartner
- From course selection right on through to course completion. Most importantly, use learning analytics so that instructors can mentor students before they even know they need help (that is, no one wants to be centered out as needing help or failing – which is what student services strategies do today. Students wouldn’t even know that they need help with the earliest and ongoing predictions that S3 offers.At-risk does not mean alone.
- Group 1 – depending on the audience, perhaps a lively debate around the challenges of galvanizing a disparate and typically silo’ed organization to enable this kind of change - it might be a good discussion driver for foundational planning and resourcing needsGroup 2 – I like the approach of the discussion around self-serve and enabling end-users – this would be a good discussion driver re: giving custos the right questions to ask vendors so that they can ensure that they get easy-to-deploy tools and technology
- Beauty of analytics is that its all about the data. You can dice it, slice it, etc. and get a whole new view every time . . . if the reporting tool is done *right* (don’t laugh ) It will give you a new perspective on the typical one-dimensional reporting that we are used to.
- Both represent the same dataCharts easier to interpret quicklyTables better for granular details
- Summary infoDrill-down into details
- Q#1 – what are some examples in your personal life of a rich visualization done right? (Maybe you could have some samples to show the groups to aid the discussion? . . mobile apps??, financial apps?, etc. )Q#2 – how would you apply that visualization to learning analytics?Q#3 – do instructors have specific visualization needs? That is, does everyone love a pie chart ;)Q#4 – what about accessibility needs re: visualizations?
- Standards based gradingEach category represents a standardSize of category represents the weight of activity around that standard in the course
- Drill down to show details for each activity related to that standard
- Stats from Complete College AmericaGetting students into HE has never been easierGI Bill (1944), Civil Rights Act (1964), Higher Education Act (1965, 2009)Over $150B in student federal financial aid a yearKeepingstudents in HE has never been harderIncreased enrollment ≠ increased degree attainmentUS dropped in world rank from 1st to 16th for young adults holding degrees Despite value of college degree less than 30% of Americans have a college degree (27%2009 US Census Bureau)
- Acquired and integrated with Knowillage Learning Path technology to extend the ILP.Adaptive learning platform that extends the capabilities of your LMS by powering your content. Imagine if the LMS could learn from the student – right in the learning moment?
- imagine how it would be almost impossible for someone to summarize what they read in hundreds of medical research papers. Now imagine reading millions and millions of research papers and then summarizing the most important information for a particular patient in less than 10 seconds. This sheds light on both the and the machines can make interconnections humans would have a hard time making because they can remember everything and make connections across different sources in very large collections of documents. In fact they can beat groups of humans who split up the work if the collection is very large. scalability and value of text analytics. Text mining is a way of using computers to read through text and associate the terms and phrases into common areas. Businesses use text mining to help identify the types of subjects and topics people are sharing with each other via their computer. Like people do when they are good listeners; with text mining companies identify, ideas, opinions, things people like or do not like. Then we can interact based on their interests.
- When you connect with your contacts, here is the messaging to focus on: 1) D2L is the first vendor everto be asked to presentin this capacity at LAK. It has always been a scholarly event with no formal vendor presence. D2L was selected to attend as an example of a good vendor role model who is actively bridging the gap between the research and software vendor worlds2) D2L will be the only vendor present in this capacity at LAK 2014. Committee members have made it clear that D2L were chosen not only because of our support for the analytics research community but also because we deliver research-backed solutions to market. This is a huge differentiator for D2L!3) D2L was selected based on our own research work as well as support of the research community of which we are well-known – not just in the learning analytics world (see below) but also in other education areas as well (to name a few): a. OERb. Teacher Resources/Successc. Accessibility 4) Examples of learning analytics research you can cite if you like: a. Predictive analytics algorithm and data modeling techniques for Insights S3 and Degree Compass (past research works)b. Semantic Content Analysis and Big Data (current research works)c. Learning Objectives and Unstructured Text (future research works)Detailed Deets:OER = Open Educational Resources .. this was the project :“Investigating Open Scholarship and Inclusive Practices in Higher Education” http://www.mitacs.ca/cat/partner/desire2learnWhat came out of this research was a survey as well as a question bank of questions that could be used by institutions to determine/validate the feelings and understanding of their staff regarding OER and Inclusive Practices. We would be happy to have clients work with us to get these questions into a survey for their institutions. Teacher Resources/Success“Why some teachers easily learn to use a new virtual learning environment: a technology acceptance perspective”http://www.tandfonline.com/doi/abs/10.1080/10494820.2014.881394 - this is what others will see if they look it up (and will have to pay for) .. I’ve attached the actual paper from Bart .. please do not distribute the paper to anyone not working at D2L. AccessibilityPlease remove the CAST wording .. you can say “We have just kicked off research with a couple of partners for K-12 Students with Disabilities”Essa/Ayad – S3 research paperhttp://www.researchinlearningtechnology.net/index.php/rlt/article/viewFile/19191/pdf_1Past Product Based ResearchDesire2Learn Student Success System (S3)TMS3 is an application that uses predictive models, advanced statistics, and interactive visualizations for identifying at-risk students pre-emptively, understanding why they are at risk, designing interventions to mitigate that risk, and closing the feedback loop by tracking the efficacy of the applied intervention. S3 uses machine learning to train a regression model from past student activity in the LMS (e.g. logins, content visits, discussion posts, grades, etc.) and specific Student Information Data (SIS) data in order to predict grades for students currently enrolled within a specific course.Desire2Learn Degree Compass TMDegree Compass is a recommendation engine. The tool helps students select the courses most suited to their strengths and interests. Its algorithms consider a particular student’s transcript and test scores, as well as the performance of hundreds of thousands of previous students, to generate a personalized curriculum. It uses a collaborative filtering technique to predict student grades in courses they have not taken yet. These algorithms rely on Student Information System (SIS) data.Desire2Learn LeaP TMCurrently in BETA release, LEArning Path (LeaP) is an LTI-integrated tool that gives users the power of personalization and adaptive learning. LeaP gives instructors the ability to extend their students learning beyond the traditional course content structure. When enabled and utilized by the instructor, LeaP provides students with personalized learning paths using existing course content as well as open educational resources. The LeaP recommendation engine suggests the most effective learning paths through each course's materials, and its activity and feedback engines use objective results to identify the most effective learning materials and adapt the learning paths for each student. The tool offers pre- and post-tests to assess student mastery of material, and LeaP uses those assessment results to deliver appropriate learning paths for each student.AccessibilityAlthough standards provide a great starting point for learning and testing accessible designs, years of close interaction with client experts and students with disabilities have led to a user-experience culture at Desire2Learn. We are committed to designing products that are not only standards compliant, but also flexible and intuitive. We regularly review our designs and products with our client-led Accessibility Interest Group (http://collaborate.athenpro.org/group/d2l/) and students and instructors with disabilities to ensure we are supporting equal, inclusive learning. For example, many of our tools provide shortcuts, settings and layout options that enable users to customize and simplify their experience with the system. Many of our newer tools use advanced interface controls, such as drag-and-drop functionality, that also have fully integrated keyboard and assistive technology alternatives. Current Research InterestsContent AnalysisCleaning and preprocessing: this process aims to provide efficient tools for text extraction from the different types of the textual data sets. It also aims to clean the textual data from un-useful information and transform text into format that is ready for preprocessing using text analytics specifically Semantic Analysis: analyze text using natural language processing including syntactic and semantic parsing, named entity recognition, text mining, machine learning, semantic role labelling, and building social graphs. Then, generate comprehended concepts based on the output of the text analyticsVisualization: make use of interactive visualizations which will allow for better presentation and evaluation of relevant content concepts and better presentation for various volumes and modes of social activity, which in turn will aid in better decision-making.Hybrid Cloud – Big Data AnalyticsCloud Architecture: create a cloud for data analytics with significant amounts of computing power and robust storage to conduct analytics using on-demand, scalable infrastructure. Data Extraction and Transformation: extract data from multiple data sources and load it into an optimized cloud-based data warehouse. Hybrid Cloud: Introduce a mechanism for hybrid clouds that facilitates the communication among different clouds specifically public and private clouds. The mechanism will provide the opportunity to interchange data sources and computing facilities among the disparate clouds.Product Based ResearchWe continue to conduct Product based research and development in S3, Degree Compass, LeaP and accessibility. Future Research InterestsWe are interested in the relationships among learning objectives, content and discussions in order to identify which learning objectives students struggle.
- Detailed Deets:OER = Open Educational Resources .. this was the project :“Investigating Open Scholarship and Inclusive Practices in Higher Education” http://www.mitacs.ca/cat/partner/desire2learnWhat came out of this research was a survey as well as a question bank of questions that could be used by institutions to determine/validate the feelings and understanding of their staff regarding OER and Inclusive Practices. We would be happy to have clients work with us to get these questions into a survey for their institutions. Teacher Resources/Success“Why some teachers easily learn to use a new virtual learning environment: a technology acceptance perspective”http://www.tandfonline.com/doi/abs/10.1080/10494820.2014.881394 - this is what others will see if they look it up (and will have to pay for) .. I’ve attached the actual paper from Bart .. please do not distribute the paper to anyone not working at D2L. AccessibilityPlease remove the CAST wording .. you can say “We have just kicked off research with a couple of partners for K-12 Students with Disabilities”Essa/Ayad – S3 research paperhttp://www.researchinlearningtechnology.net/index.php/rlt/article/viewFile/19191/pdf_1Past Product Based ResearchDesire2Learn Student Success System (S3)TMS3 is an application that uses predictive models, advanced statistics, and interactive visualizations for identifying at-risk students pre-emptively, understanding why they are at risk, designing interventions to mitigate that risk, and closing the feedback loop by tracking the efficacy of the applied intervention. S3 uses machine learning to train a regression model from past student activity in the LMS (e.g. logins, content visits, discussion posts, grades, etc.) and specific Student Information Data (SIS) data in order to predict grades for students currently enrolled within a specific course.Desire2Learn Degree Compass TMDegree Compass is a recommendation engine. The tool helps students select the courses most suited to their strengths and interests. Its algorithms consider a particular student’s transcript and test scores, as well as the performance of hundreds of thousands of previous students, to generate a personalized curriculum. It uses a collaborative filtering technique to predict student grades in courses they have not taken yet. These algorithms rely on Student Information System (SIS) data.Desire2Learn LeaP TMCurrently in BETA release, LEArning Path (LeaP) is an LTI-integrated tool that gives users the power of personalization and adaptive learning. LeaP gives instructors the ability to extend their students learning beyond the traditional course content structure. When enabled and utilized by the instructor, LeaP provides students with personalized learning paths using existing course content as well as open educational resources. The LeaP recommendation engine suggests the most effective learning paths through each course's materials, and its activity and feedback engines use objective results to identify the most effective learning materials and adapt the learning paths for each student. The tool offers pre- and post-tests to assess student mastery of material, and LeaP uses those assessment results to deliver appropriate learning paths for each student.AccessibilityAlthough standards provide a great starting point for learning and testing accessible designs, years of close interaction with client experts and students with disabilities have led to a user-experience culture at Desire2Learn. We are committed to designing products that are not only standards compliant, but also flexible and intuitive. We regularly review our designs and products with our client-led Accessibility Interest Group (http://collaborate.athenpro.org/group/d2l/) and students and instructors with disabilities to ensure we are supporting equal, inclusive learning. For example, many of our tools provide shortcuts, settings and layout options that enable users to customize and simplify their experience with the system. Many of our newer tools use advanced interface controls, such as drag-and-drop functionality, that also have fully integrated keyboard and assistive technology alternatives. Current Research InterestsContent AnalysisCleaning and preprocessing: this process aims to provide efficient tools for text extraction from the different types of the textual data sets. It also aims to clean the textual data from un-useful information and transform text into format that is ready for preprocessing using text analytics specifically Semantic Analysis: analyze text using natural language processing including syntactic and semantic parsing, named entity recognition, text mining, machine learning, semantic role labelling, and building social graphs. Then, generate comprehended concepts based on the output of the text analyticsVisualization: make use of interactive visualizations which will allow for better presentation and evaluation of relevant content concepts and better presentation for various volumes and modes of social activity, which in turn will aid in better decision-making.Hybrid Cloud – Big Data AnalyticsCloud Architecture: create a cloud for data analytics with significant amounts of computing power and robust storage to conduct analytics using on-demand, scalable infrastructure. Data Extraction and Transformation: extract data from multiple data sources and load it into an optimized cloud-based data warehouse. Hybrid Cloud: Introduce a mechanism for hybrid clouds that facilitates the communication among different clouds specifically public and private clouds. The mechanism will provide the opportunity to interchange data sources and computing facilities among the disparate clouds.Product Based ResearchWe continue to conduct Product based research and development in S3, Degree Compass, LeaP and accessibility. Future Research InterestsWe are interested in the relationships among learning objectives, content and discussions in order to identify which learning objectives students struggle.
- Include this slide in your presentation as it has all the necessary Desire2Learn trademark information that needs to be included in external materials.