Information Systems Success Awareness for Professional Long Tail Communities of Practice
•Download as PPTX, PDF•
1 like•422 views
Doctoral Thesis Defense, July 12, 2016, Aachen, Germany
Recommended
Recommended
More Related Content
Similar to Information Systems Success Awareness for Professional Long Tail Communities of Practice
Similar to Information Systems Success Awareness for Professional Long Tail Communities of Practice (20)
More from Dominik Renzel
More from Dominik Renzel (6)
Recently uploaded
Recently uploaded (20)
Information Systems Success Awareness for Professional Long Tail Communities of Practice
- 1. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-1 This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International Public License. Information Systems Success Awareness for Professional Long Tail Communities of Practice Dipl.-Inform. Dominik Renzel Doctoral Thesis Defense Chair of Computer Science 5 – Information Systems & Databases RWTH Aachen University Aachen, Germany July 12, 2016
- 2. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-2 Agenda Introduction MobSOS CIS Success Awareness Framework – Methodology – Formalization – Technical Infrastructure Longitudinal Studies & Findings Conclusion & Outlook
- 3. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-3 Introduction Introduction Motivating Examples Terminology Research Problem & Questions
- 4. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-4 Small-Scale Example – Aphasics Group Chat Tool (DESRIST 2015) Success? Quality? Impact? Simple? Responsive? Reliable? Deployable? Helps regain skills? Avoids isolation? Augments therapy? Unforeseen effects? Success Success Success Introduction
- 5. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-5 How to measure & stay aware of success in/across highly distributed & heterogeneous communities using mobile Web technologies on a massive scale over time? Large-Scale Heterogeneity & Dynamicity – ROLE Sandbox (EC-TEL 2015) Introduction Success? Success? Success?
- 6. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-6 Community of Practice, Long Tail & Community Information Systems Community of Practice (CoP) [Weng98] Technology People Rules “A group of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly.” [Ande06] Introduction Learning requires reflection & awareness of CIS Success!
- 7. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-7 Related Work Web Science IS Success IS Design Research Framework [HMPR04] Methodology [PTRC07,SHP*11] Models [DeMc92,GSCh08,CKAb14] Formalization [JMPo03,SBGe04] Web Engineering [GJHV95,Fiel00] Web Analytics [Kaus10,CCSt12,Evan12] Introduction
- 8. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-8 Related Work Web Science IS Success IS Design Research Framework [HMPR04] Methodology [PTRC07,SHP*11] But: implicit success notion Models [DeMc92,GSCh08,CKAb14] Formalization [JMPo03,SBGe04] But: positivist paradigm, myopic financial focus Web Engineering [GJHV95,Fiel00] Web Analytics [Kaus10,CCSt12,Evan12] But: analytics infrastructure & strategy corporate secret Introduction
- 9. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-9 General Idea Introduction From main-stream, centralized & secret corporate business analytics to long-tail, scalable, distributed, open negotiation of CIS success
- 10. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-10 Research Questions & Contributions RQ1: How can professional long tail communities of practice achieve a shared sense of CIS success awareness with the help of a media-supported and negotiation- based methodology? RQ2: What formalization should communities use to model CIS success? How can communities formalize the assessment, negotiation, and validation of different notions of CIS success? RQ3: How should technical infrastructure for community information systems be designed to achieve integrated support for CIS success awareness? MobSOS CIS Success Awareness Framework Introduction
- 11. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-11 MobSOS Methodology MobSOS Methodology Community-Oriented Design Research Media-Supported CIS Success Negotiation RQ1: How can professional long tail communities of practice achieve a shared sense of CIS success awareness with the help of a media-supported and negotiation-based methodology?
- 12. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-12 Design Science Research in Information Systems [HMPR04] MobSOS Methodology
- 13. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-13 Community-Oriented Design Science Research (DESRIST 2015) MobSOS Methodology
- 14. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-14 CIS Success Modeling & Negotiation as Media Transcription [JJKS08,Klam10] MobSOS Methodology
- 15. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-15 MobSOS Formalization Framework CIS Success Model Formalization CIS Success Model Validation RQ2: What formalization should communities use to model CIS success? How can communities formalize the assessment, negotiation, and validation of different notions of CIS success? MobSOS Formalization Framework
- 16. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-16 Formal CIS Success Models – Structural Model (DESRIST 2015) MobSOS Formalization Framework Explore ValidateAdapt IS-Impact Model, Formative Index [GSCh08,JMPo02]
- 17. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-17 Formal CIS Success Models – Measurement Model Model: Metrics: MobSOS Formalization Framework
- 18. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-18 CIS Success Model Validation – Ordinal Logistic Regression Literature: multiple (linear) regression [DiWi01,PSRa07] But: metrics violate assumptions (linearity, homoskedacity, normality, scales) Better: ordinal logistic regression [Agre01] Interpretation: odds & probabilities Statistics: – Model: fit – Effect Strengths – Significances MobSOS Formalization Framework
- 19. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-19 MobSOS Technical Infrastructure CIS Success-Aware Community Platform MobSOS Service Tool Kit RQ3: How should technical infrastructure for community information systems be designed to achieve integrated support for CIS success awareness? MobSOS Technical Infrastructure
- 20. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-20 Success-Aware CIS Platform (WISMA 2008, ICSC 2009) MobSOS Technical Infrastructure
- 21. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-21 Usage Data Collection & Enrichment – MobSOS Pipeline (LAK 2013) MobSOS Technical Infrastructure Reverse Proxy & Pipeline Pattern: [Somm02,GHJV95]
- 22. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-22 Exploration by Visual Analytics – MobSOS QV (I-Know 2012) Visualization Authoring Visual Analytics Widgets & Dashboards MobSOS Technical Infrastructure
- 23. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-23 Evaluation Products for Awareness – MobSOS Success Model Service MobSOS Technical Infrastructure
- 24. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-24 Longitudinal Studies & Findings Case Studies & Findings 2004 – 2015: SOCRATES – Aphasia Therapy (DFG SFB 427) 2008 – 2016: ROLE SDK – Self-Regulated Learning (ROLE) (6 more studies in thesis)
- 25. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-25 SOCRATESX – Aphasia Therapy (DESRIST 2015) Case Studies & Findings
- 26. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-26 SOCRATESX – Aphasia Therapy (DESRIST 2015) Group chat tool incl. word completion German aphasia patients (N>200) + therapists, linguists, computer scientists Chat protocol logs & survey responses (1 yr.) Compact success model? Mineable from protocol logs? Benefits? Case Studies & Findings
- 27. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-27 SOCRATESX – Optimizing Design with System Quality Metrics Case Studies & Findings High hit rate (HR) & low keystrokes until prediction (KUP) 5 candidates good size for word completion list
- 28. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-28 SOCRATESX – Measuring Individual & Community Impacts Patients improve individual performance Community as whole improves performance Positive impacts traceably measurable Case Studies & Findings
- 29. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-29 SOCRATESX CIS Success Model (DESRIST 2015) Case Studies & Findings
- 30. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-30 CIS Success Models – General Target Characteristics Sharply focused scope Handful of most relevant factors Balance universal vs. community-specific factors Majority of quality & impact dimensions reflected Actionable support for agency Case Studies & Findings
- 31. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-31 ROLE Sandbox – Analyzing Self- Regulated Learning (LAK 2013, ICWL 2015) Background: planning, building & reflection of personal learning environments (PLE) essential activities for self-regulated learning (SRL) [Zimm02,NKR*14a]. Case Studies & Findings
- 32. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-32 ROLE Sandbox – Analyzing Self- Regulated Learning (LAK 2013, ICWL 2015) PLE management platform, learning spaces & learning widgets Widget developers, learners, teachers, researchers (N>10.000) Context-enriched PLE management logs (>3 yrs.) Self-regulated learning activity traceable in PLE? Learner classification? Recommendations? Case Studies & Findings
- 33. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-33 Category-Based Recommendations for Self-Regulated Learners (ICWL 2015) In SRL-Spaces: Intense use of recommendations Interesting evaluation contexts Improve recommendations by complete categorization Case Studies & Findings
- 34. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-34 Measuring Networking Impact of ROLE SDK with SNA Metrics Case Studies & Findings
- 35. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-35 Conclusion & Outlook Conclusion & Outlook
- 36. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-36 Major Contributions Anti-positivist methodology for media-based CIS success negotiation as guidance for CIS design Digital media representation & validation of fluid CIS success models capturing qualitative & quantitative aspects from multiple heterogeneous data sources Robust, scalable & extensible Open Source Software framework based on established patterns & cutting- edge mobile Web technologies MobSOS CIS Success Awareness Framework
- 37. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-Renzel-0716-37 Possible Avenues for Future Work Conclusion & Outlook Standard ACIS thesis evaluation framework Social software for CIS success negotiation Integrated early-phase RE & CIS success modeling CIS success-driven recommender systems Automated metric exploration by machine learning Cross-community CIS success analytics
Editor's Notes
- Dear Prof. Jarke thank you for the kind introduction, dear Dr. Klamma, dear Prof. Spaniol, respected members of the examination committee, fellow researchers, dear family, ladies and gentlemen, good afternoon and thanks for being here. Today I will present the results of my PhD dissertation titled „Information Systems Success Awareness for Professional Long Tail Communities of Practice“. Before diving into the details, let me start with a brief real-world example which motivates the topic and identifies many of the challenges.
- Throughout Presentation: references to own works and external works
- To give idea, Look at example What is Aphasia? Cause? Effects? Essential: start early, frequently Problem: insurance coverage SOCRATES Describe features If design, take resources, but scarce Guide efforts or justify external funding requires constant reflection and awareness To give you an idea of what this topic is about, lets have a look at a small example from year-long participation in an Aphasia therapy group at RWTH Aachen university hospital. Aphasia is a medical condition caused by brain lesions and the effect of severe language impairments for tasks like reading, writing, finding the right words, and so on. To avoid known negative consequences like social isolation and depression, group therapy must apply as early and frequently as possible. Problem is that health insurance only covers infrequent group meetings, so the community developed the idea to use networked communication technology to effectively augment regular f2f therapy. In this community we worked together with patients, therapists and other researchers and co-developed the SOCRATES group chat tool custom tailored to the needs of its members, mainly the patients. (describe features) If you design such tools, they take up resources, which are inherently scarce. In order to guide community efforts in the right direction or to justify funding by health insurance, this community must be able to reflect and constantly stay aware if and why such a chat tool is successful for the different stakeholders in the community.
- If you scale out of this very particular example and consider the big picture, there arise very concrete problems of heterogeneity (wide variation of domain, scale, structure, practice) and dynamicity (emergent and mutually influencing change between technology and people in information systems we know from all kinds of socio-technical systems). Especially in the long tail, we find different notions of success, with different focus on relevance and valuation for universal success factors and very community specific success factors.
- With examples we have seen a couple instances of the more general problem tackled in my work. For generalization we need abstracting theories capturing all possible contexts we want to consider. For this work we therefore use three theories: CoP Long Tail CIS
- In my further literature research on related works, I mainly focused on three rather disjoint research areas in IS and CS. The IS Design research community provides a research framework and methodologies capturing design processes in CIS with an emphasis on a build & evaluate cycle and a community memory for learning how to do it better. However, although quite strongly related to evaluation, a notion for CIS success remains implicit in these works. The IS success research community provides explicit models, formalizations, and validation techniques. However, as follower of the traditional IS positivist research paradigm and with a myopic financial focus on EIS, such models clash with our understanding of the different notions and facets of success in long tail commuinities. Finally, the Web Science community provide the necessary Web engineering and analytics technologies to build up infrastructures capable of creating CIS success awareness. However, mostly designed for data-driven enterprises, such infrastructures and strategies are considered corporate secrets and are thus not accessible to end-user communities.
- In my further literature research on related works, I mainly focused on three rather disjoint research areas in IS and CS. The IS Design research community provides a research framework and methodologies capturing design processes in CIS with an emphasis on a build & evaluate cycle and a community memory for learning how to do it better. However, although quite strongly related to evaluation, a notion for CIS success remains implicit in these works. The IS success research community provides explicit models, formalizations, and validation techniques. However, as follower of the traditional IS positivist research paradigm and with a myopic financial focus on EIS, such models clash with our understanding of the different notions and facets of success in long tail commuinities. Finally, the Web Science community provide the necessary Web engineering and analytics technologies to build up infrastructures capable of creating CIS success awareness. However, mostly designed for data-driven enterprises, such infrastructures and strategies are considered corporate secrets and are thus not accessible to end-user communities.
- From carved-in-stone company secrets to open community billboards!! From universal pre-built CIS success models to community negotiation of shared CIS success awareness
- Alle Fragen durchgehen und kurz vorab vorstellen, was die contributions sein werden Am Ende: all diese Contributions zusammengefasst zu MobSOS Basis Rahmenwerk
- TODO: reformulate RQ1.
- TODO: reformulate RQ1.
- Notice that exploration is a time consuming task! Always better to reuse already established and validated measures/instruments.
- Data integration by attribute correspondence! Use of unique identifiers, usually provided by the CIS and its core services (user, artifact, tool, etc.) In the end, the measurement model Delta_M is then a simple join of the individual metrics views over the respective user. One data vector per user.
- What is the purpose of validation? The primary purpose of validation is to test if a model really measures CIS success as it is understood by the community. Validation also provides information on the most relevant core of such measurement, so we make models more compact and robust. Intuitively, a community will usually keep track of only few factors that are most relevant for it. Once valid, you community members can ask the model questions of predictive nature. A typical question would be: “Assuming that the community improves success Metric Mj by one unit, what is the probability that overall success is rated k afterwards?”.
- TODO: reformulate RQ1.
- Today, such browser-based visual analytics services are offered by modern cloud-based analytics providers (e.g.
- In my thesis you find pointers to quite some studies in which we used the MobSOS approach for evaluation in different communities in the areas of healthcare, multimedia, technology-enhanced learning, digital entertainment, and Web engineering. For this presentation, I have selected two rather longitudinal studies from healthcare and technology-enhanced learning.
- Notice that this improvement is almost nothing for a person with normal language skills, but for an aphasia patient this is a very significant improvement, if you keep in mind that producing one line of text sometimes costs them some minutes.
- Self-regulated learning: learners actively follow four learning phases: planning
- In the same sense as word prediction helped tune system quality in SOCRATES, the knowledge about category coverage here helped to identify room for significant improvements regarding category-based recommendations.
- TODO: reformulate RQ1.
- New order: from small-scale (chair) to large-scale (long tail) For some topics, new theses have been advertised already on our chair website.