O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
Evaluating Comments in
MOOC Discussion Forums
Tim O’Riordan
Web and Internet Science (WAIS)
www.ecs.soton.ac.uk/people/tor...
Motivations
Image: Boxing Mail – N.Y. Post Office c. 1910.
Library of Congress, NKCR
@studywbv
2
Evaluating Comments in
MO...
Content analysis
Image: ALTC2015, Association for Learning Technology ©2015, CC BY-NC-
SA 2.0
@studywbv
Evaluating Comment...
4
Bloom score Bloom descriptor
0 - Off-topic There is written content, but not relevant to the subject under discussion.
1...
5
CoI score CoI descriptor
0 - Off-topic There is written content, but not relevant to the subject under discussion.
1 – T...
Data
Image: University of Southampton, Understanding Language, Exploring Oceans
and Contract Management MOOCs. FutureLearn...
• 500 comments each MOOC
• Rated according to 2 methods by 7 raters
• Linguistic Inquiry and Word Count (LIWC 2015)
• Corr...
Results
Correlation between Bloom and Cognitive Presence
r = 0.909
p = <0.001
@studywbv
8
Evaluating Comments in
MOOC Disc...
Results
Correlation between Bloom and Word Count
r = 0.687
p = <0.001
@studywbv
9
Evaluating Comments in
MOOC Discussion F...
Results
Correlation between Bloom and 1st person singular
r = -0.321
p = <0.001
@studywbv
10
Evaluating Comments in
MOOC D...
Images: Weka Logo/Weka interface, University of Waikato ©2015
11
Evaluating Comments in
MOOC Discussion Forums
Preliminary ML – CP
@studywbv
12
Evaluating Comments in
MOOC Discussion Forums
• 16 attributes:
like, word count, WPS, Six...
Preliminary ML
@studywbv
13
Evaluating Comments in
MOOC Discussion Forums
Predicted
Actual Off-topic Triggering Exploratio...
@studywbv
tim.oriordan@soton.ac.uk
Thank you…
www.slideshare.net/studywbv/flan-presentation-7-june-2016
14
Evaluating Comm...
Próximos SlideShares
Carregando em…5
×

FLAN Presentation, 7 June 2016

My talk at FutureLearn Academic Network day at CALRG Conference 2016, Open University, Milton Keynes.

Programme: http://cloudworks.ac.uk/cloudscape/view/2975

Livros relacionados

Gratuito durante 30 dias do Scribd

Ver tudo
  • Seja o primeiro a comentar

FLAN Presentation, 7 June 2016

  1. 1. Evaluating Comments in MOOC Discussion Forums Tim O’Riordan Web and Internet Science (WAIS) www.ecs.soton.ac.uk/people/tor1w07 @studywbv
  2. 2. Motivations Image: Boxing Mail – N.Y. Post Office c. 1910. Library of Congress, NKCR @studywbv 2 Evaluating Comments in MOOC Discussion Forums
  3. 3. Content analysis Image: ALTC2015, Association for Learning Technology ©2015, CC BY-NC- SA 2.0 @studywbv Evaluating Comments in MOOC Discussion Forums 3
  4. 4. 4 Bloom score Bloom descriptor 0 - Off-topic There is written content, but not relevant to the subject under discussion. 1 - Remember Recall of specific learned content, including facts, methods, and theories. Verbs: name, describe, relate, find, list, write, tell. 2 - Understand Perception of meaning and being able to make use of knowledge, without understanding full implications. Verbs: explain, compare, discuss, restate, predict, translate, outline. 3 - Apply Tangible application of learned material in new settings. Verbs: show, complete, use, classify, examine, illustrate, implement, solve. 4 - Analyse Deconstruct learned content into its constituent elements in order to clarify concepts and relationships between ideas. Verbs: explain, compare, contrast, examine, identify, investigate, categorise, differentiate, organise. 5 - Evaluate Assess the significance of material and value in specific settings. Verbs: check, decide, rate, choose, recommend, justify, assess, prioritise, critique. 6 - Create Judge the usefulness of different parts of content, and producing a new arrangement. Verbs: synthesise, invent, plan, compose, construct, design, imagine, generate. A popular and well-respected aid to curriculum development that maps learning to six categories of knowledge acquisition. Bloom’s Taxonomy Evaluating Comments in MOOC Discussion Forums
  5. 5. 5 CoI score CoI descriptor 0 - Off-topic There is written content, but not relevant to the subject under discussion. 1 – Triggering event A contribution that exhibits a sense of puzzlement deriving from an issue, dilemma or problem. Includes contributions that present background information, ask questions or move the discussion in a new direction. Verbs: evoke, induce, contradict 2 - Exploration A comment that is seeking a fuller explanation of relevant information. This can include brainstorming, questioning and exchanging information. Contributions are unstructured and may include: unsubstantiated contradictions of previous contributions, different unsupported ideas or themes, personal stories, and descriptions or facts that are not used as evidence. Verbs: inquire, diverge, search 3 - Integration Previously developed ideas are connected. Contributions include: references to previous messages followed by substantiated agreements or disagreements; developing and justifying established themes; cautious hypotheses; combining different sources; providing a tentative solution to an issue. Verb: test, conjecture, check 4 - Resolution New ideas are applied, tested and defended with real world examples. This involves methodically testing hypotheses, critiquing content in a systematic manner, and expressing supported intuition and insight. Verb: commit, settle, confirm A highly cited method that categorises 4 phases of discussion. Cognitive Presence (Community of Inquiry) Evaluating Comments in MOOC Discussion Forums
  6. 6. Data Image: University of Southampton, Understanding Language, Exploring Oceans and Contract Management MOOCs. FutureLearn Ltd 2015 @studywbv 6 Evaluating Comments in MOOC Discussion Forums
  7. 7. • 500 comments each MOOC • Rated according to 2 methods by 7 raters • Linguistic Inquiry and Word Count (LIWC 2015) • Correlated by: i) ≥50 word comments ii) Aggregated batches of 10 contiguous comments iii) All individual comments Method @studywbv 7 Evaluating Comments in MOOC Discussion Forums Image: University of Southampton, Understanding Language, Exploring Oceans and Contract Management MOOCs. FutureLearn Ltd 2015.
  8. 8. Results Correlation between Bloom and Cognitive Presence r = 0.909 p = <0.001 @studywbv 8 Evaluating Comments in MOOC Discussion Forums
  9. 9. Results Correlation between Bloom and Word Count r = 0.687 p = <0.001 @studywbv 9 Evaluating Comments in MOOC Discussion Forums
  10. 10. Results Correlation between Bloom and 1st person singular r = -0.321 p = <0.001 @studywbv 10 Evaluating Comments in MOOC Discussion Forums
  11. 11. Images: Weka Logo/Weka interface, University of Waikato ©2015 11 Evaluating Comments in MOOC Discussion Forums
  12. 12. Preliminary ML – CP @studywbv 12 Evaluating Comments in MOOC Discussion Forums • 16 attributes: like, word count, WPS, Sixltr, ppron, prep, auxverb, negate, posemo, negemo, cogproc, cause, differ, affiliation, power • 10 fold Cross-validation • Classifiers: Naive Bayes, J48, Random Forest • Accuracy: 60% • Cohen’s K: 0.48
  13. 13. Preliminary ML @studywbv 13 Evaluating Comments in MOOC Discussion Forums Predicted Actual Off-topic Triggering Exploration Integration Resolution Off-topic 135 41 36 6 0 Triggering 41 114 115 18 1 Exploration 13 60 295 92 9 Integration 0 8 120 184 70 Resolution 0 0 9 54 291 Confusion matrix for best model – Cognitive presence
  14. 14. @studywbv tim.oriordan@soton.ac.uk Thank you… www.slideshare.net/studywbv/flan-presentation-7-june-2016 14 Evaluating Comments in MOOC Discussion Forums timswww.wordpress.com

    Seja o primeiro a comentar

    Entre para ver os comentários

  • ignatia

    Jun. 7, 2016
  • MiguelBallesteros10

    Jun. 7, 2016

My talk at FutureLearn Academic Network day at CALRG Conference 2016, Open University, Milton Keynes. Programme: http://cloudworks.ac.uk/cloudscape/view/2975

Vistos

Vistos totais

1.531

No Slideshare

0

De incorporações

0

Número de incorporações

575

Ações

Baixados

2

Compartilhados

0

Comentários

0

Curtir

2

×