This document describes the polyphonic model of communication and its applications. The polyphonic model views communication as involving multiple voices or perspectives that interact in parallel threads. The document discusses how the polyphonic model and analysis method can be used to study collaboration in chat conversations, online discussions, and face-to-face settings. It also describes various computer systems that have been developed to conduct polyphonic analysis and detect collaboration patterns in text-based communications.

1. Polyphonic Analysis of Discourse in Texts and
in Collaborative Learning Chats
Ştefan Trăuşan-Matu
University Politehnica of Bucharest
Computer Science Department

2. Contents
 The Polyphonic Model
 Polyphonic Analysis
 Implementations of the Polyphonic Analysis
 The PolyCAFe Analysis System
 Other Applications
 Conclusions
2
Stefan Trausan-Matu
12/14/2013

3. Polyphony- An Unitary Model of
Human Communication
 Mediation:
 Using natural language (words) in
 texts
 hypertexts
 discussion forums
 conversations
but also
 Non verbal communication (e.g.
gestures)
3
Stefan Trausan-Matu
12/14/2013

4. Polyphony- An Unitary Model of
Human Group Communication
 Considering rather a dialogistic, post-
structuralist (Bakhtin, Kristeva) than a
mechanistic perspective on communication
 Rather a socio-cultural (Vygotsky) approach
than a cognitivist one (like in Artificial
Intelligence) but considering the both
 Ethnomethodology (Garfinkel), Conversation
Analysis (Sacks, Schegloff, Jefferson)
4
Stefan Trausan-Matu
12/14/2013

5. Polyphony- An Unitary Model of
Group Communication
 Applicable to:
 Small groups (e.g. virtual teams
collaborating by chat or forums – the
INTER-ANIMATION phenomenon
appears)
 Large groups – social networks
 Global level - intertextuality
5
Stefan Trausan-Matu
12/14/2013

6. Polyphony- An Unitary Model of
Group Communication and
Intertextuality
 It may be used in IT implementations
using
 Natural language Processing
 Machine learning
 Social Network analysis
 Specific techniques (inter-animation
and collaboration analysis)
6
Stefan Trausan-Matu
12/14/2013

7. The Polyphonic Model
7
Stefan Trausan-Matu
12/14/2013

8. Polyphony
 Appears in music (e.g. J.S.Bach) and in texts (Bakhtin)
 The Polyphonic
 Model (Trausan-Matu, Handbook of Hybrid Learning, 2010)
 Analysis method (Trausan-Matu and Rebedea, 2010)
 Computer support tools for the polyphonic analysis of F2F,
online and offline conversations:
 The “Polyphony” system (Trausan-Matu and all, 2007)
 ASAP (Dascalu, Chioasca and Trausan-Matu, 2008)
 PolyCAFe (Trausan-Matu, Rebedea and Dascalu, 2011; Rebedea,
Dascalu, Trausan-Matu and all, 2010)
 Collaboration regions detection (Banica, Trausan-Matu and Rebedea,
2011)
 Detection of the Important moments (Chiru and Trausan-Matu, 2012)
8
Stefan Trausan-Matu
12/14/2013

9. Polyphony
 A group of participants that, each of them
keeps their individuality, personality, creativity,
but also collaborate to achieve a common
goal, trying to solve dissonances
 A merge of:
 Unity vs. Difference
 Melody (longitudinal) and Harmony (transversal)
Cycles –
centrifugal/centripetal forces
 Inter-animation of voices – inter-animation
Stefan Trausan-Matu
patterns
12/14/2013
 Dissonance – Consonance 
9

10. The Polyphonic Model
 Polyphony = Model of collaboration and interaction
(Trausan-Matu, Stahl and Zemel, 2005)
 Human communication in knowledge construction and
collaboration are processes in which words and other
utterances are linked in parallel threads which interact
similarly to voices in polyphonic music
 Repetition and rhythm are essential
10
Stefan Trausan-Matu
12/14/2013

11. 11
Stefan Trausan-Matu
12/14/2013

12. Polyphony
12
Stefan Trausan-Matu
12/14/2013

13. Dialogism and Polyphony (Bakhtin)
 Mikhail Bakhtin:
• Utterances (not sentences) should be the unit of analysis
• “These are different voices singing variously on a single theme. This
is indeed 'multivoicedness,' exposing the diversity of life and the
great complexity of human experience. 'Everything in life is
counterpoint, that is, opposition,' “ (Bakhtin, 1984)
• “… Any true understanding is dialogic in nature” (VoloshinovBakhtin, 1973)
• Speech genres
• Polyphony Inter-animation of voices
• Basis for the CSCL paradigm (Koschman, 1999)
• Opposed to de Saussure ideas:
• Real life dialog should be the focus, not written text
• Words are not arbitrary
13
Stefan Trausan-Matu
12/14/2013

14. Bakhtin’s Polyphony
 Everything is a dialog (applying not only to
speech and text)
 Utterances
 Voices
 Inter-animations among voices
14
Stefan Trausan-Matu
12/14/2013

15. Utterances
 Utterances (not sentences, as in ‘classical’ linguistics) should
be the unit of analysis (Bakhtin)
 Utterances are acts
 An utterances may be a:







Word
Turn, a reply in a conversation, chat or forum
Sentence
Text
Image (picture, diagramatic representation, etc.)
Gesture (individual or group)
Thought – inner utterances – inner speech
 Utterances should be considered at different granularities
 Utterances are linked in threads formed by:
 Explicit links (VMT chat environment; forum’s replies) - uptakes
(Suthers, 2010)
 Implicit links, detected by Natural Language Processing techniques
– contingencies, uptakes (Suthers, 2010)
 Utterances may become voices
15
Stefan Trausan-Matu
12/14/2013

16. Voices
 Distinctive presences in a group, influencing the
other voices
 Generated by utterances (singular or repeated)
 Correspond to:
 participants (may also be inner voices)
 groups of participants (e.g. collective or collaborative
utterances)
 chains or threads of words or concepts:





repeated words
lexical chains
co-references
reasoning or argumentation
rhetorical schemas
 Each utterance may contain multiple voices
 Voices continue and influence each other through
explicit or implicit links.
16
Stefan Trausan-Matu
12/14/2013

17. Inter-animation patterns
 Longitudinal
 Adjacency pairs
 Repetitions
 Elaboration
 Convergence
 Cumulative talk
 Repair
 Transversal, differential
 Dissonance
17
Stefan Trausan-Matu
12/14/2013

18. The Polyphonic Method applications
 Chat conversations with multiple participants for:
 CSCL:
K-12 students solving mathematics problems both individually and
collaboratively in the VMT project at Drexel University, Philadelphia,
US
 CS students at University Politehnica of Bucharest , Romania at
o CHI course in Romanian and French – role playing and debate
o Natural Language Processing - role playing and debate
o Algorithm Design – problem solving

 Fostering creativity – brainstorming, synectics
 F2F collaborative learning (Suthers & all, 2011)
 Analysis of Rhythm
 Metacognition (conversation & essays)
 OpenSimDeveloper dataset
18
Stefan Trausan-Matu
 Intertextuality
12/14/2013

19. Computer Support for the
Polyphonic Analysis
19
Stefan Trausan-Matu
12/14/2013

20. LTfLL - EU FP7 Project (2008-2011) and
NSF Virtual Math Teams Project
http://www.ltfll-project.org/ http://mathforum.org
 Language Technologies for Lifelong Learning
 Netherlands, France, United Kingdom, Germany, Ausria, Romania,
Bulgaria
 PolyCAFe system (Polyphony-based Collaboration
Analysis and Feedback generation)
 The system has been validated with students and tutors in
 University of Manchester, UK
 Politehnica University of Bucharest, Romania
20
Stefan Trausan-Matu
12/14/2013

21. Chat-based CSCL
 K-12 students solving mathematics problems both
individually and collaboratively in the Virtual Math
Teams (VMT) project at Drexel University,
Philadelphia, US (Directed by Gerry Stahl)
 Computer Science students at University
Politehnica of Bucharest (UPB), Romania at
 Human-Computer Interaction course in Romanian and
French – role playing and debate
 Natural Language Processing - role playing and debate
 Algorithm Design – problem solving
21
Stefan Trausan-Matu
12/14/2013

22. Theories for analysing
multi-parties conversation
 Discourse analysis (Tannen)
 Conversation analysis (Sacks, Jefferson, Schegloff)
 Accountable talk (Resnick)
 Transactivity (Teasley, Berkowitz & Gibbs, Joshi &
Rose)
 Events/contingencies, coordinations/uptakes
(Suthers)
 Inter-animation (Bakhtin, Wegerif, Trausan-Matu)
 Polyphony (Bakhtin, Trausan-Matu)
22
Stefan Trausan-Matu
12/14/2013

23. Analysis methods
 TF-IDF
 Latent Semantic Analysis
 Naïve Bayes
 Social Network Analysis
 WordNet (wordnet.princeton.edu)
 Support Vector Machines
 Collin’s perceptron
 TagHelper environment
23
Stefan Trausan-Matu
12/14/2013

24. Analyis methods
 TF-IDF
 Latent Semantic Analysis
Almost all are
based also
on
 Naïve Bayes
a two
interlocutors
 Social Network Analysis
model, in which
 WordNet (wordnet.princeton.edu)
one person
speaks
 Support Vector Machines
at a time,
resulting
 Collin’s perceptron
one discussion
thread
 TagHelper environment
24
Stefan Trausan-Matu
12/14/2013

25. 25
Stefan Trausan-Matu
12/14/2013

26. 26
Stefan Trausan-Matu
12/14/2013

27. NLP pipe
 spelling correction, stemmer, tokenizer, Named Entity
Recognizer, POS tagger and parser, and NP-chunker.
Stanford NLP software
(http://nlp.stanford.edu/software)
 Spellchecker : Jazzy
http://www.ibm.com/developerworks/java/library/jjazzy/
 Alternative NLP pipes are under development,
 GATE (http://gate.ac.uk)
 LingPipe (http://aliasi.com/lingpipe/).
27
Stefan Trausan-Matu
12/14/2013

28. Social network analysis
 Consider explicit and implicit referencing as arcs
between participants, which are the nodes
 A kind of page-rank algorithm – an utterance is
important if it is referred by important utterances;
The strength of a voice (of an utterance)
depends on the strength of the utterances that
refer to it
 Determines if a person is central/peripheral
28
Stefan Trausan-Matu
12/14/2013

29. Polyphony, Inter-animation and
Collaboration analysis
 Assign an importance value for each utterance
considering several indicators of inter-animation
(collaboration)
 Detection of voices (chains) inter-animation patterns
(Trausan-Matu) in the chat
 Consider several criteria such as the presence in the
chat of questions, agreement, disagreement
 Presence of others’ voices
 Social Networks metrics
 Machine learning approach (genetic algorithms and
neural networks) for tuning the
29
Stefan Trausan-Matu
12/14/2013

30. Computational details
(Trausan-Matu, Dascalu and Dessus, ITS 2012;
Dascalu, Trausan-Matu and all, 2010, 2011))
30
Stefan Trausan-Matu
12/14/2013

31. Representations:
Conversation graph
 For each participant there is a separate
horizontal line in the representation
 Each utterance is placed in the line
corresponding to the issuer of that utterance,
according to the emission time, alligned from
left to right
 The explicit references among utterances are
depicted using connecting lines distinctively colored
 The implicit references (deduced by the system) are
represented using other color that the explicit ones
 An estimation of the strength of each
31
utterance (when available) is represented as a
bar chart
Stefan Trausan-Matu
12/14/2013

32. Representations:
Weaving of Voices
 Voices in the conversation graph
 Participants = horizontal lines
 Threads of repeated words or phrases = differently
colored threads
32
Stefan Trausan-Matu
12/14/2013

33. 33
Stefan Trausan-Matu
12/14/2013

34. 34
Stefan Trausan-Matu
12/14/2013

35. 35
Stefan Trausan-Matu
12/14/2013

36. 36
Stefan Trausan-Matu
12/14/2013

37. 37
Stefan Trausan-Matu
12/14/2013
37

38. 38
Stefan Trausan-Matu
12/14/2013
38

39. 39
Stefan Trausan-Matu
12/14/2013
39

40. 40
Stefan Trausan-Matu
12/14/2013
40

41. 41
Stefan Trausan-Matu
12/14/2013
41

42. Validation
(Rebedea, Dascalu, Trausan-Matu and all, ECTEL 2010;
Rebedea, Dascalu, Trausan-Matu and all, ECTEL 2011)
42
Stefan Trausan-Matu
12/14/2013

43. Other applications
43
Stefan Trausan-Matu
12/14/2013

44. Analysis Dimensions (types of voices)
in Face-to-Face Settings
(Trausan-Matu, in Suthers & all, (eds.) 2013)
 Spoken dialog
 Body language
 Individual
 Collective
 The visual dimension
 Visual data on the blackboard
 What others participants do
 Others’ body language
 Internal dialogue (at an intra-mental level)
 Echoes
44
Stefan Trausan-Matu
12/14/2013

45. Inner utterances, inner speech
 “Mead (1934) called thought a <<conversation
with the generalized other,>> implying that when
we think individually we attempt to respondinternally and vicariously-to the imagined
responses of others to our ideas and arguments.”
(Resnick & all, 1993)
 “There are no ontological differences between
inner and outer speech” (Clark and Holquist,
1984).
45
Stefan Trausan-Matu
12/14/2013

46. Detecting important moments
(Chiru and Trausan-Matu, ITS 2012)
46
Stefan Trausan-Matu
12/14/2013

47. Graphical representation of topic’s
rhythmicity
(Chiru, Cojocaru, Trausan-Matu and Rebedea, ISMIS 2011)
47
High
rhythmicity for
all topics –
these were
debated in
parallel as it
can be seen by
the lack of flat
lines near the
left side of the
Stefan Trausan-Matu
representation.
Low rhythmicity
– flat lines on
the left side of
the graphic
showing that
the topic that
they represent
has not been
debated in
those parts of
the chat. 12/14/2013

48. Determination of collaborative
regions
(Banica, Trausan-Matu and Rebedea, CSCL 2011)
48
Stefan Trausan-Matu
12/14/2013

49. Metacognition
(Trausan-Matu, Dascalu and Dessus, ITS 2012)
 Combining Chats and Texts
 Intertextuality
49
Stefan Trausan-Matu
12/14/2013

50. ReaderBench
(Dascalu, Trausan-Matu, Dessus, 2012)
50
Stefan Trausan-Matu
12/14/2013

51. Creativity fostering
(e.g. brainstorming)(Trausan-Matu, 2011)
51
Stefan Trausan-Matu
12/14/2013

52. Opinion mining
(Musat, Velcin, Rizoiu, and Trausan-Matu,
2011)
52
Stefan Trausan-Matu
12/14/2013

53. Topic Modeling
(Musat and Trausan-Matu, 2011)
 No generally accepted definition for a “topic”
 Document clusters
 Abstractions based on document clusters
 Labels;
 Centroids, etc
 (Word, Probability) pairs
 Bayesian statistical models
 Topics – distributions over words
 Documents – distributions over topics
 Generative model
 Topic Intertwining
 Conceptually similar to the ideas of Mikhail Bakhtin
 Topics and voices
53
Stefan Trausan-Matu
12/14/2013

54. Topic Modeling
(Musat and Trausan-Matu, 2011)
 LDA/pLSA/hLDA/CTM
 Each newer version corrects some flaws of the earlier
ones
 However the traditional means of testing the accuracy
have been proven wrong
 Even more reason to look into the problem of evaluating
the models
 LDA
 Readily available
 Mallet
 Easily reproducible experiments
 Well known topic model;
54
Stefan Trausan-Matu
12/14/2013

55. Intertextuality analysis
(Ghiban & Trausan-Matu, 2012)
Voice I
Voice I
Voice II In dialog
Voice III
Voice II
Voice III
Text 1
Text 2
Text 3
Text 1
Text 2
Text 3
In dialog in text 4
Text 4
55
Stefan Trausan-Matu
12/14/2013

56. Intertextuality analysis
(Ghiban & Trausan-Matu, 2012)
56
Theme 2 and Theme
3 may have the
Stefan Trausan-Matu
same words but only
Section 1 and 6 are dialogical or
polyphonical. They may present a
12/14/2013
higher force of expresivity.

57. Analysis of interethnic discourse
(Trausan-Matu, 2012)
 Needed a corpus of texts with time stamps
 Extract recurrent concepts in texts
 Identify historical events
 Generate time series
 Analysis of correlations between time series
 Analysis of the polyphonic structure
57
Stefan Trausan-Matu
12/14/2013

58. Time Series Analysis of News
(Badea & Trausan-Matu, 2013)
58
Stefan Trausan-Matu
12/14/2013

59. Music Composition at K-Teams Laboratory
(Master and Bachelor Thesis coordinated by Prof. Trausan-Matu)
 Genetic Algorithms
 Celular automata
 Artificial chemistry
 Constraint-based systems
 Accompaniments generation with Markov Models
 Random generation
 Automatic counterpoint generation according to Fux rules
 Chat sonification
59
Stefan Trausan-Matu
12/14/2013

60. Chat sonification
(Stefan Trausan-Matu and Alexandru Calinescu)
60
Stefan Trausan-Matu
12/14/2013

61. Chat sonification
61
Stefan Trausan-Matu
12/14/2013

62. 62
Stefan Trausan-Matu
12/14/2013

63. Chat sonification
(Orchestration by Serban Nichifor)
63
Stefan Trausan-Matu
12/14/2013

64. Conclusions
 The polyphonic model may apply to non-verbal
collaboration and intra-subjective (inner thinking)
as well as inter-subjective levels
 A combination of Conversational Analysis with
Natural Language Processing is possible
(cognitive and socio-cultural)
 Learning analytics tools that combine the two
perspectives and the Polyphonic Model may be
developed
64
Stefan Trausan-Matu
12/14/2013

65. Thank you!
stefan.trausan@cs.pub.ro
http://www.racai.ro/~trausan
Questions?
65
Stefan Trausan-Matu
12/14/2013