Machine Learning and Argumentation: A Tutorial

ARGUMENTATION AND MACHINE LEARNING: WHEN THE WHOLE
IS GREATER THAN THE SUM OF ITS PARTS
A Tutorial @ IJCAI 2019
Federico Cerutti <federico.cerutti@acm.org>

P. Baroni T. Bench-Capon K. Budzynska P. Dunne
M. Giacomin A. Hunter T. Norman S. Parsons
C. Reed A. Toniolo M. Vallati S. Woltran
3

• Introduction to Formal Argumentation Theory
• Why it is important?
• Supporting scientiﬁc enquiry
• Structured argumentation
• Abstract Argumentation
• Algorithms and Implementations
1030H, in ROOM: 2405
Third International Competition on Computational Models of Argumentation
(ICCMA’19) Award Ceremony
• Machine learning for argumentation
• Argumentation mining
• Machine learning for evaluating argumentation framework
• Argumentation for machine learning
• Are we using quality data?
• Arguing about the model: explanations and tellability
• Arguing about the algorithmic presence
4

6 https://archive.org/stream/in.ernet.dli.2015.228218/2015.228218.English-Dictionary#page/n269/mode/2up/search/epistemology

Cave
Anamnesys
7 Image: Wikipedia

Empiricism
All hypotheses and theories must be tested against observations of the natural
world, rather than resting solely on a priori reasoning, intuition, or revelation .
8

Paradigm Shift
12 Image: Wikipedia

15 https://www.jpl.nasa.gov/spaceimages/details.php?id=PIA02210

The path of the planet Uranus did not
conform to the path predicted by Newton’s
law of gravitation in presence of the known
planets.
Explanations:
• Human/instrument measure error
• Newton’s laws are mistaken
• An invisible magic teapot caused the
perturbation in order to show the
hubris of modern science
• …
• Newton’s laws—conﬁrmed by a
signiﬁcant amount of evidence—are
correct and the perturbation is caused
by another, unknown, planet
17 Image: Wikipedia

Scientific theories are capable of being
refuted: they are falsifiable
Verification and falsification are
different processes:
• No accumulation of confirming
instances is sufficient
• Only one contradicting instance
suffices to refute a theory
Scientific theories are tentative
18 Image: Wikipedia

computational models of argumentation

Supporting Reasoning with Different Types of Evidence in
Intelligence Analysis
Alice Toniolo_ Anthony Etuk Robin Wentao Ouyang
Tlmothy J-
N0Fman Federico Cerutti Mani Srivastava
DBPL 0f_C0ml3U“”Q SCIENCE Dept. of Computing Science University of California
University of Aberdeen, UK University of Aberdeen, UK Los Angeles, CA, USA
Nir Oren Timothy Dropps Paul Sullivan
Dept. of Computing Science John A_ Allen INTELPOINT Incorporated
University of Aberdeen, UK Honeywell, USA Pennsylvania, USA
Appears in: Proceedings of the 14th International
Conference on Autonomous Agents and ll/Iultiayent
Systems (AAJWAS 2015), Bordim, Elkind, Was.-3, Yolum
(ed5.), Mlay 4 8, 2015, Istcmbttl, Turkey.
[Ton+15]
20

Does MMR vaccination cause autism?
22

Douglas Walton
Chris Reed
Fabrizio Macagno
ARGUMENTATION
SCHEMES
[WRM08]
23

Argumentation scheme for argument from correlation to cause
Correlation Premise: There is a positive correlation between A and B.
Conclusion: A causes B.
Critical questions are:
CQ1: Is there really a correlation between A and B?
CQ2: is there any reason to think that the correlation is any more than a
coincidence?
CQ3: Could there be some third factor, C, that is causing both A and B?
24

The Knowledge Engineering Review, Vol. 26:4, 487—51 1. © Cambridge University Press, 2011
doi:10.1017/S0269888911000191
Representing and classifying arguments on the
Semantic Web
IYAD RAHWAN1‘2, B_ITA BANIHASHEMI3, CHRIS REED4,
DOUGLAS WALTON” and SHERIEF ABDALLAH”
[Rah+11]
25

Node Graph
(argument
network)
has-a
Information
Node
(I-Node)
is-a
Scheme Node
S-Node
has-a
Edge
is-a
Rule of inference
application node
(RA-Node)
Conflict application
node (CA-Node)
Preference
application node
(PA-Node)
Derived concept
application node (e.g.
defeat)
is-a
...
ContextScheme
Conflict
scheme
contained-in
Rule of inference
scheme
Logical inference
scheme
Presumptive
inference scheme
...
is-a
Logical conflict
scheme
is-a
...
Preference
scheme
Logical preference
scheme
is-a
...
Presumptive
preference scheme
is-a
uses uses uses
26 Image from [Rah+11]

MMR vaccination
causes authism
C-2-C
It is possible that
MMR vaccination
is associated to
autism
27

EARLY REPORT
Early report
lleal-lymphoid-nodular hyperplasia, non-specific colitis, and
pervasive developmental disorder in children
A J Wake eld, S H Murch, A Anthony, J Linnell, D M Casson, M Malik, M Berelowitz, A P Dhillon, M A Thomson,
P Harvey, A Valentine, 5 E Davies, J A Walker-Smith
5|-|mma|'Y Introduction
1177
" °9W several children Who, after a nP"" '
"‘ investigated a conser""' _m;mAn1".,,,
28

Support
What else should
be true if the
causal link is true?
29
From Ileal-lymphoid-nodular hyperplasia, non-speciﬁc colitis, and pervasive developmental disorder in children by Wakeﬁeld et al, The Lancet, 1998

MMR vaccination
causes authism
C-2-C
It is possible that
MMR vaccination
is associated to
autism
Behavioural symptoms
were associated by
parents of 12 children
Witn
30

The New England
Iournal of Medicine
Copyright © 2002 by the Massachusetts Medical Society
VOLUME 347 N()VEMBER 7, 2002 NUMBER 19
A POPULATION-BASED STUDY OF MEASLES, MUMPS, AND RUBELLA
VACCINATION AND AUTISM
KREESTEN MELDGAARD MADSEN, M.D., ANDERS HVIID, M.Sc., MOGENS VESTERGAARD, M.D., DIANA SCHENDEL, PH.D.,
JAN WOHLFAHRT, M.Sc., POUL THORSEN, M.D., J(ZiRN OLSEN, M.D., AND MADS MELBYE, M.D.
ABS""‘
I 7 "Tested that the measle
' +hat vaccina— ”“CCi11C C3“’
-nn- ’
31

Support
32
From A Population-based Study of Measles, Mumps, and Rubella Vaccination and Autism by Madsen et al, The New England Journal of Medicine, 2002

Support
What else should
be true if the
causal link is true?
Support
Support
33

MMR vaccination
causes authism
C-2-C
It is possible that
MMR vaccination
is associated to
autism
were associated by
Witn
CQ1: There is no
correlation between
MMR vaccination
and autism
CON
E-2-H
No statistical
correlation over
440,655 children
34

ASPIC+
[Pra10] [MP13] [MP14]
35 From [MP13]

ASPIC+
An argumentation system is as tuple AS = ⟨L, R, ν, ⟩ where:
• : L → 2L
: a contrariness function s.t. if φ ∈ ψ and:
• ψ /∈ ϕ, then ϕ is a contrary of ψ;
• ψ ∈ ϕ, then ϕ is a contradictory of ψ (ϕ = –ψ);
• R = Rd ∪ Rs: strict (Rs) and defeasible (Rd) inference rules s.t. Rd ∩ Rs = ∅;
• ν : Rd → L, is a partial function.*
P ⊆ L is consistent iff ∄φ, ψ ∈ P s.t. φ ̸∈ ψ, otherwise is inconsistent.
A knowledge base in an AS is Kn ∪ Kp = K ⊆ L; {Kn, Kp} is a partition of K; Kn contains
axioms that cannot be attacked; Kp contains ordinary premises that can be attacked.
An argumentation theory is a pair AT = ⟨AS, K⟩.
*Informally, ν(r) is a wff in L which says that the defeasible rule r is applicable.
36 From [MP13]

ASPIC+
An argument a on the basis of a AT = ⟨AS, K⟩, AS = ⟨L, R, ν, ⟩ is:
1. φ if φ ∈ K with: Prem(a) = {φ}; Conc(a) = φ; Sub(a) = {φ};
Rules(a) = DefRules(a) = ∅; TopRule(a) = undefined.
2. a1, . . . , an −→ / =⇒ ψ if a1, . . . , an, with n ≥ 0, are arguments such that there exists a
strict/defeasible rule r = Conc(a1), . . . , Conc(an) −→ / =⇒ ψ ∈ Rs/Rd.
Prem(a) =
∪n
i=1 Prem(ai); Conc(a) = ψ;
Sub(a) =
∪n
i=1 Sub(ai) ∪ {a};
Rules(a) =
∪n
i=1 Rules(ai) ∪ {r};
DefRules(a) = {d | d ∈ Rules(a) ∩ Rd};
TopRule(a) = r
a is strict if DefRules(a) = ∅, otherwise defeasible; ﬁrm if Prem(a) ⊆ Kn, otherwise
plausible.
37 From [MP13]

MMR vaccination
causes authism
C-2-C
It is possible that
MMR vaccination
is associated to
autism
were associated by
Witn
CQ1: There is no
correlation between
MMR vaccination
and autism
CON
E-2-H
No statistical
correlation over
440,655 children
α
β
γ
δ
ε
38

MMR vaccination
causes authism
C-2-C
It is possible that
MMR vaccination
is associated to
autism
were associated by
Witn
CQ1: There is no
correlation between
MMR vaccination
and autism
CON
E-2-H
No statistical
correlation over
440,655 children
α
β
γ
δ
ε
β =⇒ α
γ =⇒ β
ϵ =⇒ δ
δ ∈ β
39

ASPIC+
Given a and b arguments, a defeats b iff a undercuts, successfully rebuts or successfully
undermines b, where:
• a undercuts b (on b′
) iff Conc(a) /∈ ν(r) for some b′
∈ Sub(b) s.t.
r = TopRule(b′
) ∈ Rd;
• a successfully rebuts b (on b′
) iff Conc(a) /∈ φ for some b′
∈ Sub(b) of the form
b′′
1 , . . . , b′′
n =⇒ –φ, and a ⊀ b′
;
• a successfully undermines b (on φ) iff Conc(a) /∈ φ, and φ ∈ Prem(b) ∩ Kp, and
a ⊀ φ.
AF is the abstract argumentation framework deﬁned by AT = ⟨AS, K⟩ if A is the smallest
set of all ﬁnite arguments constructed from K; and → is the defeat relation on A.
40 From [MP13]

γε
ε, ε ⇒ δ γ, γ ⇒ β
γ, γ ⇒ β, β ⇒ α
41

Artificial
Intelligence
Arti cialIntelligence 77 (1995) 321v357
On the acceptability of arguments and its fundamental
role in nonmonotonic reasoning, logic programming and
n-person games*
Phan Minh Dung*
[Dun95]
42

Deﬁnition
A Dung argumentation framework AF is a pair
⟨A, → ⟩
where A is a set of arguments, and → is a binary relation on A i.e. →⊆ A × A.
43 Deﬁnitions from [Dun95]

A semantics is a way to identify sets of arguments (i.e. extensions)
“surviving the conﬂict together”
44 Deﬁnitions from [Dun95]

(Some) Semantics Properties
wailah-la unlina at 1-Iwmnscianca-dira+:t.corn
':.i; Science-.Direct Ani gal
Intelligence:1
E.LSI:'."v'lI:'.R. .eu:i:'.u'.-in Jnl::||igI:n»;::: m izrocm n75—:':m
www.r:I:i::1.r'icr.r:nn1.-'|m::3n:.':3r1iI11
On principle-based evaluation of extension-based
argumentation semantics ii’
Pietra Bamni, Massimiliano Giacomin *
[BG07]
The Kn0w[ed'ge Engineering Review, Vol. 26:4, 365-410. © Cambridge University Press, 2011
doi:10.1017J/S0269888911000166
An introduction to argumentation semantics
PIETRO BARONI‘, MARTIN CAMINADA2 and
MASSIMILIANO GlACOMIN'
[BCG11]
45 Deﬁnitions from [BG07]

• Conﬂict-freeness
an attacking and an attacked argument can not stay together (∅ is c.f. by def.)
• Admissibility
• Reinstatement
• I-Maximality

• Admissibility
the extension should be able to defend itself, „fight fire with fire” (∅ is adm. by def.)
• Reinstatement
• I-Maximality

• Admissibility
• Reinstatement
if you defend some argument you should take it on board (∅ satisﬁes the principle
only if there are no unattacked arguments)
• I-Maximality

• Admissibility
• Reinstatement
• I-Maximality
no extension is a proper subset of another one

Complete Extension
Admissibility and reinstatement
Set of conﬂict-free arguments s.t. each defended argument is included
b a
c
d
f e
gh



{a, c, d, e, g},
{a, b, c, e, g},
{a, c, e, g}



50 Deﬁnitions from [Dun95,BG07]

Preferred Extension
Admissibility and maximality
Maximum complete extensions
b a
c
d
f e
gh



{a, c, d, e, g},
{a, b, c, e, g}



51 Deﬁnitions from [Dun95,BG07]

Stable Extension
Complete extensions attacking all the arguments outside
When exist, stable extensions are also preferred extensions
b a
c
d
f e
gh



{a, c, d, e, g},
{a, b, c, e, g}



52

Complete Labellings
An argument is IN if all its attackers are OUT
An argument is OUT if at least one of its attackers is IN
Otherwise is UNDEC
53 Deﬁnitions from [BCG11]

Complete Labellings
Max. IN ≡ Preferred
b a
c
d
f e
gh



{a, c, d, e, g}



54 Deﬁnitions from [Dun95,BCG11]

Complete Labellings
Max. IN ≡ Preferred
b a
c
d
f e
gh



{a, b, c, e, g}




Complete Labellings
No UNDEC ≡ Stable
b a
c
d
f e
gh



{a, c, d, e, g}




Complete Labellings
No UNDEC ≡ Stable
b a
c
d
f e
gh



{a, b, c, e, g}




Chapter 5
Complexity of Abstract Argumentation
Paul E. Dunne and Michael Wooldridge
I. Rahwan, G. R. Simari (cds.), Argunzerztarion in Ar‘!1j‘icial Intelligence,
DO] 10.1007/978—0—387—98197'-0-5. © Springer SCience+Business Media. LLC 2009
[DW09]
58 From [DW09]

σ = CO σ = GR σ = PR σ = ST
existsσ trivial trivial trivial np-c
caσ np-c polynomial np-c np-c
saσ polynomial polynomial Πp
2 -c conp-c
verσ polynomial polynomial conp-c polynomial
neσ np-c polynomial np-c np-c
59 From [DW09]

algorithms and implementations

(Handbook of Formal Argumentation, Pietro Baroni, Dov
Gabbay, Massimiliano Giacomin and Leendert van der
Torre, eds, 978-1-84890-275-6)
61

Non-Reduction Based Procedures
62

CSP-based approach
Given an AF:
1. create a variable for each argument whose domain is
always {0, 1} — ∀ai ∈ A, ∃xi ∈ X such that Di = {0, 1};
2. describe constraints associated to different
deﬁnitions of Dung’s argumentation framework: e.g.
{a, b} ⊆ A is D-conﬂict-free iff ¬(x1 = 1 ∧ x2 = 1);
3. solve the CSP problem.
63

ASP-based approach
πST = { in(X) ← not out(X), arg(X);
out(X) ← not in(X), arg(X);
← in(X), in(Y), defeat(X, Y);
defeated(X) ← in(Y), defeat(Y, X);
← out(X), not defeated(X)}.
Tests for subset-maximality exploit the metasp
optimisation frontend for the ASP-package
gringo/claspD.
64

Second Order Logic
TCF = {∄N, M | r(N, M) ∧ s(N) ∧ s(M).}
TAD =
{
∀N | att(N) ⇐⇒ ( a(N) ∧ ∃M | r(M, N) ∧ s(M) ).
∀N | def(N) ⇐⇒ ( a(N) ∧ ∀M | r(M, N) =⇒ att(M) ).
}
TST = {TAD. ∀N | a(N) =⇒ ( s(N) ⇐⇒ ¬att(N) ).}
65

SAT-based approaches
∧
a→b
(¬xa ∨ ¬xb)∧
∧
b→c

¬xc ∨
∨
a→b
xa


66

SAT-based approaches
[CGV19]
67 https://www.sciencedirect.com/science/article/pii/S0004370218302650

Complete labelling: C→
in ∧ C←
in ∧ C→
out ∧ C←
out ∧ C→
undec ∧ C←
undec, where
• C→
in ≡ (Lab(a) = in ⇒ ∀b ∈ a−
Lab(b) = out);
• C←
in ≡ (Lab(a) = in ⇐ ∀b ∈ a−
Lab(b) = out);
• C→
out ≡ (Lab(a) = out ⇒ ∃b ∈ a−
: Lab(b) = in);
• C←
out ≡ (Lab(a) = out ⇐ ∃b ∈ a−
: Lab(b) = in);
• C→
undec ≡ (Lab(a) = undec ⇒ ∀b ∈ a−
Lab(b) ̸= in ∧ ∃c ∈ a−
: Lab(c) = undec);
• C←
undec ≡ (Lab(a) = undec ⇐ ∀b ∈ a−
Lab(b) ̸= in ∧ ∃c ∈ a−
: Lab(c) = undec).
Let us also deﬁne C↔
in ≡ C→
in ∧ C←
in, C↔
out ≡ C→
out ∧ C←
out, C↔
undec ≡ C→
undec ∧ C←
undec.
Ci
o
u
, with i, o, u ∈ {→, ←, ↔, ∼}, i.e. Ci
o
u
≡ Ci
in ∧ Co
out ∧ Cu
undec, where
C∼
in = C∼
out = C∼
undec = ⊤ (in other words, the symbol ∼ denotes that the corresponding
term is not present in the encoding). For instance, C→↔∼ ≡ C→
in ∧ C↔
out.
69 From [CGV19]

0
1
2
3
4
5
6
C→→→ C←←←
C↔→← C↔←→ C→↔← C←↔→ C→←↔ C←→↔C∼↔↔ C↔∼↔ C↔↔∼
C→→↔ C→↔→ C↔→→ C←←↔C←↔←C↔←←
C→↔↔ C↔→↔ C↔↔→ C↔↔←C←↔↔
C↔↔↔
70 From [CGV19]

C↔↔↔
C↔→↔C←↔↔C→↔↔ C↔←↔ C↔↔→ C↔↔←
C↔∼↔C↔→→ C↔←←C←←↔C∼↔↔C→→↔ C→↔→ C↔↔∼ C←↔←
C←←← C→→→
71 From [CGV19]

C←
in ≡
∧
{a∈A}

Ia ∨


∨
{b | b→a}
(¬Ob)




C→
in ≡
∧
{a∈A}


∧
{b | b→a}
¬Ia ∨ Ob


…
72 From [CGV19]

SAT-EL-ST
Input: Γ “ xA, Ry, C˚ P C, xπS
All, πS
All;O, πS
All;I, πS
O, πS
I y P tJ, Ku5
Output: xLs Ď LSTpΓq, blockingy
1: if πS
All then
2: return ALLSatS
˜
cnfpC˚, Γq ^
ľ
aPA
Ua, πS
All;I, πS
All;O
¸
3: end if
4: Ls :“ H, blocking :“ J
5: do
6: stb :“ SatS
˜
cnfpC˚, Γq ^
ľ
aPA
Ua ^ blocking
¸
7: if stb ‰ H then
8: if πS
O then
9: blocking :“ blocking ^
ł
aPI-ARGSpstbq
Oa
10: end if
11: if πS
I then
ł
aPO-ARGSpstbq
Ia
13: end if
14: Ls :“ Ls Y tstbu
15: end if
16: while stb ‰ H
17: return xLs , blockingy
SAT-EL-PR
Input: Γ “ xA, Ry, C˚ P C, xπP
S , πS
All, πS
All;O, πS
All;I, πS
O, πS
I , πP
iO, πP
iI , πP
eO, πP
eIy P
tJ, Ku10
Output: Ls Ď LSTpΓq
1: Lp :“ H, blocking :“ J
2: if πP
S then
3: xLp, blockingy :“ SAT-EL-ST
`
Γ, C˚, xπS
All, πS
All;O, πS
All;I, πS
O, πS
I y
˘
4: end if
5: do
6: iblock :“ J, prf :“ H
7: do
8: cmp :“ SatS pcnfpC˚, Γq ^ blocking ^ iblockq
9: if cmp ! “ H then
10: prf :“ cmp
11: iblock :“
ľ
Ia
aPI-ARGSpcmpq
^
ľ
Oa
aPO-ARGSpcmpq
12: if πP
iO then
13: iblock :“ iblock ^
ł
Oa
aPU-ARGSpcmpq
14: end if
15: if πP
iI then
16: iblock :“ iblock ^
ł
Ia
aPU-ARGSpcmpq
17: end if
18: end if
19: while cmp ‰ H ^ U-ARGSpcmpq ‰ H
20: if prf ‰ H then
21: Lp :“ Lp Y tprf u
22: if πP
eO then
ł
Oa
aPI-ARGSpprf q
24: end if
25: if πP
eI then
ł
Ia
aPO-ARGSpprf q
27: end if
28: end if
29: while prf ‰ H
30: if Lp “ H then
31: Lp “ tLUu
32: end if
33: return Lp
73 From [CGV19]

IPC Score
IPC(s, P) =



0 if P is unsolved
1
1 + log10
(
TP(s)
T∗
P
) otherwise
tP(s) denotes the time needed by solver s to solve P
T∗
P is the minimum amount of time required by any considered solver to solve P
74

C→→→ ≡ C→
in ∧ C→
out ∧ C→
undec is the most efﬁcient encoding of complete labellings
C→→→ C←←← C→→↔ C→↔→ C←←↔ C←↔← C↔→→ C↔←← C↔↔∼ C↔∼↔ C∼↔↔ C→↔↔ C←↔↔ C↔→↔ C↔←↔ C↔↔→ C↔↔← C↔↔↔
155.00
160.95
166.89
172.84
178.79
184.73
190.68
Minisat
EE-ST, ICCMA2015
(highest best)
240.00
250.20
260.41
270.61
280.82
291.02
301.23
Minisat
EE-ST, ICCMA2017
(highest best)
75 From [CGV19]

C→→→ ≡ C→
in ∧ C→
out ∧ C→
150.00
156.78
163.56
170.34
177.12
183.91
190.69
Minisat
EE-PR, ICCMA2015
(highest best)
160.00
169.13
178.26
187.38
196.51
205.64
214.77
Minisat
EE-PR, ICCMA2017
(highest best)
76 From [CGV19]

C→→→ ≡ C→
in ∧ C→
out ∧ C→
¬C←
undec ≜ {C→→→, C→↔→, C↔→→, C↔↔∼ , C↔↔→}
C←
undec ≡
∧
{a∈A}
(∧
{b | b→a}
(
Ua ∨ ¬Ub ∨
(∨
{c | c→a} Ic
)))
Only two non-redundant encodings in ¬C←
undec: C→→→ and C↔↔∼ , but C↔↔∼ de facto becomes
redundant for stable semantics (difference in the case of preferred semantics less
noticeable)
77 From [CGV19]

More in the paper (spoiler alert):
• Using an ALLSAT solver leads to a significant improvement for enumerating stable
labellings
• Enumerating stable labellings first can lead to a significant improvement for
enumerating preferred labellings
• The improved algorithm for skeptical acceptance w.r.t. preferred semantics leads to
significant improvements (and was decisive for ICCMA 2017)
78 From [CGV19]

MMR vaccination
causes authism
C-2-C
It is possible that
MMR vaccination
is associated to
autism
were associated by
Witn
CQ1: There is no
correlation between
MMR vaccination
and autism
CON
E-2-H
No statistical
correlation over
440,655 children
α
β
γ
δ
ε
β =⇒ α
γ =⇒ β
ϵ =⇒ δ
δ ∈ β
79

γε
ε, ε ⇒ δ γ, γ ⇒ β
γ, γ ⇒ β, β ⇒ α
80

MMR vaccination
causes authism
C-2-C
It is possible that
MMR vaccination
is associated to
autism
were associated by
Witn
CQ1: There is no
correlation between
MMR vaccination
and autism
CON
E-2-H
No statistical
correlation over
440,655 children
α
β
γ
δ
ε
81

This research was sponsored by the U.S. Army Research Laboratory and the U.K. Ministry
of Defence and was accomplished under Agreement Number W911NF-06-3-0001. The
views and conclusions contained in this document are those of the author(s) and should
not be interpreted as representing the ofﬁcial policies, either expressed or implied, of
the U.S. Army Research Laboratory, the U.S. Government, the U.K. Ministry of Defence or
the U.K. Government. The U.S. and U.K. Governments are authorized to reproduce and
distribute reprints for Government purposes notwithstanding any copyright notation
hereon.
The current development of CISpaces2.0 is funded by the DSTL Defence Accellerator
programme under agreement number ACC102157. This project is led by Timothy Norman,
and is a collaborative effort with Alice Toniolo, Federico Cerutti and Stuart Middleton.
82

CISpaces has been developed and tested with the help of professional intelligence
analysts
Analysts have highlighted that CISpaces is useful for training and provides an
effective means to record an audit trail that includes important elements of the
reasoning processes involved in the analysis of competing hypotheses
CISpaces has been deployed on a dedicated machine at the ARL Adelphi Laboratory
Center
CISpaces.org has been deployed on the UK Joint Forces Intelligence Group
experimental servers
CISpaces.org has been used in an in-depth analysis of the case of Prosecutor v.
Karadžić, MICT-13-55-A, that lead to the submission of an Amicus Curiae* to the UN
International Criminal Tribunal
*http://jrad.unmict.org/webdrawer/webdrawer.dll/webdrawer/rec/240941/view/
83

http://52.56.97.213:8080/
Username: demo
Password: demo
84

Karadžić was knowledgeable
of the intent to kill Bosnian
Muslims
Generally, if the Accused knew
that Bosnian Muslims had been
recently killed by Bosnian Serb
forces [in Kravica Warehouse]
then he might have known that
it may occur that Bosnian Serb
forces would kill other Bosnian
Muslim in the future
Cause
to Effect
In this case Karadžić knew
that Bosnian Muslim
have been killed by Bosnian
Serb Forces
The Chamber finds it
inconceivable that Kovač
did not discuss the
developments on the
ground in Srebrenica on
13 July (Para 5767)
Opinion
At 2010h on 13 July 1995
Karadžić talked on the phone
with Deronjić [about moving
prisoners to Zvornik, ed.]
(Para 5772)
Evidence
to hypho-
thesis
At 2010h on 13 July 1995
Karadžić talked on the phone
with Deronjić [moving
prisoners to a place different
from Zvornik ed.] (Para 5772)
CON CON
The Chamber therefore finds
that [...] the Accused conveyed
to Deronjić the direction that
the detainees should be
transferred to Zvornik
(Para 5773)
CON
Evidence
to hypho-
thesis
Davidović had urged Deronjić
to “use [his] connections”
with the Accused in order
to have the buses moved
(Para 5773)
Before speaking to the
Accused Deronjić had
previously complained to
Beara about the detainees’
presence in Bratunac
(Para 5773)
Beara and Deronjić later
argued about whether the
detainees would be killed in
Bratunac or would be
transferred to Zvornik
for that purpose (Para 5773)
Deronjić [...] [said] that the
Accused had instructed him
that all detainees should
be transferred to Zvornik
(Para 5773)
Evidence
to hypho-
thesis
Evidence
to hypho-
thesis
The Chamber has no doubt
that [...] [on 14 July]
Deronjić and the Accused ,
they both discussed the
killings [...], and the
implementation of the
Accused’s order to transport
the detainees [...] to Zvornik
(Para 5808)
Opinion
Evidence
to hypho-
thesis
Evidence
to hypho-
thesis
During the second meeting
[with the Srebrenica
representatives, ed.],
Deronjić reported on the
situation in Srebrenica
(Para 5808)
The Chamber received
evidence that there was no
mention or discussion about
the executions of detainees
in Srebrenica during the
meeting with the Srebrenica
representatives (Para 5808)
CON
Evidence
to hypho-
thesis
Deronjić’s participation in
the efforts to bury the bodies
of those killed at the Kravica
Warehouse, starting in the
early hours of 14 July
(Para 5808)
Witness
Testimony
Simić testified that Deronjić
told him that he had
informed the Accused about
the events at the Kravica
Warehouse the day after the
incident (Para 5808)
The only reasonable inference
is that Bajagić reported the
events in Srebrenica he had
witnessed [...] to the Accused
during their meeting on 15
July. (Para 5783)
Opinion
The extremely late hour
of their meeting (Para 5783)
The Accused had invited
Bajagić to Pale (Para 5783)
Bajagić had substantive
knowledge of the events in
Srebrenica (Para 5783)
Meeting Bajagić in Pale at an
extremely late hour, given that
Bajagić had substantive
knowledge of the events in
Srebrenica implies that
Bajagić reported the events he
had witnessed
Unstated
Unstated
Abductive
Inference
Opinion
The Chamber finds it
incredible that Kovač would
not have discussed these
matters with the Accused
(Para 5782)
Kovač gathered additional
important information that
he ultimately relayed back to
the Accused when he returned
to Pale on 14 July. (Para 5806)
Opinion
On 13 July Mladić informed
Karadžić that Srebrenica “[wa]s
done” (Para 5768).
Srebrenica had fallen on 11
July, hence Karadžić should
have known by 13 July
(Para 5770, fn 19596)
Abductive
Inference
Inference 4.a
Inference 4.b
Inference 4.cX
Inference 4.d
Inference 4.e
Unstated
Mladić informing Karadžić, on 13 July,
that Srebrenica “[wa]s done” when
Srebrenica had fallen on 11 July, hence
Karadžić should have known by 13 July
implies that Karadžić was knowledgeable
of the intent to kill Bosnian Muslims
Nikolić testimony
(Para 5312, fn 18025)
Witness
Testimony
Witness
Testimony
McDermott Rees and Cerutti 2018
http://jrad.unmict.org/webdrawer/webdrawer.dll/webdrawer/rec/240941/view/
85 http://jrad.unmict.org/webdrawer/webdrawer.dll/webdrawer/rec/240941/view/

Gatete was responsible for the
killings of Tutsis in Rwankuba
sector, Murambi commune, on 7
April 1994 (1).
Gatete participated in a joint
criminal enterprise to conduct
killings on 7 April1994 (2).
A meeting took place on 7 April
1994 in the Rwankuba sector
office courtyard (4).
Gatete made a significant
contribution to the achievement
of the common plan (32).
Gatete ordered the Interhamwe
at the meeting on 7 April 1994 to
‘work relentlessly’ (33).
Gatete told those present to
‘sensitise’ others to the killings
(35).
No meeting took place in the
sector office on 7 April 1994 (15).
BBR testimony (5).
BBR testimony (34).
AIZ testimony (36).
LA40 testimony (18)
LA40 testimony (43).
LA41 testimony (39). LA43 testimony (41)
LA40 could not see
the sector office at
all times on 7 April
1994 (42).
LA41 could not see
1994 (38).
LA43 could not see
1994 (40).
LA41 testimony (16) LA43 testimony (17)
AIZ testimony (6).
PRO
PRO
PRO
PRO
CON
CON
CON CON
CON
CON
CON
LPK
LPK
LPK
LPK LPK
LPK LPK
LPK
LPK
LPK
89 From [CNT18]

• We can help humans in structuring their reasoning
• We can reduce the effect some biases, in particular conﬁrmation bias, via
argumentation schemes and critical questions
• We have machineries to derive arguments and attacks
• We have machineries to derive acceptable arguments on the basis of different
criteria
• The performance of such machineries vary dramatically on the basis of a variety of
parameters
92

1030H
ROOM: 2405
Third International Competition on Computational Models of
Argumentation (ICCMA’19) Award Ceremony
93

machine learning for argumentation

Magic Box 1
Creation of Arguments
Magic Box 2
Evaluation of Arguments
95

[BV18]
96 Handbook of Formal Argumentation, Pietro Baroni, Dov Gabbay, Massimiliano Giacomin and Leendert van der Torre, eds, 978-1-84890-275-6

large resources
of NL texts
97 Image from [BV18]

large resources
of NL texts
annotation
scheme (theory)

Node Graph
(argument
network)
has-a
Information
Node
(I-Node)
is-a
Scheme Node
S-Node
has-a
Edge
is-a
Rule of inference
application node
(RA-Node)
Conflict application
node (CA-Node)
Preference
application node
(PA-Node)
Derived concept
application node (e.g.
defeat)
is-a
...
ContextScheme
Conflict
scheme
contained-in
Rule of inference
scheme
Logical inference
scheme
Presumptive
inference scheme
...
is-a
Logical conflict
scheme
is-a
...
Preference
scheme
Logical preference
scheme
is-a
...
Presumptive
preference scheme
is-a
uses uses uses
99 Image from [Rah+11]

Bob says: Lower taxes
stimulate the economy
Bob says: The government
will inevitably lower the tax
rate.
Wilma says: Why?
Challenging
Substantiating
Asserting
Asserting
Challenging
Lower taxes stimulate
the economy
An application of the
argument scheme for
Argument from Positive
Consequences
The government will
inevitably lower the tax
rate.
Arguing
100 Inference Anchoring Theory, IAT, Katarzyna Budzynska and Chris Reed. Whence inference. Technical report, University of Dundee, 2011.

Various argument structures
101

[Cer+16]
103 Proceedings of SAFA 2016, http://ceur-ws.org/Vol-1672/paper_6.pdf

Should contraception be covered by health insurance?
105

Y N
#1
+
#2
-
#3
#4
-
+
#5
2
+
+
-
-
1. Noes because that’s not something you need.
2. You probably shouldn’t make that blanket statement, without
any qualiﬁers or exceptions. For many women, birth control
pills are very important and are necessary to daily life
3. What about Viagra, should that be covered by health
insurance?
4. Women have the right to choose what to do with their bodies.
5. It is true that women have the right to choose what they wish
to do with their bodies, but they have absolutely no power to
force insurance companies to pay for them. That should be
left up to the insurance company, and not the woman.
106 From [Cer+16]

large resources
of NL texts
annotation
scheme (theory)

large resources
of NL texts
annotation
scheme (theory)
annotated
corpus
training
test

large resources
of NL texts
annotation
scheme (theory)
annotated
corpus
training
test
kappa

Deﬁnition (Cohen’s kappa)
The agreement between two annotators who each classify N items into C mutually
exclusive categories†
κ =
po − pe
1 − pe
= 1 −
1 − po
1 − pe
where:
• po is the relative observed agreement among annotators
• pe is the probability of chance agreement (the probabilities of each annotator
randomly saying each category)
• 0.41–0.60 moderate, 0.61–0.80 substantial, 0.81–1 almost perfect agreement‡
†Jacob Cohen. A coefﬁcient of agreement for nominal scales. Educational and Psychological Measurement,
20:3746, 1960.
‡J.R. Landis and G.G. Koch. The measurement of observer agreement for categorical data. Biometrics,
33:159174, 1977.
111

large resources
of NL texts
annotation
scheme (theory)
annotated
corpus
training
test
kappa
grammars +
classifiers

http://arg.tech/~chris/acl2019tut/index.html
http://acl2016tutorial.arg.tech/index.php/tutorial-materials/
http://www.i3s.unice.fr/~villata/tutorialIJCAI2016.html
114

• Argumentative discourse unit segmentation
Unit size ranges from single-word to paragraph
• Typed segmentation
Facts? Opinions?
• Relations, directions, type
115

‡Iyad Rahwan, Fouad Zablith, and Chris Reed. Laying the foundations for a World Wide Argument Web.
Artiﬁcial Intelligence, 171(10), pages 897–921, 2007.116

large resources
of NL texts
annotation
scheme (theory)
annotated
corpus
training
test
kappa
grammars +
classifiers
automatically
annotated
arguments

large resources
of NL texts
annotation
scheme (theory)
annotated
corpus
training
test
kappa
grammars +
classifiers
automatically
annotated
arguments
cf.
precision
& recall

• Recall
tp
tp + fn
How often arguments have been missed out?
• Precision
tp
tp + fp
How often magical arguments appeared
• F1 score 2
precision · recall
precision + recall
119

large resources
of NL texts
annotation
scheme (theory)
annotated
corpus
training
test
kappa
grammars +
classifiers
automatically
annotated
arguments
cf.
precision
& recall
.
.
.
real
arguments
.
.
.
automatically
extracted
arguments

• Argumentation mining is hard
• It relies on a substantial number of assumptions, including which theory you want to
use
• It is a fast dynamic world across formal argumentation and linguistics
121

Magic Box 1
Creation of Arguments
Magic Box 2
Evaluation of Arguments
122

I have a set of AFs that want to analyse, I know the problem I am working on, I picked up
a solver that works decently.
...but, in order to deploy the system, I need it to be faster.
Let’s learn something then.
124 Slide courtesy of Mauro Vallati

Learning: idea
Generic solver
Knowledge
(about the
problem, solver,
...)
Knowledge-boosted approach

However...
Extracting additional knowledge could, in principle, be easy. But...

Which Kind of Knowledge?
• Combination and Selection of solvers
• Conﬁguration of solvers
• Conﬁguration (Reformulation) of AFs
Here we focus on knowledge that can be automatically extracted.

Combining and Selecting Solvers
(Solver selection can be seen as a particular case of portfolio conﬁguration)
• Static: the same portfolio is used for analysing any AF
• Dynamic: portfolio is conﬁgured according to some characteristics of the AF

Static Portfolio: Process

Static Portfolio
Deﬁned by:
1. the selected solvers;
2. the order in which solvers will be run; and
3. the runtime allocated to each solver.

Static Portfolio: Approaches
Shared-k
Each component solver has been allocated maxRuntime
k seconds. Solvers selected/ordered
according to overall PAR10
FDSS
From an empty portfolio, we iteratively add either a new solver component, or extend the
allocated CPU-time of a solver already added to the portfolio, depending on what
maximises the increment of the PAR10 score of the portfolio
Penalised Average Runtime 10.
PAR10(s, P) =
{
10 ∗ T if P is unsolved
tP(s) otherwise
T indicates the considered timeout; tP(s) denotes the time needed by solver s to solve P
131 From [CVG17], Slide courtesy of Mauro Vallati

Dynamic Portfolio: Process

Dynamic Portfolio
For each AF, a vector of features is computed.
Similar instances should have similar feature vectors.
Portfolios are conﬁgured using empirical performance models

Dynamic Portfolio: Features
Features can be extracted from different representations of an AF§
E.g., Directed graph representation.
• Graph size features: number of vertices, number of edges, ratios vertices–edges and
inverse, and graph density
• Degree features: average, standard deviation, maximum, minimum degree values
across the nodes in the graph.
• SCC features: number of SCCs, average, standard deviation, maximum and
minimum size.
• Graph structure: presence of auto-loops, number of isolated vertices, etc
Similarly, features can be extracted by considering undirected graph, or matrix
representation.
§F. Cerutti, M. Giacomin, and M. Vallati. Algorithm selection for preferred extensions enumeration. In
Computational Models of Argument - Proceedings of COMMA, pages 221–232, 2014

Dynamic Portfolio: Approaches
Classiﬁcation-based
Classify
It classiﬁes a given AF into a single category which corresponds to the single solver predicted to be the fastest
and allocates it all the available CPU-time
Regression-based
1-Regression
Given the predicted runtime of each solver, the solver predicted to be the fastest is selected and it has
allocated all the available CPU-time
M-regression
Initially we select the solver predicted to be the fastest, but we allocate only its predicted CPU-time +10%. If
such a solver does not solve the given AF in the allocated time, it is stopped and no longer available to be
selected, and the process iterates by selecting a different solver
135 [CVG17], Slide courtesy of Mauro Vallati

Some interesting results
when using representative
training instances..
EE-PR
System Cov. PAR10
VBS 91.4 562.9
Classify 89.7 665.2
1-Regression 88.6 734.7
M-Regression 82.8 1068.3
FDSS 80.0 1311.4
Cegartix 79.1 1350.4
Shared-2 73.2 1678.0
Shared-3 69.4 1892.0
ArgSemSAT 69.1 1916.2
LabSATSolver 66.8 2050.3
prefMaxSAT 66.8 2057.2
Shared-4 65.7 2105.5
Shared-5 63.3 2240.3
DIAMOND 61.0 2417.0
…

Selection of Solvers
EE-PR
System Class. M-Reg.
ArgSemSAT 0 253
ArgTools 311 305
ASGL 6 36
ASPARTIX-D 2 80
ASPARTIX-V 1 99
Cegartix 221 403
Conarg 157 122
CoQuiAas 43 44
DIAMOND 0 65
GRIS 153 278
LabSATSolver 13 208
prefMaxSAT 297 301

Leave-one-set-out Scenario: Can We Generalise?
EE-PR
Barabasi-Albert Erdös-Rényi StableM Watts-Strogatz
System Cov. PAR10 Cov. PAR10 Cov. PAR10 Cov. PAR10
Classify 78.9 1321.4 88.6 745.0 74.4 1574.3 89.5 677.8
1-Regression 76.3 1479.0 63.0 2255.2 76.5 1453.9 83.0 1079.9
M-Regression 70.4 1828.4 67.3 2039.7 77.0 1434.7 79.6 1267.6
FDSS 69.1 1916.2 80.9 1245.5 79.1 1341.9 78.6 1380.0
Shared-2 73.2 1678.0 73.2 1678.0 74.2 1620.4 73.2 1678.0
Shared-3 69.4 1892.0 67.3 2007.9 69.5 1896.7 69.4 1892.0
Shared-4 65.7 2106.2 65.7 2101.1 65.7 2108.1 65.7 2103.9
Shared-5 63.3 2240.9 63.4 2235.8 63.3 2242.9 63.3 2242.9

• Portfolio systems generally outperform basic solvers;
• If the training instances are representative of testing AFs, the existing set of features
is informative for selecting most suitable solvers;
• Classiﬁcation-based portfolios show good generalisation performance;
• Static portfolios are usually the approaches which are less sensitive to different
training sets.
139

Configuration of Algorithms
Solvers can be configured to improve performance on a class of problems / instances.
§F. Hutter, H. H. Hoos, K. Leyton-Brown, and T. Stützle. Paramils: An automatic algorithm configuration
framework. J. Artif. Intell. Res. (JAIR), 36:267–306, 2009

Configuration of Algorithms
There exists several configuration approaches, based on different underlying ideas.
For the sake of this talk, we focus on SMAC,¶
used for configuring ArgSemSAT
¶F. Hutter, H. H. Hoos, K. Leyton-Brown, and K. P. Murphy. Time-bounded sequential parameter optimization.
In Learning and Intelligent Optimization, 4th International Conference, LION, pages 281–298, 2010

Conﬁguration of the Solver
Parameter Domain Default
SOLVER-ExtEnc {001111, 010101, 010111, ......, 111111} 101010
GLUCOSE-gc-frac [0.0, 500.0] 0.2
GLUCOSE-rnd-freq [0.0, 1.0] [0.0
GLUCOSE-cla-decay [0.0, 1.0] 0.999
GLUCOSE-max-var-decay [0.0, 1.0] 0.95
GLUCOSE-var-decay [0.0, 1.0] 0.8
GLUCOSE-phase-saving 0,1,2 2
GLUCOSE-ccmin-mode 0,1,2 2
GLUCOSE-K [0.0, 1.0] 0.8
GLUCOSE-R [1.0, 5.0] 1.4
GLUCOSE-szTrailQueue [10,10000] (int) 5000
GLUCOSE-szLBDQueue [10,10000] (int) 50
GLUCOSE-simp-gc-frac [0.0, 5000.0] 0.5
GLUCOSE-sub-lim [-1,10000] (int) 20
GLUCOSE-cl-lim [-1,10000] (int) 1000
GLUCOSE-grow [-10000,10000] (int) 0
GLUCOSE-incReduceDB [0,10000] (int) 300
GLUCOSE-ﬁrstReduceDB [0,10000] (int) 2000
GLUCOSE-specialIncReduceDB [0,10000] (int) 1000
GLUCOSE-minLBDFrozenClause [0,10000] (int) 30

Conﬁguration of the Framework
Order arguments/attacks according to:
1. The number of attacks received;
2. The number of attacks to other arguments;
3. The presence of self-attacks;
4. The difference between the number of received attacks and the number of attacks to
other arguments;
5. Being an argument in a mutual attack.
+ arguments can be listed following a direct or inverse order
Ordering of arguments and attacks are independent

Conﬁguration of the Framework (2)
a1 a3 a2
arg(a1).
arg(a2).
arg(a3).
att(a1,a3).
att(a2,a2).
att(a3,a1).
att(a3,a2).
arg(a2).
arg(a3).
arg(a1).
att(a2,a2).
att(a3,a2).
att(a3,a1).
att(a1,a3).
List of arguments ordered according to the number of
received attacks and, subsequently, the number of outgoing
attacks; and the list of attacks ordered prioritising self-attacks
and, subsequently, the number of outgoing attacks

Parametrisation
Parameter Domain Default
args_ingoingFirst [-1.0,1.0] 0
args_outgoingFirst [-1.0,1.0] 0.2
args_autoFirst [-1.0,1.0] -1
args_eachOther [-1.0,1.0] -1
args_differenceFirst [-1.0,1.0] -1
atts_ingoingFirst [-1.0,1.0] 0
atts_outgoingFirst [-1.0,1.0] 0
atts_autoFirst [-1.0,1.0] 0.2
atts_eachOther [-1.0,1.0] 0
atts_differenceFirst [-1.0,1.0] 0
atts_orders {0,1,2,3,4} 0
0 Same ordering applied to the first argument of the attack pair
1 Same ordering applied to the second argument of the attack pair
2 Inverse ordering applied to the first argument of the attack pair
3 Inverse ordering applied to the second argument of the attack pair
4 Attack-specific ordering

Results: Representative Training Instances
Set Configuration IPC Score PAR10 Fastest ( )
Barabasi-Albert Default 78.0 1921.0 2.5
Configured 125.2 1863.1 60.5
Erdös-Rényi Default 56.8 3426.5 16.5
Configured 60.4 3329.2 18.0
Watts-Strogatz Default 116.6 1967.3 28.0
Configured 118.1 1967.9 23.5
General Default 110.0 1665.4 11.0
Configured 143.0 1376.8 62.5

Results: Cross-Validation
Training sets Test sets
Barabasi-Albert Erdös-Rényi Watts-Strogatz General
Barabasi-Albert 119.2 6.9 34.5 42.8
Erdös-Rényi 92.3 58.6 105.3 125.7
Watts-Strogatz 116.2 52.6 115.6 129.2
General 87.5 57.6 113.5 133.2

Conﬁguration: Most Important Single Parameters
Set 1st 2nd 3rd
Barabasi-Albert S-ExtEnc (011111) G-ﬁrstReduceDB (1528) G-cla-decay (0.32)
Erdös-Rényi F-autoFirst (-1.00) G-rnd-freq (0.00) G-K (0.26)
Watts-Strogatz S-ExtEnc (101010) G-Grow (0) G-rnd-freq (0.08)
General S-ExtEnc (101010) G-R (2.09) G-cla-decay (0.99)

Conﬁguration: Interaction Between Parameters

• We demonstrate that joint AF-solver configuration has a statistically significant
impact on the performance of ArgSemSAT;
• We demonstrate the synergies between AFs configuration and SAT solvers behaviour;
• We open new, exciting possibilities in the area of learning for improving performance
of abstract argumentation solvers.
150

Can we predict the number of extensions?
152

Groups of Feature
DG UG Graph Matrix All
Level 1
Accuracy 90.2 84.6 90.1 91.1 91.4
Precision (E∆(PR) = {∅}) 93.7 91.0 93.7 93.7 93.5
Precision (|E∆(PR)| = 1, E∆(PR) ̸= {∅}) 79.4 63.3 79.4 84.9 85.6
Precision (|E∆(PR)| > 1) 90.7 87.3 90.7 90.5 91.2
Level 2
Accuracy 85.4 64.9 85.3 86.7 86.3
Precision (|E∆(PR)| ≤ χ) 89.1 71.5 88.9 90.0 90.0
Precision (|E∆(PR)| > χ) 78.3 47.5 85.3 80.0 79.2
154 From [VCG19]

Groups of Feature
RMSE 1.18 2.45 1.18 1.24 1.81
155 From [VCG19]

AF derived from r : ¬a, ¬b → c
156 From [VCG19]

Groups of Feature
Level 1
Accuracy 100.0 92.4 100.0 100.0 100.0
Level 2
Accuracy 94.4 56.3 94.4 78.8 86.5
Precision (|E∆(PR)| ≤ χ) 33.3 0.0 33.3 0.0 0.0
Precision (|E∆(PR)| > χ) 96.8 92.8 96.8 94.7 95.2
157 From [VCG19]

Groups of Feature
RMSE 12.9 1.3 5.8 15.5 38.0
158 From [VCG19]

• We can develop a model able to predict, with an overall accuracy of 91.4%, whether
or not an AF has a unique empty preferred extension;
• We can distinguish when there is one or more preferred extensions;
• If more than one preferred extension is predicted, predictive models are able to
predict their log-number;
• It is possible to discriminate around a pivotal number of extension with an accuracy
≥ 90.0% .
159

argumentation for machine learning

1010101
0101101
1110001
0101010
1010100
1010101
0101101
1110001
0101010
1010100
1010101
0101101
1110001
0101010
1010100
λ
λ
π
Data Model Inferencing
Algorithmic
PresenceDirect Human Involvement
162 From [CGV18], Figure based upon Diakopoulos N. Accountability in algorithmic decision making. Communications of the ACM. 2016; 59(2): 56-62.

Are we using quality data?
163

1. it should follow an argumentation process constructing reasons for/against competing claims;
2. evidential arguments should increase/reduce confidence in claims;
3. ceteris paribus, the more independent and sound arguments for a given claim, the greater our
confidence in such a claim;
4. a single argument can be conclusive for confirming or refuting a claim;
5. arguments and theories can themselves be questioned;
6. some arguments can be stronger than others;
7. in the absence of information about relative strength, contradictory arguments still play an
important role in decision making;
8. it is desirable to develop systems using sound, formal languages for argumentation but that
can be translated to and from intuitive natural language interfaces;
9. a rational agent can choose the hypothesis that has the greatest confidence among all the
competing hypotheses, unless there are grounds to argue against such a confidence;
10. a rational agent not forced to choose may defer a decision on the grounds that the arguments
are unwarranted.
¶J. Fox, 2011. Arguing about the evidence: a logical approach. In Proc. of the British Academy, Vol. 171. 151–182.
164

Please refer to the ﬁrst part of this tutorial
165

1010101
0101101
1110001
0101010
1010100
1010101
0101101
1110001
0101010
1010100
1010101
0101101
1110001
0101010
1010100
λ
λ
π
Algorithmic

Walton and Krabbe
[WK95]
• Information seeking
• Inquiry
• Persuasion
• Negotiation
• Deliberation
• Eristic
171 Slide courtesy of Simon Parsons

Information-seeking dialogue
One participant obtains information from another.
172 From [WK95], Slide courtesy of Simon Parsons

• Walton and Krabbe discuss a number of kinds of information-seeking dialogue.
• Expert consultation
Layman elicits the expert’s opinion.
• Didactic
Dialogue aims to turn the layman into an expert.
• Interview
Obtain the opinion of one party.
• Interrogation
Extract information from one party.
• …

Inquiry dialogue
Participants collaborate in ﬁnding a proof.

Persuasion dialogue
One participant tries to convince another about a proposition.

Negotiation dialogue
Participants discuss how to divide a scare resource.

Deliberation dialogue
Participants discuss a course of action.

Eristic dialogue
Participants quarrel. Argument as a substitute for ﬁghting.

Initial Situation
Conﬂict Open Problem Unsatisfactory
Spread of
Information
MainGoal
Stable Agree-
ment/Resolution
Persuasion Inquiry Information
Seeking
Practical
Settlement /
Decision (Not) to
Act
Negotiation Deliberation
Reaching a
(Provisional)
Accommodation
Eristic
179 From [WK95]

Other forms of dialogue
• Walton and Krabbe made no claim that this set of dialogue types was exhaustive.
…it has not been our aim to give a complete or comprehensive account of all the
different types of dialogue …

Non-cooperation dialogue
One party aims to prolong the dialogue as long as possible.

How can we operationalise it?
182

• Protocols, in the sense we consider them here, are a means of restricting the bounds
of an interaction.
• Set context.
• Place constraints.
• If one agent operates using a different protocol from another, confusion is likely.

When entering a railway compartment,
make sure to shake hands with all the
passengers.
(Gerard Hoffnung)

What is a protocol?
• One way to deﬁne a protocol is in terms of the dialogues that it permits.
• An utterance is an instantiated locution:
µ = assert(p)
• A dialogue is a sequence of utterances:
δ = assert(p), challenge(p), assert(q, q → p), accept(q), . . .
• A protocol π is then a function from a dialogue to a set of utterances:
π : δ → U
since a protocol, in general, doesn’t restrict an agent’s choice of utterance to a
singleton.

Here is a more explicit protocol
1. A asks question(p).
2. B replies with either assert(p) or assert(¬p) if it can, and assert(U) if it cannot.
3. A accepts B’s response if it can, or challenges.
4. B replies to a challenge with an assert(S), where S is the support of an argument for
the last proposition challenged by A.
5. Go to 3 for each proposition in S in turn.
6. A accepts p if it can.
187 From [PWA03], Slide courtesy of Simon Parsonss

• U indicates that B cannot give an answer.
• U cannot be challenged and as soon as it is asserted, the dialogue terminates
without the question being resolved.
• Note that A accepts whenever possible.
• A is only able to challenge when unable to accept.
• In such a case challenge is the only locution other than accept that it is allowed to
make. B then has to give a response if it has one.
• Note also that there is a general prohibition against an agent making the same
utterance twice. Doing so terminates the dialogue.
188 From [PWA03], Slide courtesy of Simon Parsons

(image by Sumon Azhar)

• Clearly this is a rather restrictive protocol — doesn’t allow that much ﬂexibility in
what agents can do.
• That was actually what we intended.
• Wanted a set of simple protocols in order to be able to analyse them relatively easily.
• Turns out that they are still rather interesting.

German Stock Market
https://gsm.dais-ita.org
DAIDAI BMW PAH3 VOW3
CON
BAYN HEN3 BEI3
How can I help you?
Explain Henkel in detail
Henkel (HEN3) stock price depends on Bayer
(BAYN) stock price.
When BAYN stock price changes, there is high
confidence that HEN3 stock price is very likely to
change <0.8, 0.1, 0.1>.
When BAYN stock price does not change, there
is high confidence that HEN3 stock price is
somewhat unlikely to change <0.3, 0.6, 0.1>.
Please remember that the three numbers above
represent respectively the belief, disbelief, and
uncertanty of the conditional opinions.
192 From [Bra+18]

German Stock Market
CON
BAYN HEN3 BEI3
How can I help you?
What happens in detail if both Porsche
and Continental stock prices change?
When both Porsche and Continental stock prices change:
There is high confidence that BMW stock price is unlikely
to change <0.2, 0.7, 0.1>.
There is low confidence that Porsche (PAH3) stock price is
somewhat likely to change <0.2, 0.1, 0.7>.
There is low confidence that Volkswagen (VOW3) stock
price is somewhat likely to change. <0.1, 0.05, 0.85>.
There is high confidence that Daimler (DAI) stock price is
unlikely to change <0.15, 0.75, 0.1>.
Please remember that the three numbers above represent
respectively the belief, disbelief, and uncertanty of the
marginal opinions computed on the basis of the given
knowledge.
193 From [Bra+18]

German Stock Market
CON
BAYN HEN3 BEI3
How can I help you?
Bayer depends on Daimler
Did you mean:
the stock ‘BAYN’ depends on the stock ‘DAI’.?
Yes
Ok, I have updated the graph to the right with
that information (please note the red arrow) and
recomputed the opinions.
194 From [Bra+18]

• Explaining a model is an act of communication
• An autonomous agent can engage with a human using a formal protocol
• Interfaces using controlled natural languages can support such a conversation
195

p0
p1
p2
p3
One-Hot
Encoding
Preference
Ranking
Path in
Graph
1. Sake
2. Unagi
3. Ika
198 Image from [Xu+18]

x1 ¬x2 ¬x3 ¬x1 x2 x3
A compiled decomposable and
deterministic circuit for the
exactly-one constraint with 3
variables
Pr(x1) Pr(¬x2) Pr(¬x3) Pr(¬x1) Pr(x2) Pr(x3)
× × ×
+
The corresponding arithmetic circuit for the
exactly-one constraint with 3 variables
199 Images from [Xu+18]

Argumentation should be closer to human experience, right?
200

The Experiment
• Presenting each participant with a text followed by a questionnaire
• Each participant is shown a single (randomly selected) text
• Four domains:
1. weather forecast
2. political debate
3. used car sale
4. romantic relationship
• Two KBs: base case, and extended case
• The base case always consider two arguments a1 and a2 with two contradicting
conclusions; and a preference in favour of a2
• The extended case reinstates a1 (somehow)
• Participants are asked to determine which of the following positions they think is
accurate:
• I agree with a1
• I agree with a2
• I can’t agree with either a1 or a2
202 From [CTO14]

Hypotheses
H1: In the base cases the majority of participants will agree with a2
H2: In the extended cases the majority of participants will agree that they cannot
conclude anything from the text
203 From [CTO14]

Analysis
0
15
30
45
60
PA PB PU
%
Distribution of acceptability of actors’ positions
Base cases Extended cases
PA = a1; PB = a2; PU = neither
Distribution of the ﬁnal conclusion is statistically signiﬁcant
Base cases, χ2
analysis (2, N=77)=37.74, p < 0.001;
Extended cases χ2
(2, N=84)=8.0, p < 0.02
204 From [CTO14]

In this experiment participants seem to have a skeptical
attitude
205

Steps Person Statement Content
1 to 5 P1 A Hospital staff members do not need to receive flu
shots.
1 to 5 P2 B Hospital staff members are exposed to the flu virus
a lot. Therefore, it would be good for them to
receive flu shots in order to stay healthy.
2 to 5 P1 C The virus is only airborne and it is sufficient to wear
a mask in order to protect yourself. Therefore, a
vaccination is not necessary.
3 to 5 P2 D The flu virus is not just airborne, it can be
transmitted through touch as well. Hence, a mask is
insufficient to protect yourself against the virus.
4 to 5 P1 E The flu vaccine causes flu in order to gain immunity.
Making people sick, who otherwise might have
stayed healthy, is unreasonable.
5 P2 F The flu vaccine does not cause flu. It only has some
side effects, such as headaches, that can be
mistaken for flu symptoms.
207 From [PH18]

Tasks
• Agreement: the participants were asked to state how much they agree or disagree
with a given statement.
• Relation: the participants were asked to state how they viewed the relation between
the statements.
208 From [PH18]

Conclusions of [PH18]
Observation 1 The data supports the use of the constellation approach to probabilistic
argumentation for modelling the argument graphs representing the views
of dialogue participants.
…
Observation 4 The data supports the use of bipolar argumentation frameworks.
209

• Kind of true that argumentation semantics are intuitive for humans
• There are plenty of caveats
• See also Ruth Byrne “Counterfactuals in Explainable Artiﬁcial Intelligence (XAI):
Evidence from Human Reasoning” @ IJCAI
210

1010101
0101101
1110001
0101010
1010100
1010101
0101101
1110001
0101010
1010100
1010101
0101101
1110001
0101010
1010100
λ
λ
π
Algorithmic

212 https://www.law.cornell.edu/supremecourt/text/534/266

Human experts are often allowed
“
to draw on their own experience and specialised training to make inferences
from and deductions about the cumulative information available to them that
might well elude an untrained person
„
213

Argument from Autonomous Inferencing
Major Premise: A is an autonomous system trained in subject domain S containing
proposition P.
Minor Premise: A asserts that proposition P is true (or false).
Conclusion: P is true (or false).
Critical questions
CQ1: What are A’s maker interests?
CQ2: Is A’s assertion internally consistent?
CQ3: Is A training adequate to make a judgement about P?
CQ4: Is the provenance of A’s judgement about P sound?
CQ5: Is A’s assertion consistent with the known fact of the case (based on evidence
independent from A)?
CQ6: Is A’s assertion consistent with other, independent autonomous systems’
assertions?
214 From [CGV18]

CONCLUSIONS
215 Image: https://goo.gl/RcCSbb

• Introduction to Formal Argumentation Theory
• Why it is important?
• Supporting scientiﬁc enquiry
• Structured argumentation
• Abstract Argumentation
• Algorithms and Implementations
• Machine learning for argumentation
• Argumentation mining
• Machine learning for evaluating argumentation framework
• Argumentation for machine learning
• Are we using quality data?
• Arguing about the model: explanations and tellability
• Arguing about the algorithmic presence
216

Machine Learning and Argumentation: A Tutorial

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Machine Learning and Argumentation: A Tutorial

Semelhante a Machine Learning and Argumentation: A Tutorial (20)

Mais de Federico Cerutti

Mais de Federico Cerutti (20)

Último

Último (20)

Machine Learning and Argumentation: A Tutorial