A Multidisciplinary Approach to Crowd Studies

4th Summer School AACIMP-2009
Achievements and Applications of Contemporary Informatics,
Mathematics and Physics

A multidisciplinary approach to crowd
studies
Lecture 1 – 11.08.2009

Dr. Sara Manzoni
Complex Systems and Artificial Intelligence research center
Department of Computer Science, Systems and
Communication
University of Milano-Bicocca

What is a Crowd?
Contributions from social sciences, social
psychology on Human behavior and social
collectivities

Some definitions

• “Too many people in too little space” (Kruse,
1996)

• “A gathering of individuals that influence one
another and share a purpose, intent or
emotional state in a limited space” [Blumer]

• A crowd is a form of collective action: “two or
more persons engaged in one or more actions
(e.g. locomotion, orientation, vocalization,
verbalization, gesticulation, and or
manipulation), judged common or concerted
on one or more dimensions (e.g. direction,
velocity, time, or substantive content”
(McPhail, 1991)

What is a Crowd?
Contributions from social sciences, social
psychology on Human behavior and social
collectivities
Theories

• Contagion/Transformation Theory Gabriel
(Tarde; Le Bon; Blumer, Canetti) Tarde
Gustave LeBon

• Convergence Theory
Elias Canetti
(Berk, Floyd Allport, Neal Miller, John
Dollard)
• Emergent Norm Theory Robert Ezra Park
Herbert Blumer
(Turner, Killian)
• Value Added Theory
(Smelser)

Neil Smelser

Traditional Theories of Crowd Behavior

Contagion T heory  Crowd behaviour is irrational

The crowd
• exert an effect on its members
• forces individuals to action thanks to anonymity that
encourages people to abandon rationality and
responsibility
• helps emotion propagation that can drive to irrational and
suitably violent action


Convergence theory  Crowd behavior is rational

Crowd behavior is
• not the result of the crowd itself
• carried inside the crowd by specific individuals

• People that would like to behave and act in a certain way
come together in order to form and constitute a crowd

• Crowd behavior expresses values and beliefs that are
already present in the population (i.e. racist feelings)
• The mob is a rational product of rational values


• E m ergent/ nor m theory  Crowd behavior is not fully
predictable but it is not irrational

• People in a crowd assume different roles (e.g. some participants
become leaders, other lieutenants, followers, inactive bystanders or
even opponents)
• Common interests can bring people together in a crowd, but
different patterns (of behavior) can emerge inside the crowd itself
• Norms inside a crowd can be vague and changing in the process of
aggregation (people state their own rules while participating at the
crowd)
• Decision-making has a preponderant role in the behavior of the
crowd although external observers may find it difficult to realize

“ the reason why good data on
crowd and collective behavior are
so scarce is that data are function
of theorethical guidance and Herbert Blumer
1900-1987
existing theories provide no
guidance; but, useful theories
cannot be built in the absence of
empirical data ...”

Freely taken from:Clark McPhail, Blumer’s theory of collective behavior: The development of a Non-symbolic interaction
explanation, The Sociological Quaterly, Volume 30, Number 3, JAI press 1989

Crowd study:
Contributions/Open issues from computer
science
Better comprehension of crowd phenomena (crowd
study) and development of tools (for crowd study and
crowd management)
– Data acquisition techniques and technologies
Data acquisition techniques and technologies
• Direct observation (Stalking, Questionnaires)
• Scene analysis
• Proximity detection (RF-ID)
• Localization systems (GPS, sensor networks)
– Crowdmodeling andand simulation
Crowd modeling simulation
• Modeling, computational, analysis tools
• Simulation and visualization tools
– Knowledge representation
• Data representation and analysis M ultidisciplina
• Experts’ knowledge ry F ield of
• Available theory/results from social sciences S tudy

Data Acquisition

HOW CAN CROWDS AND INDIVIDUALS BE MEASURED?
WHAT CAN BE MEASURED?
HOW CAN AVAILABLE TECHNOLOGIES MEASURE CROWDS?

S. Bandini, M.L. Federici, S. Manzoni
“A qualitative evaluation of Technologies and Techniques for Data
Acquisition on Pedestrians and Crowded”
Proc. of Special session “At man’s step”@SCSC07, San Diego, CA

What are we interested in Measuring:
Data on the crowd
• Number of people
(static or inside a
march)
• Density of the crowd
• Flow, Pressure and
times of ingress/egress
from a place
• Groups movement
inside a crowd
• …

What are we interested in Measuring:
Data on individuals
• Trajectories in a specific
environment
• Walking speed in
different situations
• Physical Behavior:
– Queuing
– Streaming
– Group formation
– Separation
– Cohesion
– Imitation
– …

Measuring Crowds: HOW?
Mature and emergent technologies for data
acquisition
Stalking (following people
• Direct Observation, interviews, without being seen!)
questionnaires, stalking

• Technologies for people
positioning and counting
– Scene analysis: TV Camera
– Global Positioning System (GPS)
– Proximity technologies (Radio
Frequency IDentification – RFID)
– Sensor Networks
– PDAs, SmartPhones (GPRS, Wi-
Fi)
– Dead reckoning (portable inertial
platform)

Data acquisition: an example (2005)

Application of
GIS/GPS to track
pedestrian movements
– Position, velocity,
trajectories
– Critical areas
identification

N. Koshaak (Makkah - Saudi Arabia)

Comparing Data Acquisition
Technologies and Techniques (1)
Scalability
5

Continuous Localization Single Individual Monitoring
4

3

2
Precise Localization Data Entire Crowd/groups Monitoring
1

0

Outdoor Indoor
0: null quality
1: insufficient
2: just sufficient
3: discrete Cheap Large Scale

4: good
5: best Available Small Scale

Absolute Position System (GPS) Proximity Tech. (Passive RFiD) Scene Analysis (Video Analysis)

Comparing Data Acquisition
Technologies and Techniques (2)
Scalability
5

Continuous Localization Single Individual Monitoring
4

3

2
Precise Localization Data Entire Crowd/groups Monitoring
1

0

0: null quality Outdoor Indoor

1: insufficient
2: just sufficient
3: discrete Cheap Large Scale

4: good
5: best Available Small Scale

Dead Reckoning (Portable Inertial Platform) Sensor Network (ZigBee) Direct Observation (People Counting)

San Diego (CA)
At man’s Step special track at Summer
Computer Simulation Conference 2007
Jul, 13-14 2007

S. Bandini, M.L. Federici, S. Manzoni, “A
qualitative comparison of technologies for Data
Acquisition on Pedestrians and Crowded
Situations”

Crowd modeling and
HOW DO PEOPLE BEHAVE IN CROWDED SPACES AND
simulation
SITUATIONS?

SIMULATIONS ARE EXPERIMENTAL LABORATORIES FOR
HUMAN SCIENCES

Crowd modeling and application
directions
• Support the study of
pedestrians/crowds behavior
– Envisioning of different behavioral
models in realistic environments
– Possibility to perform ‘in-machina’
experiments
• Decision makers might not be
experts neither on crowd
dynamics nor on software and
math tools
– Need of effective ways to edit,
execute, visualize and analyze
simulations (what-if scenarios)
• Indoor (Buildings, Shopping
Centers, Stadiums) vs Outdoor
(Urban spaces for public
events/transport, Parades,
Marches, Fairs, Sport Events,
Concerts)

Examples of Crowd Dynamics

• Evacuation dynamics
– normal vs panic
– open/structured spaces

• Lane formation and other self-
organization phenomena

• Crowd formation/dispersion
• Crowd movement and behavior (e.g. in
shopping center)

Schema of the abstract levels involved in a
simulation
Phases1: observation canpassages implied in Hypothesissimulationsimulation
IfPhase 2: at the to 4 of beTarget System  The the (Model),athat could
Phase from 1 different the seen as of an theof Target System has to be
Formulation phases Building
Abstraction Phaseconcerned with models implicitly we areconstruction the Abstract
we look Modelization
Phase Computational or informal. can’t be separated of the next).
3:a data on it vague that and abstraction. Translation models.
and aresimplyMas
even be 4: we can see Modeling
Software Implementation working withfrom of the
observed and intuitive, collected (this phase
model using Translation of many

Levels Computational Model the of Model (language/entities). This of the
Model into something selective instead and an Hypothesis on what is
To observe thephasesinlevel Software Code
Each model represents a concernabstraction.the Decoding Levels Phase
The next Computational attention How many levels of abstraction
constitutes the point of “no return”
observed must be already present in to reality (an interpretation key that
are involved in a simulation process? the observer)
simulation
explains how translate back entities of the computational model in
entities of reality is needed) 4) Phase 4: Software
The Software Model (operational model)
implementation
Software 4 SW of the Computational Model
4

The computational model is what we use
3) Phase to Computational abstract model. The
3: represent the
Comp. MAS
3 3
modeling computational model is always a formal
Model model. It can be a model Agent Based or
Cellular Automata based etc.
The abstract model of the target system
Abstract M 2) Phase 2: Model / Theory Construction language,
can be expressed in natural
2 Model mathematical formulas etc. but usually it is
2 not computational. It can be, at first, also
anThe target system is the object of study
intuitive set of rules.
(Physical model). It is a specific point of
Target view on a portion of reality that we consider
1 TS 1) Phase 1: Observation the context. The target
“isolated” from
System 1
system is determined by our observation
perspective on reality, and by the aspects
of reality that we want to capture (i.e. atom
level; molecular; macro-level)
0
Reality

1) Observation 5) Computation (sim running)
2) Model / Theory Construction 6) Visualization
3) Computational modeling 7) Verification of Sim in respect to the Theory
4) Software implementation 8) Validation of Theory in respect to Real Data
9) Prediction

Abstraction Phase 5: 7:assumptions Simulation is Validatedthe it Decoding
Output:Phase Displaying:(calculus-computation)
Phase9: prediction  of The theory meaningful
Theory:8:Verification  the theory ofhas then for
Phasesimple results and thesoftware appropriate
Simulation 6: Visualization anrunning and
Software run If the envisioning of the
Real Data: collection constitute to
Validation rules that
Levels
execution thatin simulationonly to initialobservable
available data then itrespect to our on the base
description of is often the outputs in theory
hascheckedof relation to reality makeoperate of
measurements domain
be to be
outputs verifiedin is target system
translation the in the possible way to predictions Levels
for future simulation dynamic data
objects states of the Target System
indicators
with the chosen in precedent phase
5
4 SW O 6 Output
4 Software

Simulation
3 Comp. 3
MAS SD
7
Model Displaying

Abstract M T
2 8 Theory
Model 2

1 Target TS R Real Data
System 1 9

0
Reality

Decoding and correcting
• Abstract Model Revision: If Check: a Campaign results are that gave capture
Computational Model Change: Computational Model is judged obtained a deeper
Software Implementation the After If no suitableof Simulationadequate tonegative
the theory assumptions, but realbe needed in order to my theory in translates
results a check of the assumptionsdata don’t give translate verify that itthe model is
check into the software may that I made to a positive feedback, abstract
model must undergo a revision that will Model can eventually on new hypothesis and
needed. Athe computational model lie on the constructionbe necessary.
properly change of the computational
empirical observations

4 Software SW O
So
f
Ch tware
eck Imp
lem
Comp. MAS en t SD
3 Com atio
Model put n
atio
nal
Abstract Mod
M el C
2 han T
Model Abs ge
trac
t Mo
Target del
1 TS Rev R
System isio
n

Mapping of the phases of Design, Inference,
Interpretation and Analysis in the Schema

4 Software SW Inference O

Comp. MAS SD
3
Model Interpretation
Design
Abstract
2 M T
Model

Target Analysis
1 TS R
System

Steps in Physical Scientific Practice

Experiment/ Hypothesis/
Validation Observation Theory Building

Physical
Prediction Model

Mathematical
Inference/ Model Translation in
Deduction Mathematical
Framework

From: Modeling Games in the Newtonian World, by David Hestenes

Comparison Between Abstraction Implied in Physic
Scientific Practice and MAS Simulation Practice

Abstraction in Mabs Abstractions in Physical
Theories

One Step More
of Abstraction
4 Software SW
is implied
Mas Model
Comp. MAS Corresponds to
3 Mat
Model Mathematical Model
Abstract
2 M M
Model

Target
1 TS TS
System
reality

Pedestrian Movement at the Micro-Scale: Social
Force Model [Batty, Helbing (2001)]

• Four principles “guide” movement
– Agents avoid obstacles present in the environment
– Agents consider repulsive the presence of other pedestrians when
space is congestioned
– Agents also attract each other (principle of the flocking)
– Agents “desire” to follow a direction

• To each of these components it is associated a force that
pushes the agent towards a specific direction

New Position = Old
Position
+ Desired
Position
+ Geometric
Repulsion
+ Social
Repulsion
+ Social
Attraction
+ ε

Crowd modeling
Analytical (physical) approach
Lane formation

• Pedestrians  particles
subject to forces
• Goals: forces of attraction
generated by points/reference
point in the space
• Interaction among
pedestrians: forces generated
by particles
• Social forces ‘Freezing by heating’
– Repulsive  tendency to stay
at a distance
– Attractive  imitative
mechanisms

D. Helbing, I. J. Farkas, T. Vicsek: Freezing by Heating
in a Driven Mesoscopic System, PHYSICAL REVIEW
LETTERS, VOLUME 84, NUMBER 6, 2000

Crowd modelling: Cellular Automata
• Environment  bidimensional lattice of
cells
• Pedestrian  specific state of a cell (e.g.
occupied, empty)
• Movement  generated thanks to the
transition rule
– an occupied cell becomes empty and an
adjacent one, which was previously
vacant, becomes occupied

• Choice of destination cell in a transition
generally includes information on
– Benefit-Cost/Gradient: information
about “cell desirability”
– Magnetic Force: models the effect of
presence of other agents in the
environment (attraction/repulsion
of crowds)

Crowd modelling: From CA to Situated MAS

• Individuals are separated from the
environment
– Agents, not just cell occupancy states
– may have different behaviors: several
action deliberation models can be
integrated
– heterogeneous system

• Agents interact by means of
mechanisms not necessarily related
to underlying cell’s adjacency
– Action at a distance is allowed

Situated MAS action and interaction
• Agents are situated
– they perceive their context and
situation
– their behaviour is based on their
local point of view
– their possibility to interact is
influenced by the environment
• Situated Agents Interaction models
– Often inspired by biological
systems (e.g. pheromones,
computational fields)
– Generally provide a modification of
the environment, which can be
perceived by other entities
– May also provide a direct
communication (as for CAs
interaction among neighbouring
cells)

Situated Cellular Agents (SCA)
Multi Agent model providing react(s,ab,s’) react(s,ac,s’)
• Explicit representation of
agents’ environment
• Interaction model strongly
related to agents’ positions
in the environment
– Among adjacent agents
(reaction)
– Among distant agents,
through field emission- emit(f)

diffusion-perception
mechanism
• Possibility to model
heterogeneous agents, with
different perceptive
capabilities and behaviour

CompareT(f×c,t) = true

Situated Cellular Agents (SCA)
• Formal and computational framework to
represent and study of dynamics in pedestrian
systems
– autonomous interacting entities
– situated in an environment whose spatial
structure represents a key factor in their behaviors
(i.e. actions and interactions)
• Based on MMASS (Multilayered Multi-Agent
Situated Systems)
[S. Bandini, S. Manzoni, C. Simone, Dealing with Space in Multi-
Agent System: a model for Situated MAS, in Proc. of AAMAS
2002, ACM Press, New York, 2002]
• MMASS relaxes constraints on uniformity, locality
and closure of CA
[S. Bandini, S. Manzoni, C. Simone: Enhancing Cellular Spaces
by Multilayered Multi Agent Situated Systems, Proc. of ACRI
2002: 156-167]
– Open systems can be modeled
– Not homogeneous agent environment
– Heterogeneous agents
– Interaction involving spatially not adjacent agents

SCA model

Spatial
structure

Agents and
behaviours At-a-distance
interaction

SCA Space
• Space: set P of sites arranged in a
network
• Each site p∈P is defined by <ap, Fp, Pp>
where
– ap∈A ∪{⊥}: agent situated in p
– Fp⊂F: set of fields active in p
– Pp⊂P: set of sites adjacent to p

SCA Fields
• <Wf, Diffusionf, Comparef, Composef>
– Wf: set of field values
– Diffusionf: P X Wf X P Wf X…X Wf
field diffusion function
– Composef: Wf X …X Wf Wf
field composition function
– Comparef: Wf X Wf  {True, False}
field comparison function
• Fields are generated by agents to interact at-a-distance and
asynchronously

SCA Agents
• a∈A : <s,p,T>
• T  < ∑T, PerceptionT, ActionT>
– ∑T: set of states that agents can assume
– ActionT: set of allowed actions for agents of
type T
– PerceptionT:
∑T [N X Wf1] … [N X Wf|F|]
• PerceptionT(s) = (cT(s), tT(s))
• cT(s): perception modulation
• tT(s): sensibility threshold
• An agent a = <s,p,T> perceives the field value wfi
of a field fi = <Wf, Diffusionf, >i, *i>when ciT(s)*fi
wfi,>fi tiT(s)) and

SCA Agent Actions
• ActionsT: set of actions that agents of type T can perform
• Agent behavior: perception-deliberation-action cycle
– Perception of local environment (e.g. free sites, fields)
– Action selection based on agent state, position and type
– Action execution
• Four basic actions
– intra-agent actions: triggerT(), transportT()
– inter-agent actions: emitT(), reactionT()

action: trigger(s,fi,s’) action: reaction(s, ap1, ap2, …, apn,s’)
condit: state(s), perceive(fi) condit: state(s), agreed(ap1, ap2,…,
effect: state(s’) apn)
action: transport(p,fi,q) effect: state(s’)
action: emit(s,f,p)
condit: position(p), empty(q), near(p,q), condit: state(s)
perceive(fi) effect: present(f, p)

Situated MAS and crowd modeling
• Pedestrians  agents

• Environment  graph, as an
abstraction of the actual environmental
structure

• Movement  generated thanks to
perception-action mechanism
– Sources of signals: relevant objects transport(p,q)
(gateways, reference points), but
also other agents
– Agents are sensitive to these signals
and can be attracted/repelled by
them (amplification/contrast)
– Possible superposition of different
such effects

SCA Crowd Modelling Approach
Abstract scenario Computational model for Experiment-specific
specification the scenario parameters

Definition of the MMASS
spatial structure

Definition of active Definition of monitored
elements of the parameters and
environment specification
and field types of monitoring mechanisms

Specific simulation
Definition of mobile agents
configuration (number, type,
(types, states, perceptive
position and initial state of
capabilities and behavioural
mobile agents, other
specification)
parameters)

Underground scenario
• An underground station (several interesting crowd
behaviors can be studied)

• Passengers' behaviors are difficult to predict: crowd
dynamics emerges from single interactions
– between passengers
– between single passengers and parts of the
environment (signals, constraints)

• Passengers (actions)
– on board may
• have to get off
• be looking for a seat or try standing beside a handle
• be seated
– on the station platform may
• try to reach for the exit door
• get on the train

• Passengers have to match their goals with
– Environment obstacles
– other passengers goals
– implicit behavioral rules that govern the social
interaction in underground stations

SCA model of Underground Station Scenario
Spatial structure of the environment

Spatial structure 
discrete abstraction of
simulation environment

Active Elements of the Environment and Field Types

Agent types
An agent type t is a triple
<∑t , Perceptiont, Actiont> where A Passengers Dynamic
g Agents
• ∑t : set of agent states e
• Perceptiont : specifies for every agent n
state and field type t
Seat
– a sensitivity coefficient c modulating
(amplifies/attenuates) field values T
Station Exit
– a sensibility threshold t filtering out y
Static Agents
fields that are considered too faint p
Wagon Exit (active
– An agent perceives a field fT when e
objects)
CompareT(f*c,t) = true s
• Actiont : behavioral specification for Handles
agents of type t

Passengers behavior as state-transition diagram and
attitudes towards movement
E
Example based on designed behaviors for
the case study. G
Should be calibrated SCA platform
editor of agents’ behaviors
W S

P

Seated: agent seated on a seat
W Waiting: passengers on the platform S of the wagon
waiting for a train

G Get Off: people on the wagon that have to State Transition
get off the train

Passenger: agent on the train that has no
P immediate necessity to get off

Exit: passenger that has got down the train
E and goes away from the station

Mobile Agents – Movement

When multiple signals are perceived, agents
evaluate next-destination site according to
weighted sum of perceived values

transport(p,q)

S tate E xits D oors S eats H andles P resence E xit press.

∑ W - Attract (2) - - Repel (3) Repel (1)

P - - Attract (1) Attract (2) Repel (3) Repel (2)

G - Attract (1) - - Repel (2) -

S - Attract (1) - - - -

E Attract (1) - - - Repel (2) -

SCA model of Underground Station Scenario (demo)

Field “Handle”
1) Pedestrian agent in site P (on the
wagon)


E mit (s , f, p)
Field “Handle”
wagon)
1) Available seats (only one) emit a
field
that is perceived as attractive by
the agent

Field “Seat”


E mit (s , f, p)

wagon)
2) Available seats (only one in site Q)
emit a field that is perceived as
attractive by the agent if
distance(P,Q)<ped_threshold
3) The agent perceives the field and
moves (by a transport action) to the
adjacent site, e.g. where the field is
more intense Tras port (p,q)


E mit (s , f, p)

wagon)
more intense Tras port (q,t)
1) Process iterated until the agent
reaches the site where the local
max of intensity is perceived


E mit (s , f, p)

wagon)
more intense Tras port (t,r)


E mit (s , f, p)

wagon)
more intense Trasport (r,g)


E mit (s , f, p)

wagon)
moves (by a transport action) to the Tras port (g,k)
more intense

R eact (a,s s,o)
wagon)
more intense
R eact (p,s g,s )
reaches the site where the local max
of intensity is perceived
3) Available_seat agent and pedestrian
agent change their state
simultaneously (agent seat turns into
the occupied state and stops emitting
fields while passenger turns into
state seated)

Underground Case Study: model execution

• Simulation configuration
– 6 agents getting off
– 8 agents getting on

Visualization of system dynamics
3D rendering of 2D simulations (offline animation)
• Java based bidimensional simulator
• Exported log of the simulation including
– Definition of the spatial structure
– System dynamics
• MaxScript that allows 3D Studio Max to generate an
animation representing the simulated scenario

Avatar#001#001#001#004#003#000@
Avatar#002#002#001#003#005#000@
Avatar#001#002#010#003#002#000@
Avatar#002#001#001#003#004#000@
Space#001#001#001#001#006#000@
Space#001#002#001#002#005#000@
Space#001#003#001#004#005#000@
Space#001#004#001#004#004#000@
Space#001#005#001#003#004#000@
Space#001#006#001#003#002#000@
Space#001#007#001#004#002#000@
Space#001#008#001#005#002#000@

Why 3D?
• To obtain an effective visualization of simulation dynamics
• To obtain a machine-readable spatial abstraction in a semi-
automatic way from existing models of the environment
• To exploit the rich information of a 3D model to implement highly
realistic perception

Freezing by Heating - Experiments
• 3 densities (20-40-60%)
• 10 simulations for each densities
• After few turns pedestrians-agents get stacked in
front of the exit (80-90% of pedestrian population
can’t move for the turn)

Schadschneider
Helbing

Lane Formation: Experiments
• Simple Behavioural Model:
pedestrians are attracted by
the desired exit

(Without collision avoidance the
phenomenon of freezing by heating
is detected also at low densities)

• Introduction of repulsion:

– Each pedestrian at the beginning of
the turn emits a presence field that is
spread in adjacent sites

– Each pedestrian evaluates negatively
the sites where it is perceived the
presence of other pedestrians
(presence of pedestrians with
opposite direction is evaluated more
negatively)

• Lane Formation only at low densities

Lane Formation: Experiments
• Introduction in the model of the 1,2

possibility of an exchange of
position between pedestrians that 1

want to occupy one the site of the Togawa
Simulation 1

Pedestrian Speed (Sites Per Turn)
other
0,8

Simulation 2
Simulation 3

• Introduction of a concept of 0,6 Simulation 4

Blue&Adler

“irritation” that leads pedestrians to: 0,4

– Search for new paths (empty sites
become more desirable after some 0,2

turns of immobility)
0

– Attempt to exchange position with 10 20 30 40 50 60 70 80

Pedestrian Density (Pedestrians / Total Sites)
90 95 100

other agents (less sensitivity to
presence field of other
pedestrians) 4 different configurations of
3 different corridor
• Experimentations performed parameters (2 experiments for
geometries
each density, 500 turns each)
with density from 10 to 90%:

Evacuation scenarios

S ingola uscita D ue uscite

…… continues ……

I Thank You

A Multidisciplinary Approach to Crowd Studies

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a A Multidisciplinary Approach to Crowd Studies

Semelhante a A Multidisciplinary Approach to Crowd Studies (20)

Mais de SSA KPI

Mais de SSA KPI (20)

Último

Último (20)

A Multidisciplinary Approach to Crowd Studies