Engineering human-based services in complex processes

Engineering human-based services in
complex processes
Hong-Linh Truong
Distributed Systems Group,
Vienna University of Technology
truong@dsg.tuwien.ac.at
http://dsg.tuwien.ac.at/staff/truong
1ASE Summer 2014
Advanced Services Engineering,
Summer 2014 – Lecture 9
Advanced Services Engineering,
Summer 2014 – Lecture 9

Outline
 Human capabilities for complex processes
 Human service units
 Incorporating human units into complex services
 Frameworks
 Some issues and solutions
ASE Summer 2014 2

Human-based services for solving
complex problems (1)
3
We need to actively finding and binding human
capabilities into the application!
We need to actively finding and binding human
capabilities into the application!
Humans and software for
solving complex problems
ASE Summer 2014

Human-based services for solving
complex problems (2)
4
But how to program human-based services and
software-based services together?
But how to program human-based services and
software-based services together?
ASE Summer 2014

Example: some common tasks in
data analytics
ASE Summer 2014 5
Human activities in Data Analytics
Labeling
Annotating
Cleansing
Evaluating
data/content
Dectecting
patterns
Classifying
objects
Steering
analytics

Human service units in data
analytics -- functions
 Evaluating: is the quality of picture good?
 Classifying: is it a man‘s or a woman‘s picture?
 Decting: any unidentified object in a picture?
 Labeling: adding location information of a picture
 Cleansing: remove duplicated pictures
 Steering: the quality of picture is bad, should we
continue to merge it with others?
ASE Summer 2014 6
How to model such functions for human units ?
E.g., with WSDL or REST?
How to model such functions for human units ?
E.g., with WSDL or REST?

HUMAN SERVICE UNITS
ASE Summer 2014 7

Human service units
ASE Summer 2014 8
Human
service unit
Human
service unit
Functions
Non-
functional
parameters
Service
models
Technical
interfaces
Provisioning
mechanisms
Interaction
models
Human acting as a „service unit“Human acting as a „service unit“

Human service units
9
Modeling type of
units (e.g.,
computation, data,
monitor,) and their
dependencies
Consumption,
ownership, provisioning,
price, etc.
Elastic
Service
Unit
Service
model
Unit
Dependency
Elastic
Capability
Function
The functional
capability of the unit
and interface to
access the function
Capabilities to be elastic
under different
requirements
SoftwareSoftware PeoplePeople
VolunteersProfessionals
ThingThing
Resources
ASE Summer 2014

Forms of human service
 Individual Compute Unit
 An individual is treated like „a processor“ or “functional
unit“. A service can wrap human capabilities to support
the communication and coordination of tasks
 Social Compute Unit
 A set of people and software that are initiated and
provisioned as a service for solving tasks
 Web services interfaces can be built
 Different pricing models and different
quality models
ASE Summer 2014 10

Human service units – provisioning
mechanisms (1)
 An infrastructure can be introduced for
accessing many ICUs in a crowd
 Allow people to register their service unit capabilities
 Facilitate communication, task bidding, retrieval and
result delivery
 Act like a marketplace: multiple providers and multiple
consumers
ASE Summer 2014 11
ICU ICU
ICUICU
ICU
ICU ICU
CrowdCrowd

mechanisms (2)
 An „infrastructure-as-a-service“ for ICUs
 Facilitate communication, task retrieval and result
delivery
 Single ICUaaS provider and multiple consumers
ASE Summer 2014 12
ICU ICU
ICUICU
ICU
ICU ICU
Cloud
provisioning
models ICUICU

mechanisms (3)
 An „infrastructure-as-a-service“ for SCUs
 Facilitate communication, task retrieval and result
delivery
 Single SCUaaS provider and multiple consumers
ASE Summer 2014 13
SCU SCU
SCUSCU
SCU
SCU SCU
Cloud
provisioning
models SCUSCU

Human service units – technical
interfaces (1)
ASE Summer 2014 14
People
Web Servicce
(REST/SOAP)
People
Web page
Analytics
Activity
Analytics
Activity

Human service units – technical
interfaces (2)
ASE Summer 2014 15
Email/SMS/Instant
Messaging
People
Web page
Web Servicce
(REST/SOAP)
Communication
Services
People
Analytics
Activity
Analytics
Activity

Human service units – interaction
model
ASE Summer 2014 16
ICU/SCU
Task queue
Scheduler
ICU/SCU
Task queue
Scheduler
Analytics
Activity
Analytics
Activity
ICU/SCU
Analytics
Activity

Human service units -- NfPs
ASE Summer 2014 17
Which are important
considerations when
interpreting NfP for human
units?
Which are important
considerations when
interpreting NfP for human
units?

Incorporating human units into
complex processes
 How to provision and employ human compute
units?
 How to select human units?
 Where to place human units in data analytics
and why?
 How to monitor and test human units in data
analytics?
ASE Summer 2014 18

PROVISIONING AND
EMPLOYING HUMAN UNITS--
SOME FRAMEWORKS
ASE Summer 2014 19

Qurk system architecture (1)
ASE Summer 2014 20
Source: Adam Marcus, Eugene Wu, David Karger, Samuel Madden, and Robert Miller. 2011. Human-powered sorts and joins. Proc. VLDB
Endow. 5, 1 (September 2011), 13-24.
Endow. 5, 1 (September 2011), 13-24.

Qurk system architecture (2)
ASE Summer 2014 21
Endow. 5, 1 (September 2011), 13-24.
Endow. 5, 1 (September 2011), 13-24.

Jabberwocky approach (1)
ASE Summer 2014 22
Source: Salman Ahmad, Alexis Battle, Zahan Malkani, Sepandar D. Kamvar: The jabberwocky programming environment for structured
social computing. UIST 2011: 53-64

Jabberwocky approach (2)
ASE Summer 2014 23

Automan approach
ASE Summer 2014 24
Source: Daniel W. Barowy, Charlie Curtsinger, Emery D. Berger, Andrew McGregor: AutoMan: a platform for integrating human-based
and digital computation. OOPSLA 2012: 639-654
Source: Daniel W. Barowy, Charlie Curtsinger, Emery D. Berger, Andrew McGregor: AutoMan: a platform for integrating human-based
and digital computation. OOPSLA 2012: 639-654

SW4H approach (1)
ASE Summer 2014 25
Karastoyanova, Dimka; Dentsas, Dimitrios; Schumm, David; Sonntag, Mirko; Sun, Lina; Vukojevic, Karolina: Service-based Integration of
Human Users in Workflow-driven Scientific Experiments. In: Proceedings of the 8th IEEE International Conference on eScience (eScience
2012
Karastoyanova, Dimka; Dentsas, Dimitrios; Schumm, David; Sonntag, Mirko; Sun, Lina; Vukojevic, Karolina: Service-based Integration of
Human Users in Workflow-driven Scientific Experiments. In: Proceedings of the 8th IEEE International Conference on eScience (eScience
2012

SW4H approach (2)
 Similar concepts in
collaborative working
environments but
integrated into
workflows
 Do not discuss about
where and how to
select human units
ASE Summer 2014 26
Karastoyanova, Dimka; Dentsas, Dimitrios; Schumm,
David; Sonntag, Mirko; Sun, Lina; Vukojevic, Karolina:
Service-based Integration of Human Users in Workflow-
driven Scientific Experiments. In: Proceedings of the 8th
IEEE International Conference on eScience (eScience
2012
Karastoyanova, Dimka; Dentsas, Dimitrios; Schumm,
David; Sonntag, Mirko; Sun, Lina; Vukojevic, Karolina:
Service-based Integration of Human Users in Workflow-
driven Scientific Experiments. In: Proceedings of the 8th
IEEE International Conference on eScience (eScience
2012

HUMAN SERVICE UNITS IN
VIENNA ELASTIC COMPUTING
MODEL
ASE Summer 2014 27

VieCOM -- incorporate humans into a
programming paradigm (1)
• Abstracting human compute units as program elements
• Extending programming languages to support human
compute units
• Data/control flows via extensible APIs
Programming
languages
• Shared memory (e.g., human –software – human),
message passing (human-to-human), artifact-centric,
etc., via APIs working atop the compute unit abstraction
layer
Multiple
programming
models
• Computing capability /profile management: human
computing power, reputation and incentive models
• Monitoring and enforcing incentives/rewards, quality of
results, availability
• Communication between human-middleware, among
Individual Compute Units (SCU)/Social Compute Units
(SCU) for exchanging artifacts and comprehensing l tasks
Execution
environment
28 http://dsg.tuwien.ac.at/research/viecomASE Summer 2014

VieCOM-- incorporate humans into a
programming paradigm (2)
Volunteers ProfessionalsTeamIndividual
Service-based Middleware
CommunicationCommunicationMonitoringMonitoring
Capability/Profile
Management
Capability/Profile
Management
Provisioning/Negotiation/Execution APIProvisioning/Negotiation/Execution API
Abstraction of Human-based Compute Units
SCU
SC
U
SC
U SCU SCU
Program languages and programming models
Program
elements
Software
Compute
Units
program human actions
and dependencies
program incentive condition
and rewarding action
program result evaluation
method
Human-to-middleware
interfaces:
•visualization of collective tasks
•embedding of common forms
•mobile app
29ASE Summer 2014

Cloud of hybrid service units
Cloud of HBS: A cloud of HBS includes HBS that can be
provisioned, deployed, and utilized on-demand based on
different pricing and incentive models.
Cloud of HBS: A cloud of HBS includes HBS that can be
provisioned, deployed, and utilized on-demand based on
different pricing and incentive models.
Cloud of hybrid services: A cloud of hybrid services
includes SBS and HBS that can be provisioned, deployed
and utilized on-demand based on different pricing and
incentive models.
Cloud of hybrid services: A cloud of hybrid services
includes SBS and HBS that can be provisioned, deployed
and utilized on-demand based on different pricing and
incentive models.
Sevilla

SCU for independent tasks
Application /
Workflow
SCU
Provisoning Middleware
Human-Task
Requests
ICU Clouds
● Task-based crowdsourcing platforms
● Collections of experts on SN
● Enterprise ICU pools
Consumer
Problems
- Complex tasks
- Quality control
- Flexible quality requirements
Problems
- Complex tasks
- Quality control
- Flexible quality requirements
SCU
ICU
Human-Task
Requests
- ICU: individuals
- SCU: a collective of
collaborative individuals
- ICU: individuals
- SCU: a collective of
collaborative individuals

Elastic SCU provisioning atop ICUs
Elastic profile
SCU (pre-)runtime/static formation
Cloud APIs
Muhammad Z.C. Candra, Hong-Linh Truong, and Schahram
Dustdar, Provisioning Quality-aware Social Compute Units in
the Cloud, ICSOC 2013.
Muhammad Z.C. Candra, Hong-Linh Truong, and Schahram
Dustdar, Provisioning Quality-aware Social Compute Units in
the Cloud, ICSOC 2013.
Algorithms
 Ant Colony
Optimization
variants
 FCFS
 Greedy
SCU
extension/reduction
 Task reassignment
based on trust, cost,
availability
Mirela Riveni, Hong-Linh Truong, and Schahram
Dustdar, On the Elasticity of Social Compute Units,
CAISE 2014
Mirela Riveni, Hong-Linh Truong, and Schahram
Dustdar, On the Elasticity of Social Compute Units,
CAISE 2014

Selecting human units
 Do not select at all
 Let human units bid the tasks
 E.g., in crowdsourcing platforms
 Static/fix mapping
 E.g., using static information for human-task mapping
 Simple selection techniques
 Using the requirement of the task to find the suitable
human units based on their capabilities
 Complex selection techniques
 Utilizing complex dependency graphs to find suitable
human units
ASE Summer 2014 33

Selecting SCU based on task
graphs
34
SCU Formation
Algorithms
Business As
Usual
Corrective
Action
HBS
Constraints
Hong Linh Truong, Schahram Dustdar, Kamal Bhattacharya: Programming Hybrid Services in the Cloud. ICSOC 2012: 96-110Hong Linh Truong, Schahram Dustdar, Kamal Bhattacharya: Programming Hybrid Services in the Cloud. ICSOC 2012: 96-110
ASE Summer 2014

Utilizing hybrid services for
evolving/dependent task graphs
Hong-Linh Truong, Schahram
Dustdar, Kamal Bhattacharya
"Programming Hybrid Services
in the Cloud", 10th International
Conference on Service-oriented
Computing (ICSOC 2012),
November 12-16, 2012, Shanghai,
China. Best Paper Award.
Hong-Linh Truong, Schahram
Dustdar, Kamal Bhattacharya
"Programming Hybrid Services
in the Cloud", 10th International
Conference on Service-oriented
Computing (ICSOC 2012),
November 12-16, 2012, Shanghai,
China. Best Paper Award.

Placement techniques for human
units
 Usually at design time the developer/designer decides
 Where to put human units
 Where some triggers should be put in order to invoke
human units if needed
 At runtime
 Find suitable human units
 Invoke human units
 Placement of human units
 Application-specific
 Needs automatic algorithms and supporting tools
ASE Summer 2014 36

Configuring iSCU
 Establish „connectedness“ based on compliance
constraints and network topology
 Addional cost might occur!
 Program SBS and HBS for the iSCU to have a
complete working environment.
 Different connectedness
 E.g., ring-based, star-based, and master-slave
topologies

Hybrid compute units
38
Hybrid compute unit (HCU): a set of service units
includes software-based services, human-based
services and things-based services that can be
provisioned, deployed and utilized as a collective
on-demand based on different quality, pricing and
incentive models.
Hybrid compute unit (HCU): a set of service units
includes software-based services, human-based
services and things-based services that can be
provisioned, deployed and utilized as a collective
on-demand based on different quality, pricing and
incentive models.
ASE Summer 2014

Hybrid compute unit design –
fundamental elements
39ASE Summer 2014
Hong-Linh Truong, Hoa Khanh Dam, Aditya Ghose, Schahram Dustdar "Augmenting Complex Problem Solving with
Hybrid Compute Units",9th International Workshop on Engineering Service-Oriented Application (WESOA's 2013), In
conjunction with ICSOC 2013, Dec 2, 2013, Berlin, Germany, (c)Springer-Verlag
Hong-Linh Truong, Hoa Khanh Dam, Aditya Ghose, Schahram Dustdar "Augmenting Complex Problem Solving with
Hybrid Compute Units",9th International Workshop on Engineering Service-Oriented Application (WESOA's 2013), In
conjunction with ICSOC 2013, Dec 2, 2013, Berlin, Germany, (c)Springer-Verlag

Hybrid compute unit design --
Relationships
40
Relationship Type HBS SBS TBS HCU
Similarity Yes Yes Yes Yes
Composition Yes Yes Yes Yes
Data Dependency Yes Yes Yes Yes
Control Dependency Yes Yes Yes Yes
Location Dependency Yes Yes Yes Yes
Forwarding Yes Yes No Yes
Delegation Yes Yes No Yes
Social Relation Yes No No Yes
Elasticity Yes Yes No Yes
ASE Summer 2014

Towards programming framework
for HCU
41
Coordination and
Composition Models
Coordination and
Composition Models
[ICSOC12]
ASE Summer 2014

Illustrating Example
 Evaluating quality of data (QoD) in
simulations/data analytics processes
 Typical components in simulation workflows/data
analytics
 Goal using hCAS to support quality of data
control of the simulations/data analytics
 Reasons: reducing wasting time and computational
cost, dealing with tacit quality evaluation, etc.
42
pre-
processing
solving
post-
processing
data
analysis
ASE Summer 2014

HCU and simulations
43
 Only SBS is needed,
 for example, in the QoD evaluation step
before pre-processing
 SBS or HBS is used interchangeably or in
combination
 for example, in the QoD evaluation after pre-
processing,
 Only HBS is used
 e.g., in QoD evaluation after solving.
ASE Summer 2014

hCAS and simulation
workflows/data analytics
44
HCU
ASE Summer 2014

HOW DO WE EVALUATE
HUMAN ACTIVITIES IN DATA
ANALYTICS
ASE Summer 2014 45

Turkalytics
 Develop „Interaction
Model“ for human
activities
 Monitor and analyze
metrics, e.g.,
performance and
location
ASE Summer 2014 46
Paul Heymann and Hector Garcia-Molina. 2011. Turkalytics: analytics for human computation. In Proceedings of the 20th international
conference on World wide web (WWW '11). ACM, New York, NY, USA, 477-486. DOI=10.1145/1963405.1963473
http://doi.acm.org/10.1145/1963405.1963473
http://doi.acm.org/10.1145/1963405.1963473

Turkalytics
 Track client
detail at client
side (in browser)
 Store tracked
information in a
log server
 Analyze events
ASE Summer 2014 47
http://doi.acm.org/10.1145/1963405.1963473
http://doi.acm.org/10.1145/1963405.1963473

VieCOM
 SCU execution model and lifecycle
management
 Metrics for ICUs and SCUs
Mirela Riveni, Hong-Linh Truong, and Schahram Dustdar, On the Elasticity of Social Compute
Units, CAISE 2014
Mirela Riveni, Hong-Linh Truong, and Schahram Dustdar, On the Elasticity of Social Compute
Units, CAISE 2014

Exercises
 Read mentioned papers
 Analyze pros and cons of existing frameworks
for data analytics
 Survey existing algorithms for matching human
units to data analytics tasks
 Examine requirements for locating places for
human units and implement some algorithms
 Examine monitoring techniques for cloud of
human compute units
ASE Summer 2014 49

50
Thanks for
your attention
Hong-Linh Truong
Distributed Systems Group
Vienna University of Technology
truong@dsg.tuwien.ac.at
http://dsg.tuwien.ac.at/staff/truong
ASE Summer 2014

Engineering human-based services in complex processes

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Engineering human-based services in complex processes

Semelhante a Engineering human-based services in complex processes (20)

Mais de Hong-Linh Truong

Mais de Hong-Linh Truong (20)

Último

Último (20)

Engineering human-based services in complex processes