This is a presentation of research done within the EU Community project and its evaluation, combining reputation management and sentiment analysis techniques for policy modelling

1. Aggeliki Androutsopoulou, Francesco Mureddu,
Euripidis Loukis, Yannis Charalabidis
PASSIVE EXPERT-SOURCING FOR POLICY
MAKING IN THE EUROPEAN UNION
15th IFIP Electronic Government (EGOV) and
8th Electronic Participation (ePart) Conference 2016
5th - 8th September 2016 Guimarães, Portugal

2. Introduction
• Crowd-sourcing: a web-based business model that harnesses
the creative solutions of a distributed network of individuals’, in
order to exploit ‘collective wisdom’ and mine fresh ideas from
large numbers of individuals (Brabham, 2012)
• Citizen-sourcing: Application of the crowdsourcing ideas by
the public sector, the politicians and the citizens, during a policy
making process
• Further research required on public sector citizen-sourcing, to
develop a considerable knowledge base, comparable to private
sector crowd-sourcing

3. Introduction
• Citizen-sourcing initiatives provide valuable insights into the
perceptions of the general public, concerning important social
problems and government activities for addressing them
• Targeting to more knowledgeable communities having strong
interest and good expertise on the particular topic/policy under
discussion to collect information and knowledge of higher
quality

4. Public Sector Citizen-Sourcing
• Active citizen-sourcing: use of government agencies’
web-sites or social media accounts, in order to pose a
particular social problem or public policy direction, and
solicit relevant information, knowledge, opinions and
ideas from citizens.
• Challenge.gov initiative, U.S. Office of Management and Budget
• The PADGETS project, EU
• Passive citizen-sourcing: exploit political content that
has been developed by citizens freely, without any direct
stimulation or direction by government, in various external
web-sites or social media
• The NOMAD project, EU

5. Democracy vs Technocracy in Public Policy Making
• Increased complexity of social problems leads to the
establishment of expert bodies to support policy formulation
• Two fundamental bases of public policy making:
• Democracy - political consultation with stakeholder groups (diverse
values and concerns, perspectives and ideologies )
• Technocracy - knowledge of experts
• Necessity of combined and balanced input and knowledge
exchange between the two, which, luckily, can be supported
by ICT

6. Research Objectives
• ICT-based method for ‘passive expert-sourcing’, which allows the
collection of high quality policy relevant information, knowledge and
ideas from knowledgeable experts with the in order to support policy
making by the EU by leveraging its large policy community, based on:
• EU policy experts’ profiling and reputation management,
• relevant documents’ opinion mining and relevance rating,
• Several means of visualisation
• Theoretical foundation:
• relationships between democracy and technocracy
• policy networks in public policy making

7. Policy Networks
• Sets of formal and informal institutional linkages between
governmental actors and non-government actors associations,
professions, labor unions and interest groups) around shared
interests in public policy-making
• Non-state actors provide to state actors information,
knowledge, expertise and support for the formulation and
implementation of public policies. In return they have the
opportunity to influence the public policies
• ICT increases density of policy networks interactions and
supports the exchange of diverse expertise and knowledge
among participants

8. Design and Evaluation Methodology
13 workshops (EurActiv)
• EU policy stakeholders
and thematic experts
• Understanding the EU
policy community
structure
• Requirements
Elicitation
Evaluation Session
• Usage scenarios
execution
• Questionnaires on the
evaluation framework
• Qualitative discussion
on questions

9. Passive Expert-sourcing Method
• Automatic retrieval of information
from various sources concerning
policy experts and content generated
by them
• Opinion mining and sentiment
classification of content to identify
subjectivity, opinions and their
polarity, relevance to a policy topic
• Digital Reputation Management to
asses the credibility of experts
• Modelling of policy processes (EU
legislative procedures or complex
political debates)
• Interconnection and structured
visual presentation of all relevant
information

10. ICT Platform - EurActory
• Maintains a directory of profiles of people
having an active role in EU policy making
• Crawls at regular time intervals external
sources and SM and stores information in
CurActory DB
• Provides ranking of expert profiles per topic
through Reputation Management
• Provides capabilities of searching, filtering,
curation and activation of user profile

11. Digital Reputation Criteria
1. Self-evaluation: direct input from the user on his/her own area of
expertise.
2. Peer-assessment: based on endorsements from other users made
through EurActory
3. Business Card Reputation: based on the reputation ranking of the
organization and the user’s position in the organization’s hierarchy
4. Document Assessment: results of authored documents’
assessment by their readers
5. Network Value: level of influence as the sum of network
connections
6. Proximity trust: level of connection in social media
7. Past Measurements: taking into account reputation in previous
months (its stability means credibility).
8. Offline Reputation: manually added for persons with no online
presence

12. ICT Platform - PolicyLine
• Timeline visualisation of main
documents (based on
relevance and author’s
reputation) per policy process
• Clusters documents under user
defined stages of the policy
process
• Classifies documents per
authorship (sub)categories
• Provides sentiment
classification results and users’
feedback per document
• Provides statistical Information
for each policy process

13. Evaluation Framework
Technology
Acceptance
Model
Intention to
use
Ease of Use
Usefulness

14. Evaluation Results – EurActory
Ease of Use
EurActory can be easily used without assistance
Creating a profile is easy
It is easy to access topic listings
It is easy to rate peers
Using EurActory has been a positive experience
3.46
4.08
4.15
3.75
4.08
Usefulness
EurActory puts together information not found or collected under one roof elsewhere
EurActory allows me to be more productive
EurActory improves the quality of my work
EurActory assists me in identifying relevant experts
EurActory provides me with all the needed information on relevant experts
EurActory enables me to reinforce my expert positioning
3.15
3.38
3.46
3.85
3.54
3.54
Intention to Use
I expect to use EurActory on a regular basis in the future
I will advise colleagues to use EurActory
3.85
3.62

15. Evaluation Results – PolicyLine
Ease of Use
PolicyLine can be easily used without assistance
I can easily create a ‘policy process’
I can easily add a document in the ‘policy process’
I can easily rate/comment a document
I can easily get an overview of the process
Using PolicyLine has been a positive experience
3.64
3.69
3.79
3.5
3.73
3.71
Usefulness
PolicyLine puts together information not found or collected under one roof elsewhere
PolicyLine allows me to be more productive
PolicyLine improves the quality of my work
3.29
3.29
3.43
Intention to Use
I expect to use EurActory on a regular basis in the future
I will advise colleagues to use EurActory
4.14
3.71

16. Conclusions
• Passive expert-sourcing, combined with automated reputation
management and opinion mining can assist the public debate and
policy making.
• Two tools were developed: Euractory and PolicyLine, for finding high
quality information and opinions on important policy-related topics and
policy formulation processes in European Commission and European
Parliament
• The users (politicians, experts, citizens) had an initial positive
response towards the provided tools, in many different occasions.
• Promotes the debate and communication among EU policy
stakeholders, allowing expression of opinions and criticism on EU
policy initiatives
• Future improvements on the ICT platform concerning the graphical
interface and timeline visualisation are now being

17. Further Research
• Further evaluation on realistic pilot applications
• to what extent it enables and supports the transfer of information, knowledge and
proposals from experts to the participants in the democratic processes of modern
policy making, and under what conditions?
• to what extent it can enable and support the exchange of information, knowledge
and proposals among the participants in public policy networks, and under what
conditions ?
• to what extent can this method can assist the EU institutions to collect high quality
information, knowledge, opinions and proposals from their policy networks
• Further research concerning the reverse transfer of knowledge, from
the democratic process to experts, towards more multi-dimensional
comprehensive experts’ analysis and plans on social problems and
public policies

18. The research presented in this paper has been conducted as part of
the European research project ‘EU-Community’, partially funded
by the ‘ICT for Governance and Policy Modelling’ research
programme of the EU
http://project.eucommunity.eu/
Contact me at:
Email ; yannisx@aegean.gr
Twitter : @yannisc