The Digital Transformation Team in Italy developed a Data & Analytics Framework (DAF) to help public sector organizations make more data-driven decisions. DAF includes a big data platform to centralize, store, and analyze data. It also includes a data team of data scientists and engineers. The goals of DAF are to break down data silos, make more data open and accessible via APIs, and build data products and applications. DAF aims to support public policymaking and monitoring through centralized data infrastructure and analytics capabilities shared across organizations.

1. Data & Analytics Framework:
how public sector can profit
from its immense asset, data
RAFFAELE LILLO
Chief Data Officer
@ Digital Transformation Team
—

2. Who we are and what we do
• Digital Transformation Team
• Projects
Data & Analytics Framework (DAF)
• Goals
• Ok, but what is DAF?
• Projects
• Some Architectural Highlights
Q&A
—
DAF

3. Digital
Transformation
Team

4. The “operating system” of the
country: a series of
fundamental blocks upon
which services for citizens, the
Public Administration, and
enterprises are built with
modern digital products.
Vision

5. Make public services for citizens and
businesses accessible in an easy
manner, via a mobile first approach,
with reliable, scalable and fault
tolerant architectures, based on
clearly defined APIs; support the
different central and local
government departments in making
the best and most data driven
decisions, thanks to the adoption of
big data and machine learning
techniques.
Mission

6. —

7. ● DAF: Data & Analytics F.
● ANPR
● Security - Resp. Disclosure
● SPID
● PagoPA
● API Ecosystem
● E-Procurement
● Developer Community
● Digital Citizenship
● LexDatafication
● ...
Projects
● 7 "Comuni": Bari, Firenze,
Milano, Palermo, Roma, Torino,
Venezia
● Software House in-house:
Sogei, InfoCamere, ACI
Informatica, IPZS
● PAC: Agenzia Entrate, ISTAT,
MISE, MIUR, ANAC, CdC…
● PAL: Roma, Milano, Torino,
Firenze...
Partners

8. DAF: Data & Analytics
Framework

9. Data & Analytics Framework (DAF) is a combination of:
● A Big Data Platform to centralize and store (data lake),
manipulate and standardize (data engine), re-distribute (API
& Data Applications) data and insights.
● A Data Team (data scientists + data engineers) which uses
and evolve the Big Data Platform to analyze data, create ML
models and build data applications and data viz.
Give us data and a platform...

10. Interoperability (aka Get out of the Silos!)
Public data is… public and all PP.AA. should have access to it
Democratizing Data (aka Open Data, API & Data Viz)
Data should be open (when legally possible), accessible by anyone (and
anything) and insightful
Data Products (aka deliver value & insights)
Machine Learning in interconnected software applications
Crowdsourcing (aka data is everywhere, let’s help us out)
Citizens (esp. civic hackers) contribute to the surfacing of knowledge
… and we shall move the PA

11. Organizational and Managerial Challenge
Central Data Office and federated analytics teams
Human Resources
Data Scientists & Data Engineers to get knowledge from data
Technology
This is the least complicated one, but still fundamental.
Legislative Challenge
Balancing Privacy and Public Interest
Data Driven Policy needs… Data (and Data
Scientists)

12. Introduction of DAF in Piano Triennale 2017-2019
DAF is one of the building blocks of the official document setting the strategy for
digitalization of the PA, and signed by the Prime Minister
DAF prototype development
TD started the development of the platform from scratch around March ‘17, and
released an Alpha version the first week of October ‘17
Experimental phase
We started working with a selected number of PA to showcase DAF, test it and
listen to PA’s needs so to fine-tune the platform before final release
Institutionalization of DAF
Introduce by law the role of a central data office for the entire PA
Our Strategy

13. Mission: Data driven decision making in efficient ways
Support PA at all levels to implement informed policies, both ex ante (policy
formulation) and ex post (policy monitoring and fine tuning).
Centralize common & non-domain specific tasks
Provide general purpose data platform once and for all, efficiency in standard
data processes, let PA focus on domain specific tasks / analysis
Economy of scope towards a center of excellence
Reach proper dimension to develop and acquire expensive and idiosyncratic
capabilities, and share them with all PA
Design and coordinate implementation of Data Policies
Help interoperability and usage of state-of-the-art standards and processes in
data management and analysis. Stimulates research and collaboration.
End Goal: Chief Data Office for the PA

14. High-level Architectural Design
Data Ingestion
Persistence & Offline
analysis (analysis, data
processing, model training)
Real-Time data
processing
Operational DB for API and
Data applications
Communication layer - API

15. (Less) High-Level Architectural Design
Hadoop cluster for
distributed persistence and
processing
Kubernetes cluster
manages dockerized
microservices and external
applications.
Core Managers:
microservices managing
core functionalities of DAF
External applications
natively integrated in DAF
Unique identity
management system,
integrated with HDFS

16. Dataportal - Public Version
https://dataportal.daf.teamdigitale.it

17. Dataportal - Private Version
https://dataportal-private.daf.teamdigitale.it

18. Data Sources
People Firms Smart Cities
Fiscality
Pension
Education
Healthcare Cultural
events
Public
Transportatio
n
Corporate
events
Energy Traffic
Corporate
info
Balance
Sheet
Crime
Real estate
registry

19. Machine Learning Based Applications (aka Data Products)
Lex Datafication & Citizen Assistant, Fraud Detection, Citizen
Recommendation Engine, Spending Check, Leading Indicators, etc.
Data Visualization
Thematic dashboards and infographics for citizens and firms
API for Interoperability and Open Data
Easy and standard access to data within PP.AA. and citizens
And much more… The limit is imagination
Smart city, analysis for data driven policy making, etc.
Use Cases (examples)

20. It… Could… Work!

21. Raffaele Lillo
Chief Data Officer
raffaele@teamdigitale.governo.it
Twitter, Medium: @lilloraffa
—
Grazie!
Cooperate with us, please :)
Website
http://teamdigitale.governo.it
Forum
https://forum.italia.it/c/daf
Twitter
#DatiPubblici #DAF
Google Group - Open Data & Analytics
data@teamdigitale.governo.it