This document discusses project management and big data analytics. It covers two main topics: 1) Project management of analytics and analytics of project management. It discusses the roles of data scientists and project managers. It also addresses common myths and facts about big data/analytics projects and project management. Key reasons for big data project failure include unclear objectives, lack of talent, wrong tool selection, poor planning, and ownership issues. The document emphasizes getting a data scientist involved, properly defining business objectives, using appropriate tools and methodologies, and accepting requirements volatility.
2. INTRODUCTION
IT Strategy & Business Transformation
Industries:
Media & Advertising, Telecom, BFSI,
FMCG, Manufacturing, BPO/KPO
Services:
Shared Service Delivery, GIC & Back
Office, PMO, Lean Six Sigma, Continuous
Improvement, Enterprise IT, ERP, Cloud &
Infrastructure, Development, Outsourcing
and IT Security & Governance
3. TWO STREAMS
Project Management of
Analytics
• Big Data
• Data Warehousing
• Lean Six Sigma
• In-memory computing
• Internet of things
• Social Media
Analytics of Project
Management
• Time-Cost-Spec Analytics
• Feasibility Analytics
• Resource Analytics
• Management of Collaboration
• Agile & SCRUM
4. APOLOGIES, DISCLAIMERS, ET AL
• Big Data is over-hyped
• Big Data is still evolving
• Analytics is old, the tools are new!
• Project Management solves most of the problems
• Its importance is usually understated
• The success of Big Data initiative lies primarily on the management, then
on the PM & the DS
• Hold the PM responsible only if you know what you want!
• The roles I talk about here are essentially with respect Big Data Projects
If you have
Questions,
I will try &
Answer
them!
5. UNDER SCANNER !
Big Data / Analytics
Myths
• It is mature and cool
• Is an extension of EDW
• Data Quality can be slightly compromised
• A single pre-built technology (e.g. Hadoop) will suffice
• Data scientists are easy to get
• Virtualization / Clustering will take care of infra needs
• If you have huge data, every solution is a Big Data solution
Project Management
Myths
• Managing only activities
• Just time and cost management
• Mere resource allocation
• Reaching the finishing-line!
• General management suffices
• Have time to learn
6. UNDER SCANNER !
Big Data / Analytics
Facts
• Responsible for Business Case and ROI
definitions
• Executive Sponsorship & Funds
• ‘Real’ Resource Provisioning
• Based on Enterprise Architecture
• Highly complex and iterative process
• Loads of scientific knowledge required
• Source of data increases every day
• Should be able to adapt with time
Project Management
Facts
• Responsible for Scope & acceptance by all
parties
• Direction setting & KPI-SF definitions
• ‘Real’ Resource Management
• Right to procure & deploy the appropriate
resource
• Stakeholder & Communication management
• Accountability and Responsibility for the success
(and failure)
7. THE DATA SCIENTIST
Key skills of a Data Scientist – the hard skills guy
• Basic Tools: Knowledge of statistical programming languages, like R or Python, and SQL
• Basic Statistics: Familiar with statistical tests, distributions, maximum likelihood estimators, etc.
• ETL Tools: Best in class like Informatica, IBM Infosphere, SAP BO, Oracle or SAS Data Integrator, Penta-ho, AB-Initio
• Machine Learning / Artificial Intelligence / Pattern Recognition: Methods for Classification and Regression like k-nearest
neighbours, random forests, etc.
• Multivariable Calculus & Linear Algebra: Specially required where data is used for predictive performance or algorithm
optimization
• Data Munging / Scrubbing or Cleanliness: For example inconsistent string formatting as ND or Del for New Delhi; date
alignment as [mm-dd-yyyy] or [dd-mm-yyyy] or [yyyy-dd-mm]
• Data Visualization & Communication tools: Principles of and tools of Data Visualization like ggplot and d3.js.
• Software Engineering: Strong software engineering background, SDLC, Agile, Scrum, DB techniques, Data intensive product development
• Software Testing skills – To make sure the output delivers what the business needs
• Basic Project Management Skills: Thinking like a Project Manager
8. THE PROJECT MANAGER
Key skills of a Project Manager – the soft skills guy
• Project Charter: Project Stakeholders and Objectives documented and signed-off
• Business Case: Asks the ‘whats’ and ‘whys’ of the business requirement
• Scheduling Tools: Creates a Plan of Action to answer the ‘hows’ of the project
• Vendor Management: Links up all 1st and 3rd party resources
• Risk Management: The real management tool, with the mitigant
• Communication Management: The core of collaboration and Management
• Software Engineering: Software engineering background with fair knowledge of tools
• Software Testing skills: To make sure the output delivers what the business needs
• Basic Data Management Skills: Thinking like a Data Scientist
9. MERGING ROLES
The Data Scientist
The Requirements guy
The Data Tools guy
The Resource guy
The Specialist
The Enterprise Architect
The Software guy
Breaking the Technical Barriers
The Project Manager
The Requirements & Scope guy
The Project Tools guy
The Resource guy
The Generalist
The Program Manager
The hardware & Software guy
Breaking the Cultural Barriers
10. WHY DO ANALYTICS
PROJECTS FAIL ?
When do Projects fail, in general?
• Not completed within budgets
• Not completed on time
• Not completed as per specifications
Whys and Wherefores…
• Poor scoping – unclear objective
• Inadequate resources – lack of talent
• Inappropriate Solution – wrong tool selection
• Bad planning – Insufficient analysis
• Bad execution – poor Project Management
“3 out of 4 Big Data Projects fail”
• Inaccurate Project Scope
• Lack of Talent
• Challenging Tools
• Even more Challenging Concepts
• Poor Planning
• Ownership Issues – Business
Initiative or IT Project?
Apache Hadoop vs. Apache Cassandra
55% of Big Data projects don’t get completed; in case of IT projects in general, it is only 25%.
11. THE SUCCESS MANTRA!
• Get a Data Scientist at the PM or SME
• Or, at least get the Data Scientist as one of the leads
• Ask very tough questions to sponsors for a Business Case
• Get the CFO and the End-User on your side – get the expectations right
• Take your time – use appropriate Project Management methodologies
• Use the most appropriate Platform / Tool
• Accept requirements’ volatility
• Scrum: accepting that the problem cannot be fully understood or defined, focusing instead on maximizing the team's ability to deliver quickly
• POC Deployment Acceptance Enhancement Deployment … …
• Agile: adaptive planning, evolutionary development, early delivery, continuous improvement
• Document and Handover to End-User at every stage
PM answers “How” as long as the Business knows “Why & What”