According to recent research report by Wall Street Journal, AI project failure rates near 50%, more than 53% terminates at proof of concept level and does not make it to production. Gartner report says that nearly 80% of the analytics projects are not delivering any business value. That means for every 10 projects, only 2 projects are useful to the organization. Let us pause here a moment, rather than looking at what makes AI projects to fail, let’s look at the challenges involved in AI projects and find a solution to overcome these challenges.
AI projects are different from traditional software projects. Typical software projects, as shown in Figure 1, consist of well-defined software requirements, high level design, coding, unit testing, system testing, and deployment along with beta testing or field testing. Now, organizations are adopting Agile process instead of traditional V or waterfall model, but still steps mentioned are valid.
However, AI and Machine Learning projects’ methodology is different from the above. Our experience working on many AI/ML projects has given us insights on some of the challenges of executing AI projects. Also, we are in regular touch with senior executives and thought leaders from different industries who understand the success formula. The following discussion is based on our practical experience and knowledge gained in the field.
Successful execution of AI projects depends on the following factors:
1. Clearly aligned Business Expectations
2. Clarity on Terminologies
3. Meeting Data Requirements
4. Tools and Technology
5. Right Resources
6. Understanding Output Results
7. Project Planning and the Process
Best VIP Call Girls Noida Sector 22 Call Me: 8448380779
Challenges of Executing AI
1. N-U Sigma U2 Analytics Lab web: www.businessanalyticsr.com email: umesh@mytechnospeak.com Ph: +1 408 757 0093
Umesh Rao Hodeghatta, Ph.D
Chief Data Scientist
Challenges of Executing AI &
Machine Learning Projects
10/18/2020
2. Outline
AI Project Success Reports
AI and ML Project Execution Challenges
AI Project Vs Software Projects
Best Practices For Executing AI Projects
Q & A
2
3. Brief Introduction about me
I have more than 20 years of experience
I have worked for AT&T Bell Labs, Cisco Systems, McAfee, Wipro
I have provided AI and ML solutions to some of the leading retail
companies, Human Resource company and healthcare company.
I have authored Books titled “Business Analytics Using R and “The
InfoSec Handbook: An Information Security”, published by
Springer Apress.
I have taught at Walden University, Kent State University, Xavier
University (XIMB).
3
4. AI Project Statistics
AI project failure rates near 50%,
More than 53% terminates at proof of concept level and
does not make it to production
Gartner report says that nearly 80% of the analytics projects
are not delivering any business value
4
5. AI Project Execution Challenges
Business Expectations
Data
Tools and Technology
Resources
Results and Outcome
Project Planning and the Process
5
7. Leverage AI
Deployment Timeline
Quick deployment
Smooth Integration
Cybersecurity
Value Add
% development efforts
Risks
Agile
7
“What percentage of business decisions are we
making with help from AI?”
10. Tools and Technology
Simple Analytics and Data Visualization
Data Plumbing
Data cleaning
Data cleaning utilities
Data Preparation
Reading pdf files and search certain key words?
Data scaling utilities
Feature selection
AI and ML Predictive Analytics
Image recognition
Recommendation Engine
Sentiment Analysis
10
11. Resources
Important roles (not in the order of
importance)
Data Engineer
Data Analyst
Machine Learning
Data Scientist
Software Programer/Coder Programs the solution as per the direction of the Data Scientist
Data Analyst analyses the data using data analysis tools and techniques like
MSBI, Tableau, etc. and finds out patterns and what the data broadly suggests
Data Plumbing Engineer: Capturing the data, collating the data, cleaning up
data, etc.
11
12. Output Results
Wrong interpretation of the results
• Misguided Decisions
• Few false positives or few false negatives can seriously undermine the use of the model
in spite of high accuracy
Selecting Metrics
• Different algorithms have different but relevant metrics
Overfit models may not be relied upon as they throw up different
results on the unseen / new data.
Trust and Ethics
12
15. AI Machine Learning Project Process
ASSESS
DATA MODEL
Deploy
Science
Initiation
• Requirements
• Aligning to Business
• Data Availability
• Quality Of Data
• Data vis-à-vis Business
Requirement
• Data Collection/Collation & Cleaning
• Descriptive
• Predictive
• Prescriptive
• NLP, Deep Learning
• Ph.Ds and Coders
• Experiments
• Measure & Validate
• Vary Parameters
• Re-validate
• Errors
• Deploy
• Verify
• Calibrate
15
16. E X E C U T I O N O F A I A N D M L P R O J E C T S
Best Practices
17. AI is Science!!
Clarity on the data
Label Data
Distinguish supervised or unsupervised ML
Composition of the classes/categories
Do Not Directly jump into the model building
Coding without science
Understand the features and their contribution to the
model
Business Objectives, Requirements, Features and
Data
17
18. Developing AI Solutions
Sufficient and reliable training data
Sample size or Population?
Training data cover all the relevant and possible results?
Is the training data in tune with the basic underlying fundamentals of the field under
study?
Validating Algorithms
validations carried out using sound principles of validation?
Results available?
Results are sufficiently challenged to bring out the weaknesses if any?
Transparent process of generating the model?
The process worked in tune with the requirements to get the requisite confidence in
the results?
No Secrets
18
19. Project Planning
Agile Stories
Scrum Meetings
Sprints
Weekly discussions
Weekly updates
Code reviews or process
Demos
Discussion with Business
Guide Business and Team
19
20. Summary
W.r.t. Data science project, you don’t know if it’s going to work
Data science requires an experimental process that allows for
uncertainty
Businesses and all the resources involved need to clearly
understand the roles and responsibilities of various stakeholders
Transparency is critical for the success
20
21. References
21
Hodeghatta, U. R., & Nayak, U. (2016). Business analytics using R-a
practical approach. Apress.
https://www.wsj.com/articles/ai-project-failure-rates-near-50-but-it-doesnt-
have-to-be-that-way-say-experts-11596810601
https://www.forbes.com/sites/gilpress/2019/07/19/this-week-in-ai-stats-up-to-50-
failure-rate-in-25-of-enterprises-deploying-ai/#34ea003272ce;
https://www.forbes.com/sites/gilpress/2020/01/13/ai-stats-news-only-146-of-firms-
have-deployed-ai-capabilities-in-production/#4da0ee526500
22. THANK YOU
E M A I L : U M E S H @ B U S I N E S S A N A L Y T I C S R . C O M
: U M E S H @ M Y T E C H N O S P E A K . C O M
W E B : H T T P : / / W W W . M Y T E C H N O S P E A K . C O M
P H : + 1 4 0 8 7 5 7 0 0 9 3
Q & A