2. Agenda
Truck Roll reduction AI Use case
Continuous Development and Deployment of AI Model
Data Scientist
Driverless AI
GAIA
Demo
3. Opportunity
Identified about 14% of truck rolls were resulted ‘No Fault
found’, in the system over 1 year.
Minimum of 5% of these TRs can be avoided
Annual potential savings of about $ 10M
• From existing dataset performed n-gram analysis on technician
comments to identify meaningful sentences like -
• “no fault found”, “clear dial tone”, “localized high workload”, “line
ok lead”, “line tested ok”, “fast test pass” etc.
• From testing and diagnosis dataset identify first round of test
results where successful
Data Analysis Approach 2
“COMPLETED no fault found on network working on the pillar line ok tested at customer
premise fast test ok”
“Complete nff tested with customer line sounds clear customer advised issue only
happens when raining intermittent issue”
“COMPLETED no fault found in network customer is 4 8 km from exchange and getting 3
85 mbps speed FFS”
“COMPLETED dial tone at house pair gain system working no fault found ”
Sample Customer Technician Closure Comments
Test &
Diagnosis
Technician
Comments
Network
Health &
Alarms
Remotely
closable
Tickets
Network
Topology
Truck Roll reduction Use case :
Remotely Closable Tickets
1
3
4
Customer
Profile
Agent
Profile
5. Data Preparation :
Candidate Input Data Sources for TR Prediction
Input to AI/ML
Model
Ticket ProfileCustomer
Customer Profile
(N/W)
Agent Behavior
Test & Diagnostics
Network Health
6. Training the Model :
Machine Learning Automation - H2O Driverless AI
Driverless AI
Expert Data Scientist in a Box
Delivers Insights and Interpretability
Automatic feature engineering
Automatic scoring pipeline
Automatic Visualization
Flexibility of data & deployment
NLP with Tensorflow
No code AI/ML
7. Training the Model :
Truck Roll Prediction Demo
Domain: Service Assurance
Scope: Predict avoidable Truck Roll vs Valid Truck Rolls.
Driverless AI
1 Month
Tickets data
Test Data
Train Data
Compare Predicted Vs
Actual Values
8. AI Development
Service and tools
Executable
Predictor
Data Sources Training
Dataset
Training / Testing
Lifecycle
Runtime Systems
Create & On-Board Models Execute InTarget Environment
Local Learning
Train
Deploy
Model
Docker
Images
Onboard
Publish
Review
Search
Chaining
Sharing Models In Marketplace
Rating
AI Platform &
Marketplace
Enhancing ModelWith Application Data Sets
Continuous
Learning
Design Studio
Construct chained AI
applications
GAIA : Platform Overview
9. Demo: Output Summary (sample)
Actual vs Predicted Matrix – Test Dataset (48k)
Summary:
• ML model used is XGBoostModel by H20.ai
• Training dataset contains historical data
• Test dataset used to validate
• The accuracy (test dataset) is : ~92 % (higher is better)
• Driverless AI performed the various steps(ID-SP) to find the optimal final
model, depicted in picture
• Driverless AI has come up with a final model in 32 iterations.
Steps performed to reach to final Model
Predicted :
No Truck Roll
Predicted :
Truck Roll
Actual :
No Truck Roll 93% 7%
Actual :
Truck Roll 9% 91%