[DSC Europe 22] Business Value through XAI - Ozgur Akarsu

1. SEPTEMBER 2022 Business Value through XAI Explainability Usecases in Real World AI Solutions
2. 1 Gizlilik Sınıflandırması : HİZMETE ÖZEL Ozgur Akarsu is the AI & Data Analytics Group Manager at KocDigital. He graduated from ITU Industrial Engineering and finished his Ph.D. in Organizational Studies in 2016. Ozgur has more than 18 years of experience in digital transformation and advanced analytics in various companies. He is also teaching business analytics and value chain management classes at Istanbul Bilgi University. Özgür Akarsu AI & Data Analytics Group Manager KoçDigital ozgur.akarsu@kocdigital.com
3. 2 Objectives for Today • Introduce KoçDigital • Business Usecase: Demand Forecasting in Retail • Our XAI Approach: Usecase for retail
5. 4 Gizlilik Sınıflandırması : HİZMETE ÖZEL KoçDigital is a world class Digital Center, combining strengths of BCG and Koç • Digital capabilities and infrastructure • Investment for future capability build (e.g. people, infrastructure, products and tools) • Commercial relationships • World class capabilities in advanced data analytics (GAMMA) • Access to global network of industry and topic experts • State-of-the-art training and enablement Offerings KoçDigital Academy Advanced Analytics Data Platform Design & Development IoT Solutions
6. 5 Gizlilik Sınıflandırması : HİZMETE ÖZEL 6 COUNTRIES KoçDigitalProjects&ProductsOverview Significant capability built in four years since inception IN SALES 200M+ TRL 2-9 MONTHS PAY BACK PERIOD 12 IoT ANDANALYTICS PRODUCTS 200+ PROJECTS 220+ EMPLOYEES
7. 6 Objectives for Today • Introduce KoçDigital • Business Usecase: Demand Planning in Retail • Our XAI Approach: Usecase for retail
8. 7 Gizlilik Sınıflandırması : HİZMETE ÖZEL Demand Planning & Auto Replenishment engine was built based on three key components What will be right inventory level in stores/WHS? What will be Daily orders from WHS/supplier? How will be optimised inventory level in stores with transfer algorithm? What will be the demand in short / long term period? Demand Forecasting • Segmented product category approach • Tailored forecasting model • Demand sensing on complex data • Model Integration Auto Replenishment & Inventory Planning • Parameters Setting (with UI) • Constraints & Business Rules integration • Statistical Inventory Control • Ordering Mechanism Optimization • KPI (such as lost sales, in-stock rate etc.) monitoring S2S Transfer Optimization • Transfer scenario selection (with UI) • Constraints & Business rules integration or selection (wtih UI) • Profit & logistics cost optimization • Transfer sales conversion monitoring
9. 8 Gizlilik Sınıflandırması : HİZMETE ÖZEL A Machine Learning based Forecasting Model to sense the market demand Accurate predictions using Machine Learning algorithms Scalable for wide range of SKUs and channels Sensing the market by wide range of internal and external variables 1 2 3
10. 9 Gizlilik Sınıflandırması : HİZMETE ÖZEL We achieved accurate forecast results for an extremely complex product-channel portfolio 262 Categories 5 Channels 120 Stores Weekly forecast 4 weeks + 20 weeks horizon Store-location level Lightgbm for short-term forecast (4 weeks) Fbprophet for 20 weeks Error calculation (WAPE) for category and store level Scope Forecast Unit Accuracy 70K Sku Coverage 44K Weekly forecast 74% Average Accuracy* *Results for test period for last 6 months 200+ Tailored Features Historic sales Campaigns-Price Stock Special Events Calendar Covid Features Currency rates Macro-economic indicators …. Features
11. 10 «How do external factors affect sales?» «What is the effect of campaigns for model’s outputs in different locations?» «Model failed to predict air conditioner sales in May. Why???» «Why are the models’ predictions lower than actuals for location X?» Erosion of trust for model outputs Resistance to adopting data-driven decision making
12. 11 3 Major Challenges in practice Unexpected weekly deviations between predictions and actuals Shortfall of the model for some products or stores. 1 Difficulty to explain effect of +100 model features by analyzing the feature importance 2 3
14. 13 Explainable AI Enable accountability Accelerate AI adoption Ensure ethics & regulatory compliance Provide strategic insights Our XAI value proposal includes not only better and fair models, but better business outcomes as well
15. 14 Gizlilik Sınıflandırması : HİZMETE ÖZEL Our Explainable AI Intelligence Approach- XAI Explaining the blackbox models High accuracy whitebox models Local Explanations Counterfactuals Linear Models Trees & Rules Model Agnostic Explanations Interpretable Models Explain each prediction locally by using a local model that approximates the predictive model Set of features that should be changed in order to flip a model’s prediction Algorithms for extracting or generating rules by using linear programming with ML Learning with subset stacking (LESS) algorithm tuned with lasso
16. 15 Gizlilik Sınıflandırması : HİZMETE ÖZEL XAI for Retail: Simple Explanations to Complex Problems Global Local Complex Non-linear Simple Linear Shapley Values Local Explanations XAI Dashboard for SCM Interpretable Linear Ensembles Lasso (tuned) Learning with subset stacking (LESS) MSE: 296.42 MSE: 315.51 Only with 20 Features LightGBM (tuned) MSE: 293.59
17. 16 Gizlilik Sınıflandırması : HİZMETE ÖZEL
20. 19 Outcomes The client SCM team was able to interpret the weekly model outputs and increased decision- making quality for stock orders Strategic insights for product and campaign management • Proven effect of changes in air temperature on AC sales for different regions. • Visibility of how campaigns effect sales in different regions and products. Target increase product availability in stores, automated order rate, and %10 decrease in warehouse costs. XAI as a business enabler and analytics accelerator!
21. 20 Next Steps  Monitor critical SCM KPIs and apply necessary revisions.  Apply counterfactual explanations (CFEs) and rule generator (RUG) and rule extractor (RUX) modules  Transform XAI solution to a model and domain agnostic product
22. Thank you Özgür Akarsu ozgur.akarsu@kocdigital.com

Notas do Editor

KocDgital was founded in 2018 with the cooperation of KocGrup and BCG. We are a technology company developing AA and IOT products and serving end-to-end AI solutions for various industries.
In these 4 years we have successfully completed 200+ projects in 6 different countries. Our product portfolio contains 12 unique products. Our greatest strength is our 220 employees who are specialized in different roles of advanced analytics and AI development cycles.
Last year we finished a Supply Chain digital transformation Project with Koçtaş which is the leading retail company in Turkey.
The Project consist of 3 modules. Demand forecasting was the heart and beginning point whose outputs was used for AutoReplenishment and S2s Transfer optimization
In this Koçtaş Project we built a demand forecasting pipeline based on Machine Learning using a wide range of internal and external variables to ensure accurate forecast especially for the short-term
Our solutions was scalable for wide range of SKUs, channels and product categories.
At the same time it was flexible to include internal and external data which can explain the fluctations in demand.
The Project covered 70K Skus, 5 channels and 120 sales point
Every week it is producing forecasts for 44K units
Using 200+ features, which include internal sales Dynamic +external data such as covid indexes, currency rates and macro-economic indicators such as currency, inflation or interest rate.

The model gave amazing results. Our accuracy score for Lightgbm ML model %74 on average..

The solution was deployed on MS Azure and SCM team started to monitör the outputs and determine the orders
During the incubation period of the Project we started to receive some feedback and questions from the field. To use the outputs of the model in practice, they need to deep dive to the model outputs. And the questions rose.
Our consultants and data Scientist working closely with the client team started to conduct ad hoc analysis to answer each question which is time consuming.
And we identified there is huge risk for erosion of trust for model outputs, that if people are not convinced with how model behaves and predicts, they have a tendency to work as the way they are used to instead of transforming processes using AI.
During the incubation period we saw that the client teams has difficulty to intrepret the affect of features on forecasts.

They was a certain need for explain unexpected weekly deviations for forecasts.
And for some products or stores model was constantly giving lower or higher forecasts than the actuals. We have to analyze the reasons for underfitting.
Answers for those 3 questions was XAI . To accelerate AI adoption, enable accountability for AI Outputs, provide strategic insights for business and to handle the upcoming regulatory compliance requirement, XAI capabilities should be build and embeedded in AI solutions.
Our value proposal includes not only better and fair models but better businesss outcomes as well. XAI has huge potential for business owners to understand how AI systems work and behave.
Our XAI solution portfolio contains 4 dffrerent methods, local explanations, counterfactuals, linear models and trees and rules.
We applied 2 of them for in this koctas case.
Local explanations, transform highly non-linear decision boundaries to locally explained linear models by zooming in every single prediction. But if we zoom in and look at a single prediction, the behaviour of the model in that locality can be explained by a simple interpretable model (mostly linear).

Interpretable linear ensembles, we build several tuned linear models and compared the model power with our deployed model.
So to sum up we could say that XAI has a huge potential to transform organizations. For years our focus was creating more robust and powerful models but we believe that switcing the focus to explanability and interpretability can trigger and fasten the AI revolution.
As data Scientist our focus is business outcomes and XAI can help us to achieve these goals as much as complex and sophisticated algos.

[DSC Europe 22] Business Value through XAI - Ozgur Akarsu

Esté a ler uma amostra.

Confira estes a seguir

[DSC Europe 22] Business Value through XAI - Ozgur Akarsu

Mais Conteúdo rRelacionado

Semelhante a [DSC Europe 22] Business Value through XAI - Ozgur Akarsu (20)

Mais de DataScienceConferenc1 (20)

Mais recentes (20)

[DSC Europe 22] Business Value through XAI - Ozgur Akarsu

Notas do Editor

[DSC Europe 22] Business Value through XAI - Ozgur Akarsu

Esté a ler uma amostra.

[DSC Europe 22] Business Value through XAI - Ozgur Akarsu

Recomendadas

Mais Conteúdo rRelacionado

Semelhante a [DSC Europe 22] Business Value through XAI - Ozgur Akarsu (20)

Mais de DataScienceConferenc1 (20)

Mais recentes (20)

[DSC Europe 22] Business Value through XAI - Ozgur Akarsu

Notas do Editor