SigOpt's Fay Kallel, Head of Product, and Jim Blomo, Head of Engineering, describe the latest updates to SigOpt, a suite of features that help you manage your modeling process.

1. SigOpt. Confidential.
Fay Kallel
Head of Product
Jim Blomo
Head of Engineering

2. SigOpt. Confidential.
Accelerate and Amplify the Impact of
Modelers Everywhere

3. SigOpt. Confidential.
Productivity
75%
of enterprises build their
own AI models
Sources:
1) 75% of advanced enterprises develop their own models - O’Reilly Strata State of ML Survey
2) AI projects take 6 months and 2 in 3 of them fail - Databricks 2019 CIO Survey
3) 58% will put AI in prod by 2021 - Gartner 2019 CIO Survey | Gartner: Predicts 2020: Artificial Intelligence — the Road to Production
Performance
2 of 3
modeling projects fail to be
productionalized
Scale
58%
of teams will deploy 20+
models in production

4. “Companies lose patience and fail before they go to production
because they have no ability to understand where the real
issues are in the life cycle.”
Global Online Retailer

5. “ML organizations and engineering biggest problem is the
art of finding how offline metrics may fail and therefore what
can we do to better design it so the learning system can
perform and achieve our business metrics.”
Robotics Startup

6. “With DL and neural networks, HPO is absolutely necessary,
sometimes swapping a grid search for HPO is sufficient to drive
a 1% improvement and that is a game changer in some
problems.”
Cloud Services Provider

7. “We hired experts from Google, Amazon, Facebook, Microsoft and they
have diverse skills across tools, we can’t afford to standardize on one
stack so we need to make them productive by giving them the tools to
manage all of their models regardless of their modeling environment,
and that challenge took us a few years to begin solving”
Global Hospitality Services Provider

8. “we have decentralized organization and as a result, we have
many teams building models, leveraging each other’s work is
crucial yet very hard to do today.”
Global Enterprise Solution Provider

9. SigOpt. Confidential.
Problems
Solutions
Modeling is a siloed
artisana process
Track & Organize
Modeling is about
tradeoffs
Visualize & Analyse
Modeling is tough to
get right
Train, Tune &
Collaborate

10. SigOpt. Confidential.
SigOpt enabling MLOps in the Enterprise
Prepare
Data
Data
Repository
Train
Model
FEATURIZE TRAIN EVALUATE TUNE
Model
Registry
MODELING PIPELINE Data
Scientist
ML Engineer
/ DevOps
Release
Model
APPROVE VALIDATEDEPLOY
MONITOR
DEPLOYMENT PIPELINE
COLLECTData
Source
Data
Engineer

11. DEMO
Applying SigOpt to a fraud detection problem

12. Benefits: Explore, Understand, Advance
Realized by global consulting firm
with 3,000+ modelers
30% Productivity Boost
Realized by Two Sigma in
comparison to open source
8x Faster Experimentation
Custom visualization informs
intuition and explains progress
Interactively visualize learning
Seamless training to tuning to
reaching the best model
Train & Tune Any Model
Retrace and reproduce modeling
decisions with perfect recall
Transcribe modeling workflow

13. Technical Deep Dive

14. SigOpt. Confidential.
How it Works: Integrate with a few lines of code
Install SigOpt1
Instrument model2
Track runs3
Run optimization loop4
Analyze experiments5
Install SigOpt1

15. SigOpt. Confidential.
How it Works: Integrate with a few lines of code
Install SigOpt1
Instrument model2
Track runs3
Run optimization loop4
Analyze experiments5
Install SigOpt1

16. SigOpt. Confidential.
How it Works: Integrate with a few lines of code
Install SigOpt1
Instrument model2
Track runs3
Run optimization loop4
Analyze experiments5
Install SigOpt1

17. SigOpt. Confidential.
How it Works: Integrate with a few lines of code
Install SigOpt1
Instrument model2
Track runs3
Run optimization loop4
Analyze experiments5
Install SigOpt1

18. SigOpt. Confidential.
How it Works: Analyzing Results
Install SigOpt1
Instrument model2
Track runs3
Run optimization loop4
Analyze experiments5
Install SigOpt1

19. SigOpt. Confidential.
How it Works: Analyzing Results
Install SigOpt1
Instrument model2
Track runs3
Run optimization loop4
Analyze experiments5
Install SigOpt1

20. SigOpt. Confidential.
Get Free Beta
Access
sigopt.com/beta
Next Steps
Join our EDU
Program
sigopt.com/edu

21. “Thank you so much for your work on this, the cool factor is high, the
fact that you are hovering over a metric point and showing me how
that point is linked to all metrics and hyperparameters, that dashboard
is giving me in one view great intuition across two types of metrics, I
commend you for coming up with an interface like this.”
Beta tester from a Financial Insurance Provider

22. SigOpt. Confidential.
Thank you!
Fay Kallel - Head of Product
Jim Blomo - Head of Engineering

23. “Saved views after filtering is awesome! It solved my
current workaround where I write a script to pull from
hundreds of directories.”
Beta tester from the U.S. Federal Government

24. SigOpt. Confidential.24
What is “Optimize”?
get_parameter
Suggestion
log_metric
SigOpt

25. SigOpt. Confidential.
ML, DL or
Simulation Model
Model Evaluation or
Backtest
Testing
Data
Training
Data
Never
accesses your
data or models
Iterative, automated optimization
Built specifically
for scalable
enterprise use
cases
25
What is “Optimize”?
REST API

26. SigOpt. Confidential.
API documentation available at
https://docs.sigopt.com

27. SigOpt. Confidential.27
Deep Dive - Enterprise Reliability

28. SigOpt. Confidential.
● API Performance Suggestions and observations are produced and recorded in
milliseconds.
● 100+ Parameters per Model Support for your most complex models, with
continuous, integer, and categorical types.
10,000+ Observations per Experiment Run thousands of simulations with no
slowdown of the API.
● 1,000,000+ Experiments Optimize and re-tune every model in production to
ensure continuous optimal performance.
● 100s of Simultaneous Experiments & Observations With our Cloud solution, high
concurrent workload is automatically scaled and supported.
● 1000s of modelers Enable your entire modeling organization.
Enterprise Platform: Scalability
28