The goal of this talk to lay out a framework for what algorithms work best in which situations, and why. Drawing on results of hundreds of crowd-sourced predictive modeling contests, this talk shows examples of how structure informs a choice in algorithm. As an illustration of these concepts, ZestFinance's work with China's retail giant, JD.com is used to describe how the right algorithms were applied to the right datasets to turn shopping data into credit data -- creating credit scores from scratch.

1. © 2016 ZestFinance, Inc.
The Right Tool for the Job: Guidelines for
Algorithm Selection in Predictive Modeling
Derek Wilcox
ZestFinance

2. 2 © 2016 ZestFinance, Inc.
About ZestFinance
• Founded by Douglas Merrill, the former CIO of Google
• Nearly $65M in funding from Lightspeed, Matrix and others
• Additional $150M funding from Fortress for Basix
• The team is mostly data geeks, math whizzes, and financial
analysts from prestigious universities and top companies
• Based in Los Angeles

3. 3 © 2016 ZestFinance, Inc.
Our mission
Make fair and transparent credit available to everyone

4. 4 © 2016 ZestFinance, Inc.
Zest is built to achieve that mission
We built a technology platform that is transforming how credit decisions
are made.
We are using that platform to:
• Partner with high-volume lenders worldwide to extend credit to their
customers
• Provide convenient, online loans that help millions of middle-class
Americans move from near-prime to prime

5. 5 © 2016 ZestFinance, Inc.
Technology platform
ZestFinance has developed an underwriting platform that:
• Ingests data from tens of thousands of disparate sources
• Cleans, scrubs, and normalizes the data
• Runs the data through ensembled Machine Learning
algorithms, enhanced with a touch of Machine Learning artistry
• To deliver scores/ratings that best predict
– Probability of fraud
– Likelihood of default
– Overall creditworthiness
All in under 5 seconds

6. 6 © 2016 ZestFinance, Inc.
ZestFinance: more data is better
The world is flooded with information that’s currently being
overlooked.
Why use only a bit of data when there is an infinite amount available?
We are always striving to use even more data and really advanced
math to change the world.

7. 7 © 2016 ZestFinance, Inc.
Turning shopping data into credit data
• Now, let’s talk about China
• Only 240 million of the more than 1 billion Chinese citizens have a
credit history
• On the other hand, China has more data than most any other place
in the world
– eCommerce at $275B and growing over 30%
– 33% of eCommerce via mobile phones
• This data has tremendous potential to create the most accurate
credit history and decisioning system in the world

8. 8 © 2016 ZestFinance, Inc.
JD.com
• We’ve partnered with JD.com -- the largest e-tailer in China
• We’re working together to turn shopping data into credit data,
creating credit histories from scratch
• Our approach also identifies fraud

9. 9 © 2016 ZestFinance, Inc.
Applying Deep Learning to structured
data
• Among 29 challenge winning solutions on Kaggle’s blog in 2015, 17
solutions used XGBoost and 11 used Deep Neural Networks
• Problems with more inherent structure like image, audio, and nlp
seem to favor Deep Neural Nets
• When problems don’t have this sort of structure we can use
XGBoost

10. 10 © 2016 ZestFinance, Inc.
Neural Network
http://cs231n.github.io/neural-networks-1/

11. 11 © 2016 ZestFinance, Inc.
Neural Network
http://arxiv.org/pdf/1509.07627.pdf

12. 12 © 2016 ZestFinance, Inc.
Deep Learning - ImageNet
http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-
convnet-on-imagenet/

13. 13 © 2016 ZestFinance, Inc.
What kind of structure does my data
have?
• Is there some sort of invariance or equivariance?
• Can we effectively learn representations?
• Examples: Image, Speech, Sequences
https://arxiv.org/pdf/1602.02660v2.pdf

14. 14 © 2016 ZestFinance, Inc.
Where does my data actually live?
http://colah.github.io/posts/2014-07-NLP-RNNs-Representations/

15. 15 © 2016 ZestFinance, Inc.
Text Embedding - word2vec
http://www.offconvex.org/2015/12/12/word-embeddings-
1/http://colah.github.io/posts/2014-07-NLP-RNNs-Representations/

16. 16 © 2016 ZestFinance, Inc.
Learning representations
http://colah.github.io/posts/2014-03-NN-
Manifolds-Topology/

17. 17 © 2016 ZestFinance, Inc.
Deep Learning
• Composition of many different functions
• Combining lower level features to create more complicated ones
http://www.iro.umontreal.ca/~bengioy/talks/DL-Tutorial-NIPS2015.pdf

18. 18 © 2016 ZestFinance, Inc.
Deep Learning - Convolutional Nets
http://www.slideshare.net/matsukenbook/deep-learning-chap6-
convolutional-neural-net

19. 19 © 2016 ZestFinance, Inc.
Deep Learning - Recurrent Nets
• Sequences like text
http://colah.github.io/posts/2015-08-Understanding-LSTMs/

20. 20 © 2016 ZestFinance, Inc.
Long Short-Term Memory (LSTMs)
http://colah.github.io/posts/2015-08-Understanding-LSTMs/

21. 21 © 2016 ZestFinance, Inc.
Convolutional and Recurrent Networks
• Speech systems
https://arxiv.org/pdf/1512.02595v1.pdf

22. 22 © 2016 ZestFinance, Inc.
Deeper Networks
http://icml.
cc/2016/tutorials/icml2016_tutorial_deep_residu
al_networks_kaiminghe.pdf

23. 23 © 2016 ZestFinance, Inc.
How do we make networks deeper/longer?
• Exploding/Vanishing Gradient Problem
http://deepdish.io/2015/02/24/network-initialization/

24. 24 © 2016 ZestFinance, Inc.
How do we make networks deeper/longer?
http://arxiv.org/pdf/1512.03385v1.pdf

25. 25 © 2016 ZestFinance, Inc.
Thanks
• Christopher Olah and Andrej Karpathy for their amazing blogs that inspired
and provided some of the visuals in this presentation