Building ML platforms in Financial Services with serverless technology - FSV202 - New York AWS Summit

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Building ML platforms in Financial
Services with serverless technology
Soofi Safavi
Managing Director of Applied AI Division
Black Knight, Inc.
F S V 2 0 2

Black Knight Inc.
Confidential, Proprietary and/or Trade Secret
TM SM ® Trademark(s) of Black Knight IP Holding Company, LLC, or an affiliate.
© 28 September 2017 Black Knight Financial Technology Solutions, LLC. All Rights Reserved.
2
Agenda
■ What’s a mortgage and how important the industry is
■ Introducing AIVA...the impact and the lessons learned
■ Architecture before and after
■ What helped us transition
■ What didn’t work
■ Principles we practice

Black Knight Inc.
3
The mortgage industry makes up over 13% of the US GDP

Black Knight Inc.
4
Despite its economic significance, the mortgage industry hasn’t improved at the rate we’d expect
Historical investment in mortgage technology has simply not paid off
2007
$2500PER MORTGAGE ORIGINATED
2017
$8000PER MORTGAGE ORIGINATED
INCREASED
COST
STAGNANT
PERFORMANCE
400K
PEOPLE DOING REDUNDANT
LOW-COGNITION TASKS
45–60
DAYS TO CLOSE
$170
BILLION
SPENT YEARLY ON
LOAN ORIGINATIONS*
INCREASED
INVESTMENT

Black Knight Inc.
5
Human labor
Due to lack of authorization,
people often perform
redundant work
Operational redundancy
One classification performed 7 times
results in $2 billion dollars of costs
Untrusted data
Check the checkers; data sources often
have to be verified in person or multiple
times over
WASTE
In fact, costs in the mortgage industry are on the rise
Why does it cost $20,000 to produce a mortgage?

Black Knight Inc.
6
The right solution should be able to:
➔ Make work at least 70% cheaper than
a human worker alternative
➔ Focus on repetitive tasks
➔ Obtain skills based on a series of
cognitive capabilities
➔ Learn new skills in days
➔ Charge customer per data point
So we asked ourselves: What would a disrupter in this industry look like?
A technology that eliminates operational costs and provides visibility into the data

Black Knight Inc.
7
Meet AIVA
A computer that understands business grammar
who learns skills and becomes an alternative workforce

Black Knight Inc.
8
Comprehend
knowing the lexicon
Read
business content
& images
Learn
the unique styles, language
& content from feedback
Conclude
based on data
provided
Remember
historic data &
alternative sources
A computer is as effective as a human at performing a task, if the computer can:

Black Knight Inc.
9
Lender LOS
System
LO collects
documents & data
AIVA SKILLS
performs 1 or more
Human reviews
HW data scientists
Structured data points
1
2
train and iterate to
improve AIVA accuracy
only low confidence or
missing data
3
structured & unstructured
Map to specific fields
in LOS
We trained AIVA to use lenders’ tools to provide data conclusions

Black Knight Inc.
10
Architecture principles for AIVA
➔ Each transaction should cost $0.002
➔ Learning can be shared, but differences should be considered
➔ Should work both in back office and front office (AI on the edge)
➔ No interruption as the machine learning (ML) platform evolves
We defined our own principles to make AIVA stand out from the crowd

Black Knight Inc.
11
COST/TRANSACTION
$11
DEPLOYMENT
2 WEEKS
TRANSACTION
CAPACITY/CUSTOMER
100K
TIME/TRANSACTION
2O MIN
KPIs before serverless
We chose our metrics to ensure that AIVA’s fees didn’t exceed costs to maintain and train her

Black Knight Inc.
12
ConcludeIntake Read Comprehend Remember TranslateAsk Know Learn
Pre-Qualify
Borrowers
Review Bank
Statements
Order
Assets
Order Tax
Transcripts
Compile
Loan
Verify
InsuranceSKILLS
WORK
PATTERNS Review
SERVERS and
SELF-MANAGED Amazon SQSAmazon
DynamoDB
AWS
CloudFormation
Amazon API
Gateway
Amazon S3Amazon EC2 Amazon EC2
Auto scaling
Amazon ECR Amazon EBS
Jenkins Elasticsearch Logstash
Our original ontology included the following architecture… and like many starters, it wasn’t perfect
Amazon
Redshift
Amazon
QuickSight
Amazon ECS
Amazon Simple
Workflow Service

Black Knight Inc.
13
We ran over 800 Amazon EC2 instances... not cheap

Black Knight Inc.
14
With Amazon SWF, state transitions became unmanageable – ⅓ of transactions got stuck in transit
Too many servers, too many failures, hard to manage

Black Knight Inc.
15
When we implemented infrastructure to collect metrics, we actually blocked transactions in production
Too many calls to S3
In production, we couldn’t write
the data we needed to capture in
time with all the calls made, which
slowed us down
Amazon Redshift had too many
small insertions
We were moving too many small files
sequentially instead of batching large
objects
AWS Glue was slow passing data
to Redshift
Our data wasn’t partitioned at first, so
moving the data was costly and time-
consuming

Black Knight Inc.
16
SERVERLESS
Amazon KinesisAWS GlueAmazon
Elasticsearch Service
Amazon
Athena
AWS Step
Functions
AWS LambdaAWS CodeBuildAWS
CodePipeline
Amazon
CloudWatch
Amazon
CloudFront
Our new serverless architecture looked like this: Notice the differences?
SERVERS and
SELF-MANAGED Amazon SQSAmazon
DynamoDB
AWS
CloudFormation
Amazon API
Gateway
Amazon S3Amazon EC2 Amazon EC2
Auto Scaling
Amazon ECR Amazon EBS
Jenkins Elasticsearch Logstash Amazon Simple
Workflow Service
Amazon
Redshift
Amazon
QuickSight
Amazon ECS
AWS Certificate
Manager
Amazon
DynamoDB
Amazon S3 Amazon SQS
AWS
CloudFormation
Amazon API
Gateway

Black Knight Inc.
17
Then, we introduced some new changes, and our old KPIs drastically improved!
COST/TRANSACTION
$0.002
DEPLOYMENT
8*/DAY
TRANSACTION
CAPACITY/CUSTOMER
1M
TIME/TRANSACTION
3 SEC
COST/TRANSACTION
$11
DEPLOYMENT
2 WEEKS
TRANSACTION
CAPACITY/CUSTOMER
100K
TIME/TRANSACTION
2O MIN
BEFORE:
AFTER:

Black Knight Inc.
18
What made the transition easier?
The principles that apply to everything

Black Knight Inc.
19
Every service packs its own backpack
Minimum set of dependencies that account provides:
name, root-url, acm cert, S3 bucket name, S3 bucket

Black Knight Inc.
20
If it’s not in GitHub, it doesn't exist

Black Knight Inc.
21
Avoid a traditional database; use data as a reference model
We use DyanmoDB and S3 to access all our data; DynamoDB contains metadata about objects in S3

Black Knight Inc.
22
Find the orchestration tool that fits
Moving to Step Functions instead of Amazon SWF brought success
Amazon SWF had too many servers, too many failures Step Functions is better for us

Black Knight Inc.
23
Build it for a BYOS

Black Knight Inc.
24
Anatomy of a service/module in serverless

Black Knight Inc.
25
Test-driven development should dictate costs
CUCUMBER TEST AWS Cost Explorer
Write tests first, and make sure that your solution fits the bill

Black Knight Inc.
26
Measure your:
1. Lead time to deploy
2. Change success rate
3. Change frequency

Black Knight Inc.
27
This isn't wine; time does not make it better
Shrink changes and ship faster

Black Knight Inc.
28
Aggressive destruction
90-day lifecycle policy on buckets,
launch a new account each week,
delete all CFTs in an account each
week, month, quarter

Black Knight Inc.
29
With all our lessons learned, we’re excited to share how AIVA is already impacting the mortgage industry
AIVA has read and classified millions of
pages of loan documents in 2019, working
with industry-leading banks and lenders
● Page accuracy rates of up to 97% without
human remediation
● Turn times measured in minutes, not hours
● Indexes hundreds of loan document types with
over 3 top lenders in the mortgage industry
● 70% cost savings from your current OCR/RPA,
BPO, and internal processes

Black Knight Inc.
30
Questions?

Building ML platforms in Financial Services with serverless technology - FSV202 - New York AWS Summit

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Building ML platforms in Financial Services with serverless technology - FSV202 - New York AWS Summit

Semelhante a Building ML platforms in Financial Services with serverless technology - FSV202 - New York AWS Summit (20)

Mais de Amazon Web Services

Mais de Amazon Web Services (20)

Building ML platforms in Financial Services with serverless technology - FSV202 - New York AWS Summit