1.
Overcoming the Challenges of
Experimentation on a Service
Oriented Architecture
Lupin Campos & Stephen Schwahn
September 2020

2.
Hi! We are Lupin and Steph :)
Lupin Campos
Director of Product Growth
Tech Startup Founder (Funnely)
Software Development Company Founder (Leanbirds & Condiminds)
Former Venture Partner @ 500Startups
Steph Schwahn
Growth Team Tech Lead
Tech Startup Founder (Crescendo)
Mobile App Development Company Founder (Plutonium Apps)
Georgia Tech Alumni, graduated with high honors.

3.
So far:
- 1 complete backend refactor
- 2 frontend refactors
- 36 Registration flow experiments (web + fullstack)
- 40% Improvement on Registration Conversion Rate

4.
1Let’s Get Into
Context
A quick note on who we are, what we do, and the
challenges that we found in the way.

5.
This is our registration Process...
Phone
Number
Role
Verification
Code
Email &
Password
Child Info
Personal
Info
Sync
Bank Acc
(Website)

6.
….And here are the services behind it.
A simplified version, you don’t want to see the entire thing. Trust us.

7.
1Service Oriented Architectures with
Optimizely
(the wrong way)

8.
Greenlight Financial Technology, Inc.
SOA with Optimizely Fullstack on Frontend
Web
App
Marketing
Site
Mobile
App Edge
Service
1
Edge
Service
2
Nested
Service
1
Nested
Service
2
Helper
Service
1
Backend

9.
Greenlight Financial Technology, Inc.
SOA with Optimizely Fullstack on Frontend
Web
App
Marketing
Site
Mobile
App Edge
Service
1
Edge
Service
2
Nested
Service
1
Nested
Service
2
Helper
Service
1
Backend
Variant: Variant 1
Variable: FlowId = 6

10.
Greenlight Financial Technology, Inc.
SOA with Optimizely Fullstack
Web
App
Marketing
Site
Mobile
App Edge
Service
1
Edge
Service
2
Nested
Service
1
Nested
Service
2
Helper
Service
1
Backend
User_id: '1234_4321'

11.
Greenlight Financial Technology, Inc.
SOA with Optimizely Fullstack
Web
App
Marketing
Site
Mobile
App Edge
Service
1
Edge
Service
2
Nested
Service
1
Nested
Service
2
Helper
Service
1
Backend
User_id: '1234_4321'
30s 40s
400s 200s
5s

12.
Greenlight Financial Technology, Inc.
SOA with Optimizely Fullstack
Web
App
Marketing
Site
Mobile
App Edge
Service
1
Edge
Service
2
Nested
Service
1
Nested
Service
2
Helper
Service
1
Backend
User_id: '1234_4321'
30s 40s
400s 200s
5s

13.
1Service Oriented Architectures with
Optimizely
(our way)

14.
Greenlight Financial Technology, Inc.
Solution: SOA with Optimizely Agent
Marketing
Site
Mobile
App Edge
Service
1
Edge
Service
2
Nested
Service
1
Nested
Service
2
Helper
Service
1
Backend
Feature or Experiment
Key: 'featureTest’
User_id: '1234_4321'
Optimizely
Agent
30s
Web
App

15.
Greenlight Financial Technology, Inc.
In Practice….
Client 1
Client 2
Client n
.
.
.
Ambassador
(API Gateway)
Pod (3-30 running based on traffic)
Optimizely
Service
(Node.js)
Optimizely
Agentlocalhost:8080
Datafile sync
(every 30s)
localhost:3000
Exposes Optimizely Service
on service port 80
Service
grpc:80
pod-ip:3000
NodeJS Container Go Container (sidecar)
Datafile
pod-ip:3000pod-ip:3000

16.
1Let’s talk about “User ID”
And how do we pass it everywhere!

17.
Greenlight Financial Technology, Inc.
Used for bucketing
MurmurHash(“user_1234”)
Rollout to 60%

18.
Greenlight Financial Technology, Inc.
Used for bucketing
MurmurHash(“user_5678”)
Rollout to 60%

19.
Greenlight Financial Technology, Inc.
What if we don’t have the user id?
MurmurHash(“steph@greenlight.me”)
?
Rollout to 60%

20.
Greenlight Financial Technology, Inc.
Introducing the Experimentation ID!
MurmurHash(“exp_user_1234”)
Rollout to 60%

21.
1Introducing
Experimentation-ID
Service
How we keep a consistent user id regardless of
context

22.
Greenlight Financial Technology, Inc.
Experimentation
-ID service

23.
Greenlight Financial Technology, Inc.
Optimizely Service orchestrates

24.
Greenlight Financial Technology, Inc.
New API Contract!
export type Context = UserIdContext | EmailContext | AnonymousIdContext | RegistrationIdContext | DeviceInfoContext
export type EmailContext = {
email: string
}
export type AnonymousIdContext = {
anonymousId: string
}
export type DeviceInfoContext = {
deviceOs: string,
deviceToken: string
}
export type UserIdContext = {
userId: string
}
export type RegistrationIdContext = {
registrationId: string
}
export function isFeatureEnabled(featureKey: string, context: Context, attributes?: Attributes)
export function getEnabledFeatures(context: Context, attributes?: Attributes)
...

25.
1Truly Full Stack
Experimentation
Through the ENTIRE user journey, from the
marketing landing page, through registration
services, and all the way to fundamental service
primitives

26.
Thank You
greenlightcard.com #greenlight

27.
Science of delivering it right