1.
Optimizely Agent:
Scale Feature Delivery and
Testing Across Service-
Oriented Architectures
Mike Davis
Sr Staff Engineer
Optimizely

2.
Optimizely Agent: Scale Feature Delivery...
Mike Davis
Sr Staff Engineer
Optimizely

3.
3
Defining "Progressive Delivery"

4.
Progressive Delivery
What
• Blue/Green
• Canarying
• Rollouts
• A/B Testing

5.
Progressive Delivery
What
• Blue/Green
• Canarying
• Rollouts
• A/B Testing
Why
• Mitigate risk
• Minimize blast radius
• Reduce the guesswork
• Ship faster

6.
Progressive Delivery
What
• Blue/Green
• Canarying
• Rollouts
• A/B Testing
Why
• Mitigate risk
• Minimize blast radius
• Reduce the guesswork
• Ship faster
How
• CI/CD
• SOA
• Feature Flags
• Observability

7.
“Our success at Amazon is a
function of how many
experiments we do per year,
per month, per week, per day.”
“Our company culture
encourages experimentation
and free flow of ideas.”
“We use experimentation
and testing to inform as
much of the business as we
can."
Culture of
experimentation
Ship every
11 Seconds
Ship every
22 seconds
Continuous
Code Deploys
Progressive Delivery and Experimentation
Greg Peters / Netflix CPO
Jeff Bezos / Amazon Founder
Larry Page / Google Founder
Ship every
3.5 Seconds

8.
9
Feature Flags with Optimizely Full Stack

9.
Starts with feature flags

10.
Key Feature Flagging Use Cases with Optimizely
Develop features concurrently by implementing
flags set to “off”. Deploy to master at any time
while development is still in on-going.
Trunk-based development
Use a flag to send a small percentage of traffic
to a new build of your application to discovery
any unforeseen issues prior to rolling out to all
users.
Canary Releases
Give business users the power to grant
exclusive access to new features or beta
programs for your best customers or
prospects in a sales cycle.
Beta or Exclusive Access
With feature flags, you can quickly turn off
problem features without rolling back an entire
release.
Kill Switch
Deploy features behind “off” flags, then give
marketers or PMs the ability to turn on the
feature independent of a new deploy.
Dark Launches
Add parameterized variables to feature flags to
update features in real time, without a code
push or app update. Variables can also be
used to A/B test variations of features.
Remote Configuration

11.
Full Stack Architecture Overview
JSON
Datafile
Client SDKs
Server SDKs
Remote
Config
Data
Warehouse
Remotely configure
flags / experiments in Optimizely.
(UI and/or REST API)
Update the JSON datafile
SDKs make decisions locally,
track events asynchronously
for zero latency
Analyze results through
Optimizely and/or exports
and integrations into your
other systems
Optimizely
Event Tracking
Event Tracking

12.
17
Alternative: Build a Service around SDK

13.
18
Alternative: Build a Service around SDK
Drawback: Requires months of engineering time

14.
19
Benefits of Optimizely Agent

15.
20
Faster implementation time vs. building a service
● Get started in minutes
● Reach production scale in days vs.
months
● Customizable and open source
● Cloud native

16.
21
Maintenance and monitoring
100s to 1000s of
SDK instances
1 Service
● Single location for
upgrades
● Fits microservices pattern
● Built-in admin API to track
health, application
performance

17.
22
Governance and Onboarding
Team A Team B Team C
Optimizely Agent

18.
23
Security
● One implementation for
security review
● Decreased network surface
area with less external network
requests
● Predictable outbound requests

19.
24
Explore Optimizely Agent APIs

20.
25
Agent APIs
CLIENT APIs
● POST /v1/activate
● POST /v1/track
● GET /v1/config
● POST /v1/override
ADMIN APIs
● GET /health
● GET /metrics
● GET /info
● POST /webhooks/

21.
26
Activate API
POST /v1/activate
Activate iterates over all qualifying features and
experiments and returns decisions for the given user.
Activate returns variables where applicable.
REQUEST BODY
● userId — string
● userAttributes — object
QUERY PARAMETERS
● featureKey — Array of strings
● experimentKey — Array of strings
● disableTracking — boolean
● type — enum array (“feature”, “experiment”)
● enabled — boolean
Example Request
{
"userId": "string",
"userAttributes": {
"country": "string",
"customer": false
}
}
Example Response
[
{
"featureKey": "string",
"experimentKey": "string",
"variationKey": "string",
"type": "feature",
"enabled": true,
"variables": {
"property1": "string",
"property2": "string"
}
}
]

22.
27
Track API
POST /v1/track
Example Request
{
"userId": "string",
"userAttributes": {
"country": "string",
"customer": false
}
"eventTags": {
"revenue": int,
"product": "string"
}
}
Example Response
{
"userId": "string",
"eventKey": "string"
}
Track sends event and user details to Optimizely’s
analytics backend for experiment analysis.
REQUEST BODY
● userId — string
● userAttributes — object
● eventTags — object
QUERY PARAMETERS
● eventKey (required) — string

23.
28
Config API
GET /v1/config
The config API allows you to access
the contents of your datafile — the
experiments and features required by
your application.
Example Response
{
"revision": "string",
"experimentMap": {
"experiment-key1": {
"id": 0,
"key": "string",
"variationsMap": {...}
},
"experiment-key2": {
"id": 0,
"key": "string",
"variationsMap": {...}
}
},
"featureMap": {
"feature-key1": {
"id": 0,
"key": "string",
"variablesMap": {...},
"experimentsMap": {...}
},
"feature-key2": {
"id": 0,
"key": "string",
"variablesMap": {...},
"experimentsMap": {...}
}
}
}

24.
29
Override API
POST /v1/override
Override an experiment or feature test decision. The
override is stored in local memory. Should be used
debugging and testing purposes. Should not be
used for production overrides.
REQUEST BODY
● userId — string
● experimentKey — string
● variationKey — string
Example Request
{
"userId": "string",
"experimentKey": "string",
"variationKey": "string"
}
Example Request
{
"userId": "string",
"experimentKey": "string",
"variationKey": "string",
"prevVariationKey": "string"
}

25.
30
Built for Enterprise Scale
● Native OAuth2 support
● Optional TLS encryption
● Network isolation between client and
admin APIs
● Support for multiple environments
● Open source software

26.
31
Optimizely Dogfooding Case Study:
The Results Page

27.
33
Optimizely Results API: SDK Implementation
= 70 servers

28.
34
Optimizely Results API: SDK Implementation
= 700 SDK Instances

29.
36
Agent and the Optimizely Results API
How it’s deployed:
● AWS Fargate
● 6 vCPUs
● 12GB RAM
What it’s doing:
● >700 clients
● 2.5K requests per second
● P95 latency* ~1ms per request
*All performance data depends on the connection speed, network, device, web page, and many other factors; actual results will vary.

30.
38

31.
39

32.
40
How to get started with Agent

33.
41
Deployment Options
Deploy as a container Compile from source
> make run

34.
42
Recap on Optimizely Agent
1. Faster implementation in service-oriented architectures
2. Easier to maintain, monitor, and upgrade
3. Better governance and easy onboarding
4. Centralized networking and security best practices

35.
Thank you!
Join us on
Slack for Q&A
optimize.ly/dev-community