Unraveling Multimodality with Large Language Models.pdf
Programming proxies to do what we need so we don't have to talk to the network guys again
1. PROGRAMMING PROXIES
TO DO WHAT WE NEED SO
WE DON'T HAVE TO TALK
TO THE NETWORK GUYS
AGAIN
@lmacvittie from @f5networks at #gluecon
Lori MacVittie
Sr. Product Manager, Emerging Technologies
F5 Networks
4. ROUTING IS A NETWORK THING
Router Switch FirewallDDoS Protection Load BalancingDNS
CORE NETWORK (SHARED)
THE NETWORK GUYS ARE GENERALLY RESPONSIBLE FOR LAYER 7 ROUTING
@lmacvittie #gluecon
7. Go forward and
backwards.
PROXIES
A Reverse Proxy sits between the user and an
application and can do things like caching, load
balancing, and security on behalf of the app.
A Forward Proxy sits between the user and an
application and does things like caching and
stopping you from using Facebook at work.
Today we’re (mostly) talking about the Reverse kind of Proxy.
@lmacvittie #gluecon
8. Proxies are
application-
aware with
network chops.
They are fluent
in both the
language of
applications
and networks.
PROXIES
THIS IS WHERE NETWORK STUFFS LIVE
THIS IS WHERE PROXIES LIVE
THIS IS WHERE APPLICATIONS LIVE
DATA
NETWORK
TRANSPORT
SESSION
PRESENTATION
APPLICATION
MAC ADDRESS
IP ADDRESS
TCP
SOCKS
SSL
HTTP / SPDY
L2-3 SERVICES
L4-7 SERVICES
HTML JSON XMLCSS
@lmacvittie #gluecon
12. A/B TESTING
Devices
Internet
Service Pool A
Service Pool B
serverGroupA
serverGroupB
vs1
vs2
• Transparently direct users to either version “A” or version “B”
• Increase or decrease traffic to each version in an instant
• Customize the selection criteria to your needs with a short Node.js script
• Use resources like databases or web APIs as part of the decision
@lmacvittie #gluecon
MySQL
Database
13. var assert = require('assert');
var os = require('os');
var http = require('http');
var fpm = require('lrs/forwardProxyModule');
var vsm = require('lrs/virtualServerModule');
var mysql = require('mysql');
var Cookies = require('cookies');
var proxyhost = os.hostname();
var vs = vsm.find('Bugzilla');
var vs_a = vsm.find('Bugzilla-A');
var vs_b = vsm.find('Bugzilla-B');
var logged_in = false;
// Log to a database
var connection = mysql.createConnection({
host : '192.168.22.22',
user : ‘xxxx',
password : ‘yyyyyyyyy',
database : 'abtesting'
});
var onRequest = function(request, response, next ) {
var cookie = new Cookies( request, response );
var bugz_login = cookie.get("Bugzilla_login");
if( !logged_in || !bugz_login ) {
// Default action: Send to A
vs_a.newRequest(request, response, next);
return;
}
// Add the user to the database automatically if they don't already exist
connection.query('INSERT INTO abtest (userid, ip) select * FROM (SELECT ' +
bugz_login + ', "' + request.connection.remoteAddress + '") as tmp
WHERE NOT EXISTS(SELECT userid from abtest where userid=' +
bugz_login + ')', function(err, rows, fields) {
if (err) throw err;
// Use the database to decide which server to send this request to
connection.query('SELECT opt_in from abtest where userid=' + bugz_login,
function(err, rows, fields) {
if (err) throw err;
var opt_in = rows[0].opt_in;
if( !opt_in ) { vs_a.newRequest(request, response, next);
return;
} else { vs_b.newRequest(request, response, next);
return;
}
});
});
};
// onRequest
var onExist = function(vs) {
if(vs.id == 'Bugzilla') {
vs.on('request', onRequest);
connection.connect();
logged_in = true;
setInterval(keepAlive, 60000);
}
};
vsm.on('exist', 'Bugzilla', onExist);
14. URI MANAGEMENT (REDIRECTION)
Devices
Internet
• Manage hundreds of redirects/rewrites
(www.example.com/app2 www.example.com/app/v2)
• Update redirects without incurring potential outages
• Turn over management to the business folks because updating http conf files
every other day isn’t exactly the job you signed up for @lmacvittie #gluecon
serverGroupA
serverGroupB
vs1
vs2
17. function forwardRequest(request, response, next) {
"use strict";
var vsm = require('lrs/virtualServerModule');
var http = require('http');
var mgmt = require('lrs/managementRest');
function ReplicateTraffic(scenarioName, primaryVSName, secondaryPort) {
var self = this;
self.scenarioName = scenarioName;
self.primaryVS = primaryVSName;
self.port = secondaryPort;
//We need a secondary port that we expect is a loopback virtual IP that
//goes to the secondary virtual server
vsm.on('exist', primaryVSName, function(vs) {
vs.on('request', function(req, res, next) {
self.replicate(req, res, next);
});
});
}
ReplicateTraffic.prototype.cloneReq = function(req) {
var newReq = http.request({ host: "127.0.0.1",
port: this.port,
method: req.method,
path: req.url,
headers: req.headers},
function() {});
return newReq;
}
ReplicateTraffic.prototype.replicate = function(req, res, next) {
if(req.method == 'GET' || req.method == 'HEAD') {
// Only do GET and HEAD
var newReq = this.cloneReq(req);
// I want to do vsB.newRequest(newReq) but cannot
// so I loop it through a dummy vip in cloneReq
newReq.on('response', function(res)
{ console.log('saw B resp'); });
newReq.end();
}
next();
}
var repl = new ReplicateTraffic("xxx",
'vsAandB',
15000);
19. How to choose
between proxy
and app
NETWORK
STUFFS
• chooses an application instance based on HTTP header
• Content-type, URI, device (user-agent), API version, HTTP
CRUD operation, etc…
• chooses an application instance based on payload
• Value of a key in a JSON payload, XML element value,
HTML form data, etc…
• would force you to use an HTTP redirect
• Changing URLs
• Deprecated API calls
• is enforcing a quota (rate limiting) to avoid overwhelming
applications
• needs to do a network thing (e.g. app routing, load balancing,
service chaining) that requires application data from an
external source (database, API call, etc…)
Put the logic in a proxy if the logic ….
@lmacvittie #gluecon
20. Use
programmable
proxies to
implement
deployment
patterns that
require more
logic than basic
conditionals or
data from
external sources
DEVOPS
PATTERNS
@lmacvittie #gluecon
Canary Deployments
Blue/Green Deployments
A/B Testing
v.1
v.2
v.3
API Management
Redirection
Replication
(Dark Architecture)
21. If you can code
it, you can do it
(probably)
PROGRAMMABLE
PROXIES
More things you can do with a programmable proxy
Application
security
Broker
authentication
Identity
devices and
users
v1.04
API version
matching
Rate Limiting /
API quota
enforcement
@lmacvittie #gluecon
22. Programmability in the Network: Traffic Replication
Programmability in the Network: Canary Deployments
Programmability in the Network: Blue-Green Deployment Pattern
Devops.com - Code in Flight
Gluecon 2013 - Dark Architecture and How to Forklift Upgrade Your System
Dyn's CTO Cory von Wallenstein:
LineRate Proxy Download (https://linerate.f5.com/)
@lmacvittie #gluecon
Editor's Notes
All of these deployment patterns require dynamically changing the route through the network. They require layer 7 routing.
A programmable proxy is not the same as a web server proxy. A web server proxy separates the proxy from the application. The application can’t modify the config or behavior of the proxy. A programmable proxy brings it all together and code can interact with “config” and network stuffs as well as with application stuffs.
Managing redirects (www.directv.com/NFL -> www.directv.com/entertainment/something) can quickly become a coordination nightmare
5 or 15 are easy, but what about hundreds?
How do you respond to marketing campaigns quickly without incurring potential outages? (A typo in http.conf can bring down a web server)
How can we get better control of “redirect sprawl”?