Epita pres

Turn Inc. ConfidentialTurn Inc. Confidential
Thursday, September 19, 2013
Turn Tracker
Epita’s presentation
1Turn Inc. Confidential 1

Turn Inc. Confidential
Outline
• Internship context
– Company
– Location
• Internship organization
• Projects presentation
– Turn tracker
– Hack@Turn project
2

Internship Context – Company
• Company name: Turn Inc
• Founded in 2006
• Based in Redwood City, CA
• Fastest growing software company in Silicon Valley in 2012
(Deloitte)
• Manage between 800k and 1M bid opportunities per second
• Now around 300 employees
3

Internship Context – Working at Turn
• Lunch deliver
• Many offsite events
• Human scale engineer team
• Knowledge sharing culture
• Weekly company information update
4

Internship Context – Location
• Company location: In the heart of the Silicon Valley
• Live in San Francisco
• Huge French community
• Join while42, a group of former Epita/Epitech students
– Monthly meeting in a San Francisco start-up
• Beneficial environment for a software Engineer
5

Company presentation – Business
• Turn: Online Advertisement
Company
• Specialize in RTB (Real Time
Bidding)
• Main strength: offering both a
DSP (Demand Side Platform)
and a DMP (Data Management
Platform)
• Multi Channel
– Display
– Mobile
– Video
– Social (Facebook)
• Two Main Product
– Audience Suite (DMP)
– Campaign Suite (DSP)
6

Company presentation – Organization
7
Engineer
teams
Project
Manageme
nt
Product
Manageme
nt
Q&A
Define the product
Define tasksRealize technical tasks
Realize Verification tasks

Company presentation – Communication and
hardware provided
• Monthly Company Meeting
– General update on the company
• New Clients
• Financial updates
• Engineering updates
• Product update
• New hired presentation
• Weekly engineer meeting
– Questions and answers
• Weekly team meeting
– Update on the team organization
8
• Personal hardware
– Laptop (MacBook pro)
– Desktop
– Dual screen
• Company hardware and
software
– Dev environment
– Staging, mirror and production
environment
– Project Management tool: Jira
– Knowledge sharing: Intranet
– Code quality: Crucible
– Development tools: Whatever I want
to use, no standard tools

Tracker – Introduction
• Goal
– Monitoring
– Reporting
– Analytics
9
I want to I have a real-
time display of a given
metric.
I want to raise an alert if
a metric goes over a
defined limit.
I want to know what is the
impact of the new hardware
on the application
performances.
I want to know the usage
of the application hour
per hour.
I want to be able to know
what are the most costly
points in a given function
I want to see the
performances of the SQL
requests for a given table

Tracker – Organization
• Split in three parts:
– Java part
• Get the data
• Save them on mongoDb
– Back-end
• Generate the real-time data
• Query the db
– Display
• Display different views
• Search views
• Monitoring views
10

Tracker – Architecture and technologies
• N-Tiers
• Dependency injection
• Easily maintainable
• Single entry point library
• Not Turn Dependent
11
• Keep it simple, keep it light
– As few dependencies as possible
Architecture Technologies

Tracker – Java
• Annotation
– @Trace annotation
– Tree structure
– Time Related metrics
– Possibilities to retrieve functional
data
– Easy to personalize
• Exceptions
– The exceptions inside a @Trace
annotated function
– Different type of logs
12
• SQL
– Only inside a call to a @Trace
function
– Table name is save
– Parameter of the request are saved.
– Track the JDBC
• Not DB dependent
• Http Response Status
– Saved if the code correspond to an
error.

Tracker – Java – How is it working?
• Base on Aop
– Do something around advised functions
– Build a tree of calls
– Save it to mongo DB,
– Easy to Enable/disable
• Configuration file
• Java agent
13
.class
Class
Loader
Java
agent1
aspectJ
weaver
Java
agent2
BCI .class
Before
After
• What is advised?
- @Trace Functions
- JDBC
- HttpServletResponse

Tracker – Display
• 3 applications
– Browser Application
– Basic back-end
• Socket.io
• REST
• Non-real-time data
– Generic Request Maker
• Real-time data
14
Browser
App
Basic
Back-end
Generic
request
Maker

Tracker – Display – Generic request maker – 1
• Why?
– Generic way to retrieve, emit, save real-time data
– Fun to do
– Db performance efficient
• Request only done once
• Principles
– Node.js based application
– JavaScript configuration (not JSON!)
– Socket.io endpoint
– Run some requests each and every X seconds and do specific actions with the
result
15

• Configuration
– end-points
– Connection to mongoDB
– Query
16

• What are the request?
– Aggregation only
• What can you define?
– $match (condition)
– $group (aggregation)
– $order (try to guess)
– $limit (try to guess)
• What happens if it is not
defined?
– It is not in the query
• Why aggregation
– You can do everything with it
17
$match
• Apply the condition(s)
$group
• Aggregate the result
$order
• Order the result
$limit
• Limit the size of the result
set
Query Pipeline

18

Tracker – Display – Browser application – 1
19
Data Driven SVG
Generator
Better Code
Organization
Animations
Easier CSS
Compatibility
+ ~ ?

• Monitoring
– Real Time
– Line-graph
– Alert Buttons
– Tree-view
– Add a new line graph or alert button
• New configuration in the generic request
maker
• ~ 15-20 lines of code
20
Functionalities
• Searching
– SQL Search
• By table Name
• Display List of request
– Function Search
• Help for the developers
• Profiling
• Performance info
– JavaScript Information
• Exploration of the user actions

21

Tracker – Conclusion
• Current problems
– MongoDb performances
– Lack of tests for the JavaScript part
• Not integrated yet
• Possible improvement
– More generic search
– Enrich the info get through the annotation
– Set up more functional related Visual elements
– More endpoints for the generic request maker (Mail for example)
22

Hack@turn - presentation
• 24 hours rush
• In the Turn HQ Building
• Project have been improve after
the rush
• Directly business related
• Log analyzer
• 2 different parts
– Java indexer
– Display application
• Presentation of the result to the
entire product team

Hack@turn - Business problematic
Which are the most
viewed page ?
How is page
viewed across
day and week ?
Who are the
main referrers
?
Identify the weak
points and work on
them.

Hack@turn - Implementation
Databas
e Display
D3.js
Display
Implementation

Hack@turn - Details
• Cron task executed once every
day
• Download the logs files from
servers
• “Clean” the logs
• Index them
• Available for display
General workflow Java workflow
Save into elastic search
Bulk insertion
Prepare for save
Not save directly
Mapping between URL and business functionality
Retrieve from a mapping file Generic mark-up parsing
Retrieve missing information
In memory save Save when encounter
Check that the file haven’t been already index
Md5 check

Hack@turn - Results
• Working project
• Used by some Product
Managers
• More functionalities through an
elastic search plugin
27

Internship - Conclusion
• Learn a lot!
– JavaScript
– Css3
– Aop
– Node.js
• New databases exploration
– OpenTsdb
– ElasticSearch
– MongoDb
• Improve skills
– Spring Core
– Spring MVC
– General CS skills
• True conclusion
– Hire by Turn
– Will work in London for 1 year (visa
issue)
– Will come back in the Silicon Valley
– Successful Internship
28

Turn Inc. Confidential 29
Thank you for your attention

Epita pres

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Epita pres

Semelhante a Epita pres (20)

Último

Último (20)

Epita pres