SlideShare uma empresa Scribd logo
1 de 69
Baixar para ler offline
Making the entire world accessible via a single API
Ed Freyfogle
Barcelona Data Engineering - 30 Nov. 2017
http://opencagedata.com
@opencagedata
- Some background
- Technical overview
- Build versus Buy?
- Questions
Overview
$whoami
$whoami
- Co-founder of OpenCage
- Before that, co-founded a real estate metasearch engine called
Nestoria. 10 years, 20 employees, €4.5M in revenue
- Way, way before that, first developer at yahoo.de
- Moved to BCN to live the good life
opencagedata.com
WTF is geocoding?
Where are we?
Barcelona
Barcelona
Ironhack BCN
Barcelona
IronHack BCN
08018
Barcelona
IronHack BCN
08002
Carrer de Pamplona
Poblenou
Barcelona
IronHack BCN
08002
Carrer de Pamplona
Poblenou
Europe
Barcelona
IronHack BCN
08002
Carrer de Pamplona
Poblenou
Europe
Spain / Rep. of Catalonia
NEW!
41.39778,2.19011
placename lat/long
placename long/lat
forward
placename long/lat
forward
reverse
Why Geocode?
- Vehicle tracking
- Routing
- Data enrichment
- Put points on a map
- Image processing
Yes!
Yes! but …
T&Cs
expensive at volume
OpenStreetMap
Yes!
Yes! but …
not for high volume use
variable coverage
Yes!
Yes! and …
one API, many geocoders
free as in freedom
enterprise level reliability
- Some background
- Technical overview
- Build versus Buy?
- Questions
Overview
OpenStreetMap
www.openstreetmap.org
OpenStreetMap
- Crowdsourced geodata. Anyone can edit. 4 million reg. users
- Bad name. Is a database, not a map.
- 3 million edits per day. And growing.
- Operations run by volunteers.
- Master DB is hosted at a university in London
- Data is under ODbL License
- Many, many different tools for editing, querying, visualizing the data
Example
$ curl ‘https://api.opencagedata.com/geocode/v1/json?q=41.39778,2.19011&key=...&pretty=1’
Example
$ curl ‘https://api.opencagedata.com/geocode/v1/json?q=41.39778,2.19011&key=...&pretty=1’
"components": {
"ISO_3166-1_alpha-2": "ES",
"_type": "pub",
"city": "Barcelona",
"city_district": "Sant Martí",
"country": "Spain",
"country_code": "es",
"county": "BCN",
"house_number": "96",
"postcode": "08018",
"pub": "Sonora Sport Tavern",
"road": "Carrer de Pamplona",
"state": "Catalonia",
"suburb": "el Parc i la Llacuna del Poblenou"
},
"formatted": "Sonora Sport Tavern, Carrer de Pamplona, 96, 08018 Barcelona, Spain",
https://geocoder.opencagedata.com/code
Libraries for
- Perl, PHP, Python, Ruby, R, Java, .NET, elixir,
Haskell, Go, Erlang, nodejs, Stata, perl6, iOS
- Leaflet, OpenLayers
TMTOWTDI
Architecture
HTTP request
Architecture
HTTP request LB
LB
- HAProxy
- Two different physical locations
Architecture
HTTP request LB
LB
HTTP request LB
meta
- ubuntu
- apache
- mod_perl
Architecture
HTTP request LB
LB
HTTP request LB
meta
Kong
HTTP request
- authentication
- validity checking
- routing
Architecture
HTTP request LB
LB
HTTP request LB
meta
Kong
HTTP request
geocoder geocoder geocoder
Architecture
HTTP request LB
LB
HTTP request LB
meta
Kong
HTTP request
geocoder geocoder geocoder
OSM
Continually
updating
HTTP request
Architecture
HTTP request LB
LB
HTTP request LB
meta
Kong
HTTP request
geocoder geocoder geocoder
OSM
Continually
updating
- accept response, error check
- standardize format
- rank
- add annotations
- send to client in XML, JSON, etc
- timezones
- geohash, MGRS, Maidenhead, DMS
- what3words
- calling codes, currency
- and more ...
Annotations
Formatting
$ curl ‘https://api.opencagedata.com/geocode/v1/json?q=41.39778,2.19011&key=...&pretty=1’
"components": {
"ISO_3166-1_alpha-2": "ES",
"_type": "pub",
"city": "Barcelona",
"city_district": "Sant Martí",
"country": "Spain",
"country_code": "es",
"county": "BCN",
"house_number": "96",
"postcode": "08018",
"pub": "Sonora Sport Tavern",
"road": "Carrer de Pamplona",
"state": "Catalonia",
"suburb": "el Parc i la Llacuna del Poblenou"
},
"formatted": "Sonora Sport Tavern, Carrer de Pamplona, 96, 08018 Barcelona, Spain",
115 New Cavendish Street, London W1T 5DU, United Kingdom
223 William Street, Melbourne VIC 3000, Australia
Rosenthaler Straße 1, 10119 Berlin, Germany
Via Pisacane, 13, 76121 Barletta BT, Italy
3 Upper Alma Road, Rosebank, Cape Town, 7700, South Africa
Formatting
Architecture
HTTP request LB
LB
HTTP request LB
meta
Kong
HTTP request
geocoder geocoder geocoder
OSM
Continually
updating
- logging
- monitoring
- emails
metrics
scp
Monitoring
We use StatusCake
- Dev/testing process
- Deployment
- User registration / onboarding
- Billing
Not covered
- Some background
- Technical overview
- Build versus Buy?
- Questions
Overview
- Unable to buy
- Lack of finance
- Lack of affordable, reliable vendor
When to build
- Unable to buy
- Core to your service
When to build
- Unable to buy
- Core to your service
- Critical differentiator
- Privacy / data protection issues
- Legal requirement
When to build
- Unable to build
When to buy
- Unable to build
- Need to move fast
When to buy
- Unable to build
- Need to move fast
- Is non-core
When to buy
- Unable to build
- Need to move fast
- Is non-core
- Unable to maintain
When to buy
- Financial cost
- Opportunity costs
- Staffing costs
Cost analysis
- Industry trend has been towards buying
- Servers (AWS, Dropbox, etc)
- Login / Identity (FB, google, etc)
- Billing (Stripe)
- 1000s of SaaS businesses
The trend
- Engineers love to build
MAIN POINT
- Engineers love to build
- Building is (comparatively) easy
MAIN POINT
- Engineers love to build
- Building is (comparatively) easy
- MAINTENANCE IS REALLY
HARD!!!!
MAIN POINT
- Some background
- Technical overview
- Build versus Buy?
- Questions
Overview
Three more things
Three more things
1. Lifetime 10% discount
use code: XXXXXX
Three more things
1. Lifetime 10% discount
use code: XXXXXX
2. Happy to pay anyone who wants to write
a wrapper library: F#, Scala, Lua, etc
Three more things
My friends at AVUXI are hiring
http://www.avuxi.com
Ed Freyfogle
Barcelona Data Engineering - 30 Nov. 2017
https://opencagedata.com
@opencagedata
Thanks!
Questions?

Mais conteúdo relacionado

Semelhante a Making the entire world accessible via a single API - the OpenCage Geocoder

ITCamp 2012 - Alessandro Pilotti - Web API, web sockets and RSignal
ITCamp 2012 - Alessandro Pilotti - Web API, web sockets and RSignalITCamp 2012 - Alessandro Pilotti - Web API, web sockets and RSignal
ITCamp 2012 - Alessandro Pilotti - Web API, web sockets and RSignal
ITCamp
 
CV_NguyenQuangViet.DOC
CV_NguyenQuangViet.DOCCV_NguyenQuangViet.DOC
CV_NguyenQuangViet.DOC
Viet Nguyen
 
Top ten it companies and the Skills they work on
Top ten it companies and the Skills they work onTop ten it companies and the Skills they work on
Top ten it companies and the Skills they work on
Yashu dali
 
HTML Hypermedia APIs and Adaptive Web Design - jDays 2013
HTML Hypermedia APIs and Adaptive Web Design - jDays 2013HTML Hypermedia APIs and Adaptive Web Design - jDays 2013
HTML Hypermedia APIs and Adaptive Web Design - jDays 2013
Gustaf Nilsson Kotte
 

Semelhante a Making the entire world accessible via a single API - the OpenCage Geocoder (20)

ITCamp 2012 - Alessandro Pilotti - Web API, web sockets and RSignal
ITCamp 2012 - Alessandro Pilotti - Web API, web sockets and RSignalITCamp 2012 - Alessandro Pilotti - Web API, web sockets and RSignal
ITCamp 2012 - Alessandro Pilotti - Web API, web sockets and RSignal
 
Building modern web sites with ASP .Net Web API, WebSockets and RSignal
Building modern web sites with ASP .Net Web API, WebSockets and RSignalBuilding modern web sites with ASP .Net Web API, WebSockets and RSignal
Building modern web sites with ASP .Net Web API, WebSockets and RSignal
 
CV_NguyenQuangViet.DOC
CV_NguyenQuangViet.DOCCV_NguyenQuangViet.DOC
CV_NguyenQuangViet.DOC
 
Top ten it companies and the Skills they work on
Top ten it companies and the Skills they work onTop ten it companies and the Skills they work on
Top ten it companies and the Skills they work on
 
OrientDB for real & Web App development
OrientDB for real & Web App developmentOrientDB for real & Web App development
OrientDB for real & Web App development
 
Big data at United Airlines
Big data at United AirlinesBig data at United Airlines
Big data at United Airlines
 
PHP is the King, nodejs is the Prince and Lua is the fool
PHP is the King, nodejs is the Prince and Lua is the foolPHP is the King, nodejs is the Prince and Lua is the fool
PHP is the King, nodejs is the Prince and Lua is the fool
 
IoT databases - review and challenges - IoT, Hardware & Robotics meetup - onl...
IoT databases - review and challenges - IoT, Hardware & Robotics meetup - onl...IoT databases - review and challenges - IoT, Hardware & Robotics meetup - onl...
IoT databases - review and challenges - IoT, Hardware & Robotics meetup - onl...
 
Choisir entre une API RPC, SOAP, REST, GraphQL? 
Et si le problème était ai...
Choisir entre une API  RPC, SOAP, REST, GraphQL?  
Et si le problème était ai...Choisir entre une API  RPC, SOAP, REST, GraphQL?  
Et si le problème était ai...
Choisir entre une API RPC, SOAP, REST, GraphQL? 
Et si le problème était ai...
 
HTML Hypermedia APIs and Adaptive Web Design - jDays 2013
HTML Hypermedia APIs and Adaptive Web Design - jDays 2013HTML Hypermedia APIs and Adaptive Web Design - jDays 2013
HTML Hypermedia APIs and Adaptive Web Design - jDays 2013
 
Perchè potresti aver bisogno di un database NoSQL anche se non sei Google o F...
Perchè potresti aver bisogno di un database NoSQL anche se non sei Google o F...Perchè potresti aver bisogno di un database NoSQL anche se non sei Google o F...
Perchè potresti aver bisogno di un database NoSQL anche se non sei Google o F...
 
Rank | Analyse | Lead | Search
Rank | Analyse | Lead | SearchRank | Analyse | Lead | Search
Rank | Analyse | Lead | Search
 
Oracle Code Capgemini: API management & microservices a match made in heaven
Oracle Code Capgemini: API management & microservices a match made in heavenOracle Code Capgemini: API management & microservices a match made in heaven
Oracle Code Capgemini: API management & microservices a match made in heaven
 
IoT interoperability
IoT interoperabilityIoT interoperability
IoT interoperability
 
Semantic-assisted Analysis and Search in Customer Specifications
Semantic-assisted Analysis and Search in Customer SpecificationsSemantic-assisted Analysis and Search in Customer Specifications
Semantic-assisted Analysis and Search in Customer Specifications
 
Semantic technologies in practice - KULeuven 2016
Semantic technologies in practice - KULeuven 2016Semantic technologies in practice - KULeuven 2016
Semantic technologies in practice - KULeuven 2016
 
Past, Present and Future of APIs of Mobile and Web Apps
Past, Present and Future of APIs of Mobile and Web AppsPast, Present and Future of APIs of Mobile and Web Apps
Past, Present and Future of APIs of Mobile and Web Apps
 
Introduction to Polyglot Persistence
Introduction to Polyglot Persistence Introduction to Polyglot Persistence
Introduction to Polyglot Persistence
 
Varun-CV-J
Varun-CV-JVarun-CV-J
Varun-CV-J
 
"Fintech inside of a SaaS powered by 2000+ Microservices", Volodymyr Malyk
"Fintech inside of a SaaS powered by 2000+ Microservices", Volodymyr Malyk"Fintech inside of a SaaS powered by 2000+ Microservices", Volodymyr Malyk
"Fintech inside of a SaaS powered by 2000+ Microservices", Volodymyr Malyk
 

Mais de Ed Freyfogle (7)

Competing with an API giant, lessons learned: OpenCage vs. Google
Competing with an API giant, lessons learned: OpenCage vs. GoogleCompeting with an API giant, lessons learned: OpenCage vs. Google
Competing with an API giant, lessons learned: OpenCage vs. Google
 
The missing feature - simplicity
The missing feature - simplicityThe missing feature - simplicity
The missing feature - simplicity
 
The OpenCage Geocoder
The OpenCage Geocoder The OpenCage Geocoder
The OpenCage Geocoder
 
The OpenCage geocoder - geoinquiets 2 July 2015
The OpenCage geocoder  - geoinquiets 2 July 2015The OpenCage geocoder  - geoinquiets 2 July 2015
The OpenCage geocoder - geoinquiets 2 July 2015
 
The OpenCage Geocoder
The OpenCage GeocoderThe OpenCage Geocoder
The OpenCage Geocoder
 
The OpenCage Geocoder #lpw2014
The OpenCage Geocoder #lpw2014The OpenCage Geocoder #lpw2014
The OpenCage Geocoder #lpw2014
 
Ucladvances entrepreneurship freyfogle nestoria
Ucladvances entrepreneurship freyfogle nestoriaUcladvances entrepreneurship freyfogle nestoria
Ucladvances entrepreneurship freyfogle nestoria
 

Último

audience research (emma) 1.pptxkkkkkkkkkkkkkkkkk
audience research (emma) 1.pptxkkkkkkkkkkkkkkkkkaudience research (emma) 1.pptxkkkkkkkkkkkkkkkkk
audience research (emma) 1.pptxkkkkkkkkkkkkkkkkk
lolsDocherty
 
Production 2024 sunderland culture final - Copy.pptx
Production 2024 sunderland culture final - Copy.pptxProduction 2024 sunderland culture final - Copy.pptx
Production 2024 sunderland culture final - Copy.pptx
ChloeMeadows1
 

Último (17)

I’ll See Y’All Motherfuckers In Game 7 Shirt
I’ll See Y’All Motherfuckers In Game 7 ShirtI’ll See Y’All Motherfuckers In Game 7 Shirt
I’ll See Y’All Motherfuckers In Game 7 Shirt
 
The Rise of Subscription-Based Digital Services.pdf
The Rise of Subscription-Based Digital Services.pdfThe Rise of Subscription-Based Digital Services.pdf
The Rise of Subscription-Based Digital Services.pdf
 
Development Lifecycle.pptx for the secure development of apps
Development Lifecycle.pptx for the secure development of appsDevelopment Lifecycle.pptx for the secure development of apps
Development Lifecycle.pptx for the secure development of apps
 
Reggie miller choke t shirtsReggie miller choke t shirts
Reggie miller choke t shirtsReggie miller choke t shirtsReggie miller choke t shirtsReggie miller choke t shirts
Reggie miller choke t shirtsReggie miller choke t shirts
 
AI Generated 3D Models | AI 3D Model Generator
AI Generated 3D Models | AI 3D Model GeneratorAI Generated 3D Models | AI 3D Model Generator
AI Generated 3D Models | AI 3D Model Generator
 
GOOGLE Io 2024 At takes center stage.pdf
GOOGLE Io 2024 At takes center stage.pdfGOOGLE Io 2024 At takes center stage.pdf
GOOGLE Io 2024 At takes center stage.pdf
 
Free scottie t shirts Free scottie t shirts
Free scottie t shirts Free scottie t shirtsFree scottie t shirts Free scottie t shirts
Free scottie t shirts Free scottie t shirts
 
Registry Data Accuracy Improvements, presented by Chimi Dorji at SANOG 41 / I...
Registry Data Accuracy Improvements, presented by Chimi Dorji at SANOG 41 / I...Registry Data Accuracy Improvements, presented by Chimi Dorji at SANOG 41 / I...
Registry Data Accuracy Improvements, presented by Chimi Dorji at SANOG 41 / I...
 
TORTOGEL TELAH MENJADI SALAH SATU PLATFORM PERMAINAN PALING FAVORIT.
TORTOGEL TELAH MENJADI SALAH SATU PLATFORM PERMAINAN PALING FAVORIT.TORTOGEL TELAH MENJADI SALAH SATU PLATFORM PERMAINAN PALING FAVORIT.
TORTOGEL TELAH MENJADI SALAH SATU PLATFORM PERMAINAN PALING FAVORIT.
 
Statistical Analysis of DNS Latencies.pdf
Statistical Analysis of DNS Latencies.pdfStatistical Analysis of DNS Latencies.pdf
Statistical Analysis of DNS Latencies.pdf
 
Cyber Security Services Unveiled: Strategies to Secure Your Digital Presence
Cyber Security Services Unveiled: Strategies to Secure Your Digital PresenceCyber Security Services Unveiled: Strategies to Secure Your Digital Presence
Cyber Security Services Unveiled: Strategies to Secure Your Digital Presence
 
Thank You Luv I’ll Never Walk Alone Again T shirts
Thank You Luv I’ll Never Walk Alone Again T shirtsThank You Luv I’ll Never Walk Alone Again T shirts
Thank You Luv I’ll Never Walk Alone Again T shirts
 
audience research (emma) 1.pptxkkkkkkkkkkkkkkkkk
audience research (emma) 1.pptxkkkkkkkkkkkkkkkkkaudience research (emma) 1.pptxkkkkkkkkkkkkkkkkk
audience research (emma) 1.pptxkkkkkkkkkkkkkkkkk
 
Premier Mobile App Development Agency in USA.pdf
Premier Mobile App Development Agency in USA.pdfPremier Mobile App Development Agency in USA.pdf
Premier Mobile App Development Agency in USA.pdf
 
Production 2024 sunderland culture final - Copy.pptx
Production 2024 sunderland culture final - Copy.pptxProduction 2024 sunderland culture final - Copy.pptx
Production 2024 sunderland culture final - Copy.pptx
 
Bug Bounty Blueprint : A Beginner's Guide
Bug Bounty Blueprint : A Beginner's GuideBug Bounty Blueprint : A Beginner's Guide
Bug Bounty Blueprint : A Beginner's Guide
 
iThome_CYBERSEC2024_Drive_Into_the_DarkWeb
iThome_CYBERSEC2024_Drive_Into_the_DarkWebiThome_CYBERSEC2024_Drive_Into_the_DarkWeb
iThome_CYBERSEC2024_Drive_Into_the_DarkWeb
 

Making the entire world accessible via a single API - the OpenCage Geocoder