SlideShare uma empresa Scribd logo
1 de 37
Baixar para ler offline
API and Big data Solution
Patterns
Srinath Perera
Senior Software Architect
About	
  WSO2	
  
•  Providing	
  the	
  only	
  complete	
  open	
  source	
  componen:zed	
  cloud	
  
pla=orm	
  
–  Dedicated	
  to	
  removing	
  all	
  the	
  stumbling	
  blocks	
  to	
  
enterprise	
  agility	
  
–  Enabling	
  you	
  to	
  focus	
  on	
  business	
  logic	
  and	
  business	
  
value	
  
•  Recognized	
  by	
  leading	
  analyst	
  firms	
  as	
  visionaries	
  and	
  leaders	
  
–  Gartner	
  cites	
  WSO2	
  as	
  visionaries	
  in	
  all	
  3	
  categories	
  of	
  	
  
applica:on	
  infrastructure	
  
–  Forrester	
  places	
  WSO2	
  in	
  top	
  2	
  for	
  API	
  Management	
  
•  Global	
  corpora:on	
  with	
  offices	
  in	
  USA,	
  UK	
  &	
  Sri	
  Lanka	
  
–  200+	
  employees	
  and	
  growing	
  
•  Business	
  model	
  of	
  selling	
  comprehensive	
  support	
  &	
  
maintenance	
  for	
  our	
  products	
  
	
  
150+	
  globally	
  posi:oned	
  support	
  customers	
  
Outline
•  What	
  is	
  API	
  
Management	
  
•  What	
  is	
  Big	
  
Data?	
  
•  API	
  manager	
  +	
  
Bigdata	
  
solu:on	
  
•  API	
  +	
  Big	
  data	
  
PaVerns	
  
•  Conclusion	
  
Photo	
  by	
  John	
  Trainoron	
  Flickr	
  hVp://www.flickr.com/photos/trainor/
2902023575/,	
  Licensed	
  under	
  CC	
  
APIs
•  An API is a business capability delivered over the
Internet to internal or external consumers
–  Network accessible function
–  Available using standard web protocols
–  With well-defined interfaces
–  Designed for access by third-parties
•  A Managed API is:
–  Actively advertised and subscribe-able
–  Available with SLAs
–  Secured, authenticated, authorized and protected
–  Monitored and monetized with analytics
Typical Use Cases
•  Expose APIs for public consumption
–  Extend your business through APIs.
•  Expose APIs for internal consumption
–  Manage the APIs used in internal applications
–  Detect Usage Patterns
–  Internal Monetization
•  Control Access to Cloud Services
–  Manage and Secure access from internal
applications to cloud services (SalesForce, Google
Apps, etc.)
WSO2 API
Management
Platform
WSO2’s API Management Vision
•  Create APIs
–  WSO2 Application Server, Data Services Server and ESB
•  Find and subscribe/buy APIs
–  API Store and Governance
•  Manage, secure and protect APIs
–  API Management and Gateway
•  Monitor and Monetize APIs
–  API Monitoring and Analytics
•  Develop, host and run API-based applications in a
Platform-as-a-Service
–  WSO2 Stratos and Cloud Development Platform
–  API Cloud
API Manager Components
API Gateway Processing Flow
API Manager Product and Platform
What is Big data?
•  There is lot of data
available
–  E.g. Internet of things
•  We have computing
power
•  We have technology
•  Goal To know
–  To Explain
–  To predict
•  Challenge is the full
lifecycle
Drivers of Big Data
Big Data Architecture
Collecting Data
•  Data collected at sensors and sent to big data
system via events or flat files
•  Event Streams: we name the events by its
content/ originator
•  Get	
  data	
  through	
  	
  
–  Point	
  to	
  Point	
  
–  Event	
  Bus	
  (WSO2	
  MB)	
  
•  E.g.	
  Data	
  bridge	
  (come	
  with	
  
WSO2	
  CEP/	
  BAM)	
  
– a	
  thrib	
  based	
  transport	
  
that	
  do	
  about	
  400k	
  
events/	
  sec	
  
Storing Data
•  Historically we used databases
–  Scale is a challenge: replication,
sharding
•  Scalable options
–  NoSQL (Cassandra, Hbase) [If
data is structured]
•  Column families Gaining Ground
–  Distributed file systems (e.g.
HDFS) [If data is unstructured]
•  New SQL
–  In Memory computing, VoltDB
•  Specialized data structures
–  Graph Databases, Data structure
servers hVp://www.flickr.com/photos/keso/
363133967/	
  
Making Sense of Data
•  To know (what happened?)
–  Basic analytics +
visualizations
–  Interactive drill down
–  E.g. MapReduce, Hive, CEP
•  To explain (why)
–  Data mining, classifications,
building models, clustering
(e.g. Mahout, Map Reduce)
•  To forecast
–  Neural networks, decision
models (e.g. Mahout)
WSO2 Big Data Story
API Gateway + Big Data
API + Big Data Patterns
•  Big Data as API enabler
–  Monitor and control: API analytics for an
organization
–  Analytics: Billing and Metering
–  API Store (Market Place)
•  Targeted Marketing (Recommendations,
Deals, Search)
•  Context sensitive prizing
–  Deployment optimization
–  Governance: Planning and Proactive
action
•  APIs as a Big Data Enabler
–  Easy way to find and get access to data
–  Data Market Place
–  API Façade for exposing Analytics/ Data
–  Controlled delegation of data
hVp://www.flickr.com/photos/ooohoooh/
1350774047/	
  
Pattern 1: Monitor and Control
Monitor and Control (Contd.)
•  API Manager as single entry point to come in and go
out
•  Capture data – client, what API, when? $$ revenue ..
•  Analyze and Monitor KPIs
–  CIO Dashboard
–  Alerts
•  React to trends faster
•  Greater control
–  Audit
–  Subscriptions policies
–  Usage tiers
•  Policy enforcement
–  Make sure everything is managed
Example: Business value driven Org
•  Customer facing site is a
composite (e.g. like Gmail page)
•  Each widget back by API or APIs
•  Each widget added removed in
runtime
•  Widget shown based on business
value
•  Each API owner get resource
compared to business value
•  We can use API management as
the tool to realize this
Pattern 2: Billing and Metering
•  API store detect all access
•  Contains information about
subscriptions, amount of data
transferred etc.
•  We can use this to
–  Meter, control and optimize
–  Enforce usage tiers (e.g. Quota
via CEP)
•  Generate Billing and reports
•  Supported by WSO2 API
manager now
Pattern 3: API Store (Marketplace)
Pattern 4: Deployment Optimization
App Factory - Applications Management
App Factory (Contd.)
•  Applications Management
–  Creation
–  Continuous Build / Deploy
–  Promotion across environments
•  Team Management
–  Add developers, QA engineers and DevOps for your
application
•  Bugs/Features Tracking
•  Integrated with API Store
•  Activity Dashboards
•  Built on top of Jaggery framework
•  Skinnable
Pattern 5:API lifecycle Governance
•  Take control of your API life cycle
(develop, test, production) with
WSO2 App Factory
•  You can apply analytics to your
API lifecycle
–  How long does it take to develop and
API?
–  How often are they updated?
–  What are the dependencies of my
API?
–  What are most effective versions?
–  What steps takes most time?
–  What APIs are actively developed?
–  What one have more issues, how fast
are they get fixed?
Pattern 6: Find datasets
•  Most big data apps need
Good data
•  Hard to find good data
–  No good way to find
–  Hard to access
–  No clear way to buy
•  APIs is a great way to find
useful data
•  Also to provide controlled
access
e.g. Andy Stanford-Clark used public data about
cruisers to track where availability.
Pattern 7: Data Marketplace
•  Find, (buy), use data
•  Great way to monetize the
output of big data projects
–  Sell the analyzed traffic data
–  Sell stock quote analytics
•  API subscribe model can help
•  Usage tiers can be based on
frequency or resolution of
data
Pattern 8: Controlled Data Delegation
•  Only expose the part of your data set of a processing
without exposing credentials of the database to the
user
Big data in API Store
APIs in Big data System
Usecase
•  Trigger an alert when new version is 10% more
slower than the older versions
–  Expose the API through API store
–  Configure API manager with BAM location
–  BAM integration will pump stats (api, version, latency,
time)
–  Write a BAM query to calculate the latency through
time and store it in a DB
–  Write a CEP query to detect if new version is slower
Define eventTable APIStatHistory ..
Select from APIStat as s join APIStatHistory as h on (h.latency -
s.latency)<=h.latency*.1
Conclusion
•  APIs and Big Data has received much
attentions
•  We looked at WSO2 offerings in both
cases
– WSO2 API Manager, WSO2 App Factory
– WSO2 BAM, WSO2 CEP
•  We looked at some patterns on how both
can work together
•  We looked at how we can implement them
Engage	
  with	
  WSO2	
  
•  Helping	
  you	
  get	
  the	
  most	
  out	
  of	
  your	
  deployments	
  
•  From	
  project	
  evalua:on	
  and	
  incep:on	
  to	
  development	
  and	
  
going	
  into	
  produc:on,	
  WSO2	
  is	
  your	
  partner	
  in	
  ensuring	
  
100%	
  project	
  success	
  
	
  
Questions?

Mais conteúdo relacionado

Mais procurados

Migrate to Azure with Confidence - Inovar Consulting
Migrate to Azure with Confidence - Inovar ConsultingMigrate to Azure with Confidence - Inovar Consulting
Migrate to Azure with Confidence - Inovar ConsultingInovar Tech
 
Apache Kafka in the Transportation and Logistics
Apache Kafka in the Transportation and LogisticsApache Kafka in the Transportation and Logistics
Apache Kafka in the Transportation and LogisticsKai Wähner
 
Azure Purview Data Toboggan Erwin de Kreuk
Azure Purview Data Toboggan Erwin de KreukAzure Purview Data Toboggan Erwin de Kreuk
Azure Purview Data Toboggan Erwin de KreukErwin de Kreuk
 
SharePoint Tutorial and SharePoint Training - Introduction
SharePoint Tutorial and SharePoint Training - IntroductionSharePoint Tutorial and SharePoint Training - Introduction
SharePoint Tutorial and SharePoint Training - IntroductionGregory Zelfond
 
Data Warehouse vs. Data Lake vs. Data Streaming – Friends, Enemies, Frenemies?
Data Warehouse vs. Data Lake vs. Data Streaming – Friends, Enemies, Frenemies?Data Warehouse vs. Data Lake vs. Data Streaming – Friends, Enemies, Frenemies?
Data Warehouse vs. Data Lake vs. Data Streaming – Friends, Enemies, Frenemies?Kai Wähner
 
Real time stock processing with apache nifi, apache flink and apache kafka
Real time stock processing with apache nifi, apache flink and apache kafkaReal time stock processing with apache nifi, apache flink and apache kafka
Real time stock processing with apache nifi, apache flink and apache kafkaTimothy Spann
 
NOVA SQL User Group - Azure Synapse Analytics Overview - May 2020
NOVA SQL User Group - Azure Synapse Analytics Overview -  May 2020NOVA SQL User Group - Azure Synapse Analytics Overview -  May 2020
NOVA SQL User Group - Azure Synapse Analytics Overview - May 2020Timothy McAliley
 
Data Streaming Ecosystem Management at Booking.com
Data Streaming Ecosystem Management at Booking.com Data Streaming Ecosystem Management at Booking.com
Data Streaming Ecosystem Management at Booking.com confluent
 
Airbyte @ Airflow Summit - The new modern data stack
Airbyte @ Airflow Summit - The new modern data stackAirbyte @ Airflow Summit - The new modern data stack
Airbyte @ Airflow Summit - The new modern data stackMichel Tricot
 
DevOps for Databricks
DevOps for DatabricksDevOps for Databricks
DevOps for DatabricksDatabricks
 
Event-Driven Messaging and Actions using Apache Flink and Apache NiFi
Event-Driven Messaging and Actions using Apache Flink and Apache NiFiEvent-Driven Messaging and Actions using Apache Flink and Apache NiFi
Event-Driven Messaging and Actions using Apache Flink and Apache NiFiDataWorks Summit
 
Data In Motion Paris 2023
Data In Motion Paris 2023Data In Motion Paris 2023
Data In Motion Paris 2023confluent
 
SharePoint Overview
SharePoint OverviewSharePoint Overview
SharePoint OverviewAmy Phillips
 
Apache Kafka for Automotive Industry, Mobility Services & Smart City
Apache Kafka for Automotive Industry, Mobility Services & Smart CityApache Kafka for Automotive Industry, Mobility Services & Smart City
Apache Kafka for Automotive Industry, Mobility Services & Smart CityKai Wähner
 
Snowflake free trial_lab_guide
Snowflake free trial_lab_guideSnowflake free trial_lab_guide
Snowflake free trial_lab_guideslidedown1
 
Structured, Unstructured and Streaming Big Data on the AWS
Structured, Unstructured and Streaming Big Data on the AWSStructured, Unstructured and Streaming Big Data on the AWS
Structured, Unstructured and Streaming Big Data on the AWSAmazon Web Services
 
Advantages of SharePoint Online
Advantages of SharePoint OnlineAdvantages of SharePoint Online
Advantages of SharePoint OnlineRishabh Software
 

Mais procurados (20)

Migrate to Azure with Confidence - Inovar Consulting
Migrate to Azure with Confidence - Inovar ConsultingMigrate to Azure with Confidence - Inovar Consulting
Migrate to Azure with Confidence - Inovar Consulting
 
Apache Kafka in the Transportation and Logistics
Apache Kafka in the Transportation and LogisticsApache Kafka in the Transportation and Logistics
Apache Kafka in the Transportation and Logistics
 
Azure Purview Data Toboggan Erwin de Kreuk
Azure Purview Data Toboggan Erwin de KreukAzure Purview Data Toboggan Erwin de Kreuk
Azure Purview Data Toboggan Erwin de Kreuk
 
SharePoint Tutorial and SharePoint Training - Introduction
SharePoint Tutorial and SharePoint Training - IntroductionSharePoint Tutorial and SharePoint Training - Introduction
SharePoint Tutorial and SharePoint Training - Introduction
 
Data Warehouse vs. Data Lake vs. Data Streaming – Friends, Enemies, Frenemies?
Data Warehouse vs. Data Lake vs. Data Streaming – Friends, Enemies, Frenemies?Data Warehouse vs. Data Lake vs. Data Streaming – Friends, Enemies, Frenemies?
Data Warehouse vs. Data Lake vs. Data Streaming – Friends, Enemies, Frenemies?
 
Real time stock processing with apache nifi, apache flink and apache kafka
Real time stock processing with apache nifi, apache flink and apache kafkaReal time stock processing with apache nifi, apache flink and apache kafka
Real time stock processing with apache nifi, apache flink and apache kafka
 
NOVA SQL User Group - Azure Synapse Analytics Overview - May 2020
NOVA SQL User Group - Azure Synapse Analytics Overview -  May 2020NOVA SQL User Group - Azure Synapse Analytics Overview -  May 2020
NOVA SQL User Group - Azure Synapse Analytics Overview - May 2020
 
Data Streaming Ecosystem Management at Booking.com
Data Streaming Ecosystem Management at Booking.com Data Streaming Ecosystem Management at Booking.com
Data Streaming Ecosystem Management at Booking.com
 
Airbyte @ Airflow Summit - The new modern data stack
Airbyte @ Airflow Summit - The new modern data stackAirbyte @ Airflow Summit - The new modern data stack
Airbyte @ Airflow Summit - The new modern data stack
 
DevOps for Databricks
DevOps for DatabricksDevOps for Databricks
DevOps for Databricks
 
Event-Driven Messaging and Actions using Apache Flink and Apache NiFi
Event-Driven Messaging and Actions using Apache Flink and Apache NiFiEvent-Driven Messaging and Actions using Apache Flink and Apache NiFi
Event-Driven Messaging and Actions using Apache Flink and Apache NiFi
 
Data In Motion Paris 2023
Data In Motion Paris 2023Data In Motion Paris 2023
Data In Motion Paris 2023
 
Microsoft Purview
Microsoft PurviewMicrosoft Purview
Microsoft Purview
 
SharePoint Overview
SharePoint OverviewSharePoint Overview
SharePoint Overview
 
Power Platform Governance
Power Platform GovernancePower Platform Governance
Power Platform Governance
 
Apache Kafka for Automotive Industry, Mobility Services & Smart City
Apache Kafka for Automotive Industry, Mobility Services & Smart CityApache Kafka for Automotive Industry, Mobility Services & Smart City
Apache Kafka for Automotive Industry, Mobility Services & Smart City
 
Snowflake free trial_lab_guide
Snowflake free trial_lab_guideSnowflake free trial_lab_guide
Snowflake free trial_lab_guide
 
Open Banking APIs on AWS
Open Banking APIs on AWSOpen Banking APIs on AWS
Open Banking APIs on AWS
 
Structured, Unstructured and Streaming Big Data on the AWS
Structured, Unstructured and Streaming Big Data on the AWSStructured, Unstructured and Streaming Big Data on the AWS
Structured, Unstructured and Streaming Big Data on the AWS
 
Advantages of SharePoint Online
Advantages of SharePoint OnlineAdvantages of SharePoint Online
Advantages of SharePoint Online
 

Destaque

API Strategies for Big Data - If Data Were Oil
API Strategies for Big Data - If Data Were OilAPI Strategies for Big Data - If Data Were Oil
API Strategies for Big Data - If Data Were OilDrew Bartkiewicz
 
Intelligent APIs for Big Data & IoT Create customized data views for mobile,...
Intelligent APIs for Big Data & IoT  Create customized data views for mobile,...Intelligent APIs for Big Data & IoT  Create customized data views for mobile,...
Intelligent APIs for Big Data & IoT Create customized data views for mobile,...CA API Management
 
Modernizing with microservices and fast data
Modernizing with microservices and fast dataModernizing with microservices and fast data
Modernizing with microservices and fast dataPatrick Di Loreto
 
The Practice of Big Data - The Hadoop ecosystem explained with usage scenarios
The Practice of Big Data - The Hadoop ecosystem explained with usage scenariosThe Practice of Big Data - The Hadoop ecosystem explained with usage scenarios
The Practice of Big Data - The Hadoop ecosystem explained with usage scenarioskcmallu
 
Real Time API delivering data @ Scale
Real Time API delivering data @ ScaleReal Time API delivering data @ Scale
Real Time API delivering data @ ScaleAkash Mishra
 
WSO2 Big Data Platform and Applications
WSO2 Big Data Platform and ApplicationsWSO2 Big Data Platform and Applications
WSO2 Big Data Platform and ApplicationsSrinath Perera
 
API Management and Integrated SOA Governance
API Management and Integrated SOA GovernanceAPI Management and Integrated SOA Governance
API Management and Integrated SOA GovernanceSumanth Chinthagunta
 
Using MongoDB + Hadoop Together
Using MongoDB + Hadoop TogetherUsing MongoDB + Hadoop Together
Using MongoDB + Hadoop TogetherMongoDB
 
Bringing API Management to AWS Powered Backends
Bringing API Management to AWS Powered BackendsBringing API Management to AWS Powered Backends
Bringing API Management to AWS Powered BackendsApigee | Google Cloud
 
O'Reilly author webinar "APIs: A Strategy guide": Transforming Your Business...
O'Reilly author webinar "APIs: A Strategy guide":  Transforming Your Business...O'Reilly author webinar "APIs: A Strategy guide":  Transforming Your Business...
O'Reilly author webinar "APIs: A Strategy guide": Transforming Your Business...Apigee | Google Cloud
 
API Management Demystified
API Management DemystifiedAPI Management Demystified
API Management DemystifiedSoftware AG
 
Crime Analytics: Analysis of crimes through news paper articles
Crime Analytics: Analysis of crimes through news paper articlesCrime Analytics: Analysis of crimes through news paper articles
Crime Analytics: Analysis of crimes through news paper articlesChamath Sajeewa
 
Taxonomies and Metadata in Information Architecture
Taxonomies and Metadata in Information ArchitectureTaxonomies and Metadata in Information Architecture
Taxonomies and Metadata in Information ArchitectureAccess Innovations, Inc.
 
Microsoft and Hortonworks Delivers the Modern Data Architecture for Big Data
Microsoft and Hortonworks Delivers the Modern Data Architecture for Big DataMicrosoft and Hortonworks Delivers the Modern Data Architecture for Big Data
Microsoft and Hortonworks Delivers the Modern Data Architecture for Big DataHortonworks
 
Monetizing Big Data at Telecom Service Providers
Monetizing Big Data at Telecom Service ProvidersMonetizing Big Data at Telecom Service Providers
Monetizing Big Data at Telecom Service ProvidersDataWorks Summit
 
Open API and API Management - Introduction and Comparison of Products: TIBCO ...
Open API and API Management - Introduction and Comparison of Products: TIBCO ...Open API and API Management - Introduction and Comparison of Products: TIBCO ...
Open API and API Management - Introduction and Comparison of Products: TIBCO ...Kai Wähner
 
Big Data Architecture
Big Data ArchitectureBig Data Architecture
Big Data ArchitectureGuido Schmutz
 
The Efficient Big data Platform - IDC 360, Copenhagen
The Efficient Big data Platform - IDC 360, CopenhagenThe Efficient Big data Platform - IDC 360, Copenhagen
The Efficient Big data Platform - IDC 360, CopenhagenPetri Pekkarinen
 

Destaque (20)

API Strategies for Big Data - If Data Were Oil
API Strategies for Big Data - If Data Were OilAPI Strategies for Big Data - If Data Were Oil
API Strategies for Big Data - If Data Were Oil
 
Intelligent APIs for Big Data & IoT Create customized data views for mobile,...
Intelligent APIs for Big Data & IoT  Create customized data views for mobile,...Intelligent APIs for Big Data & IoT  Create customized data views for mobile,...
Intelligent APIs for Big Data & IoT Create customized data views for mobile,...
 
Modernizing with microservices and fast data
Modernizing with microservices and fast dataModernizing with microservices and fast data
Modernizing with microservices and fast data
 
The Practice of Big Data - The Hadoop ecosystem explained with usage scenarios
The Practice of Big Data - The Hadoop ecosystem explained with usage scenariosThe Practice of Big Data - The Hadoop ecosystem explained with usage scenarios
The Practice of Big Data - The Hadoop ecosystem explained with usage scenarios
 
Real Time API delivering data @ Scale
Real Time API delivering data @ ScaleReal Time API delivering data @ Scale
Real Time API delivering data @ Scale
 
WSO2 Big Data Platform and Applications
WSO2 Big Data Platform and ApplicationsWSO2 Big Data Platform and Applications
WSO2 Big Data Platform and Applications
 
Process Maker Features
Process Maker FeaturesProcess Maker Features
Process Maker Features
 
API Management and Integrated SOA Governance
API Management and Integrated SOA GovernanceAPI Management and Integrated SOA Governance
API Management and Integrated SOA Governance
 
Using MongoDB + Hadoop Together
Using MongoDB + Hadoop TogetherUsing MongoDB + Hadoop Together
Using MongoDB + Hadoop Together
 
Bringing API Management to AWS Powered Backends
Bringing API Management to AWS Powered BackendsBringing API Management to AWS Powered Backends
Bringing API Management to AWS Powered Backends
 
O'Reilly author webinar "APIs: A Strategy guide": Transforming Your Business...
O'Reilly author webinar "APIs: A Strategy guide":  Transforming Your Business...O'Reilly author webinar "APIs: A Strategy guide":  Transforming Your Business...
O'Reilly author webinar "APIs: A Strategy guide": Transforming Your Business...
 
API Management Demystified
API Management DemystifiedAPI Management Demystified
API Management Demystified
 
Definitive Guide to API Management
Definitive Guide to API ManagementDefinitive Guide to API Management
Definitive Guide to API Management
 
Crime Analytics: Analysis of crimes through news paper articles
Crime Analytics: Analysis of crimes through news paper articlesCrime Analytics: Analysis of crimes through news paper articles
Crime Analytics: Analysis of crimes through news paper articles
 
Taxonomies and Metadata in Information Architecture
Taxonomies and Metadata in Information ArchitectureTaxonomies and Metadata in Information Architecture
Taxonomies and Metadata in Information Architecture
 
Microsoft and Hortonworks Delivers the Modern Data Architecture for Big Data
Microsoft and Hortonworks Delivers the Modern Data Architecture for Big DataMicrosoft and Hortonworks Delivers the Modern Data Architecture for Big Data
Microsoft and Hortonworks Delivers the Modern Data Architecture for Big Data
 
Monetizing Big Data at Telecom Service Providers
Monetizing Big Data at Telecom Service ProvidersMonetizing Big Data at Telecom Service Providers
Monetizing Big Data at Telecom Service Providers
 
Open API and API Management - Introduction and Comparison of Products: TIBCO ...
Open API and API Management - Introduction and Comparison of Products: TIBCO ...Open API and API Management - Introduction and Comparison of Products: TIBCO ...
Open API and API Management - Introduction and Comparison of Products: TIBCO ...
 
Big Data Architecture
Big Data ArchitectureBig Data Architecture
Big Data Architecture
 
The Efficient Big data Platform - IDC 360, Copenhagen
The Efficient Big data Platform - IDC 360, CopenhagenThe Efficient Big data Platform - IDC 360, Copenhagen
The Efficient Big data Platform - IDC 360, Copenhagen
 

Semelhante a API and Big Data Solution Patterns

Analyzing the World's Largest Security Data Lake!
Analyzing the World's Largest Security Data Lake!Analyzing the World's Largest Security Data Lake!
Analyzing the World's Largest Security Data Lake!DataWorks Summit
 
Performance monitoring in a DevOps World
Performance monitoring in a DevOps WorldPerformance monitoring in a DevOps World
Performance monitoring in a DevOps WorldSolidify
 
MLOps journey at Swisscom: AI Use Cases, Architecture and Future Vision
MLOps journey at Swisscom: AI Use Cases, Architecture and Future VisionMLOps journey at Swisscom: AI Use Cases, Architecture and Future Vision
MLOps journey at Swisscom: AI Use Cases, Architecture and Future VisionBATbern
 
Feature Store as a Data Foundation for Machine Learning
Feature Store as a Data Foundation for Machine LearningFeature Store as a Data Foundation for Machine Learning
Feature Store as a Data Foundation for Machine LearningProvectus
 
WSO2Con USA 2017: Brokerage as a Service (BaaS), Transforming Fidelity Broker...
WSO2Con USA 2017: Brokerage as a Service (BaaS), Transforming Fidelity Broker...WSO2Con USA 2017: Brokerage as a Service (BaaS), Transforming Fidelity Broker...
WSO2Con USA 2017: Brokerage as a Service (BaaS), Transforming Fidelity Broker...WSO2
 
Monitoring API Performance and Delivering a Scalable API Solution
Monitoring API Performance and Delivering a Scalable API SolutionMonitoring API Performance and Delivering a Scalable API Solution
Monitoring API Performance and Delivering a Scalable API SolutionWSO2
 
How to Manage APIs in your Enterprise for Maximum Reusability and Governance
How to Manage APIs in your Enterprise for Maximum Reusability and GovernanceHow to Manage APIs in your Enterprise for Maximum Reusability and Governance
How to Manage APIs in your Enterprise for Maximum Reusability and GovernanceWSO2
 
How to Manage APIs in your Enterprise for Maximum Reusability and Governance
How to Manage APIs in your Enterprise for Maximum Reusability and GovernanceHow to Manage APIs in your Enterprise for Maximum Reusability and Governance
How to Manage APIs in your Enterprise for Maximum Reusability and GovernanceHARMAN Services
 
João Emilio Santos Bento da Silva - Estratégia de APIs
João Emilio Santos Bento da Silva - Estratégia de APIsJoão Emilio Santos Bento da Silva - Estratégia de APIs
João Emilio Santos Bento da Silva - Estratégia de APIsDevCamp Campinas
 
WSO2 Workshop Sydney 2016 - APIs
WSO2 Workshop Sydney 2016 - APIsWSO2 Workshop Sydney 2016 - APIs
WSO2 Workshop Sydney 2016 - APIsDassana Wijesekara
 
Platform for Secure Digital Business
Platform for Secure Digital BusinessPlatform for Secure Digital Business
Platform for Secure Digital BusinessAkana
 
Modern Data Architectures for Business Insights at Scale
Modern Data Architectures for Business Insights at ScaleModern Data Architectures for Business Insights at Scale
Modern Data Architectures for Business Insights at ScaleAmazon Web Services
 
Extend soa with api management spoug- Madrid
Extend soa with api management   spoug- MadridExtend soa with api management   spoug- Madrid
Extend soa with api management spoug- MadridVinay Kumar
 
API Management in Digital Transformation
API Management in Digital TransformationAPI Management in Digital Transformation
API Management in Digital TransformationAditya Thatte
 
Modern Data Architectures for Business Outcomes
Modern Data Architectures for Business OutcomesModern Data Architectures for Business Outcomes
Modern Data Architectures for Business OutcomesAmazon Web Services
 
Extend soa with api management Doag18
Extend soa with api management Doag18Extend soa with api management Doag18
Extend soa with api management Doag18Vinay Kumar
 
#1922 rest-push2 ap-im-v6
#1922 rest-push2 ap-im-v6#1922 rest-push2 ap-im-v6
#1922 rest-push2 ap-im-v6Jack Carnes
 

Semelhante a API and Big Data Solution Patterns (20)

Smartone v1.0
Smartone v1.0Smartone v1.0
Smartone v1.0
 
Open Banking & Open Insurance
Open Banking & Open InsuranceOpen Banking & Open Insurance
Open Banking & Open Insurance
 
Analyzing the World's Largest Security Data Lake!
Analyzing the World's Largest Security Data Lake!Analyzing the World's Largest Security Data Lake!
Analyzing the World's Largest Security Data Lake!
 
Performance monitoring in a DevOps World
Performance monitoring in a DevOps WorldPerformance monitoring in a DevOps World
Performance monitoring in a DevOps World
 
MLOps journey at Swisscom: AI Use Cases, Architecture and Future Vision
MLOps journey at Swisscom: AI Use Cases, Architecture and Future VisionMLOps journey at Swisscom: AI Use Cases, Architecture and Future Vision
MLOps journey at Swisscom: AI Use Cases, Architecture and Future Vision
 
Feature Store as a Data Foundation for Machine Learning
Feature Store as a Data Foundation for Machine LearningFeature Store as a Data Foundation for Machine Learning
Feature Store as a Data Foundation for Machine Learning
 
WSO2Con USA 2017: Brokerage as a Service (BaaS), Transforming Fidelity Broker...
WSO2Con USA 2017: Brokerage as a Service (BaaS), Transforming Fidelity Broker...WSO2Con USA 2017: Brokerage as a Service (BaaS), Transforming Fidelity Broker...
WSO2Con USA 2017: Brokerage as a Service (BaaS), Transforming Fidelity Broker...
 
Monitoring API Performance and Delivering a Scalable API Solution
Monitoring API Performance and Delivering a Scalable API SolutionMonitoring API Performance and Delivering a Scalable API Solution
Monitoring API Performance and Delivering a Scalable API Solution
 
How to Manage APIs in your Enterprise for Maximum Reusability and Governance
How to Manage APIs in your Enterprise for Maximum Reusability and GovernanceHow to Manage APIs in your Enterprise for Maximum Reusability and Governance
How to Manage APIs in your Enterprise for Maximum Reusability and Governance
 
How to Manage APIs in your Enterprise for Maximum Reusability and Governance
How to Manage APIs in your Enterprise for Maximum Reusability and GovernanceHow to Manage APIs in your Enterprise for Maximum Reusability and Governance
How to Manage APIs in your Enterprise for Maximum Reusability and Governance
 
Oow2016 review--paas-microservices-
Oow2016 review--paas-microservices-Oow2016 review--paas-microservices-
Oow2016 review--paas-microservices-
 
João Emilio Santos Bento da Silva - Estratégia de APIs
João Emilio Santos Bento da Silva - Estratégia de APIsJoão Emilio Santos Bento da Silva - Estratégia de APIs
João Emilio Santos Bento da Silva - Estratégia de APIs
 
WSO2 Workshop Sydney 2016 - APIs
WSO2 Workshop Sydney 2016 - APIsWSO2 Workshop Sydney 2016 - APIs
WSO2 Workshop Sydney 2016 - APIs
 
Platform for Secure Digital Business
Platform for Secure Digital BusinessPlatform for Secure Digital Business
Platform for Secure Digital Business
 
Modern Data Architectures for Business Insights at Scale
Modern Data Architectures for Business Insights at ScaleModern Data Architectures for Business Insights at Scale
Modern Data Architectures for Business Insights at Scale
 
Extend soa with api management spoug- Madrid
Extend soa with api management   spoug- MadridExtend soa with api management   spoug- Madrid
Extend soa with api management spoug- Madrid
 
API Management in Digital Transformation
API Management in Digital TransformationAPI Management in Digital Transformation
API Management in Digital Transformation
 
Modern Data Architectures for Business Outcomes
Modern Data Architectures for Business OutcomesModern Data Architectures for Business Outcomes
Modern Data Architectures for Business Outcomes
 
Extend soa with api management Doag18
Extend soa with api management Doag18Extend soa with api management Doag18
Extend soa with api management Doag18
 
#1922 rest-push2 ap-im-v6
#1922 rest-push2 ap-im-v6#1922 rest-push2 ap-im-v6
#1922 rest-push2 ap-im-v6
 

Mais de WSO2

Driving Innovation: Scania's API Revolution with WSO2
Driving Innovation: Scania's API Revolution with WSO2Driving Innovation: Scania's API Revolution with WSO2
Driving Innovation: Scania's API Revolution with WSO2WSO2
 
Less Is More: Utilizing Ballerina to Architect a Cloud Data Platform
Less Is More: Utilizing Ballerina to Architect a Cloud Data PlatformLess Is More: Utilizing Ballerina to Architect a Cloud Data Platform
Less Is More: Utilizing Ballerina to Architect a Cloud Data PlatformWSO2
 
Modernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using BallerinaModernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using BallerinaWSO2
 
WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...
WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...
WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...WSO2
 
WSO2CON 2024 Slides - Unlocking Value with AI
WSO2CON 2024 Slides - Unlocking Value with AIWSO2CON 2024 Slides - Unlocking Value with AI
WSO2CON 2024 Slides - Unlocking Value with AIWSO2
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityWSO2
 
Quantum Leap in Next-Generation Computing
Quantum Leap in Next-Generation ComputingQuantum Leap in Next-Generation Computing
Quantum Leap in Next-Generation ComputingWSO2
 
WSO2CON 2024 - Elevating the Integration Game to the Cloud
WSO2CON 2024 - Elevating the Integration Game to the CloudWSO2CON 2024 - Elevating the Integration Game to the Cloud
WSO2CON 2024 - Elevating the Integration Game to the CloudWSO2
 
WSO2CON 2024 - OSU & WSO2: A Decade Journey in Integration & Innovation
WSO2CON 2024 - OSU & WSO2: A Decade Journey in Integration & InnovationWSO2CON 2024 - OSU & WSO2: A Decade Journey in Integration & Innovation
WSO2CON 2024 - OSU & WSO2: A Decade Journey in Integration & InnovationWSO2
 
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open SourceWSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open SourceWSO2
 
WSO2CON 2024 Slides - Open Source to SaaS
WSO2CON 2024 Slides - Open Source to SaaSWSO2CON 2024 Slides - Open Source to SaaS
WSO2CON 2024 Slides - Open Source to SaaSWSO2
 
WSO2CON 2024 - Does Open Source Still Matter?
WSO2CON 2024 - Does Open Source Still Matter?WSO2CON 2024 - Does Open Source Still Matter?
WSO2CON 2024 - Does Open Source Still Matter?WSO2
 
WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...
WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...
WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...WSO2
 
WSO2CON 2024 - Architecting AI in the Enterprise: APIs and Applications
WSO2CON 2024 - Architecting AI in the Enterprise: APIs and ApplicationsWSO2CON 2024 - Architecting AI in the Enterprise: APIs and Applications
WSO2CON 2024 - Architecting AI in the Enterprise: APIs and ApplicationsWSO2
 
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...WSO2
 
WSO2CON 2024 - Software Engineering for Digital Businesses
WSO2CON 2024 - Software Engineering for Digital BusinessesWSO2CON 2024 - Software Engineering for Digital Businesses
WSO2CON 2024 - Software Engineering for Digital BusinessesWSO2
 
WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...
WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...
WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...WSO2
 
WSO2CON 2024 - Designing Event-Driven Enterprises: Stories of Transformation
WSO2CON 2024 - Designing Event-Driven Enterprises: Stories of TransformationWSO2CON 2024 - Designing Event-Driven Enterprises: Stories of Transformation
WSO2CON 2024 - Designing Event-Driven Enterprises: Stories of TransformationWSO2
 
WSO2CON 2024 - Not Just Microservices: Rightsize Your Services!
WSO2CON 2024 - Not Just Microservices: Rightsize Your Services!WSO2CON 2024 - Not Just Microservices: Rightsize Your Services!
WSO2CON 2024 - Not Just Microservices: Rightsize Your Services!WSO2
 
WSO2CON 2024 - Cloud Native Middleware: Domain-Driven Design, Cell-Based Arch...
WSO2CON 2024 - Cloud Native Middleware: Domain-Driven Design, Cell-Based Arch...WSO2CON 2024 - Cloud Native Middleware: Domain-Driven Design, Cell-Based Arch...
WSO2CON 2024 - Cloud Native Middleware: Domain-Driven Design, Cell-Based Arch...WSO2
 

Mais de WSO2 (20)

Driving Innovation: Scania's API Revolution with WSO2
Driving Innovation: Scania's API Revolution with WSO2Driving Innovation: Scania's API Revolution with WSO2
Driving Innovation: Scania's API Revolution with WSO2
 
Less Is More: Utilizing Ballerina to Architect a Cloud Data Platform
Less Is More: Utilizing Ballerina to Architect a Cloud Data PlatformLess Is More: Utilizing Ballerina to Architect a Cloud Data Platform
Less Is More: Utilizing Ballerina to Architect a Cloud Data Platform
 
Modernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using BallerinaModernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using Ballerina
 
WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...
WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...
WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...
 
WSO2CON 2024 Slides - Unlocking Value with AI
WSO2CON 2024 Slides - Unlocking Value with AIWSO2CON 2024 Slides - Unlocking Value with AI
WSO2CON 2024 Slides - Unlocking Value with AI
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
Quantum Leap in Next-Generation Computing
Quantum Leap in Next-Generation ComputingQuantum Leap in Next-Generation Computing
Quantum Leap in Next-Generation Computing
 
WSO2CON 2024 - Elevating the Integration Game to the Cloud
WSO2CON 2024 - Elevating the Integration Game to the CloudWSO2CON 2024 - Elevating the Integration Game to the Cloud
WSO2CON 2024 - Elevating the Integration Game to the Cloud
 
WSO2CON 2024 - OSU & WSO2: A Decade Journey in Integration & Innovation
WSO2CON 2024 - OSU & WSO2: A Decade Journey in Integration & InnovationWSO2CON 2024 - OSU & WSO2: A Decade Journey in Integration & Innovation
WSO2CON 2024 - OSU & WSO2: A Decade Journey in Integration & Innovation
 
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open SourceWSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
 
WSO2CON 2024 Slides - Open Source to SaaS
WSO2CON 2024 Slides - Open Source to SaaSWSO2CON 2024 Slides - Open Source to SaaS
WSO2CON 2024 Slides - Open Source to SaaS
 
WSO2CON 2024 - Does Open Source Still Matter?
WSO2CON 2024 - Does Open Source Still Matter?WSO2CON 2024 - Does Open Source Still Matter?
WSO2CON 2024 - Does Open Source Still Matter?
 
WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...
WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...
WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...
 
WSO2CON 2024 - Architecting AI in the Enterprise: APIs and Applications
WSO2CON 2024 - Architecting AI in the Enterprise: APIs and ApplicationsWSO2CON 2024 - Architecting AI in the Enterprise: APIs and Applications
WSO2CON 2024 - Architecting AI in the Enterprise: APIs and Applications
 
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
 
WSO2CON 2024 - Software Engineering for Digital Businesses
WSO2CON 2024 - Software Engineering for Digital BusinessesWSO2CON 2024 - Software Engineering for Digital Businesses
WSO2CON 2024 - Software Engineering for Digital Businesses
 
WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...
WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...
WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...
 
WSO2CON 2024 - Designing Event-Driven Enterprises: Stories of Transformation
WSO2CON 2024 - Designing Event-Driven Enterprises: Stories of TransformationWSO2CON 2024 - Designing Event-Driven Enterprises: Stories of Transformation
WSO2CON 2024 - Designing Event-Driven Enterprises: Stories of Transformation
 
WSO2CON 2024 - Not Just Microservices: Rightsize Your Services!
WSO2CON 2024 - Not Just Microservices: Rightsize Your Services!WSO2CON 2024 - Not Just Microservices: Rightsize Your Services!
WSO2CON 2024 - Not Just Microservices: Rightsize Your Services!
 
WSO2CON 2024 - Cloud Native Middleware: Domain-Driven Design, Cell-Based Arch...
WSO2CON 2024 - Cloud Native Middleware: Domain-Driven Design, Cell-Based Arch...WSO2CON 2024 - Cloud Native Middleware: Domain-Driven Design, Cell-Based Arch...
WSO2CON 2024 - Cloud Native Middleware: Domain-Driven Design, Cell-Based Arch...
 

API and Big Data Solution Patterns

  • 1. API and Big data Solution Patterns Srinath Perera Senior Software Architect
  • 2. About  WSO2   •  Providing  the  only  complete  open  source  componen:zed  cloud   pla=orm   –  Dedicated  to  removing  all  the  stumbling  blocks  to   enterprise  agility   –  Enabling  you  to  focus  on  business  logic  and  business   value   •  Recognized  by  leading  analyst  firms  as  visionaries  and  leaders   –  Gartner  cites  WSO2  as  visionaries  in  all  3  categories  of     applica:on  infrastructure   –  Forrester  places  WSO2  in  top  2  for  API  Management   •  Global  corpora:on  with  offices  in  USA,  UK  &  Sri  Lanka   –  200+  employees  and  growing   •  Business  model  of  selling  comprehensive  support  &   maintenance  for  our  products    
  • 3. 150+  globally  posi:oned  support  customers  
  • 4. Outline •  What  is  API   Management   •  What  is  Big   Data?   •  API  manager  +   Bigdata   solu:on   •  API  +  Big  data   PaVerns   •  Conclusion   Photo  by  John  Trainoron  Flickr  hVp://www.flickr.com/photos/trainor/ 2902023575/,  Licensed  under  CC  
  • 5. APIs •  An API is a business capability delivered over the Internet to internal or external consumers –  Network accessible function –  Available using standard web protocols –  With well-defined interfaces –  Designed for access by third-parties •  A Managed API is: –  Actively advertised and subscribe-able –  Available with SLAs –  Secured, authenticated, authorized and protected –  Monitored and monetized with analytics
  • 6. Typical Use Cases •  Expose APIs for public consumption –  Extend your business through APIs. •  Expose APIs for internal consumption –  Manage the APIs used in internal applications –  Detect Usage Patterns –  Internal Monetization •  Control Access to Cloud Services –  Manage and Secure access from internal applications to cloud services (SalesForce, Google Apps, etc.)
  • 7. WSO2 API Management Platform WSO2’s API Management Vision •  Create APIs –  WSO2 Application Server, Data Services Server and ESB •  Find and subscribe/buy APIs –  API Store and Governance •  Manage, secure and protect APIs –  API Management and Gateway •  Monitor and Monetize APIs –  API Monitoring and Analytics •  Develop, host and run API-based applications in a Platform-as-a-Service –  WSO2 Stratos and Cloud Development Platform –  API Cloud
  • 10. API Manager Product and Platform
  • 11. What is Big data? •  There is lot of data available –  E.g. Internet of things •  We have computing power •  We have technology •  Goal To know –  To Explain –  To predict •  Challenge is the full lifecycle
  • 14. Collecting Data •  Data collected at sensors and sent to big data system via events or flat files •  Event Streams: we name the events by its content/ originator •  Get  data  through     –  Point  to  Point   –  Event  Bus  (WSO2  MB)   •  E.g.  Data  bridge  (come  with   WSO2  CEP/  BAM)   – a  thrib  based  transport   that  do  about  400k   events/  sec  
  • 15. Storing Data •  Historically we used databases –  Scale is a challenge: replication, sharding •  Scalable options –  NoSQL (Cassandra, Hbase) [If data is structured] •  Column families Gaining Ground –  Distributed file systems (e.g. HDFS) [If data is unstructured] •  New SQL –  In Memory computing, VoltDB •  Specialized data structures –  Graph Databases, Data structure servers hVp://www.flickr.com/photos/keso/ 363133967/  
  • 16. Making Sense of Data •  To know (what happened?) –  Basic analytics + visualizations –  Interactive drill down –  E.g. MapReduce, Hive, CEP •  To explain (why) –  Data mining, classifications, building models, clustering (e.g. Mahout, Map Reduce) •  To forecast –  Neural networks, decision models (e.g. Mahout)
  • 17. WSO2 Big Data Story
  • 18. API Gateway + Big Data
  • 19. API + Big Data Patterns •  Big Data as API enabler –  Monitor and control: API analytics for an organization –  Analytics: Billing and Metering –  API Store (Market Place) •  Targeted Marketing (Recommendations, Deals, Search) •  Context sensitive prizing –  Deployment optimization –  Governance: Planning and Proactive action •  APIs as a Big Data Enabler –  Easy way to find and get access to data –  Data Market Place –  API Façade for exposing Analytics/ Data –  Controlled delegation of data hVp://www.flickr.com/photos/ooohoooh/ 1350774047/  
  • 20. Pattern 1: Monitor and Control
  • 21. Monitor and Control (Contd.) •  API Manager as single entry point to come in and go out •  Capture data – client, what API, when? $$ revenue .. •  Analyze and Monitor KPIs –  CIO Dashboard –  Alerts •  React to trends faster •  Greater control –  Audit –  Subscriptions policies –  Usage tiers •  Policy enforcement –  Make sure everything is managed
  • 22. Example: Business value driven Org •  Customer facing site is a composite (e.g. like Gmail page) •  Each widget back by API or APIs •  Each widget added removed in runtime •  Widget shown based on business value •  Each API owner get resource compared to business value •  We can use API management as the tool to realize this
  • 23. Pattern 2: Billing and Metering •  API store detect all access •  Contains information about subscriptions, amount of data transferred etc. •  We can use this to –  Meter, control and optimize –  Enforce usage tiers (e.g. Quota via CEP) •  Generate Billing and reports •  Supported by WSO2 API manager now
  • 24. Pattern 3: API Store (Marketplace)
  • 25. Pattern 4: Deployment Optimization
  • 26. App Factory - Applications Management
  • 27. App Factory (Contd.) •  Applications Management –  Creation –  Continuous Build / Deploy –  Promotion across environments •  Team Management –  Add developers, QA engineers and DevOps for your application •  Bugs/Features Tracking •  Integrated with API Store •  Activity Dashboards •  Built on top of Jaggery framework •  Skinnable
  • 28. Pattern 5:API lifecycle Governance •  Take control of your API life cycle (develop, test, production) with WSO2 App Factory •  You can apply analytics to your API lifecycle –  How long does it take to develop and API? –  How often are they updated? –  What are the dependencies of my API? –  What are most effective versions? –  What steps takes most time? –  What APIs are actively developed? –  What one have more issues, how fast are they get fixed?
  • 29. Pattern 6: Find datasets •  Most big data apps need Good data •  Hard to find good data –  No good way to find –  Hard to access –  No clear way to buy •  APIs is a great way to find useful data •  Also to provide controlled access e.g. Andy Stanford-Clark used public data about cruisers to track where availability.
  • 30. Pattern 7: Data Marketplace •  Find, (buy), use data •  Great way to monetize the output of big data projects –  Sell the analyzed traffic data –  Sell stock quote analytics •  API subscribe model can help •  Usage tiers can be based on frequency or resolution of data
  • 31. Pattern 8: Controlled Data Delegation •  Only expose the part of your data set of a processing without exposing credentials of the database to the user
  • 32. Big data in API Store
  • 33. APIs in Big data System
  • 34. Usecase •  Trigger an alert when new version is 10% more slower than the older versions –  Expose the API through API store –  Configure API manager with BAM location –  BAM integration will pump stats (api, version, latency, time) –  Write a BAM query to calculate the latency through time and store it in a DB –  Write a CEP query to detect if new version is slower Define eventTable APIStatHistory .. Select from APIStat as s join APIStatHistory as h on (h.latency - s.latency)<=h.latency*.1
  • 35. Conclusion •  APIs and Big Data has received much attentions •  We looked at WSO2 offerings in both cases – WSO2 API Manager, WSO2 App Factory – WSO2 BAM, WSO2 CEP •  We looked at some patterns on how both can work together •  We looked at how we can implement them
  • 36. Engage  with  WSO2   •  Helping  you  get  the  most  out  of  your  deployments   •  From  project  evalua:on  and  incep:on  to  development  and   going  into  produc:on,  WSO2  is  your  partner  in  ensuring   100%  project  success