Do you often tell your potential customer or investor things like “This is a PoC”, “It only works on a single host”, “We need time to install on another server”, “We need to redesign the code to handle ten thousand users” ?
You need to have solid answers to how your platform scales on growing demand if you aim for Silicon Valley.
In the first part of this seminar, we’ll focus on non-technical stakeholders like CEO and CMO co-founders, and we’ll have an overview of designing enterprise grade B2B, B2C or B2G Software as a Service (SaaS) platforms using cloud technologies. What are the best practices to scalable, highly available and reliable SaaS platforms and the store front, customer support channels? Why is an on-demand, elastic IT infrastructure setup necessary? What do scalability and automation have to do with the OPEX and the Business Case?
In the second part of this seminar, we’ll go into technical details for CTO co-founders and give examples for realizing platforms that scale from 1 to 1M User on cloud. You’ll also have an understanding of the products and services needed to realize this. You’ll also have an understanding of the products and services needed to realize this from Amazon (AWS) Cloud: Shopify (e-shop), Zendesk (Customer Support), AWS API Gateway, Lambda (Serverless Computing), EC2 (Virtual servers), ASG (Auto Scaling), SNS (Notification Service), CloudWatch (Fault and Performance Monitoring & Centralized Logging), CloudTrail (Compliance), CloudFormation (Infrastructure Automation).
Note: This session was presented in ITU Cekirdek, http://www.itucekirdek.com/en The University Startup Ecosystem in Istanbul, on December 13th 2016.
https://www.linkedin.com/pulse/creating-saas-startups-scale-millions-users-akirmak-msc-exec-mba
Powerful Google developer tools for immediate impact! (2023-24 C)
Creating SaaS Startups that Rock: Scaling to Millions of Users
1. Creating the SaaS Startups that
Scale to Millions of UsersCreating the SaaS Startups that
Scale to Millions of Users
Hasan Basri AKIRMAK
Mentor at ITU Seed, Cloud Evangelist at Ericsson
https://tr.linkedin.com/in/hasanbasriakirmak
2. This session was presented in ITU Cekirdek,
http://www.itucekirdek.com/en
The University Startup Ecosystem in Istanbul,
on December 13th 2016.
3. Do you often tell your potential customer or investor things like “This is a PoC”,
“It only works on a single host”, “We need time to install on another server”,
“We need to redesign the code to handle ten thousand users” ?
You need to have solid answers to how your platform scales on growing
demand if you aim for Silicon Valley.
In the first part of this seminar, we’ll focus on non-technical stakeholders like
CEO and CMO co-founders, and we’ll have an overview of designing
enterprise grade B2B, B2C or B2G Software as a Service (SaaS) platforms
using cloud technologies. What are the best practices to scalable, highly
available and reliable SaaS platforms and the store front, customer support
channels? Why is an on-demand, elastic IT infrastructure setup necessary?
What do scalability and automation have to do with the OPEX and the
Business Case?
In the second part of this seminar, we’ll go into technical details for CTO co-
founders and give examples for realizing platforms that scale from 1 to 1M
User on cloud. You’ll also have an understanding of the products and services
needed to realize this. You’ll also have an understanding of the products and
services needed to realize this from Amazon (AWS) Cloud: Shopify (e-shop),
Zendesk (Customer Support), AWS API Gateway, Lambda (Serverless
Computing), EC2 (Virtual servers), ASG (Auto Scaling), SNS (Notification
Service), CloudWatch (Fault and Performance Monitoring & Centralized
Logging), CloudTrail (Compliance), CloudFormation (Infrastructure Automation).
Motivation
4. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 4
What to expect from this session
› Part 1
–“Ticket to Haydarpasa or Silicon Valley” – Google Ventures Engineering Team
–“I’m gonna beat u Zuckerberg” – Industrialized Production
–“From Illusionist to Magician” – The B2B Perspective
–“From Black Friday to Godzilla” – Defining Business Performance
–“The Business Case for SaaS” – OPEX, CAC, Outsourcing
› Part 2
–“Building Infrastructure at Scale” – Well Architected Framework
–“Running it” – Tracking Performance with KPIs
–“Time vs. Space Tradeoff” – Compute, Storage, Data and Apps: The New Normal
–“Putting Pieces Together” – Scalable SaaS Platform on AWS
5. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 5
The opinions expressed here represent my
own and not those of my employer, AWS or
any other trademarks/companies
mentioned herein.
DISCLAIMER
Creative Commons License
Licensees may copy, distribute, display and
perform the work and make derivative works
and remixes based on it only if they give the
author or licensor the credits (attribution).
LICENSE
8. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 8
› Engineering
› “When a startup begins to take off, the
technical requirements for data,
computing, and networking skyrocket. At
GV, we’ve built a team that lives for these
challenges. They’re a group of experts
with a track record of working at massive
scale, and they love to help.”
› — Graham Spencer
› General Partner at GV
Why this session?
Source: Google Ventures
9. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 9
“Adidas’s pilot factory in Ansbach is an
example of how robots are revolutionising
manufacturing.” [Financial Times]
You need to be on the
right to beat Silicon
Valley.
10. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 10
In Software Context
› Single developer vs. Mass production
› Small Workshop vs. Factories
– Standardize HW, SW and Operations for Innovation
– Automate (No human interaction) for OPEX
Industrial Production
11. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 11
› “You can sell products to B2C using
emotional levers (games, fashion, etc.) but, in
B2B, getting customers excited will not get
you a check.”
› “B2C consumers are accustomed to ‘what
you see is what you get.’ They can choose to
use it or not. Businesses have higher
expectations (security, customization,
integration, performance, etc.)*
Source: Lean B2B: Build Products Businesses Want, Garbugli, Étienne.
Image: IMDB
B2B also has higher expectations
13. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 14
› Revenue Drivers
– Consider upfront payments
– Contract Length & discounts
– Monthly Recurring Revenues
– Churn effect
› CAC – Cost of Acquiring
Customer
– Digital marketing to help you
› LTV – Customer’s Life Time Value
– Retaining the subscriber
– Add-on sales (new features, or
increase consumption)
SaaS Business Case – Revenues
Source: http://www.forentrepreneurs.com/saas-sales-compensation-plan/
14. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 15
› Revenue Drivers
– Consider upfront payments
– Contract Length & discounts
– Monthly Recurring Revenues
– Churn effect
› CAC – Cost of Acquiring
Customer
– Digital marketing to help you
› LTV – Customer’s Life Time Value
– Retaining the subscriber
– Add-on sales (new features, or
increase consumption)
SaaS Business Case – Revenues
Source: http://www.forentrepreneurs.com/saas-sales-compensation-plan/
15. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 16
SaaS Business Case – Costs
› COGS
– IT infrastructure components
› CAPEX: Acquisition Costs
- Server, Storage, Networking
- SW Development: Reliability,
Availability, Security
– OPEX: Operation Costs
– Support Costs
› 1st line: Web, mail or Call Center
› 2nd line: Software Development Team
› 3rd line: 3PP (AWS, Shopify, …)
› Risk Mitigation Costs
– How much Data can we afford to lose?
– How much downtime can we accept?
Diagram Source: AWS
16. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 17
SaaS Business Case – Costs
› COGS
– IT infrastructure components
› CAPEX: Acquisition Costs
- Server, Storage, Networking
- SW Development: Reliability,
Availability, Security
– OPEX: Operation Costs
– Support Costs
› 1st line: Web, mail or Call Center
› 2nd line: Software Development Team
› 3rd line: 3PP (AWS, Shopify, …)
› Risk Mitigation Costs
– How much Data can we afford to lose?
– How much downtime can we accept?
Diagram Source: AWS
17. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 18
› Make sure customers are happy after purchase so
they will remain long term customers.
› Book as much New Recurring Revenue as
possible.
› Collect as much Cash Upfront as possible.
(Particularly important in the startup phase)
› Sign Longer Term Contracts (e.g. annual terms vs.
monthly, multi-year vs. annual)
› Drive expansion revenue with existing customers to
have a revenue retention rate of greater than 100%,
even when you lose some customers due to churn.
( “Negative churn”)
Source: http://www.forentrepreneurs.com/saas-sales-compensation-plan/
What are the Key business objectives for a
SaaS business?
18. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 19
Identify which one is
more profitable:
Adding New Users
vs. Stimulating
Consumption?
19. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 20
For Small Contract
size, use Internet as
Primary Mode of
Distribution
Source: http://www.forentrepreneurs.com/
20. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 21
› Compare your current on-premises environment to your FUTURE STATE, RE-
ARCHITECTED cloud environment. NOT a replica of what exists today.
Cloud is more than a technology
Business Agility
› Rapid experimentation.
› Faster app development
› Faster business decisions
Source: AWS ReInvent 2016
Workforce Productivity
› Self service culture
› Increased automation
› Attracting & retaining talent
Operational Resilience
› Improved Service Performance
› Less Downtime
› Less bugs
Other (In)tangible Benefits you can qualify
21. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 22
Technologies that are difficult to implement can become easier to consume by pushing
that knowledge and complexity into the cloud vendor's domain.
For example, NoSQL databases, media transcoding, and machine learning are all
technologies that require expertise that is not evenly dispersed across the technical
community. In the cloud, these technologies become services that your team can
consume while focusing on product development rather than resource provisioning and
management.
Cloud Democratizes Advanced Technologies
22. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 23
Virtualization is NOT cloud
23. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 24
› Migration (especially with redesign is complex and costly)
Never say “We move to cloud later!”
Source: AWS Cloud Adoption Framework
24. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 25
Time is Money. Performance Matters
25. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 26
Understanding End to End Performance
26. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 27
Godzilla will visit you one day
Image: IMDB
27. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 28
Acceptable Downtime and Loss of Data for
your Business
Identify where your startup
should be
28. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 29
› Backup Requirements for each workload
› HA (High Availability) requirements for each workload
› Scalability requirements for each workload
› DR (Disaster Recovery) Requirements for each workload
Performance Requirements: A Nice to Have?
29. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 30
How do we match with demand, adding/removing
resources when needed?
“Righscaling”.
Time-based, event-driven approaches
The big question
30. Part 2
Creating a software system is a lot like
constructing a building. If the foundation is
not solid there might be structural
problems that undermine the integrity and
function of the building.
When architecting
technology solutions,
do not neglect the four
pillars of security,
reliability, performance
efficiency, and cost
optimization.
31. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 32
Fallacies of Software Engineering
›Servers don’t fail
›Disks don’t fail
›Networks don’t fail
In the cloud, test how your system fails, and validate your recovery procedures.
32. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 33
SaaS Architecture – Apps vs Ops View
Data Partitioning
Tenant Isolation
Identity
Profiling&Analytics
Management&Monitoring
Metering,Billing&Tenant
Administration
Operational View
AppView
Technical and Business Agility
33. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 34
Understanding Multi-tenancy
Silo Pool
vs.
› Silo Model
› Pros
– Tenant specific tuning
– Tenant level availability
› Cons
– Cost
– Management Complexity
– Deployment Challenges
– Analytics aggregation
› Pool Model
› Pros
– Centralized management
– Cost optimization
› Cons
– Cross-tenant impacts
– Compliance challenges
– All or nothing available
34. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 35
TENANT ID ITEM ID
T1 230
t2 3434
Data partitioning
Tenant 1 Tenant 2
Schema
Schema
Separate DB for each
tenant
Single DB, multiple
schemas
Common DB, single
schema
35. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 36
› Actively track KPIs
– Resource Level: CPU load, memory usage, network traffic
– Business level: Response times, RTO and RPO, Costs …
› Use Automation: Logging in and checking the CPU manually is not scalable!
› Implement self healing
– Example Monitoring: Send an alarm when cache is 80%
– Example Action: Purge cache automatically when 90%
– Example Monitoring: Sense list of IP’s generating unsuccessful SSH attempts.
– Example Action: Block those IP’s by adding them to firewall blacklist rules automatically
› Learn to implement your “Infrastructure as Code”
Managing your infrastructure at scale
How do you monitor your instances post launch to ensure they are performing as expected?
Amazon
CloudWatch
AWS
CloudFormation
Amazon
Elasticsearch
Service
36. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 37
Performance Efficiency in the cloud is composed of four areas:
1 Compute
2. Storage
3. Database
4. Space-time trade-off
37. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 38
› Share info
› Extract insight
› Recognize patterns
› Track performance
› Ultimately make better business,
technical and operational decisions.
What Can You Do with all this Data?
Generation
Collection & Storage
Analytics & computation
Collaboration & Sharing
39. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 40
Don’t do a Monolithic DB + Search Tier
RDBMS
DB & Search Tier
Apps
40. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 41
Types of Data
› Transactional
– DB reads & writes (OLTP__
– Cache
› Search
– Logs
– Streams
› File
– Log files
› Stream
– Clickstream, Sensors, IoT data
Big Data Architectural Principles &
Understanding Types of Data
41. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 42
Use the right tool for the job. There are
multiple services for DB & Analytics in AWS
42. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 43
Data Structure and Access Patterns
ACCESS PATTERNS WHAT TO USE in AWS?
Joins, Transaction, SQL SQL: RDS
Put/Get (Key/Value) Cache: ElastiCache
(Memcached, Redis)
NoSQL: DynamoDB
Simple Relationships, No Transaction, Joins NoSQL: DynamoDB
Search Search: ElasticSearch
DATA STRUCTURES WHAT TO USE in AWS?
Fixed Schema SQL: RDS
NoSQL: DynamoDB
Schema Free (JSON) NoSQL: DynamoDB
Search: ElasticSearch
Key Value Cache: ElastiCache
(Memcached, Redis)
NoSQL: DynamoDB
43. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 44
Use the Right Tool for the Job
DB & Search Tier
Apps
Cache
MemCached, Redis
NoSQL
Cassandra, DynamoDB
Hbase, MongoDB
SQL
MySQL, Oracle,
SQL Server, Postgres
Search
ElasticSearch,
CloudSearch
45. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 46
The optimal storage solution for a particular system will vary based on the:
› Kind of access method Block, file, or object
› Patterns of access Random or sequential
› Throughput required IOPS, SSD vs. HDD
› Frequency of access Online, offline, archival
› Frequency of update WORM, dynamic
› Availability and durability constraints 99 vs. 99.9…9% and more
Understand Storage Parameters
46. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 47
› Use Stateless Design: Support 12-factor stateless apps: 12factor.net
› As Deployment Unit: Use Docker container images
Modern Software Design Goals
47. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 48
Putting Pieces Together
A Scalable SaaS Platform Implementation
AWS API
(order fulfilment triggered by e-commerce platform)
AWS CloudFormation
Infrastructure as Code
AWS Lambda
Send login and web address information
to customer
per email using SES
AWS CloudWatch
Infra & Application layer
Fault & Performance Monitoring
AWS CloudTrail
Compliance reporting (Log of all requests
to AWS account and API)
AWS Config
Resource Configuration Management
keeping track of versions of every resource
AWS DynamoDB
Metadata repository (file name, size…)
AWS S3
Storage with 99.999999999% durability
Tenant isolation by buckets
AWS SNS
Email notifications to
Operations Team
about alarms
AWS EC2
Compute service for Open Source Sync Sw
Single tenant for tenant isolation
AWS EBS
NAS Storage service, for EC2 cache
AWS ASG
Replaces EC2 in case
of sys or health check errors
AWS Route 53
DNS Service
Shopify.com
SaaS E-Commerce Platform as sales channel
Zendesk.com
SaaS Customer Support Channel
:
A Marketplace for Partner
Products, Solutions or
SaaS running on AWS
AWS IAM
MFA & Cross Account
Access to Confounders
Bitnami.com
3PP Baseline AMIAWS CloudFront
CDN for Joomla Microsite
48. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 49
SaaS Fueled by Digital Marketing
49. ReferencesThe Best Practices explained here and used in the scalable SaaS platform implementation
are based on several advanced AWS Seminars and whitepapers.
AWS ReInvent 2015 Sessions available on YouTube
1. DVO315 Log Monitor Events with AWS CloudWatch, Lambda and Dashboards. Link
2. BDT317 Building a Data Lake on AWS. Link
3. WEB401 Optimizing your Web Server on AWS. Link
4. SEC313 Security by Design. Link
5. DEV203 AWS API Gateway and Lambda. Link
6. DVO303 Scaling Infrastructure Operations AWS Config and AWS CloudTrail. Link
7. ISM402 Cost Optimization at Scale. Link
8. ARC302 Running Lean Architectures: Optimizing for Cost Efficiency. Link
AWS ReInvent 2016 Sessions (Not available on YouTube yet)
1. ARC301 Architecting Next Generation SaaS
2. ENT308 Building a Solid Business Case for Cloud Migration
APN (AWS Partner Network) Webcasts (You need to be an AWS Partner to access this content)
1. SaaS Architecture ()
2. Amazon EC2 Masterclass Webcast
AWS Whitepapers
1. AWS Well Architected Framework. Link
2. Managing Infrastructure at Scale. Link
50. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 51
›Lessons Learned from Creating a B2B Cloud SaaS Startup
in 90 Days: From Ideation to Launch
›Designing an End to End Security Architecture in Cloud
›20 AWS Services in 120 Minutes
FOR MORE INFO ABOUT
LEARNINGS & OUTCOMES, JOIN THE
UPCOMING SEMINARS:
51. İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 52
If you found this session useful, please “endorse” me in LinkedIn for areas like:
Business Development, Business Planning, Go-To-Market Strategy, Consulting,
Product Management, Product Strategy, Cloud Computing, Entrepreneurship.
THANK YOU
Hasan Basri AKIRMAK
Differentiate in Information Technology