2. 2
â˘Kevin Schroeder
ď´ Technology Evangelist for Zend
ď´ Programmer
ď´ Sys Admin
ď´ Author
⢠IBM i Programmerâs Guide to PHP
⢠You want to do WHAT with PHP?
ď´ Race Ferraris on the weekend
⢠My Honda has a dismal win record
About me
3. 3
I blog at eschrade.com
Follow us!
(good things will happen to you)
ď´ Zend Technologies
ď´ http://twitter.com/zend
ď´ http://twitter.com/kpschrade (me!)
4. 4
Could your PHP apps benefit from being able to
process data or execute asynchronously?
Twtpoll results
5. 5
â˘Performance
ď´ Execute logic apart from the main request (asynchronicity)
â˘Scalability
ď´ The ability to handle non-immediate logic as resources are
available
Why would you want to queue?
6. 6
â˘People often say that performance and scalability are two
completely different things.
This is inaccurate
â˘Performance is the speed by which a request is executed
â˘Scalability is the ability of that request to maintain its
performance as load/infrastructure increases
Performance & Scalability
7. 7
â˘DONâT!!
â˘You are not Facebook
â˘You probably wonât be
â˘Donât overcomplicate your problems by trying to be
So how do you scale to Facebook size?
8. 8
Typical anatomy of a PHP Application
|
Presentation
Application Control
Database Access
Business Logic
Presentation
Application Control
Business Logic
Presentation
Bad for
scala-
bility!
9. 9| |
Presentation
Database Access
Business Logic
Application Control
Someth
ing.pht
ml
Someth
ing.pht
ml
Someth
ing.pht
ml
Someth
ing.pht
ml
Someth
ing.pht
ml
Someth
ing.pht
ml
Someth
ing.pht
ml
Someth
ing.pht
ml
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Someth
ing.php
Good for
Scalability
12. 12
â˘Pre-caching data
â˘Data analysis
â˘Data processing
â˘Pre-calculating (preparing data for the next request)
Data is âout of dateâ once it leaves the web server
Immediacy is seldom necessary
Asynchronous execution uses
13. 13
â˘A sledgehammer can hit a machine
ď´ Scalability and High Availability are yin and yang
⢠A site that canât keep running is not scalable
⢠A site that canât scale will fail (if it gets really popular)
â˘Machines can be added and removed at will
ď´ Not âcloudyâ necessarily
â˘No single point of failure
ď´ Data exists in at least two, preferably at least three, places
Characteristics
14. 14
â˘Waste disk space
â˘Control usage (donât let users do anything they want)
â˘Pre-calculate as much as possible
ď´ Calculate and cache/store
â˘Donât scan large amounts of data
â˘Keep data processing off the front end servers
â˘Donât just cache
ď´ Donât let it substitute for thought
ď´ Cache hit rates can be meaningless if you have hundreds of cache
hits for a request
Considerations
15. 15
⢠Build a deployment mechanism with NO hardcoded values
like directory or resource locations
⢠Make as much as possible configurable/discoverable
â˘Decouple/Partition
ď´ Donât tie everything (relationships and such) into the database
â˘Use queues/messaging
ď´ Stomp interfaces are really good for PHP â Can also use Java Bridge
ď´ Zend_Queue has several interfaces
â˘Try to use stateless interfaces
ď´ (polling is more scalable than idle connections)
Considerations
16. 16
â˘Use Cron /w PHP CLI (probably shouldnât do this)
â˘Use Gearman
â˘Use home-grown (donât do this)
â˘Use pcntl_fork() (NEVER do this)
â˘Use Zend Server Job Queue
Options
17. 17
Gearman* Zend Server Job Queue
Free
Lightweight
Open Source
(mostly) language
agnostic
Distributed queuing
Ready to go
Integrates with Event Monitoring
Integrates with Code Tracing
Runs over a widely known protocol
Load distribution can be
accomplished outside of the queue
Your only real options
* I am not an expert on Gearman. Corrections will be taken in the spirit that they are given.
Very cloud friendly
For obvious reasons, I will focus on Zend Server
18. 18
â˘Schedule jobs in the future
â˘Set recurring jobs
â˘Execute immediately, as resources are available (my fav)
â˘Utilize ZendJobQueue()
Using the Zend Server Job Queue
19. 19
Job Queue Architecture â Elastic Backend
Users!
Web Server
/w JQ
Web Server
/w JQ
Web Server
/w JQ
Web Server
Web Server
Web Server
LoadBalancer
â˘Pros
ď´ Scale the backend as necessary
ď´ Default (easy) mechanism
â˘Cons
ď´ Getting the job status requires using a DB
20. 20
Job Queue Architecture â Elastic Frontend
Users!
Web Server
Web Server
Web Server
Web Server
/w JQ
Web Server
/w JQ
Web Server
/w JQ
LoadBalancer
â˘Pros
⢠Easy to communicate with the Job Queue server handling the job
ď´Cons
⢠Requires you to build your own interface
21. 21
â˘Create a task-handling controller
â˘Create an abstract task class
ď´ Understands the Job Queue
ď´ Self contained
⢠If Elastic Backend: connects to localhost
⢠If Elastic Frontend: connects to load balancer (my preferred), load
balanced JQ servers manage themselves
â˘Execute the task, have it serialize itself and send it to send
to the task handler
Kevinâs favorite way to implement it
22. 22
ď´ Q_Manager
Handles connecting to the queue and passing results back and forth
ď´ Q_JobAbstract
Abstract class that a job would be based off of
ď´ Q_Response
The response from the manager when a job is queued. Contains the
server name and job number
ď´ Job_Scandir
The actual job that scans the directory
ď´ Job_ScandirResult
An object that represents the data found
Classes involved in the demo
23. 23
ď´ Create job and set data
ď´ Execute job
⢠Job passes itself to the queue manager
⢠Manager serializes job
⢠Manager uses HTTP call through a load balancer to queue the job
⢠The queue on the other end returns the job id and server name
⢠Job ID and server name is passed to the client
ď´ Client polls the manager to get a completed job
⢠When the job is returned pass the serialized version of the executed
job
Execution Flow
24. 24
Letâs write some code
(no Iâm not copping out with slides. Weâre all told to show our work in
grade scool)