Mais conteúdo relacionado Semelhante a Gearmam, from the_worker's_perspective copy (20) Gearmam, from the_worker's_perspective copy4. “The way I like to think of Gearman is a
massively distributed fork mechanism”
-Joe Stump, Digg
“The Not Mechanical Turk”
-Don MacAskill, SmugMug
13. Server
Provides Asynchronous and Synchronous Requests
Restarts Work
Durable Requests (MySQL, Postgres,...)
Gearman Protocol/HTTP
Epoch Scheduling
Logging
(also available, native Servers in Java, Erlang, and Perl)
14. Client
# Create our client object.$gmclient= new
GearmanClient();# Add default server
(localhost).$gmclient->addServer();
$result= $gmclient->do("reverse", "Hello!");
echo "Success: $resultn";
15. Worker
# Create our worker object.$gmw= new
GearmanWorker();# Add default server
(localhost).$gmw->addServer();
$gmw->addFunction("reverse",
"reverse_fn");while ($gmworker->work())
{…}
16. Worker Function
function reverse_fn($job){ $workload= $job-
>workload(); $result= strrev($workload);
return $result;}
18. Function
gearman_return_t fetch_url(gearman_job_st *job, void*){ const char *workload=
gearman_job_workload(job); size_t workload_size=
gearman_job_workload_size(job);
gearman_job_send_status(job, 0, 100);
…
gearman_job_send_data(job, chunk, sizeofchunk);
…
gearman_job_send_status(job, 50,100);
… if (issue_warning)
gearman_job_warning(job, “I'm sorry, Dave. I'm afraid I can't do that.”, size);
return GEARMAN_SUCCESS;}
19. Worker Return
GEARMAN_SUCCESS
GEARMAN_FATAL
GEARMAN_ERROR
GEARMAN_SHUTDOWN
20. Your Client
request()
Client API
(C, PHP, Perl, Python, Java,Drizzle, ...)
Network,
Highly Available, Server
Fault Tolerant
Worker API
(C, PHP, Perl, Python, Java, ...)
Your Worker
function()
21. Your Client
resize_image(“ request()
…”);
Client API
(C, PHP, Perl, Python, Java,Drizzle, ...)
Network,
Highly Available, Provided by
Server
Fault Tolerant Gearman
Worker API
resize_image() (C, PHP, Perl, Python, Java, ...)
{
…;
return resized; Your Worker
} function()
26. Other items of
interest?
• Work status requests.
• Chunked Data.
• Exception Handling.
• Up to 4gig message sizes.
• Threaded server.
• Coalescence (the stealth killer feature)
29. reduce()
{…}
map(list[…],
map()
reduce());
{…}
reduce()
{…}
reduce()
{…}
32. Partitioning
gearman_return_t split_worker(gearman_job_st *job, void* /* context */){ const
char *workload= gearman_job_workload(job); size_t workload_size=
gearman_job_workload_size(job); const char *chunk_begin= workload; for
(size_t x= 0; x < workload_size; x++) { if (workload[x] == 0 or workload[x] ==
' ') { gearman_job_send_data(job, chunk_begin,
workload +x -chunk_begin); chunk_begin=
workload +x +1; } } return GEARMAN_SUCCESS;}
34. Aggregation
gearman_return_t cat_aggregator (gearman_aggregator_st *,
gearman_task_st *task,
gearman_result_st *result){ std::string
string_value; do { gearman_result_st *result_ptr=
gearman_task_result(task);
string_value.append(gearman_result_value(result_ptr),
gearman_result_size(result_ptr));
} while ((task= gearman_next(task)));
gearman_result_store_value(result, string_value.c_str(),
string_value.size()); return
GEARMAN_SUCCESS;}
35. Do we have to
partition?
(What other tricks exist!)
38. Gearman.inf
o
• Gearman.org (...)
• http://launchpad.net/gearmand/
• twitter: brianaker
• blog: blog.krow.net
Notas do Editor There is no difference if one of the actors is using one operating system rather than the same one used by the server. Actually, clients can get specific advantage of this architecture, by requesting tasks that are not available in their operating system but can be easily performed in the OS of one of the workers. Language, the greatest divider in the recent technology, is not an obstacle anymore. Clients can keep coding in the language they are most familiar with, and the workers will use libraries and classes that are only available to specific languages, without need for the clients to be involved with the nitpicks of such complex systems.