This presentation aims to show people that they already know how to deal with concurrency.
It argues that if we have the tools for large scale concurrency (mashups) and small scale (hardware) that midrange (normal apps) can be done in a similar way, using existing tools.
This is done by showing useful systems that have been produced in this manner using existing tools. ie from existing practice, not theory
During the actual presentation I also talked about Kamaelia projects created by novice programmers of varying ability which show high levels of concurrency.
These include: previewing PVR content on mobiles, multicast island joining, as-live streaming using bit torrent, Open GL based user interfaces & integration, seaside style webserving, speex based secure phone, IRC/IM systems, a shakespeare script player, and games tools.
Other systems created include Atom/RSS routing, memcached integration, P2P whiteboarding (with audio + mixing), gesture recognition, presentation tools, a kids development environment, topology visualisation tools, database modelling etc.
Malegaon Call Girls Service ☎ ️82500–77686 ☎️ Enjoy 24/7 Escort Service
Sociable Software
1. Sociable Software
(or “how you really already know
how to make concurrency for multicore
easy to work with”)
Michael Sparks
ms@cerenity.org
http://kamaelia.sourceforge.net/Developers/ BBC Research & Innovation
2. Note added after barcamp
This presentation aims to show people that they already know
how to deal with concurrency.
It argues that if we have the tools for large scale concurrency
(mashups) and small scale (hardware) that midrange (normal
apps) can be done in a similar way, using existing tools.
This is done by showing useful systems that have been produced
in this manner using existing tools.
ie from existing practice, not theory
http://kamaelia.sourceforge.net/Developers/ ms@cerenity.org
3. Note added after barcamp
During the actual presentation I also talked about Kamaelia
projects created by novice programmers of varying ability which
show high levels of concurrency.
These include: previewing PVR content on mobiles, multicast
island joining, as-live streaming using bit torrent, Open GL based
user interfaces & integration, seaside style webserving, speex
based secure phone, IRC/IM systems, a shakespeare script
player, and games tools.
Other systems created include Atom/RSS routing, memcached
integration, P2P whiteboarding (with audio + mixing), gesture
recognition, presentation tools, a kids development environment,
topology visualisation tools, database modelling etc.
http://kamaelia.sourceforge.net/Developers/ ms@cerenity.org
4. Sociable Software
Core idea : “building systems that
are made of many software
systems that are sociable with
each other”
(yes, this project did start at work, but I am primarily working
on it in my own time at present due to a resource crunch :)
12. Basic client side mashup
Web Server
RSS Feed
Web Server
Some
client Atom Feed
Web Server
Web API
13. Basic client side mashup
Web Server
RSS Feed
Web Server
Some
client Atom Feed
Web Server
Example: Google Ads
Web API
14. Basic client side mashup
Web Server
RSS Feed
Web Server
Some
client Atom Feed
Web Server
Example: Google Ads
Or any similar sort of system Web API
ie using Javascript for transcludes
15. Mashup as a service
Web Server
Some RSS Feed
client
Web Server
Web Server Atom Feed
Web Server
Web API
16. Mashup as a service
Web Server
Some RSS Feed
client
Web Server
Web Server Atom Feed
Web Server
Example: gmap-pedometer
Web API
17. More generally
Web Server
Some RSS Feed
client
Web Server
Web Server Atom Feed
Service
Web Server
Web API
19. Web Server
RSS Feed
Web Server
Atom Feed
Web Server
RSS Feed
Web Server
Web API Web API
Web Server
Web Server Atom Feed
Service
Web Server
Web Server RSS Feed
Web API Web API
Web Server
Web API
Web Server
Service
` Web Server
Web API
30. Both represent concurrent systems.
One is very high level.
One is incredibly low level.
The low level has the best tools and builds on
decades of experience of making things simpler.
They've made their lives easier
31. green Pipeline() =
Random() >> buffer(1) >>
Invert() >> buffer(1) >>
Invert() >> buffer(1) >> sink
end
32. green Substituter() =
chan thevalue,istwelve end
function test(value x:integer) : boolean =
x==12
end
Random() >> thevalue >> test() >> istwelve$
thevalue >> if istwelve then
sink$ source 0 >>
else
{}
end_if >> sink
end
33. green Substituter() =
chan thevalue,istwelve end
function test(value x:integer) : boolean =
Um?
x==12
end
Random() >> thevalue >> test() >> istwelve$
thevalue >> if istwelve then
sink$ source 0 >>
else
{}
end_if >> sink
end
34. green Substituter() =
chan thevalue,istwelve end
function test(value x:integer) : boolean =
Um?
x==12
end
Random() >> thevalue >> test() >> istwelve$
thevalue >> if istwelve then
sink$ source 0 >>
else
{}
end_if >> sink
end
That looks like software!
35. Can we write software that way
and still have it perform well,
and gain the same benefits?
38. Examples
I am only going to show you non-trivial
examples
For more trivial examples, please look at the
slides uploaded to slideshare.net/kamaelian
and also the Kamaelia cookbook.
39. Examples
Email Greylisting Server: Kamaelia Grey
Specifically written to deal with the ever growing
spam problem I was facing
Has eradicated my spam problem on my home
server.
43. Examples
Email Greylisting Core Code:
class GreylistServer(MoreComplexServer):
logfile = config[quot;greylist_logquot;]
...
port = config[quot;portquot;]
socketOptions=(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
class TCPS(TCPServer):
CSA = NoActivityTimeout(ConnectedSocketAdapter,
timeout=config[quot;inactivity_timeoutquot;])
class protocol(GreyListingPolicy):
servername = config[quot;servernamequot;]
smtp_ip = config[quot;smtp_ipquot;]
smtp_port = config[quot;smtp_portquot;]
...
44. Examples Selector
Email Greylisting Core Code: TCPServer
Greylist
Server
class GreylistServer(MoreComplexServer):
logfile = config[quot;greylist_logquot;] Connected
GreyListing
Socket
... Adapter Policy
port = config[quot;portquot;]
socketOptions=(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
class TCPS(TCPServer):
CSA = NoActivityTimeout(ConnectedSocketAdapter,
timeout=config[quot;inactivity_timeoutquot;])
class protocol(GreyListingPolicy):
servername = config[quot;servernamequot;]
smtp_ip = config[quot;smtp_ipquot;]
smtp_port = config[quot;smtp_portquot;]
...
45. Examples Selector
Email Greylisting Core Code: TCPServer
Greylist
Server
class GreylistServer(MoreComplexServer):
logfile = config[quot;greylist_logquot;] Connected
GreyListing
Socket
... Adapter Policy
port = config[quot;portquot;]
socketOptions=(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
class TCPS(TCPServer):
CSA = NoActivityTimeout(ConnectedSocketAdapter,
timeout=config[quot;inactivity_timeoutquot;])
class protocol(GreyListingPolicy):
servername = config[quot;servernamequot;]
smtp_ip = config[quot;smtp_ipquot;]
smtp_port = config[quot;smtp_portquot;]
...
46. Examples Selector
Email Greylisting Core Code: TCPServer
Greylist
Server
class GreylistServer(MoreComplexServer):
logfile = config[quot;greylist_logquot;] Connected
GreyListing
Socket
... Adapter Policy
port = config[quot;portquot;]
socketOptions=(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
class TCPS(TCPServer):
CSA = NoActivityTimeout(ConnectedSocketAdapter,
timeout=config[quot;inactivity_timeoutquot;])
class protocol(GreyListingPolicy):
servername = config[quot;servernamequot;]
smtp_ip = config[quot;smtp_ipquot;]
smtp_port = config[quot;smtp_portquot;]
...
47. Examples Selector
Email Greylisting Core Code: TCPServer
Greylist
Server
class GreylistServer(MoreComplexServer):
logfile = config[quot;greylist_logquot;] Connected
GreyListing
Socket
... Adapter Policy
port = config[quot;portquot;]
socketOptions=(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
class TCPS(TCPServer):
CSA = NoActivityTimeout(ConnectedSocketAdapter,
timeout=config[quot;inactivity_timeoutquot;])
class protocol(GreyListingPolicy):
servername = config[quot;servernamequot;]
smtp_ip = config[quot;smtp_ipquot;]
smtp_port = config[quot;smtp_portquot;]
...
48. Examples Selector
Email Greylisting Core Code: TCPServer
Greylist
Server
class GreylistServer(MoreComplexServer):
logfile = config[quot;greylist_logquot;] Connected
GreyListing
Socket
... Adapter Policy
port = config[quot;portquot;]
socketOptions=(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
class TCPS(TCPServer):
CSA = NoActivityTimeout(ConnectedSocketAdapter,
timeout=config[quot;inactivity_timeoutquot;])
class protocol(GreyListingPolicy):
servername = config[quot;servernamequot;]
smtp_ip = config[quot;smtp_ipquot;]
smtp_port = config[quot;smtp_portquot;]
...
49. Slightly less obvious than normal, but these
functions together form a factory.
outbox
Selector inbox
outbox
TCPServer inbox
Greylist
Server
50. Slightly less obvious than normal, but these
functions together form a factory... so as to
create new things to handle connections:
Selector
TCPServer
Greylist
Server
Connected
Socket GreyListing
Adapter Policy
51. Slightly less obvious than normal, but these
functions together form a factory... so as to
create new things to handle connections:
Selector
TCPServer
Greylist
Server
Connected
Socket GreyListing
Adapter Policy
Connected
Socket GreyListing
Adapter Policy
52. Slightly less obvious than normal, but these
functions together form a factory... so as to
create new things to handle connections:
Selector
TCPServer
Greylist
Server
Connected
Socket GreyListing
Adapter Policy
Connected
Socket GreyListing
Adapter Policy
Connected
Socket GreyListing
Adapter Policy
65. Examples
Kamaelia Macro
It records and transcodes
what is broadcast over DTT
for future viewing.
17A. (1) The making in domestic premises for private and domestic use of a recording of a broadcast
solely for the purpose of enabling it to be viewed or listened to at a more convenient time does not
infringe any right conferred by Part 2 in relation to a performance or recording included in the
broadcast.
Timeshifting as defined by copyright
68. Examples
Kamaelia Macro: Transcode progs on a Channel
inbox
inbox outbox inbox
Splitter
Restartable
_eit_
Transcoder
transcoder_factory
inbox Prog outbox next
Change
Detector
Channel Transcoder
69. Examples
Kamaelia Macro: Finding Programme Junctions
inbox
inbox
inbox
DVB Packet Now Next
Reconstructor Parser
outbox
outbox
inbox
Now Next Now Next
inbox
Changes service filter
outbox
outbox
EIT Parsing
outbox
70. Examples
Kamaelia Macro: Programme Transcoder
inbox
control (a pipeline)
inbox control
Pipethrough
(“mencoder o file options...”) _stop
signal
_transcodingcomplete signal
Programme Transcoder
71. How?
What's inside the box?
The following examples are largely simplified,
simply because slides are a terrible way to
show code.
I've generally removed shutdown code.
72. How?
What's inside the box?
class ConsoleReader(threadedcomponent):
def main(self):
while 1:
line = raw_input(quot;>>quot;)
line = line + self.eol
self.send(line, quot;outboxquot;)
73. How?
What's inside the box?
class ConsoleReader(threadedcomponent):
def main(self):
while 1: See – we're saying this
line = raw_input(quot;>>quot;) component runs inside it's
line = line + self.eol own thread. This is all we have
self.send(line, quot;outboxquot;) to write for it to work.
74. How?
What's inside the box?
class ConsoleReader(threadedcomponent):
def main(self):
while 1:
line = raw_input(quot;>>quot;)
line = line + self.eol
self.send(line, quot;outboxquot;)
As you can see here, a
component communicates
with other components by
sending things to outboxes.
75. How?
What's inside the box?
class ConsoleEchoer(component):
def main(self):
while not self.shutdown():
while self.dataReady(quot;inboxquot;):
data = self.recv(quot;inboxquot;)
sys.stdout.write(str(data)) Whereas here, you can see
sys.stdout.flush() that they listen for data in
yield 1 their inboxes. and then do
stuff based on that!
76. How?
What's inside the box?
This one however is a
class ConsoleEchoer(component): generator based component.
def main(self): This runs in the same thread
while not self.shutdown(): as all other generators,
while self.dataReady(quot;inboxquot;): making this code equivalent to
data = self.recv(quot;inboxquot;) a state machine. Note the
sys.stdout.write(str(data)) baseclass does not say
sys.stdout.flush() threaded or generator.
yield 1
77. How?
What's inside the box?
class ConsoleEchoer(component):
def main(self):
while not self.shutdown():
while self.dataReady(quot;inboxquot;):
data = self.recv(quot;inboxquot;)
sys.stdout.write(str(data))
sys.stdout.flush()
yield 1
This yield keyword is what makes
this method a generator. For
now, you can assume it acts like
a “return back here when next
called” thing. It's NOT really the
same at all as a ruby yield!
78. And that's the basic idea.
On the website there's a mini axon tutorial and a
guide to writing components, and a few
magazine articles as well which should get you
started.
There are also more tutorials on slideshare
Please also ask on #kamaelia on freenode.
