SlideShare uma empresa Scribd logo
1 de 29
Baixar para ler offline
Content based filtering,
Pub/Sub
& Bloom filter
Presented By: Yara Ali
Agenda
•
•
•
•
•
•
•
•
•
•
•

Introduction
Human Networks (HUMNETs)
Content-based PublishSubscribe
Pub-sub service network
Bloom filter-based pub-SUB “B-SUB”
B-SUB Components
Bloom Filters (BF)
How Bloom filters work?
Temporal counting Bloom filter (TCBF)
Problems with TCBF
Decaying Factor
Introduction
• Distributed system:
• A system consisting
of several connected
computers that appear
to be one computing
entity.
Introduction .. Cont,
Communication
Mechanism

Client / Server
Architecture

Remote
Procedure call
(RPC)

Message
Oriented
Middleware
(MOM)

Message Queues

Tuple Space

Publish / Subscribe
Architecture
Introduction .. Cont,
• Publish / Subscribe Architectures
1- Lists at server:
Middleware is at the servers
2- Message broker:
Middleware is in a separate unit
3- BroadCast & filter at client:
Middleware is at the clients
Human Networks (HUMNETs)
• It’s a dynamic networks composed of
human-carried wireless devices.
• Applications in HUMNETs require contentbased networking services. (style of
communication that associates source
and destination pairs based on actual
content and interests, rather than letting
source nodes specify the destination)
Content-based
PublishSubscribe (CBPS)
• Content-based matching is the problem of finding all the
subscriptions that match a given notification.
• CBPS represents a compromise between the extremes
of publisher-side filtering of messages ( with event
directly transmitted to interested subscribers ) and
subscriber-side filtering of messages ( with events
broadcasted to all subscribers ).
• Event delivery is the task of delivering the notification to
the set of interested subscribers selected with contentbased matching.
Pub-sub service network
•

Two Approaches :

1. Filter-based approach:
Performs content-based filtering on
intermediate routing servers to dynamically
guide routing decisions.
2. Multicast-based approach:
Delivers events through a few high-quality
multicast groups that are pre-constructed to
approximately match user interests.
Pub-sub service network…Cont,
Pub-sub service network…Cont,
• In the filter-based approach, Routing decisions are made
via successive content-based filtering at all nodes from source
to destination: every pub-sub server along the way matches
the event with remote subscriptions from other servers and
then forwards it only toward directions that lead to matching
subscriptions
• In the multicast-based approach, A limited number of
multicast groups are computed before event transmission
begins. For each event the routing decision is made only once
at the publisher, mapping the event into the single appropriate
group. The event is then multicast to the group assuming IP
multicast or application-level multicast support. Because only
a limited number of multicast groups can be built, servers with
different interests may be clustered into same group and
events may be sent to uninterested servers as well.
Bloom filter-based pub-SUB “BSUB”
• It’s a content-based publish-subscribe
system.
• In B-SUB, messages are identified by
using strings that summarize their
contents. ( called keys )
Bloom filter-based pub-SUB “BSUB” …Cont,
• Pub/sub paradigm is used in B-SUB
Bloom filter-based pub-SUB “BSUB” …Cont,
• Advantages:
1- Frees users from addressing & routing tasks.
(reduces the overall overhead in the system)
2- Message producers & consumers are
separated.
3- Messages are forwarded only by brokers
(Perform content matching for the users)
B-SUB Components
B-SUB

Pub – Sub
forwarding

Broker Allocation

TCBF

Interests
propagation

Message
forwarding
B-SUB Components … Cont,
1- Broker Allocation:
•

Group of socially active nodes are selected to be brokers.

•

Normal users don’t participate in interest propagation & message
forwarding

•

Brokers are responsible for collecting subscriptions and forwarding
messages

•

A Broker stores a TCBF for propagating other users’ interests.
(which is called relay filters)

2- Pub – Sub forwarding
•

It’s separated into 2 parts: interests propagation and message
forwarding
Bloom Filters (BF)
• It’s a space-efficient data structure for representing sets
which supports probabilistic membership querying.
•

is a space-efficient probabilistic data structure that is
used to test whether an element is a member of a set

• BF maps a key through multiple hash functions into a bit
vector of a few bits being set. “ User’s interests are
represented as keys – Also messages are identified by
strings that summarize their contents are called Keys “
Bloom Filters (BF) … Cont,
• The locations of the set bits are determined by
the hash functions.
• A query of a key to a
BF checks if all the
hashed bits of the key
are set, which indicates
if the key is contained
in the BF
Bloom Filters (BF) … Cont,
• A BF for a set of keys is obtained by sequentially
inserting keys into the filter.

10

10

1

1

{K0}

20
+

10

10

1

1

{K1}

{k0,K1}

1

10

10

1

1
Bloom Filters (BF) … Cont,
• To merge multiple BFs we do a bit-wise
OR on them.
10

10

1

1

{K0}

10
M

10

10

1

1

{K1}

{k0,K1}

1

10

10

1

1
Bloom Filters (BF) … Cont,
• The basic BF doesn’t support deletions since we are
unable to trace the associated keys of set bits.
• The counting bloom filter (CBF) is proposed to provide
deletion.
• In a CBF each bit is associated with a counter, which
represent the number of keys that are associated with it.
• To delete a key from a CBF we decrement the counters
of the key’s hashed bits. A bit will be reset once its
counter reaches 0.
Bloom Filters (BF) … Cont,
• The sizes of messages in B-SUB are small
which are in order of hundreds of bytes. This
assumption is true in social networking
applications.
• Ex: twitter; a popular micro-blogging application,
requires a max size of 140 bytes for each post. If
a message is wrongly injected into the network,
the wasted bandwidth is acceptable)
How Bloom filters work?
“Message Forwarding”
• When a producer meets a consumer, the
consumer reports its interests in a BF to the
producer. The producer then queries all its
messages against the filter, and forwards all the
messages that match the filter, to the consumer.
• When a broker meets a producer, it forwards a
BF to the producer. The producer queries that
filter and determines the events that need to be
transmitted.
How Bloom filters work?
“Message Forwarding” … Cont,
• When a broker meets a consumer, the broker requests a
BF containing the consumer’s interests, then forwards
the matched messages to the consumer.
• Message are removed from brokers’ memory after being
forwarded. This is to prevent excessive copies in the
network.
•

Messages’ lifetime is controlled by their time-to-live
(TTL) values, which are identical to their maximum
tolerable delay. The TTL is counted since the message
has been created.
Temporal counting Bloom filter
(TCBF)
•

Extension to BF, proposed to perform content-based networking
tasks.

•

It doesn’t support direct deletion of elements. It only supports
temporal deletion, that is, A filter constantly decrements the
counter’s values of all its set bits, which is called Decaying

•

B-SUB uses TCBF to encode user’s interest & embed information
needed for brokers to make forwarding decisions.

•

B-SUB makes forwarding decisions through querying the TCBFs
( B-SUB can propagate interests by transmitting at most two TCBFs
of dozens of bytes)

•

The operations performed are only hashing and table lookup.
Problems with TCBF
• False positive (Spam) occur because a
key’s hashed bits are accidentally set by
other keys that have been put into the
TCBF.
• Because of false postivies, B-SUB may
falsely inject useless messages into the
network.
Decaying Factor (DF)
• It’s the key for adjusting B-SUB’s behaviors.
• If decaying is not used, the counters of the set
bits don’t change after being set, then no
interests will be removed.
• An obvious consequence is that a broker will
end up with carrying the interests from the users
that it meets rarely.
Decaying Factor (DF)…Cont,
• Suppose that each message has a delay limit of time T,
we should set the DF in such a way that an interest will
get removed after T since a consumer inserted the
interest once.
• If the broker contains the interest, then that means that
the broker has met a consumer that is interested in it
within T.
• If a message is forwarded by the broker it’s likely that the
message will be delivered within T.
References

• http://temple.academia.edu/YaxiongZhao/Papers/1043038/B• http://scholar.google.com/scholar?q=
bloom+filters+in+publish+subscribe&hl=en&btnG=
Search&as_sdt=1%2C5&as_sdtp=on
• http://en.wikipedia.org/wiki/Bloom_filter

• https://www.comp.nus.edu.sg/~david/Publications/de
Thank You !

Mais conteúdo relacionado

Mais procurados

Introduction au Cloud Computing
Introduction au Cloud ComputingIntroduction au Cloud Computing
Introduction au Cloud ComputingMarc Rousselet
 
vSphere7 with Tanzu
vSphere7 with Tanzu vSphere7 with Tanzu
vSphere7 with Tanzu VMware Tanzu
 
Data Sharing using Spectrum Scale Active File Management
Data Sharing using Spectrum Scale Active File ManagementData Sharing using Spectrum Scale Active File Management
Data Sharing using Spectrum Scale Active File ManagementTrishali Nayar
 
VMware vSphere 6.0 - Troubleshooting Training - Day 1
VMware vSphere 6.0 - Troubleshooting Training - Day 1VMware vSphere 6.0 - Troubleshooting Training - Day 1
VMware vSphere 6.0 - Troubleshooting Training - Day 1Sanjeev Kumar
 
Virtualisation
VirtualisationVirtualisation
VirtualisationTsubichi
 
NComputing - A brief overview
NComputing - A brief overviewNComputing - A brief overview
NComputing - A brief overviewNkosinathi Lungu
 
Virtual machines and containers
Virtual machines and containersVirtual machines and containers
Virtual machines and containersPatrick Pierson
 
Server virtualization
Server virtualizationServer virtualization
Server virtualizationofsorganizer
 
11. operating-systems-part-2
11. operating-systems-part-211. operating-systems-part-2
11. operating-systems-part-2Muhammad Ahad
 
VMware Overview
VMware OverviewVMware Overview
VMware OverviewMadhu Bala
 
VMware Interview questions and answers
VMware Interview questions and answersVMware Interview questions and answers
VMware Interview questions and answersvivaankumar
 
VMware vSphere technical presentation
VMware vSphere technical presentationVMware vSphere technical presentation
VMware vSphere technical presentationaleyeldean
 
Vmware desktop infrastructure virtualization assessment
Vmware  desktop infrastructure virtualization assessmentVmware  desktop infrastructure virtualization assessment
Vmware desktop infrastructure virtualization assessmentsolarisyougood
 
Virtual desktop infrstructure (vdi)
Virtual desktop infrstructure (vdi)Virtual desktop infrstructure (vdi)
Virtual desktop infrstructure (vdi)Omega Peripherals
 
PCF-VxRail-ReferenceArchiteture
PCF-VxRail-ReferenceArchiteturePCF-VxRail-ReferenceArchiteture
PCF-VxRail-ReferenceArchitetureVuong Pham
 

Mais procurados (20)

Virtual machine
Virtual machineVirtual machine
Virtual machine
 
Virtualization 101
Virtualization 101Virtualization 101
Virtualization 101
 
Introduction au Cloud Computing
Introduction au Cloud ComputingIntroduction au Cloud Computing
Introduction au Cloud Computing
 
vSphere7 with Tanzu
vSphere7 with Tanzu vSphere7 with Tanzu
vSphere7 with Tanzu
 
Data Sharing using Spectrum Scale Active File Management
Data Sharing using Spectrum Scale Active File ManagementData Sharing using Spectrum Scale Active File Management
Data Sharing using Spectrum Scale Active File Management
 
VMware vSphere 6.0 - Troubleshooting Training - Day 1
VMware vSphere 6.0 - Troubleshooting Training - Day 1VMware vSphere 6.0 - Troubleshooting Training - Day 1
VMware vSphere 6.0 - Troubleshooting Training - Day 1
 
Oracle VM - the Heart of Oracle Cloud
Oracle VM - the Heart of Oracle CloudOracle VM - the Heart of Oracle Cloud
Oracle VM - the Heart of Oracle Cloud
 
Virtualisation
VirtualisationVirtualisation
Virtualisation
 
NComputing - A brief overview
NComputing - A brief overviewNComputing - A brief overview
NComputing - A brief overview
 
Virtual machines and containers
Virtual machines and containersVirtual machines and containers
Virtual machines and containers
 
Server virtualization
Server virtualizationServer virtualization
Server virtualization
 
11. operating-systems-part-2
11. operating-systems-part-211. operating-systems-part-2
11. operating-systems-part-2
 
VMware Overview
VMware OverviewVMware Overview
VMware Overview
 
What is Virtualization
What is VirtualizationWhat is Virtualization
What is Virtualization
 
VMware Interview questions and answers
VMware Interview questions and answersVMware Interview questions and answers
VMware Interview questions and answers
 
Présentation cloud computing
Présentation cloud computingPrésentation cloud computing
Présentation cloud computing
 
VMware vSphere technical presentation
VMware vSphere technical presentationVMware vSphere technical presentation
VMware vSphere technical presentation
 
Vmware desktop infrastructure virtualization assessment
Vmware  desktop infrastructure virtualization assessmentVmware  desktop infrastructure virtualization assessment
Vmware desktop infrastructure virtualization assessment
 
Virtual desktop infrstructure (vdi)
Virtual desktop infrstructure (vdi)Virtual desktop infrstructure (vdi)
Virtual desktop infrstructure (vdi)
 
PCF-VxRail-ReferenceArchiteture
PCF-VxRail-ReferenceArchiteturePCF-VxRail-ReferenceArchiteture
PCF-VxRail-ReferenceArchiteture
 

Destaque

Publish-Subscribe Middlewares
Publish-Subscribe MiddlewaresPublish-Subscribe Middlewares
Publish-Subscribe Middlewareshome
 
Publish subscribe model overview
Publish subscribe model overviewPublish subscribe model overview
Publish subscribe model overviewIshraq Al Fataftah
 
Full-Text Retrieval in Unstructured P2P Networks using Bloom Cast Efficiently
Full-Text Retrieval in Unstructured P2P Networks using Bloom Cast EfficientlyFull-Text Retrieval in Unstructured P2P Networks using Bloom Cast Efficiently
Full-Text Retrieval in Unstructured P2P Networks using Bloom Cast Efficientlyijsrd.com
 
New zealand bloom filter
New zealand bloom filterNew zealand bloom filter
New zealand bloom filterxlight
 
Event Driven Architecture
Event Driven ArchitectureEvent Driven Architecture
Event Driven ArchitectureFabien Doiron
 
Distributed Event Routing in Publish/Subscribe Systems
Distributed Event Routing in Publish/Subscribe SystemsDistributed Event Routing in Publish/Subscribe Systems
Distributed Event Routing in Publish/Subscribe SystemsRoberto Baldoni
 
Homomorphic encryption in_cloud
Homomorphic encryption in_cloudHomomorphic encryption in_cloud
Homomorphic encryption in_cloudShivam Singh
 
Publish Subscribe pattern - Design Patterns
Publish Subscribe pattern - Design PatternsPublish Subscribe pattern - Design Patterns
Publish Subscribe pattern - Design PatternsRutvik Bapat
 
Inverted Index Based Multi-Keyword Public-key Searchable Encryption with Stro...
Inverted Index Based Multi-Keyword Public-key Searchable Encryption with Stro...Inverted Index Based Multi-Keyword Public-key Searchable Encryption with Stro...
Inverted Index Based Multi-Keyword Public-key Searchable Encryption with Stro...Mateus S. H. Cruz
 
How to Build Recommender System with Content based Filtering
How to Build Recommender System with Content based FilteringHow to Build Recommender System with Content based Filtering
How to Build Recommender System with Content based FilteringVõ Duy Tuấn
 
How to build a Recommender System
How to build a Recommender SystemHow to build a Recommender System
How to build a Recommender SystemVõ Duy Tuấn
 
Recommender systems: Content-based and collaborative filtering
Recommender systems: Content-based and collaborative filteringRecommender systems: Content-based and collaborative filtering
Recommender systems: Content-based and collaborative filteringViet-Trung TRAN
 
Recommender Systems
Recommender SystemsRecommender Systems
Recommender SystemsT212
 
Collaborative Filtering Recommendation System
Collaborative Filtering Recommendation SystemCollaborative Filtering Recommendation System
Collaborative Filtering Recommendation SystemMilind Gokhale
 

Destaque (18)

Publish-Subscribe Middlewares
Publish-Subscribe MiddlewaresPublish-Subscribe Middlewares
Publish-Subscribe Middlewares
 
Publish subscribe model overview
Publish subscribe model overviewPublish subscribe model overview
Publish subscribe model overview
 
Full-Text Retrieval in Unstructured P2P Networks using Bloom Cast Efficiently
Full-Text Retrieval in Unstructured P2P Networks using Bloom Cast EfficientlyFull-Text Retrieval in Unstructured P2P Networks using Bloom Cast Efficiently
Full-Text Retrieval in Unstructured P2P Networks using Bloom Cast Efficiently
 
New zealand bloom filter
New zealand bloom filterNew zealand bloom filter
New zealand bloom filter
 
Event Driven Architecture
Event Driven ArchitectureEvent Driven Architecture
Event Driven Architecture
 
Distributed Event Routing in Publish/Subscribe Systems
Distributed Event Routing in Publish/Subscribe SystemsDistributed Event Routing in Publish/Subscribe Systems
Distributed Event Routing in Publish/Subscribe Systems
 
Homomorphic encryption in_cloud
Homomorphic encryption in_cloudHomomorphic encryption in_cloud
Homomorphic encryption in_cloud
 
Publish Subscribe pattern - Design Patterns
Publish Subscribe pattern - Design PatternsPublish Subscribe pattern - Design Patterns
Publish Subscribe pattern - Design Patterns
 
Content based filtering
Content based filteringContent based filtering
Content based filtering
 
Inverted Index Based Multi-Keyword Public-key Searchable Encryption with Stro...
Inverted Index Based Multi-Keyword Public-key Searchable Encryption with Stro...Inverted Index Based Multi-Keyword Public-key Searchable Encryption with Stro...
Inverted Index Based Multi-Keyword Public-key Searchable Encryption with Stro...
 
Publish and Subscribe
Publish and SubscribePublish and Subscribe
Publish and Subscribe
 
HiveServer2
HiveServer2HiveServer2
HiveServer2
 
How to Build Recommender System with Content based Filtering
How to Build Recommender System with Content based FilteringHow to Build Recommender System with Content based Filtering
How to Build Recommender System with Content based Filtering
 
How to build a Recommender System
How to build a Recommender SystemHow to build a Recommender System
How to build a Recommender System
 
Recommender systems: Content-based and collaborative filtering
Recommender systems: Content-based and collaborative filteringRecommender systems: Content-based and collaborative filtering
Recommender systems: Content-based and collaborative filtering
 
Recommender Systems
Recommender SystemsRecommender Systems
Recommender Systems
 
Power Quality
Power QualityPower Quality
Power Quality
 
Collaborative Filtering Recommendation System
Collaborative Filtering Recommendation SystemCollaborative Filtering Recommendation System
Collaborative Filtering Recommendation System
 

Semelhante a Content based filtering, pub sub, bloom filters

Design of an information system for HUNETs
Design of an information system for HUNETsDesign of an information system for HUNETs
Design of an information system for HUNETsrffrankzr
 
Intranet Messaging Project Report -phpapp02
Intranet Messaging Project Report -phpapp02Intranet Messaging Project Report -phpapp02
Intranet Messaging Project Report -phpapp02dvicky12
 
Lexington Bitcoin Meetup - Lightning Network.pptx
Lexington Bitcoin Meetup - Lightning Network.pptxLexington Bitcoin Meetup - Lightning Network.pptx
Lexington Bitcoin Meetup - Lightning Network.pptxPatrickUlrich10
 
AWS Study Group - Chapter 07 - Integrating Application Services [Solution Arc...
AWS Study Group - Chapter 07 - Integrating Application Services [Solution Arc...AWS Study Group - Chapter 07 - Integrating Application Services [Solution Arc...
AWS Study Group - Chapter 07 - Integrating Application Services [Solution Arc...QCloudMentor
 
Apache kafka- Onkar Kadam
Apache kafka- Onkar KadamApache kafka- Onkar Kadam
Apache kafka- Onkar KadamOnkar Kadam
 
Application Layer Protocols for the IoT
Application Layer Protocols for the IoTApplication Layer Protocols for the IoT
Application Layer Protocols for the IoTDamien Magoni
 
MQTT Protocol: IOT Technology
MQTT Protocol: IOT TechnologyMQTT Protocol: IOT Technology
MQTT Protocol: IOT TechnologyShashank Kapoor
 
Botminer Clustering Analysis Of Network Traffic For Protocol And Structure...
Botminer   Clustering Analysis Of Network Traffic For Protocol  And Structure...Botminer   Clustering Analysis Of Network Traffic For Protocol  And Structure...
Botminer Clustering Analysis Of Network Traffic For Protocol And Structure...ncct
 
AndroidThing (Internet of things)
AndroidThing (Internet of things)AndroidThing (Internet of things)
AndroidThing (Internet of things)Mayur Solanki
 
Designing Distributed Systems
Designing Distributed SystemsDesigning Distributed Systems
Designing Distributed SystemsDhananjay Singh
 
1200 wsrr & iib - advanced integration - final
1200   wsrr & iib - advanced integration - final1200   wsrr & iib - advanced integration - final
1200 wsrr & iib - advanced integration - finalsmithson.martin
 
Azure Service Bus Brokered Messaging
Azure Service Bus Brokered MessagingAzure Service Bus Brokered Messaging
Azure Service Bus Brokered MessagingBizTalk360
 
Group presentation.pptx
Group presentation.pptxGroup presentation.pptx
Group presentation.pptxYonas D. Ebren
 
IoT Reference Architecture.pptx
IoT Reference Architecture.pptxIoT Reference Architecture.pptx
IoT Reference Architecture.pptxssuserec53e73
 
Apache Kafka from 0.7 to 1.0, History and Lesson Learned
Apache Kafka from 0.7 to 1.0, History and Lesson LearnedApache Kafka from 0.7 to 1.0, History and Lesson Learned
Apache Kafka from 0.7 to 1.0, History and Lesson LearnedGuozhang Wang
 

Semelhante a Content based filtering, pub sub, bloom filters (20)

Design of an information system for HUNETs
Design of an information system for HUNETsDesign of an information system for HUNETs
Design of an information system for HUNETs
 
Intranet Messaging Project Report -phpapp02
Intranet Messaging Project Report -phpapp02Intranet Messaging Project Report -phpapp02
Intranet Messaging Project Report -phpapp02
 
Lexington Bitcoin Meetup - Lightning Network.pptx
Lexington Bitcoin Meetup - Lightning Network.pptxLexington Bitcoin Meetup - Lightning Network.pptx
Lexington Bitcoin Meetup - Lightning Network.pptx
 
AWS Study Group - Chapter 07 - Integrating Application Services [Solution Arc...
AWS Study Group - Chapter 07 - Integrating Application Services [Solution Arc...AWS Study Group - Chapter 07 - Integrating Application Services [Solution Arc...
AWS Study Group - Chapter 07 - Integrating Application Services [Solution Arc...
 
Apache kafka- Onkar Kadam
Apache kafka- Onkar KadamApache kafka- Onkar Kadam
Apache kafka- Onkar Kadam
 
Application Layer Protocols for the IoT
Application Layer Protocols for the IoTApplication Layer Protocols for the IoT
Application Layer Protocols for the IoT
 
MQTT Protocol: IOT Technology
MQTT Protocol: IOT TechnologyMQTT Protocol: IOT Technology
MQTT Protocol: IOT Technology
 
Wcf routing kt
Wcf routing ktWcf routing kt
Wcf routing kt
 
Botminer Clustering Analysis Of Network Traffic For Protocol And Structure...
Botminer   Clustering Analysis Of Network Traffic For Protocol  And Structure...Botminer   Clustering Analysis Of Network Traffic For Protocol  And Structure...
Botminer Clustering Analysis Of Network Traffic For Protocol And Structure...
 
AndroidThing (Internet of things)
AndroidThing (Internet of things)AndroidThing (Internet of things)
AndroidThing (Internet of things)
 
BOTLAB excersise
BOTLAB excersiseBOTLAB excersise
BOTLAB excersise
 
M split
M splitM split
M split
 
Designing Distributed Systems
Designing Distributed SystemsDesigning Distributed Systems
Designing Distributed Systems
 
1200 wsrr & iib - advanced integration - final
1200   wsrr & iib - advanced integration - final1200   wsrr & iib - advanced integration - final
1200 wsrr & iib - advanced integration - final
 
Azure Service Bus Brokered Messaging
Azure Service Bus Brokered MessagingAzure Service Bus Brokered Messaging
Azure Service Bus Brokered Messaging
 
Group presentation.pptx
Group presentation.pptxGroup presentation.pptx
Group presentation.pptx
 
IoT Reference Architecture.pptx
IoT Reference Architecture.pptxIoT Reference Architecture.pptx
IoT Reference Architecture.pptx
 
Apache Kafka from 0.7 to 1.0, History and Lesson Learned
Apache Kafka from 0.7 to 1.0, History and Lesson LearnedApache Kafka from 0.7 to 1.0, History and Lesson Learned
Apache Kafka from 0.7 to 1.0, History and Lesson Learned
 
Kafka.pptx
Kafka.pptxKafka.pptx
Kafka.pptx
 
SCE 2
SCE 2SCE 2
SCE 2
 

Mais de Yara Ali

Generating a time shrunk lecture video by event
Generating a time shrunk lecture video by eventGenerating a time shrunk lecture video by event
Generating a time shrunk lecture video by eventYara Ali
 
Quality enhamcment
Quality enhamcmentQuality enhamcment
Quality enhamcmentYara Ali
 
Localization in WSN
Localization in WSNLocalization in WSN
Localization in WSNYara Ali
 
Interference mitigation by dynamic self power control in femtocell
Interference mitigation by dynamic self power control in femtocellInterference mitigation by dynamic self power control in femtocell
Interference mitigation by dynamic self power control in femtocellYara Ali
 
Intel® core™ i5 700 desktop processor
Intel® core™ i5 700 desktop processorIntel® core™ i5 700 desktop processor
Intel® core™ i5 700 desktop processorYara Ali
 

Mais de Yara Ali (6)

Generating a time shrunk lecture video by event
Generating a time shrunk lecture video by eventGenerating a time shrunk lecture video by event
Generating a time shrunk lecture video by event
 
Sudoku
SudokuSudoku
Sudoku
 
Quality enhamcment
Quality enhamcmentQuality enhamcment
Quality enhamcment
 
Localization in WSN
Localization in WSNLocalization in WSN
Localization in WSN
 
Interference mitigation by dynamic self power control in femtocell
Interference mitigation by dynamic self power control in femtocellInterference mitigation by dynamic self power control in femtocell
Interference mitigation by dynamic self power control in femtocell
 
Intel® core™ i5 700 desktop processor
Intel® core™ i5 700 desktop processorIntel® core™ i5 700 desktop processor
Intel® core™ i5 700 desktop processor
 

Último

UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Governance in SharePoint Premium:What's in the box?
Governance in SharePoint Premium:What's in the box?Governance in SharePoint Premium:What's in the box?
Governance in SharePoint Premium:What's in the box?Juan Carlos Gonzalez
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Valere | Digital Solutions & AI Transformation Portfolio | 2024
Valere | Digital Solutions & AI Transformation Portfolio | 2024Valere | Digital Solutions & AI Transformation Portfolio | 2024
Valere | Digital Solutions & AI Transformation Portfolio | 2024Alexander Turgeon
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5DianaGray10
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimization100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimizationarrow10202532yuvraj
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
IEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK GuideIEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK GuideHironori Washizaki
 

Último (20)

UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Governance in SharePoint Premium:What's in the box?
Governance in SharePoint Premium:What's in the box?Governance in SharePoint Premium:What's in the box?
Governance in SharePoint Premium:What's in the box?
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Valere | Digital Solutions & AI Transformation Portfolio | 2024
Valere | Digital Solutions & AI Transformation Portfolio | 2024Valere | Digital Solutions & AI Transformation Portfolio | 2024
Valere | Digital Solutions & AI Transformation Portfolio | 2024
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimization100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimization
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
IEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK GuideIEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
 

Content based filtering, pub sub, bloom filters

  • 1. Content based filtering, Pub/Sub & Bloom filter Presented By: Yara Ali
  • 2. Agenda • • • • • • • • • • • Introduction Human Networks (HUMNETs) Content-based PublishSubscribe Pub-sub service network Bloom filter-based pub-SUB “B-SUB” B-SUB Components Bloom Filters (BF) How Bloom filters work? Temporal counting Bloom filter (TCBF) Problems with TCBF Decaying Factor
  • 3. Introduction • Distributed system: • A system consisting of several connected computers that appear to be one computing entity.
  • 4. Introduction .. Cont, Communication Mechanism Client / Server Architecture Remote Procedure call (RPC) Message Oriented Middleware (MOM) Message Queues Tuple Space Publish / Subscribe Architecture
  • 5. Introduction .. Cont, • Publish / Subscribe Architectures 1- Lists at server: Middleware is at the servers 2- Message broker: Middleware is in a separate unit 3- BroadCast & filter at client: Middleware is at the clients
  • 6. Human Networks (HUMNETs) • It’s a dynamic networks composed of human-carried wireless devices. • Applications in HUMNETs require contentbased networking services. (style of communication that associates source and destination pairs based on actual content and interests, rather than letting source nodes specify the destination)
  • 7. Content-based PublishSubscribe (CBPS) • Content-based matching is the problem of finding all the subscriptions that match a given notification. • CBPS represents a compromise between the extremes of publisher-side filtering of messages ( with event directly transmitted to interested subscribers ) and subscriber-side filtering of messages ( with events broadcasted to all subscribers ). • Event delivery is the task of delivering the notification to the set of interested subscribers selected with contentbased matching.
  • 8. Pub-sub service network • Two Approaches : 1. Filter-based approach: Performs content-based filtering on intermediate routing servers to dynamically guide routing decisions. 2. Multicast-based approach: Delivers events through a few high-quality multicast groups that are pre-constructed to approximately match user interests.
  • 10. Pub-sub service network…Cont, • In the filter-based approach, Routing decisions are made via successive content-based filtering at all nodes from source to destination: every pub-sub server along the way matches the event with remote subscriptions from other servers and then forwards it only toward directions that lead to matching subscriptions • In the multicast-based approach, A limited number of multicast groups are computed before event transmission begins. For each event the routing decision is made only once at the publisher, mapping the event into the single appropriate group. The event is then multicast to the group assuming IP multicast or application-level multicast support. Because only a limited number of multicast groups can be built, servers with different interests may be clustered into same group and events may be sent to uninterested servers as well.
  • 11. Bloom filter-based pub-SUB “BSUB” • It’s a content-based publish-subscribe system. • In B-SUB, messages are identified by using strings that summarize their contents. ( called keys )
  • 12. Bloom filter-based pub-SUB “BSUB” …Cont, • Pub/sub paradigm is used in B-SUB
  • 13. Bloom filter-based pub-SUB “BSUB” …Cont, • Advantages: 1- Frees users from addressing & routing tasks. (reduces the overall overhead in the system) 2- Message producers & consumers are separated. 3- Messages are forwarded only by brokers (Perform content matching for the users)
  • 14. B-SUB Components B-SUB Pub – Sub forwarding Broker Allocation TCBF Interests propagation Message forwarding
  • 15. B-SUB Components … Cont, 1- Broker Allocation: • Group of socially active nodes are selected to be brokers. • Normal users don’t participate in interest propagation & message forwarding • Brokers are responsible for collecting subscriptions and forwarding messages • A Broker stores a TCBF for propagating other users’ interests. (which is called relay filters) 2- Pub – Sub forwarding • It’s separated into 2 parts: interests propagation and message forwarding
  • 16. Bloom Filters (BF) • It’s a space-efficient data structure for representing sets which supports probabilistic membership querying. • is a space-efficient probabilistic data structure that is used to test whether an element is a member of a set • BF maps a key through multiple hash functions into a bit vector of a few bits being set. “ User’s interests are represented as keys – Also messages are identified by strings that summarize their contents are called Keys “
  • 17. Bloom Filters (BF) … Cont, • The locations of the set bits are determined by the hash functions. • A query of a key to a BF checks if all the hashed bits of the key are set, which indicates if the key is contained in the BF
  • 18. Bloom Filters (BF) … Cont, • A BF for a set of keys is obtained by sequentially inserting keys into the filter. 10 10 1 1 {K0} 20 + 10 10 1 1 {K1} {k0,K1} 1 10 10 1 1
  • 19. Bloom Filters (BF) … Cont, • To merge multiple BFs we do a bit-wise OR on them. 10 10 1 1 {K0} 10 M 10 10 1 1 {K1} {k0,K1} 1 10 10 1 1
  • 20. Bloom Filters (BF) … Cont, • The basic BF doesn’t support deletions since we are unable to trace the associated keys of set bits. • The counting bloom filter (CBF) is proposed to provide deletion. • In a CBF each bit is associated with a counter, which represent the number of keys that are associated with it. • To delete a key from a CBF we decrement the counters of the key’s hashed bits. A bit will be reset once its counter reaches 0.
  • 21. Bloom Filters (BF) … Cont, • The sizes of messages in B-SUB are small which are in order of hundreds of bytes. This assumption is true in social networking applications. • Ex: twitter; a popular micro-blogging application, requires a max size of 140 bytes for each post. If a message is wrongly injected into the network, the wasted bandwidth is acceptable)
  • 22. How Bloom filters work? “Message Forwarding” • When a producer meets a consumer, the consumer reports its interests in a BF to the producer. The producer then queries all its messages against the filter, and forwards all the messages that match the filter, to the consumer. • When a broker meets a producer, it forwards a BF to the producer. The producer queries that filter and determines the events that need to be transmitted.
  • 23. How Bloom filters work? “Message Forwarding” … Cont, • When a broker meets a consumer, the broker requests a BF containing the consumer’s interests, then forwards the matched messages to the consumer. • Message are removed from brokers’ memory after being forwarded. This is to prevent excessive copies in the network. • Messages’ lifetime is controlled by their time-to-live (TTL) values, which are identical to their maximum tolerable delay. The TTL is counted since the message has been created.
  • 24. Temporal counting Bloom filter (TCBF) • Extension to BF, proposed to perform content-based networking tasks. • It doesn’t support direct deletion of elements. It only supports temporal deletion, that is, A filter constantly decrements the counter’s values of all its set bits, which is called Decaying • B-SUB uses TCBF to encode user’s interest & embed information needed for brokers to make forwarding decisions. • B-SUB makes forwarding decisions through querying the TCBFs ( B-SUB can propagate interests by transmitting at most two TCBFs of dozens of bytes) • The operations performed are only hashing and table lookup.
  • 25. Problems with TCBF • False positive (Spam) occur because a key’s hashed bits are accidentally set by other keys that have been put into the TCBF. • Because of false postivies, B-SUB may falsely inject useless messages into the network.
  • 26. Decaying Factor (DF) • It’s the key for adjusting B-SUB’s behaviors. • If decaying is not used, the counters of the set bits don’t change after being set, then no interests will be removed. • An obvious consequence is that a broker will end up with carrying the interests from the users that it meets rarely.
  • 27. Decaying Factor (DF)…Cont, • Suppose that each message has a delay limit of time T, we should set the DF in such a way that an interest will get removed after T since a consumer inserted the interest once. • If the broker contains the interest, then that means that the broker has met a consumer that is interested in it within T. • If a message is forwarded by the broker it’s likely that the message will be delivered within T.