Pnuts Review

PNUTS: Yahoo!’s Hosted Data Serving Platform B.F. Cooper, R. Ramakrishnan, U. Srivastava, A. Silberstein, P. Bohannon, H. Jacobsen, N. Puz, D. Weaver and R. Yerneni Yahoo! Research Seminar Presentation for CSE 708 by Ruchika Mehresh Department of Computer Science and Engineering 22 nd February, 2011

Motivation ,[object Object],[object Object]

What does Yahoo! need? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Serializable transactions Vs Eventual consistency Serializability

PNUTS Data Storage and Retrieval Features Data and Query Model System Architecture Consistency (Yahoo! Message Broker) Query Processing Experiments Recovery Structure Future Work

Features ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data and Query Model ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data Storage and Retrieval ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data Storage and Retrieval ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Consistency model ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Related Question Insert Update Delete Update Insert Update v.1.0 v.1.1 v.1.2 v.1.3 v.2.0 v.2.1 v.2.2

Consistency model ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Animation

Yahoo! Message broker ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Yahoo! Message broker ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Related Question (Write locality) Related Question (Tablet Master)

Recovery ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Related Question

Bulk load ,[object Object],[object Object],[object Object],[object Object],Avoiding hot spots in ordered table

Query Processing ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Notifications ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Experimental setup ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Experiments ,[object Object],[object Object],[object Object],[object Object]

Experiments Zipfian Distribution

Bottlenecks ,[object Object],[object Object],[object Object]

Future Work ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Related Question

Question 1 (Dolphia Nandi) ,[object Object],[object Object],Back

Question 2 (Dolphia Nandi) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Question 3 (Dr. Murat Demirbas) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Question 4a (Dr. Murat Demirbas) ,[object Object],[object Object],[object Object],[object Object],Back

Question 4b (Dr. Murat Demirbas) ,[object Object],[object Object],Back

Question 5 (Dr. Murat Demirbas) ,[object Object],[object Object],[object Object]

Question 6 (Fatih) ,[object Object],[object Object],[object Object],[object Object]

Question 7 (Hanifi Güneş) ,[object Object],[object Object],[object Object],[object Object]

Question 8 (Yong Wang) ,[object Object],[object Object],Back

Question 9 (Santosh) ,[object Object],[object Object],[object Object],[object Object],Back (Consistency Model) Back (Bulk load)

Question 10 ,[object Object],[object Object],[object Object],[object Object]

Additional Definitions ,[object Object],[object Object],[object Object],[object Object],[object Object],Back

Zipfian distribution ,[object Object],[object Object],[object Object],[object Object],Back

Bulk loading support ,[object Object],[object Object],Related Question

[object Object],[object Object],Consistency model Time Record inserted Update Update Update Update Update Delete Time v. 1 v. 2 v. 3 v. 4 v. 5 v. 7 Generation 1 v. 6 v. 8 Update Update

Consistency model Time v. 1 v. 2 v. 3 v. 4 v. 5 v. 7 Generation 1 v. 6 v. 8 Current version Stale version Stale version Read

Consistency model Time v. 1 v. 2 v. 3 v. 4 v. 5 v. 7 Generation 1 v. 6 v. 8 Read up-to-date Current version Stale version Stale version

Consistency model Time v. 1 v. 2 v. 3 v. 4 v. 5 v. 7 Generation 1 v. 6 v. 8 Read ≥ v.6 Current version Stale version Stale version Read-critical(required version):

Consistency model Time v. 1 v. 2 v. 3 v. 4 v. 5 v. 7 Generation 1 v. 6 v. 8 Write Current version Stale version Stale version

Consistency model Time v. 1 v. 2 v. 3 v. 4 v. 5 v. 7 Generation 1 v. 6 v. 8 Write if = v.7 ERROR Current version Stale version Stale version Test-and-set-write(required version)

Consistency model Time v. 1 v. 2 v. 3 v. 4 v. 5 v. 7 Generation 1 v. 6 v. 8 Write if = v.7 ERROR Current version Stale version Stale version Back Mechanism: per record mastership

What is PNUTS? Parallel database Geographic replication Structured, flexible schema Hosted, managed infrastructure E 75656 C A 42342 E B 42521 W C 66354 W D 12352 E F 15677 E E 75656 C A 42342 E B 42521 W C 66354 W D 12352 E F 15677 E A 42342 E B 42521 W C 66354 W D 12352 E E 75656 C F 15677 E

Storage units Routers Tablet controller REST API Clients Message Broker Detailed architecture Data-path components

Storage units Routers Tablet controller REST API Clients Local region Remote regions YMB Detailed architecture

Accessing data SU SU SU Get key k 1 2 Get key k 3 Record for key k 4 Record for key k

Bulk read SU SU SU Scatter/ gather server 1 {k 1 , k 2 , … k n } 2 Get k 1 Get k 2 Get k 3

Range queries Router Apple Avocado Banana Blueberry Canteloupe Grape Kiwi Lemon Lime Mango Orange Strawberry Tomato Watermelon Storage unit 1 Storage unit 2 Storage unit 3 Grapefruit…Pear? Grapefruit…Lime? Lime…Pear? MIN-Canteloupe SU1 Canteloupe-Lime SU3 Lime-Strawberry SU2 Strawberry-MAX SU1 SU1 Strawberry-MAX SU2 Lime-Strawberry SU3 Canteloupe-Lime SU1 MIN-Canteloupe

Updates Write key k Sequence # for key k Sequence # for key k Write key k SUCCESS Write key k Routers Message brokers 1 2 Write key k 7 8 SU SU SU 3 4 5 6

Pnuts Review

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Pnuts Review

Semelhante a Pnuts Review (20)

Mais de Ruchika Mehresh

Mais de Ruchika Mehresh (7)

Último

Último (20)

Pnuts Review