DynamoDB transactions enable developers to maintain the correctness of their data at scale by adding atomicity and isolation guarantees for multi-item conditional updates. With transactions, you can perform a batch of conditional operations, including PutItem, UpdateItem, and DeleteItem, with guarantees. Come to this session to learn how DynamoDB transactions work, the primary use cases it enables, and how to build modern applications that require transactions. Speaker: Rajanikanth Bhargava Chilakapati - Solutions Architect, AWS

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Building Modern Applications Using
Amazon DynamoDB Transactions
Raj Chilakapati
Sr. Solution Architect, AWS

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
1. Introduction to atomicity, consistency, isolation, and durability
(ACID) in DynamoDB, and the new transactional API
2. Modern application use cases for the transactional API
3. Important things to know when integrating the transactional API
into your application: metering, concurrency control, and
interaction with other features

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What you will learn
• How the new transactional API works
• How you can use the new API in your application through
case studies demonstrating transactional design patterns
• How the new API interacts with DynamoDB features

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why is this important?
The transactional API can be used to
implement real-world transactional
scenarios
The transactional API can be used to
simplify the implementation of cloud
application flows

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Prerequisites
You should be familiar with DynamoDB concepts such as
tables, items, partition keys and sort keys, indexes, Time
To Live, and streams. We will only briefly introduce these
concepts in this session before using them in the use
cases.

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Transactional API at a glance
TransactWriteItems
Transact up to 10 items
Evaluate conditions
If all conditions are simultaneously true, perform write
operations
TransactGetItems
Transact up to 10 items
Return a consistent, isolated snapshot of all items

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Atomicity, consistency, isolation, and durability (ACID)
in DynamoDB
• Before: ACID support for single-item operations
• NEW: ACID support for multi-item operations
Nontransactional Transactional
New
TransactGetItems
TransactWriteItems
Existing
BatchGetItem, BatchWriteItem
Query, Scan
GetItem, PutItem, UpdateItem,
DeleteItem

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Transactions at DynamoDB scale
• Apply transactions to items in:
• Same partition key, or across partition keys
• Same table, or across tables
• Same Region only
• Same account only
• DynamoDB tables only
• Transactions scale like the rest of DynamoDB
• Limitless concurrent transactions
• Low latency, high availability
• Scales horizontally
• Your responsibility: Design to avoid hot key

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Path of success with NoSQL transactions
Design choices
• Transact any items in your account
• Service APIs vs. client-side library
• Request/response-based
• Limit of 10 items
How they help you
• Scale your data and relations within it
• Reduces complexity, and improves cost
and performance
• Keeps business logic in your application
and not in the database layer
• Consistent low latency, never deadlock
• Consistent low latency, high availability
• Addresses vast majority of use cases

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
User profile management (use case #1)
• Problem statement
• Users identified by user name and optionally email
• No two users can share the same identifier (user name or email)
• Users can change their email
• What is demonstrated
• Enforcing unique constraints
• Strongly consistent global indexing
• Heterogeneous tables

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
User profile management
• Solution
• Users table contains two kinds of records
• Profile record, keyed by opaque user name and containing user details
• Alias records, keyed by user email and referencing the profile record
Id UserNameRef Email Phone #
“John123” “john@example.com” “444-555-0110”
“john@example.com” “Joe123”
“jane456” “444-555-0199”

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Insert new profile
data = await dynamoDb.transactWriteItems({ TransactItems: [
{ Put: {
TableName: 'Users',
Item: { Id: { S: 'John123' }, Email: {S: 'john123@example.com'}, ...},
ConditionExpression: 'attribute_not_exists(Id)‘ }},
{ Put: {
TableName: 'Users',
Item: { Id: { S: 'john123@example.com' }, UserNameRef: {S: 'John123'}},
ConditionExpression: 'attribute_not_exists(Id)',}}
] }).promise();

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Update email address
data = await dynamoDb.transactWriteItems({ TransactItems: [
{ Put: { TableName: 'Users',
Item: { Id: { S: 'John123' }, Email: {S: 'john123@example.org'}, ...},
ConditionExpression: 'attribute_exists(Id) and Email = :old_email',
ExpressionAttributeValues: {':old_email' :{'S': 'john123@example.com'}}}},
{ Put: { TableName: 'Users',
Item: { Id: { S: 'john123@example.org' }, UserNameRef: {S: 'John123'}},
ConditionExpression: 'attribute_not_exists(Id)', }},
{ Delete: { TableName: 'Users',
Item: { Id: { S: 'john123@example.com' }}}).promise();

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hotel reservation management (use case #2)
• Problem statement
• Guests create room reservations
• Guests check in to rooms, fulfilling a reservation
• Guests check out of rooms, closing a reservation
• What is demonstrated
• Ensuring idempotency – operations with side effects happen at
most once
• Ensuring multi-item changes are only triggered when valid
(consistency)
• Ensuring multi-item changes happen all together (atomicity)

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hotel reservation management data model
Guests
Id (primary key)
Name, Email, etc.
Reservations (set)
OccupiesRooms (set)
Reservations
Id (primary key)
GuestId
FulfilledByRoom
State
Rooms
Id (primary key)
RentedToReservation
State

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hotel reservation management data model
Record Type Id (PK) Attributes
Guest “John”
Reservations : { “500”, “501” }
OccupiesRooms : { “20014” }
Reservation “500”
GuestId: “John”
State: “PENDING”
Reservation “501”
GuestId: “John”
State: “FULFILLED”
FulfilledByRoom: “20014”
Room “20014”
State: “OCCUPIED”
RentedToReservation : “501”

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hotel reservation management: Create a reservation
• Idempotency – how to ensure only a single reservation is
created upon retries?
• Solution
• Client generates a unique reservation ID
• Client generates TransactWriteItems request with the following operations
• Conditional Put for the new reservation
• Update the customer reservations set: Add the new reservation
• Client transaction is now idempotent and can be retried
without double booking

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hotel reservation management: Check-in
Record Type Id (PK) Attributes
Customer “John123”
Reservations : { “567” }
OccupiesRooms : {}
Reservation “567”
CustomerId: “John123”
State: “PENDING”
FulfilledByRoom: null
Room “20014”
State: “FREE”
RentedToReservation : null
Before

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hotel reservation management: Check-in
Record Type Id (PK) Attributes
Customer “John123”
Reservations : { “567” }
OccupiesRooms : { “20014” }
Reservation “567”
CustomerId: “John123”
State: “FULFILLED”
FulfilledByRoom: “20014”
Room “20014”
State: “OCCUPIED”
RentedToReservation : “567”
After

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hotel reservation management: Checkout
Record Type Id (PK) Attributes
Customer “John123”
Reservations : { “567” }
OccupiesRooms : { “20014” }
Reservation “567”
CustomerId: “John123”
State: “FULFILLED”
FulfilledByRoom: “20014”
Room “20014”
State: “OCCUPIED”
RentedToReservation : “567”
Before

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hotel reservation management: Checkout
Record Type Id (PK) Attributes
Customer “John123”
Reservations : { “567” }
OccupiesRooms : { “20014” }
Reservation “567”
CustomerId: “John123”
State: “CLOSED”
FulfilledByRoom: “20014”
Room “20014”
State: “FREE”
RentedToReservation : “567”
After

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Concurrency control
• Optimistic
• No deadlocks possible
• Low latency
• Optimized for scale-out
• Your responsibility
• Design for scale-out
• Avoid unnecessary conflicts

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Metering transactions
Transactions perform 2x the work of nontransactional
counterparts
Transactions are metered 2x their constituent
operations

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why didn’t my transaction succeed?
• Per-item failure reasons
• Precondition failure
• Insufficient capacity
• Transactional conflicts
• Other reasons
• Transaction is still in progress
• Service error
• Malformed request
• Permissions

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Retrying transactions: ClientRequestToken
• Ensures SDK can retry without “replaying” the same intent
multiple times
• ClientRequestToken – idempotency token at the SDK or API
level
• Autogenerated, you can override
• Applies for 10 minutes
• Provides information about transaction state and outcome
• Sometimes doesn’t remove the need for application-level
idempotency or intent design

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Application-level retries
• When the SDK gives up
• Build a new view of the world
• Retry with new preconditions and desirable end state
• How to build a snapshot of the world? Three options:
• Set ReturnValuesOnConditionCheckFailure = ALL_OLD
• Perform TransactGetItems
• Work from an eventually consistent view (results of queries, etc.)

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Integrating with other DynamoDB features
• General rule about DynamoDB data
• Committed state only
• But might not be isolated
• Examples
• Streams – Committed writes, sharded
• Backups and point-in-time recovery (PITR) – Subset of a committed
transaction is possible
• Global secondary index (GSI) – Committed writes are indexed
individually
• Global tables – Transactional API disabled by default; writes are
replicated individually
• Amazon DynamoDB Accelerator (DAX) support is on the roadmap

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Access control
• Individual operations are authorized separately
• (NEW) dynamodb:ConditionCheck
• dynamodb:UpdateItem
• dynamodb:PutItem
• dynamodb:DeleteItem
• No separate permissions for top-level new APIs

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Attachment management (use case #3)
• Problem statement
• A social media site allows attaching media to posts
• Posts need to share media efficiently
• Posts can have multiple attachments
• Unreferenced media needs to be deleted
• What is demonstrated
• External resource metadata and lifetime management
• Using Time To Live (TTL) and streams
• Reference counting using transactions

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Attachment management data model
Attachment item
S3Ref (PK) – Reference to an object in Amazon S3
RefCount – Number of posts referencing this attachment
TTL – Time when this attachment is eligible to be deleted
Post item
Id (PK) – Post ID
Attachments (set) – Set of attachment IDs

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Attachment management invariants
1. RefCount = number of posts referencing the attachment
2. If RefCount > 0, then S3Ref refers to a valid Amazon S3 object
3. TTL is set if, and only if, RefCount = 0

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Attachment management: Create attachment and
link it to a post
• Generate unique Amazon S3 object name
• Create attachment record in DynamoDB
• Set RefCount = 0,
• Set TTL = 24 hours from now
• Upload object to Amazon S3
• In a TransactWriteItems Request
• Check that the attachment is not already referenced by the post
• Increase RefCount by 1, remove TTL
• Add attachment reference to post

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Attachment management: Clone attachment
reference
• Copy the reference of an attachment from one post to
another
• In a TransactWriteItems Request:
• Check that the attachment is not already referenced by the new
post
• Increase RefCount by 1, remove TTL
• Add attachment reference to new post

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Attachment management: Delete unused external
resources
• Attachment records will be deleted a day after their
reference count drops to zero
• In AWS Lambda, process stream records representing items
deleted by TTL
• Delete the corresponding objects from Amazon S3

40. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Raj Chilakapati
chilakap@amazon.com