Intern – Balance Tracker
Asmita Sharma – MCA 2012-15

Profile
 Name – Asmita Sharma
 Roll No. – 06
 Year – 2012- 2015
 University – Delhi University
 University Mentor – Ms. Neelima
Gupta
 Organization – Amazon Inc.
 Manager – Mr. Ramesh Krishna
Kumar
 Mentor – Itiyama Sadana
 Team – BTS
 Platform – Ecommerce Payments
 Role – Software Development
Engineer Intern
 Internship Start Date – 16 Feb 2015
 Internship End Date – 15 Aug 2015

Development Environment
JavaS3
Perl
JSONJunit
Spring
Brazil
Coral
Git
Apollo
Odin
SQS
Easy
Mock
Log4j
Dynamo
DB
Pipelines
Version
Sets
Eclipse

Internship Road Map
I am here.

Glossary
 Pipeline : A tool for modeling your release process and adding automation to it.
 Version Set : A snapshot of application dependencies.
 SQS : Simple Queuing Service
 DynamoDB : NoSQL Database
 S3 : Simple Storage Service
 Workflow : Sequence of executable steps.
 SWF : Simple Workflow
 Environment Stages : Association between hosts and Apollo environments.
 Alpha : Environment stage receiving no traffic.
 Beta : Environment stage receiving minimal traffic for testing.
 Prod : Environment stage receiving real-time traffic.

BTS - Overview
 BTS (Balance Tracking System) is a Global Stored Value Platform of Amazon.
 BTS is a part of ecommerce payments platform of Amazon.
 It allows any business in Amazon to create instruments and track funds movement for
them.
 It also provides simple APIs for querying balance of an instrument and history of fund
movement transactions done on an instrument.

Operational Task - Overview
 Objective : To migrate packages from JAVA -
1.7 to JAVA - 1.8
 Packages :
 PerseusClient
 PerseusClientInterface
 PerseusTestClient
 PerseusTestSuite
 MonadArchivalBusinessClient
 MonadArchivalLocator
 Pipelines :
 PerseusClient-development
 MonadArchivalPerseusClient/development
 SubTasks :
 Code Change
 Merge Dependencies
 Testing
 Deployment
 Migrate Pipelines

Operational Task – Code Change
 Changes to Config file involved adding,
removing and updating versions of build
and runtime dependencies to latest versions
supporting JAVA 8.
 Config file changes also involved resolving
conflicting dependencies of the package
with it’s upstream and downstream
packages.
 Changes to build.xml involved adding
properties for classpath of new packages
and few more changes to make upstream
JAVA 7 packages compatible with JAVA 8
migrated package.
 Changes to Spring configuration xml files
involved modification of syntax for
dependencies migrated to latest versions
supporting JAVA 8.
 Changes to Spring configuration xml files
also involved using classpath approach for
integrating spring packages instead of
providing url of spring packages.

Operational Task – Merge dependencies
 Live is a version set which contains all the versions of all the packages required within
amazon.
 Every package corresponds to a version set which contains all the dependencies of the
packages in that version set.
 To upgrade or add a new version of any dependency, we need to merge that
dependency into the version set from live.

Operational Task – Testing
 A package is first tested in the workplace itself using brazil CLI commands and targets
specified in build.xml.
 After these tests are passed, any package dependent on this package is merged to
workspace and tested to run with these changes.
 When above tests pass successfully, peer review is done and suggested improvements
are incorporated.
 After peer review is complete , the package is pushed to a temporary branch of the
repository.

Operational Task – Deployment
 The package from the temporary branch is build into the version set and deployed to the
pipeline.
 It is tested at various stages (namely Alpha and Beta) of pipeline through regression
suites.
 After it passes these stages and logs of the hosts at these stages are verified, it is
deployed to Prod.

Operational Task – Migrate Pipelines
 Migrating pipeline involves merging all packages migrated to Java 8 that belong to that
pipeline or are a dependency of the packages belonging to the pipeline.
 Any conflicts arising while merging these packages have to be resolved and then
merged again.
 Logs of hosts at each stage (Alpha , Beta, Prod) have to be verified after this step also.

Minor Project
 Objective : To remove reconciliation from
PerseusClient/development-clone and
MonadBalanceServiceListener pipeline.
 Packages :
 PerseusClient
 PerseusClientInterface
 PerseusTestClient
 PerseusTestSuite
 NotificationParser
 ModelConverter
 BalanceUpdateBusinessClient
 BalanceQueryServiceSAO
 SubTasks :
 Code Change
 Modifying Regressions
 Modifying Unit Test Cases
 Resetting AWS Credentials
 Testing
 Major Version Upgrade
 Write Change Management (CM)
Document
 Deployment

Minor Project – Code Change
 Removing Tracker Table and Marker Table instances along with their interface and unit test cases.
 Removing all SWF (Simple Workflow) workflows and corresponding unit tests.
 Removing dependencies from Config file which are no longer needed.
 Removing brazil-config file properties that are no longer required.
 Removing unnecessary targets and properties from build.xml.
 Removing reconciler instances from all packages.
 Deleting spring configuration files corresponding to reconciler .
 Removing all packages used by reconcilera which are no longer required.

Minor Project – Modifying Regressions
 Removing DynamoDB instances previously used by tracker table and marker table.
 Modifying regressions to test only notification functionality and neglect reconciliation.
 Events in modified regression suite will not be dropped to test reconciler functionality.
 All regressions related to reconciler and workflows will be deleted.
 In modified regression suite, only notification listener will be tested by updating a dummy instrument
and verifying that it’s notification reaches event listener through SQS (Simple Queuing Service).
 Reconciler instances and unnecessary dependencies will be removed from Config and xml files.

Minor Project – Modifying Unit Test Cases
 Added unit test for Event Details class
 Added spring beans to set S3 (Simple Storage Service) and AWS (Amazon Web Services) credentials
and configured the same in spring configuration xml files and brazil-config.
 Removed unit test cases for workflows and dynamoDB tables.
 Removed unit test cases involving missing sequence number tests and updated event listener test for
the same.
 Modularized a function by shifting s3 upload functionality from the function to a separate method.

Minor Project – Resetting AWS Credentials
 Earlier the AWS credentials were set using Odin which did not allow the credentials to be seamlessly
retrieved dynamically.
 Hence, used AWSCredentialProvider to set AWS credentials for SQS in PerseusClient and
PerseusTestClient using spring beans and brazil-config.
 This approach eliminates the race condition where dynamic credentials could change in between
calls to getAccessKey/getSecretAccessKey/getSessionToken and leave the customer with invalid
credentials.
 This also makes it very easy for the AWSCredentialsProvider implementation to return different types of
credentials objects.

Minor Project – Testing
 Created a child environment of the Alpha stage of the PerseusClient-development
pipeline and deployed it on local machine using Apollo.
 Attached my workspace and modified packages to this child environment using brazil.
 Activated the child environment using brazil.
 Logged in to the remote host set up at Alpha stage of pipeline using SSH command.
 Verified the logs running on the host.
 Deactivated the parent environment.
 Logged in to the SWF console and verified that the workflows have stopped executing.

Minor Project – Major Version Upgrade
 Upgraded the major version of all packages to 2.0.
 Changes were made to Config file and build.xml file of all packages.

Major Project
 Objective : To support Query APIs on S3.
 API:
 GetEvent()
 GetInstrumentEvents()
 GetChildEvents()
 GetInitializeStatus()
 SubTasks :
 Data Analysis
 Design Approach
 Write Design Document
 Environment Setup
 Code Development
 Writing Unit Tests
 Deriving and Coding Regressions
 Testing
 Deployment

Major Project – Data Analysis
 Objective : To analyze up to 3 months old
logs to gather requirements.
 API :
 GetInstrumentEvents()
 SubTasks :
 Figure out data required from logs
 Write Script to gather data from logs
 Run script of hosts containing logs
 Post-process data for concluding
requirements
 Represent data graphically
 Output :
 Domains accessing more than year old
events
 Total number of events returned by
each API call
 Number of more than year old , more
than six month old and more than three
month old events returned by each API
call
 Number of calls returning more than
year old events made by each domain
 Difference between API call time and
creation time of oldest event returned
by each API call

Major Project – Design Approach
 Objective : To come up with an approach
that best fits the requirements of all the four
APIs with optimal use of resources as per the
required performance.
 SubTasks :
 Understand the functionality , input
fields and output fields of the APIs
 Figure out the required latencies
 Analyze available tools for suitability as
per the requirements
 Devise design approaches
 Calculate storage, latencies and cost
metrics for each approach
 Analyze feasibility, simplicity of
development, scalability and
maintenance of each approach
 Choose between Sync and Async path
 Compare candidate approaches
 Choose the best fit

Major Project – Design Document Index
1.1 Prior Knowledge
1.1.1 APIs
1.1.2 EventID Format
1.1.3 InstrumentID Format
1.1.4 S3 File Format
1.2 Data Analysis
1.2.1 Script
1.2.2 Requirements
1.3 Design Approach
1.3.1 Storing S3 files on hourly basis and
scanning files for required instrument
1.3.2 Creating index on instrument ID per day
1.3.2.1 Index Structure
1.3.2.1.1 Start_File and End_File field
size calculation
1.3.2.2 Metrics
1.3.2.3 Cost of storing indexes in S3
1.3.2.4 Cost of storing indexes in
DynamoDB
1.3.2.5 Comparison
1.3.3 Creating instrument level indexes
1.3.4 Using CloudSearch to index S3 files
1.3.4.1 Costing

Yet To Do…
 Minor Project
 Deployment
 Writing Change Management
Document
 Major Project
 Environment Setup
 Code Development
 Writing Unit Tests
 Deriving and Coding Regressions
 Testing
 Deployment

Thank you