1) Financial institutions must report asset trades within a short regulatory window based on changing rules. 2) A leading financial institution executes around 6 million trades daily and must report qualifying trades in a specified format within 15 minutes. 3) The institution transforms, enriches, de-duplicates trades and applies business rules to identify reportable trades for generation within 35 seconds for 30,000 records using Spark on a 10 node cluster.

1. REGULATORY
REPORTING OF ASSET
TRADING USING SPARK
Sudipto Shankar Dasgupta &
Mayoor Rao – Infosys Limited

2. Regulatory Reporting – Current Challenges
2
Financial Institutions need to report asset trade
based on defined regulations
Reporting has to comply within a time window
from the trade execution
Rules are subject to change and adjustments
based on the change in regulations
The transaction volume is high
Transactional systems are diverse

3. Case Study
3
Leading global Financial Institution engaged in around
6 million trades a day in the financial markets
Regulatory requirements necessitates reporting in a
specified reporting format within a 15 minute window
The trades are identified for reporting based on steps
• Transformation
• Enrichment
• De-duplication checks
• Business Rules

4. Report Generation Process Flow
Asset
Transactions
Transformation
Rules
Transformed
Asset Data
Master Data
Enriched Asset
Data
Business Rules
Transformation Enrichment
Report
Generation
4

5. Architecture Diagram
SQOOP
HDFS Data Lake
TemporaryStaging
ProcessedOutput
In Memory Analytic and Processing
Zone
Processing Logic
Apps Portal
5

6. Ingestion throughput
>130,000
records/second or 18.22 MB/
second (per node)
Report execution time -> 35
seconds for 30,000 trade
records
Overall Data Volume -> 596
million trade transaction
records
Hardware Configuration ->
Spark Installed in 10 node 32
GB RAM , 16 Core m/c
Statistics
6
01
02
03
04

7. DEMO
7

8. IIP - Leveraging Power of Spark
8
Open Source / Spark Components
Spark ML
Spark Streaming
HIVE / Spark SQL
Hadoop / FS Storage / Infra Mgmt
Infosys & Partner IP Components
Tools | Data Extractors | Algorithms | Packaging &
Support
Customization, Integration & Implementation
Services
Data Modeling & Cleansing | Agile App Development
Data Science & Analytics | Security & Governance
Custom Data Extractors

9. Infosys Information Platform - Architecture
IIP Data Lake!
Query Interface
BI Applications
Down Stream Apps /
Portal
IIP Data Lake!RealTime–IIPIngestionUnstructured
Data
RDBMS Data
NO SQL Data
Data Streams
JMS Queue
Social data
Machine data
Machine data
Web URL
Resource Manager
Governance
Security
Data Store
OS /LDAP/Kerberos
Authentication
Authorization Role Based Access
Metadata
Management
Data Lineage Data Audit
Cluster Resource
Manager
Cluster Monitor Disaster Recovery
Spark In Memory Analytic Zone
HDFS / Hive
Natural Language
Processing
Machine Learning
Data Views
Data Mining and
Models
Data Joins
Data
Transformation
Single Click Installer
Cluster
Maintenance
Deployment
Manager
IIP
Data
Explorer
9

10. DEMO
1
0

11. © 2015 Infosys Limited, Bangalore, India. All Rights Reserved. Infosys believes the information in this document is accurate as of its publication date; such information is subject to change without notice. Infosys
acknowledges the proprietary rights of other companies to the trademarks, product names and such other intellectual property rights mentioned in this document. Except as expressly permitted, neither this documentation nor
any part of it may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, printing, photocopying, recording or otherwise, without the prior permission of Infosys Limited
and/ or any named intellectual property rights holders under this document.
Thank You