SlideShare uma empresa Scribd logo
1 de 19
Code generation
for enterprise integration
           Max Lapshin
        max@maxidoors.ru
FIX protocol


•   Automated stock exchange trading

•   Binary protocol for receiving quotes and sending orders

•   No opensource erlang implementation
FIX protocol

•   Packet is a Key,Value list.

•   Keys are ASCII-coded integers

•   Values may vary

•   Separated with 0x01 byte
FIX protocol
•   Binary protocol with three levels: transport (TCP), packet and
    business

•   All business logic is described in protocol spec

•   Different message types with many fields and groups of fields

•   Very large protocol specification (about 100 different messages)

•   10 us limit for all parsing
FIX protocol


•   Received data is decoded to proplists

•   But we use records in code

•   How to translate? And do it fast!
FIX protocol


•   Large XML description how to compose objects from proplists

•   XML describes syntax and semantic levels
Problem

•   Every FIX user needs only small subset of whole spec

•   Impossible to work with full FIX records (more than 100 fields)

•   My code works with my records and business logic

•   How to fight impedance between our logic and FIX business logic?
Ways to go


•   Enterprise

•   Ad-hoc

•   Controlled code generation
Enterprise way

•   Object with 100 fields and 300 methods is ok

•   It is Java: abstract singleton factory will make you happy

•   Copy-pasted glue code will translate full spec to required subset

•   Impossible to use because large size of records

•   Not my way
Ad-hoc

•   Write custom code for each message type

•   Workaround for broker’s bugs

•   Ok for some situations

•   Not my way: I’m too lazy to copy-paste code by hands
Right way

•   Autogenerate proplists-to-record translator

•   Make code reuseable for others

•   Design system, that can fit into other businesses

•   Don’t want to write glue code

•   I chose configurable code generation
Code generation

•   Parse XML

•   Reduce messages according to config file

•   Generate headers

•   Generate parsers

•   Profit!
Headers

•   Translate only required FIX messages to records

•   Include only required fields in these records

•   Always include “fields” field to each record for remaining data

•   Dialyzer-compatible: types are known at generate time
Example


NewOrderSingle message with 60 fields is translated to

#new_order_single{} with 6 fields
Syntax parser

•   Translate binary fields to erlang types and back

•   Don’t forget sugar: translate 0,1,2 to atoms buy, sell, make_gift

•   Doesn’t depend on config

•   Generate C extension (blazing fast)

•   Produces readable proplist (not hash!): ordered key-value list
Semantic parser
•   Translates proplists to records

•   Need to produce records, directly used in business logic

•   Can loose some not important information (checksum)

•   Need to handle groups of fields (most cryptic part)

•   It is generated with config

•   Yet have some limited ad-hoc code to keep flexibility
Structure of semantic parser

1. Determine type of message and use compile time record info

2. Process the rest of proplist

3. If key is a known record field, write it to record

4. Else append this {Key,Value} to “fields” field

5. Use “setelement” and generated “field_index(MsgType,KeyName)”
Results

•   Simple and useable code

•   FIX datatypes are used in business logic directly

•   Configurable autogeneration of parser is really helpful

•   Working code
Questions?


    Max Lapshin

 max@maxidoors.ru

Mais conteúdo relacionado

Mais procurados

BSides LV 2016 - Beyond the tip of the iceberg - fuzzing binary protocols for...
BSides LV 2016 - Beyond the tip of the iceberg - fuzzing binary protocols for...BSides LV 2016 - Beyond the tip of the iceberg - fuzzing binary protocols for...
BSides LV 2016 - Beyond the tip of the iceberg - fuzzing binary protocols for...Alexandre Moneger
 
Harry Potter and the Daemons of Berkeley
Harry Potter and the Daemons of BerkeleyHarry Potter and the Daemons of Berkeley
Harry Potter and the Daemons of BerkeleyAlex Chistyakov
 
Ekon25 mORMot 2 Cryptography
Ekon25 mORMot 2 CryptographyEkon25 mORMot 2 Cryptography
Ekon25 mORMot 2 CryptographyArnaud Bouchez
 
My talk at Linux Piter 2015
My talk at Linux Piter 2015My talk at Linux Piter 2015
My talk at Linux Piter 2015Alex Chistyakov
 
Taming the resource tiger
Taming the resource tigerTaming the resource tiger
Taming the resource tigerElizabeth Smith
 
Doctrine 2 - Enterprise Persistence Layer For PHP
Doctrine 2 - Enterprise Persistence Layer For PHPDoctrine 2 - Enterprise Persistence Layer For PHP
Doctrine 2 - Enterprise Persistence Layer For PHPJonathan Wage
 
Erlang - Concurrent Language for Concurrent World
Erlang - Concurrent Language for Concurrent WorldErlang - Concurrent Language for Concurrent World
Erlang - Concurrent Language for Concurrent WorldZvi Avraham
 

Mais procurados (9)

Php extensions
Php extensionsPhp extensions
Php extensions
 
BSides LV 2016 - Beyond the tip of the iceberg - fuzzing binary protocols for...
BSides LV 2016 - Beyond the tip of the iceberg - fuzzing binary protocols for...BSides LV 2016 - Beyond the tip of the iceberg - fuzzing binary protocols for...
BSides LV 2016 - Beyond the tip of the iceberg - fuzzing binary protocols for...
 
Harry Potter and the Daemons of Berkeley
Harry Potter and the Daemons of BerkeleyHarry Potter and the Daemons of Berkeley
Harry Potter and the Daemons of Berkeley
 
Ekon25 mORMot 2 Cryptography
Ekon25 mORMot 2 CryptographyEkon25 mORMot 2 Cryptography
Ekon25 mORMot 2 Cryptography
 
My talk at Linux Piter 2015
My talk at Linux Piter 2015My talk at Linux Piter 2015
My talk at Linux Piter 2015
 
Taming the resource tiger
Taming the resource tigerTaming the resource tiger
Taming the resource tiger
 
Ekon24 mORMot 2
Ekon24 mORMot 2Ekon24 mORMot 2
Ekon24 mORMot 2
 
Doctrine 2 - Enterprise Persistence Layer For PHP
Doctrine 2 - Enterprise Persistence Layer For PHPDoctrine 2 - Enterprise Persistence Layer For PHP
Doctrine 2 - Enterprise Persistence Layer For PHP
 
Erlang - Concurrent Language for Concurrent World
Erlang - Concurrent Language for Concurrent WorldErlang - Concurrent Language for Concurrent World
Erlang - Concurrent Language for Concurrent World
 

Destaque

Fix protocol an introduction (r motie)
Fix protocol   an introduction (r motie)Fix protocol   an introduction (r motie)
Fix protocol an introduction (r motie)Dr Richard Motie
 
Intro To The FIX Protocol presented at BarCampNYC3
Intro To The FIX Protocol presented at BarCampNYC3 Intro To The FIX Protocol presented at BarCampNYC3
Intro To The FIX Protocol presented at BarCampNYC3 Brian Driscoll
 
FIX Protocol Overview.
FIX Protocol Overview.FIX Protocol Overview.
FIX Protocol Overview.aiQUANT
 
Algorithmic Trading and FIX Protocol
Algorithmic Trading and FIX ProtocolAlgorithmic Trading and FIX Protocol
Algorithmic Trading and FIX ProtocolEXANTE
 
Yapılan ağ saldırılarına karşı önlemler
Yapılan ağ saldırılarına karşı önlemlerYapılan ağ saldırılarına karşı önlemler
Yapılan ağ saldırılarına karşı önlemlerIlkin Azizov
 

Destaque (7)

Fix 32 bit
Fix 32 bitFix 32 bit
Fix 32 bit
 
Fix
FixFix
Fix
 
Fix protocol an introduction (r motie)
Fix protocol   an introduction (r motie)Fix protocol   an introduction (r motie)
Fix protocol an introduction (r motie)
 
Intro To The FIX Protocol presented at BarCampNYC3
Intro To The FIX Protocol presented at BarCampNYC3 Intro To The FIX Protocol presented at BarCampNYC3
Intro To The FIX Protocol presented at BarCampNYC3
 
FIX Protocol Overview.
FIX Protocol Overview.FIX Protocol Overview.
FIX Protocol Overview.
 
Algorithmic Trading and FIX Protocol
Algorithmic Trading and FIX ProtocolAlgorithmic Trading and FIX Protocol
Algorithmic Trading and FIX Protocol
 
Yapılan ağ saldırılarına karşı önlemler
Yapılan ağ saldırılarına karşı önlemlerYapılan ağ saldırılarına karşı önlemler
Yapılan ağ saldırılarına karşı önlemler
 

Semelhante a Code generation in Erlang

NetWork - 15.10.2011 - Applied code generation in .NET
NetWork - 15.10.2011 - Applied code generation in .NET NetWork - 15.10.2011 - Applied code generation in .NET
NetWork - 15.10.2011 - Applied code generation in .NET Dmytro Mindra
 
Design Like a Pro: Scripting Best Practices
Design Like a Pro: Scripting Best PracticesDesign Like a Pro: Scripting Best Practices
Design Like a Pro: Scripting Best PracticesInductive Automation
 
Design Like a Pro: Scripting Best Practices
Design Like a Pro: Scripting Best PracticesDesign Like a Pro: Scripting Best Practices
Design Like a Pro: Scripting Best PracticesInductive Automation
 
Pipiot - the double-architecture shellcode constructor
Pipiot - the double-architecture shellcode constructorPipiot - the double-architecture shellcode constructor
Pipiot - the double-architecture shellcode constructorMoshe Zioni
 
FPL - Part 1 (Sem - I 2013 )
FPL - Part 1  (Sem - I  2013 ) FPL - Part 1  (Sem - I  2013 )
FPL - Part 1 (Sem - I 2013 ) Yogesh Deshpande
 
Programming Languages #devcon2013
Programming Languages #devcon2013Programming Languages #devcon2013
Programming Languages #devcon2013Iván Montes
 
RSYSLOG v8 improvements and how to write plugins in any language.
RSYSLOG v8 improvements and how to write plugins in any language.RSYSLOG v8 improvements and how to write plugins in any language.
RSYSLOG v8 improvements and how to write plugins in any language.Rainer Gerhards
 
5 things you need to know about the Scala compiler
5 things you need to know about the Scala compiler5 things you need to know about the Scala compiler
5 things you need to know about the Scala compilerIulian Dragos
 
Understanding Character Encodings
Understanding Character EncodingsUnderstanding Character Encodings
Understanding Character EncodingsMobisoft Infotech
 
General Programming Concept
General Programming ConceptGeneral Programming Concept
General Programming ConceptHaris Bin Zahid
 
Intro to Programming Lang.pptx
Intro to Programming Lang.pptxIntro to Programming Lang.pptx
Intro to Programming Lang.pptxssuser51ead3
 
VoltDB and Erlang - Tech planet 2012
VoltDB and Erlang - Tech planet 2012VoltDB and Erlang - Tech planet 2012
VoltDB and Erlang - Tech planet 2012Eonblast
 
Compiler Construction
Compiler ConstructionCompiler Construction
Compiler ConstructionSarmad Ali
 
EuroMPI 2013 presentation: McMPI
EuroMPI 2013 presentation: McMPIEuroMPI 2013 presentation: McMPI
EuroMPI 2013 presentation: McMPIDan Holmes
 
Extent3 exante broker_for_algorithmic_trading_2012
Extent3 exante broker_for_algorithmic_trading_2012Extent3 exante broker_for_algorithmic_trading_2012
Extent3 exante broker_for_algorithmic_trading_2012extentconf Tsoy
 

Semelhante a Code generation in Erlang (20)

NetWork - 15.10.2011 - Applied code generation in .NET
NetWork - 15.10.2011 - Applied code generation in .NET NetWork - 15.10.2011 - Applied code generation in .NET
NetWork - 15.10.2011 - Applied code generation in .NET
 
Design Like a Pro: Scripting Best Practices
Design Like a Pro: Scripting Best PracticesDesign Like a Pro: Scripting Best Practices
Design Like a Pro: Scripting Best Practices
 
Design Like a Pro: Scripting Best Practices
Design Like a Pro: Scripting Best PracticesDesign Like a Pro: Scripting Best Practices
Design Like a Pro: Scripting Best Practices
 
Pipiot - the double-architecture shellcode constructor
Pipiot - the double-architecture shellcode constructorPipiot - the double-architecture shellcode constructor
Pipiot - the double-architecture shellcode constructor
 
FPL - Part 1 (Sem - I 2013 )
FPL - Part 1  (Sem - I  2013 ) FPL - Part 1  (Sem - I  2013 )
FPL - Part 1 (Sem - I 2013 )
 
Programming Languages #devcon2013
Programming Languages #devcon2013Programming Languages #devcon2013
Programming Languages #devcon2013
 
RSYSLOG v8 improvements and how to write plugins in any language.
RSYSLOG v8 improvements and how to write plugins in any language.RSYSLOG v8 improvements and how to write plugins in any language.
RSYSLOG v8 improvements and how to write plugins in any language.
 
Using unicode with php
Using unicode with phpUsing unicode with php
Using unicode with php
 
Using unicode with php
Using unicode with phpUsing unicode with php
Using unicode with php
 
PPT 19.pptx
PPT 19.pptxPPT 19.pptx
PPT 19.pptx
 
5 things you need to know about the Scala compiler
5 things you need to know about the Scala compiler5 things you need to know about the Scala compiler
5 things you need to know about the Scala compiler
 
Understanding Character Encodings
Understanding Character EncodingsUnderstanding Character Encodings
Understanding Character Encodings
 
rspamd-slides
rspamd-slidesrspamd-slides
rspamd-slides
 
C# basics...
C# basics...C# basics...
C# basics...
 
General Programming Concept
General Programming ConceptGeneral Programming Concept
General Programming Concept
 
Intro to Programming Lang.pptx
Intro to Programming Lang.pptxIntro to Programming Lang.pptx
Intro to Programming Lang.pptx
 
VoltDB and Erlang - Tech planet 2012
VoltDB and Erlang - Tech planet 2012VoltDB and Erlang - Tech planet 2012
VoltDB and Erlang - Tech planet 2012
 
Compiler Construction
Compiler ConstructionCompiler Construction
Compiler Construction
 
EuroMPI 2013 presentation: McMPI
EuroMPI 2013 presentation: McMPIEuroMPI 2013 presentation: McMPI
EuroMPI 2013 presentation: McMPI
 
Extent3 exante broker_for_algorithmic_trading_2012
Extent3 exante broker_for_algorithmic_trading_2012Extent3 exante broker_for_algorithmic_trading_2012
Extent3 exante broker_for_algorithmic_trading_2012
 

Mais de Max Lapshin

Erlang railsclub - 1
Erlang   railsclub - 1Erlang   railsclub - 1
Erlang railsclub - 1Max Lapshin
 
Flussonic IPTV OTT
Flussonic IPTV OTTFlussonic IPTV OTT
Flussonic IPTV OTTMax Lapshin
 
Pulsedb — система хранения временных рядов
Pulsedb — система хранения временных рядовPulsedb — система хранения временных рядов
Pulsedb — система хранения временных рядовMax Lapshin
 
Видеостриминг на 10 ГБит/с
Видеостриминг на 10 ГБит/сВидеостриминг на 10 ГБит/с
Видеостриминг на 10 ГБит/сMax Lapshin
 
Там, где Rails не справляются
Там, где Rails не справляютсяТам, где Rails не справляются
Там, где Rails не справляютсяMax Lapshin
 
Rails, Eventmachine, Erlang
Rails, Eventmachine, ErlangRails, Eventmachine, Erlang
Rails, Eventmachine, ErlangMax Lapshin
 
Erlang for Yandex
Erlang for YandexErlang for Yandex
Erlang for YandexMax Lapshin
 
Актуальное состояние вещания видео в интернете
Актуальное состояние вещания видео в интернетеАктуальное состояние вещания видео в интернете
Актуальное состояние вещания видео в интернетеMax Lapshin
 
Что и почему писать на Erlang
Что и почему писать на ErlangЧто и почему писать на Erlang
Что и почему писать на ErlangMax Lapshin
 
Devpoint2 video in internet
Devpoint2 video in internetDevpoint2 video in internet
Devpoint2 video in internetMax Lapshin
 
Erlyvideo — сервер потокового видео.
Erlyvideo — сервер потокового видео.Erlyvideo — сервер потокового видео.
Erlyvideo — сервер потокового видео.Max Lapshin
 
Отличие Erlang от объектных языков
Отличие Erlang от объектных языковОтличие Erlang от объектных языков
Отличие Erlang от объектных языковMax Lapshin
 
кеширование в бд
кеширование в бдкеширование в бд
кеширование в бдMax Lapshin
 

Mais de Max Lapshin (16)

Erlang railsclub - 1
Erlang   railsclub - 1Erlang   railsclub - 1
Erlang railsclub - 1
 
Flussonic IPTV OTT
Flussonic IPTV OTTFlussonic IPTV OTT
Flussonic IPTV OTT
 
Pulsedb — система хранения временных рядов
Pulsedb — система хранения временных рядовPulsedb — система хранения временных рядов
Pulsedb — система хранения временных рядов
 
Видеостриминг на 10 ГБит/с
Видеостриминг на 10 ГБит/сВидеостриминг на 10 ГБит/с
Видеостриминг на 10 ГБит/с
 
Там, где Rails не справляются
Там, где Rails не справляютсяТам, где Rails не справляются
Там, где Rails не справляются
 
Rails, Eventmachine, Erlang
Rails, Eventmachine, ErlangRails, Eventmachine, Erlang
Rails, Eventmachine, Erlang
 
Erlang for Yandex
Erlang for YandexErlang for Yandex
Erlang for Yandex
 
Актуальное состояние вещания видео в интернете
Актуальное состояние вещания видео в интернетеАктуальное состояние вещания видео в интернете
Актуальное состояние вещания видео в интернете
 
Http streaming
Http streamingHttp streaming
Http streaming
 
Erlang&rails
Erlang&railsErlang&rails
Erlang&rails
 
Что и почему писать на Erlang
Что и почему писать на ErlangЧто и почему писать на Erlang
Что и почему писать на Erlang
 
Devpoint2 video in internet
Devpoint2 video in internetDevpoint2 video in internet
Devpoint2 video in internet
 
Erlyvideo — сервер потокового видео.
Erlyvideo — сервер потокового видео.Erlyvideo — сервер потокового видео.
Erlyvideo — сервер потокового видео.
 
Отличие Erlang от объектных языков
Отличие Erlang от объектных языковОтличие Erlang от объектных языков
Отличие Erlang от объектных языков
 
Erlyvideo
ErlyvideoErlyvideo
Erlyvideo
 
кеширование в бд
кеширование в бдкеширование в бд
кеширование в бд
 

Último

Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...apidays
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 
Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024SynarionITSolutions
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodJuan lago vázquez
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdflior mazor
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024The Digital Insurer
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?Igalia
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobeapidays
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CVKhem
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc
 

Último (20)

Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 

Code generation in Erlang

  • 1. Code generation for enterprise integration Max Lapshin max@maxidoors.ru
  • 2. FIX protocol • Automated stock exchange trading • Binary protocol for receiving quotes and sending orders • No opensource erlang implementation
  • 3. FIX protocol • Packet is a Key,Value list. • Keys are ASCII-coded integers • Values may vary • Separated with 0x01 byte
  • 4. FIX protocol • Binary protocol with three levels: transport (TCP), packet and business • All business logic is described in protocol spec • Different message types with many fields and groups of fields • Very large protocol specification (about 100 different messages) • 10 us limit for all parsing
  • 5. FIX protocol • Received data is decoded to proplists • But we use records in code • How to translate? And do it fast!
  • 6. FIX protocol • Large XML description how to compose objects from proplists • XML describes syntax and semantic levels
  • 7. Problem • Every FIX user needs only small subset of whole spec • Impossible to work with full FIX records (more than 100 fields) • My code works with my records and business logic • How to fight impedance between our logic and FIX business logic?
  • 8. Ways to go • Enterprise • Ad-hoc • Controlled code generation
  • 9. Enterprise way • Object with 100 fields and 300 methods is ok • It is Java: abstract singleton factory will make you happy • Copy-pasted glue code will translate full spec to required subset • Impossible to use because large size of records • Not my way
  • 10. Ad-hoc • Write custom code for each message type • Workaround for broker’s bugs • Ok for some situations • Not my way: I’m too lazy to copy-paste code by hands
  • 11. Right way • Autogenerate proplists-to-record translator • Make code reuseable for others • Design system, that can fit into other businesses • Don’t want to write glue code • I chose configurable code generation
  • 12. Code generation • Parse XML • Reduce messages according to config file • Generate headers • Generate parsers • Profit!
  • 13. Headers • Translate only required FIX messages to records • Include only required fields in these records • Always include “fields” field to each record for remaining data • Dialyzer-compatible: types are known at generate time
  • 14. Example NewOrderSingle message with 60 fields is translated to #new_order_single{} with 6 fields
  • 15. Syntax parser • Translate binary fields to erlang types and back • Don’t forget sugar: translate 0,1,2 to atoms buy, sell, make_gift • Doesn’t depend on config • Generate C extension (blazing fast) • Produces readable proplist (not hash!): ordered key-value list
  • 16. Semantic parser • Translates proplists to records • Need to produce records, directly used in business logic • Can loose some not important information (checksum) • Need to handle groups of fields (most cryptic part) • It is generated with config • Yet have some limited ad-hoc code to keep flexibility
  • 17. Structure of semantic parser 1. Determine type of message and use compile time record info 2. Process the rest of proplist 3. If key is a known record field, write it to record 4. Else append this {Key,Value} to “fields” field 5. Use “setelement” and generated “field_index(MsgType,KeyName)”
  • 18. Results • Simple and useable code • FIX datatypes are used in business logic directly • Configurable autogeneration of parser is really helpful • Working code
  • 19. Questions? Max Lapshin max@maxidoors.ru

Notas do Editor

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n