Dataflow: the concurrency/parallelism architecture you need

•

1 gostou•1,182 visualizações

This document discusses dataflow architecture. It defines dataflow as an architecture where data flows between operators through channels. Each operator executes only in response to data arriving on its input channels. The document discusses how dataflow allows for concurrency and parallelism. It provides examples of dataflow frameworks in Scala, Akka, and Java and notes their limitations. GPars is highlighted as a framework that can create true dataflow networks by using DataflowQueue. The document uses calculating mean and standard deviation as a problem to demonstrate coding in dataflow.

Tecnologia

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow:
Russel Winder
@russel_winder
http://www.russel.org.uk
russel@winder.org.uk
The Concurrency/Parallelism
Architecture You Need

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
What is Dataflow?

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
What are (in computing†
):
Concurrency:
Structuring solution and code
such that multiple parts may
execute independently and
possibly even at the same
time.
Parallelism:
Execute multiple parts of a
system at the same time on
different processors so as to
get things working faster.
†
In natural language these words have very different meanings.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
What is Dataflow?
An architecture comprising channels allowing data to flow from
one operator to another, where each operator has multiple
input channels and multiple output channels, and executes
code only in response to the arrival of data on the inputs.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Historically
Dataflow computers:
– Values flowing between…
–…operators that calculate…
–…new values to pass to…
–…other operators.
Dataflow hardware didn't take
off, but the architecture works
at various scales.
The Manchester Prototype Dataflow Computer
J R Gurd, C C Kirkham, I Watson
CACM 28(1), 1985-01.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow diagrams have been an
integral part of analysis and design of
information systems since the 1970s
T de Marco, Structured Analysis and Systems Specification,
Yourdon Press, NY, 1978.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow and Functional
Operators seem like they
might be pure functions,
but…
…they are not necessarily,
operators may have internal
state.
Operators may be referentially
transparent, but they may be
not.
Operators may even have side
effects.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow is an
event-based
architecture

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow systems are
(possibly)
reactive systems.
Which would make them exceedingly
trendy even if the idea is very old.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow systems have
no†
shared memory.
†
or at least should have no.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
operator
channel

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow systems are
message passing systems.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Each operator must†
be single threaded.
†
or at least should.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow Frameworks
Scala:
–Future
Akka:
–Dataflow variables, aka
Promise
–Deprecated in favour of Async
Java:
–Pre-8, Future
–8+, CompletableFuture, aka
Promise

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Architectural Issue
Each of the aforementioned frameworks assumes that each
operator creates a single value. Communication is by dataflow
variables: each dataflow variable is a thread-safe single
assignment variable.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
GPars…
Has dataflow variables
(promises) and tasks and so
can do everything Akka and
Java can offer.
Has DataflowQueue, and so
can create real dataflow
networks.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
One does like to code…
…doesn't one.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
We need a problem…

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
A Problem
Calculate mean and standard deviation of a data sample.
¯x =
1
n
∑i=0
n
xi
s =
√ 1
n−1
∑i=0
n
(xi−¯x)2

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Amend the Problem
s =
√ 1
n−1 ((∑i=0
n
xi
2
)−n¯x ¯x)
¯x =
1
n
∑i=0
n
xi

@YourTwitterHandle@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
C
od
e

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Switch to using an IDE for this.Switch to using an IDE for this.
Code Example

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Summary
Dataflow is an architecture:
Event-driven, single-threaded
operators communicating by
message passing using
channels.
Dataflow is an easement:
Synchronization is inherent in
the model, and there is no
shared memory, so all
deadlocks are trivial.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow is a way of harnessing
concurrency and parallelism
in easy to program ways.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
GPars is usable from Java
as well as Groovy.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Testing is really Groovy with Spock.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow is an architecture of
code you need to know.

Mais conteúdo relacionado

Semelhante a Dataflow: the concurrency/parallelism architecture you need

Dataflow, the Forgotten Way - Russel Winder

JAXLondon2014

GPars Remoting

Russel Winder

Shared memory multi-threading is an operating system technique which has no place in tool and application programming. Actors, dataflows, Communicating Sequential Processes (CSP): these are the tools for application and tool construction in this increasingly multiple multicore processor world. The Java platform has some useful tools in java.util.concurrent, but it is still focused on lower-level layers of abstraction. GPars builds on these and brings actors, dataflows, CSP, agents, etc. to people working with Java and Groovy. In this session we will look at various examples of how to use these tools to easily create concurrent and parallel programs without all the fears of deadlock and livelock inherent in shared memory multi-threaded code.

It's all about processes communicating - Russel Winder

JAX London

It's All About Processes Communicating

Russel Winder

RTI Connext 5.1.0

Jan Van Bruaene

Njug presentation

iwrigley

The Case for Kotlin and Ceylon

Russel Winder

Apache Cassandra is the leading distributed database in use at thousands of sites with the world’s most demanding scalability and availability requirements. Apache Spark is a distributed data analytics computing framework that has gained a lot of traction in processing large amounts of data in an efficient and user-friendly manner. The joining of both provides a powerful combination of real-time data collection with analytics. After a brief overview of Cassandra and Spark, this class will dive into various aspects of the integration.

Big Data Analytics with Spark

DataStax Academy

View the recording of the meet up, including the live demos, here: https://www.youtube.com/watch?v=uaJWB3K8lkg Data science holds tremendous potential for organizations to uncover new insights and drivers of revenue and profitability. Big Data has brought the promise of doing data science at scale to enterprises, however this promise also comes with challenges for data scientists to continuously learn and collaborate. Data Scientists have many tools at their disposal such as notebooks like Juypter and Apache Zeppelin & IDEs such as RStudio with languages like R, Python, Scala and frameworks like Apache Spark. Given all the choices how do you best collaborate to build your model and then work through the development lifecycle to deploy it from test into production? Why Data Science on Big Data? In this meetup you will cover the attributes of a modern data science platform that empowers data scientists to build models using all the data in their data lake and foster continuous learning and collaboration. We will show a demo of Apache Zeppelin, Apache Spark, Apache Livy and Apache Hadoop with the focus on integration, security and model deployment and management. Data Science at Scale DEMO The demo will cover the Data Science life cycle: develop model in team environment, train the model with all the data on a Hadoop cluster, deploy model into production. The model will be a Spark ML model Practical ML with Apache Spark To deliver machine learning solutions data scientists not only need to fit models but also do familiar tasks data collection & wrangling, labelling, feature extraction and transformation, model tuning and evaluation, etc. Apache Spark provide provides a unified solution for all this under the same framework. For example, one can use Spark SQL to generate training data from different sources and then pass it directly to MLlib for feature engineering and model tuning, instead of using Hive/Pig for the first half and then downloading the data to a single machine to train models in R. The latter is actually very common in practice but painful to maintain. Spark MLlib makes life easier for data scientists and machine learning engineers so that they can focus on building better ML models and applications. We will discuss the underlying principles required to develop practical machine learning and data science pipelines and show some hands-on experience using Apache Spark to solve typical machine learning and data science problem. We will also have a short discussion about how Spark MLlib faces challenges from other machine learning libraries such as TensorFlow and XGBoost.

Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017

Hortonworks

Analyzing Hadoop Data Using Sparklyr 

Cloudera, Inc.

Proxy Deep Dive Voxxed Belgrad 2015

Sven Ruppert

Everybody knows the pattern proxy, but how can you use it effectively? What kind of proxy patterns are available, and how can you build patterns more effectively with it? Why is reflection needed for this? Importantly, we need only the core JDK in most cases. This tutorial starts from the basics and continues on to DynamicProxies, DynamicObjectAdapter and DynamicStaticProxies at runtime, StaticObjectAdapters, and more. The session, based on the German book Dynamic Proxies, by Heinz Kabutz and the session’s presenter, takes a deep dive into this pattern group.

Proxy deep-dive java-one_20151027_001

Sven Ruppert

Introduction to Hadoop and Cloudera, Louisville BI & Big Data Analytics Meetup

iwrigley

GPars Workshop

Russel Winder

( Apache Spark Training: https://www.edureka.co/apache-spark-scala-training ) ( Hadoop Training: https://www.edureka.co/hadoop ) This Edureka Hadoop vs Spark video will help you to understand the differences between Hadoop and Spark. We will be comparing them on various parameters. We will be taking a broader look at: 1. Introduction to Hadoop 2. Introduction to Apache Spark 3. Spark vs Hadoop - Performance Ease of Use Cost Data Processing Fault tolerance Security 4. Hadoop Use-cases 5. Spark Use-cases

Hadoop vs Spark | Which One to Choose? | Hadoop Training | Spark Training | E...

Edureka!

ACCU 2012: Go, D, C++ and The Multicore Revolution

Russel Winder

Rust, Redis, and Protobuf - Oh My!

Nell Shamrell-Harrington

Big Data Infrastructure

Trivadis

Empowering Agile Development with Containers

Kellyn Pot'Vin-Gorman

Applications on Hadoop

markgrover

Semelhante a Dataflow: the concurrency/parallelism architecture you need (20)

Dataflow, the Forgotten Way - Russel Winder

GPars Remoting

It's all about processes communicating - Russel Winder

It's All About Processes Communicating

RTI Connext 5.1.0

Njug presentation

The Case for Kotlin and Ceylon

Big Data Analytics with Spark

Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017

Analyzing Hadoop Data Using Sparklyr 

Proxy Deep Dive Voxxed Belgrad 2015

Proxy deep-dive java-one_20151027_001

Introduction to Hadoop and Cloudera, Louisville BI & Big Data Analytics Meetup

GPars Workshop

Hadoop vs Spark | Which One to Choose? | Hadoop Training | Spark Training | E...

ACCU 2012: Go, D, C++ and The Multicore Revolution

Rust, Redis, and Protobuf - Oh My!

Big Data Infrastructure

Empowering Agile Development with Containers

Applications on Hadoop

Mais de Russel Winder

On Concurrency and Parallelism in the JVMverse

Russel Winder

On the Architectures of Microservices: the next layer

Russel Winder

Fast Python? Don't Bother

Russel Winder

Making Python computations fast

Russel Winder

Tales from the Workshops

Russel Winder

Making Computations Execute Very Quickly

Russel Winder

Java is Dead, Long Live Ceylon, Kotlin, etc

Russel Winder

Java is dead, long live Scala, Kotlin, Ceylon, etc.

Russel Winder

Spocktacular testing

Russel Winder

Spocktacular Testing

Russel Winder

Is Groovy static or dynamic

Russel Winder

Java is dead, long live Scala Kotlin Ceylon etc.

Russel Winder

Are Go and D threats to Python

Russel Winder

Is Groovy as fast as Java

Russel Winder

Who needs C++ when you have D and Go

Russel Winder

Java 8: a New Beginning

Russel Winder

Why Go is an important programming language

Russel Winder

GPars: Groovy Parallelism for Java

Russel Winder

GroovyFX: or how to program JavaFX easily

Russel Winder

Switch to Python 3…now…immediately

Russel Winder

Mais de Russel Winder (20)

On Concurrency and Parallelism in the JVMverse

On the Architectures of Microservices: the next layer

Fast Python? Don't Bother

Making Python computations fast

Tales from the Workshops

Making Computations Execute Very Quickly

Java is Dead, Long Live Ceylon, Kotlin, etc

Java is dead, long live Scala, Kotlin, Ceylon, etc.

Spocktacular testing

Spocktacular Testing

Is Groovy static or dynamic

Java is dead, long live Scala Kotlin Ceylon etc.

Are Go and D threats to Python

Is Groovy as fast as Java

Who needs C++ when you have D and Go

Java 8: a New Beginning

Why Go is an important programming language

GPars: Groovy Parallelism for Java

GroovyFX: or how to program JavaFX easily

Switch to Python 3…now…immediately

Último

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Partners Life - Insurer Innovation Award 2024

The Digital Insurer

Abhishek Deb(1), Mr Abdul Kalam(2) M. Des (UX) , School of Design, DIT University , Dehradun. This paper explores the future potential of AI-enabled smartphone processors, aiming to investigate the advancements, capabilities, and implications of integrating artificial intelligence (AI) into smartphone technology. The research study goals consist of evaluating the development of AI in mobile phone processors, analyzing the existing state as well as abilities of AI-enabled cpus determining future patterns as well as chances together with reviewing obstacles as well as factors to consider for more growth.

Exploring the Future Potential of AI-Enabled Smartphone Processors

debabhi2

Boost Fertility New Invention Ups Success Rates.pdf

sudhanshuwaghmare1

What are drone anti-jamming systems? The drone anti-jamming systems and anti-spoof technology protect against interference, jamming, and spoofing of the UAVs. To protect their security, countries are beginning to research drone anti-jamming systems, also known as drone strike weapons. The anti-jam and anti-spoof technology protects against interference, jamming and spoofing. A drone strike weapon is a drone attack weapon that can attack and destroy enemy drones. So what is so unique about this amazing system?

What Are The Drone Anti-jamming Systems Technology?

Antenna Manufacturer Coco

Join our latest Connector Corner webinar to discover how UiPath Integration Service revolutionizes API-centric automation in a 'Quote to Cash' process—and how that automation empowers businesses to accelerate revenue generation. A comprehensive demo will explore connecting systems, GenAI, and people, through powerful pre-built connectors designed to speed process cycle times. Speakers: James Dickson, Senior Software Engineer Charlie Greenberg, Host, Product Marketing Manager

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

DianaGray10

Histor y of HAM Radio presentation slide

vu2urc

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Product Anonymous

Strategies for Landing an Oracle DBA Job as a Fresher

Remote DBA Services

💉💊+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHABI}}+971581248768 +971581248768 Mtp-Kit (500MG) Prices » Dubai [(+971581248768**)] Abortion Pills For Sale In Dubai, UAE, Mifepristone and Misoprostol Tablets Available In Dubai, UAE CONTACT DR.Maya Whatsapp +971581248768 We Have Abortion Pills / Cytotec Tablets /Mifegest Kit Available in Dubai, Sharjah, Abudhabi, Ajman, Alain, Fujairah, Ras Al Khaimah, Umm Al Quwain, UAE, Buy cytotec in Dubai +971581248768''''Abortion Pills near me DUBAI | ABU DHABI|UAE. Price of Misoprostol, Cytotec” +971581248768' Dr.DEEM ''BUY ABORTION PILLS MIFEGEST KIT, MISOPROTONE, CYTOTEC PILLS IN DUBAI, ABU DHABI,UAE'' Contact me now via What's App…… abortion Pills Cytotec also available Oman Qatar Doha Saudi Arabia Bahrain Above all, Cytotec Abortion Pills are Available In Dubai / UAE, you will be very happy to do abortion in Dubai we are providing cytotec 200mg abortion pill in Dubai, UAE. Medication abortion offers an alternative to Surgical Abortion for women in the early weeks of pregnancy. We only offer abortion pills from 1 week-6 Months. We then advise you to use surgery if its beyond 6 months. Our Abu Dhabi, Ajman, Al Ain, Dubai, Fujairah, Ras Al Khaimah (RAK), Sharjah, Umm Al Quwain (UAQ) United Arab Emirates Abortion Clinic provides the safest and most advanced techniques for providing non-surgical, medical and surgical abortion methods for early through late second trimester, including the Abortion By Pill Procedure (RU 486, Mifeprex, Mifepristone, early options French Abortion Pill), Tamoxifen, Methotrexate and Cytotec (Misoprostol). The Abu Dhabi, United Arab Emirates Abortion Clinic performs Same Day Abortion Procedure using medications that are taken on the first day of the office visit and will cause the abortion to occur generally within 4 to 6 hours (as early as 30 minutes) for patients who are 3 to 12 weeks pregnant. When Mifepristone and Misoprostol are used, 50% of patients complete in 4 to 6 hours; 75% to 80% in 12 hours; and 90% in 24 hours. We use a regimen that allows for completion without the need for surgery 99% of the time. All advanced second trimester and late term pregnancies at our Tampa clinic (17 to 24 weeks or greater) can be completed within 24 hours or less 99% of the time without the need surgery. The procedure is completed with minimal to no complications. Our Women's Health Center located in Abu Dhabi, United Arab Emirates, uses the latest medications for medical abortions (RU-486, Mifeprex, Mifegyne, Mifepristone, early options French abortion pill), Methotrexate and Cytotec (Misoprostol). The safety standards of our Abu Dhabi, United Arab Emirates Abortion Doctors remain unparalleled. They consistently maintain the lowest complication rates throughout the nation. Our Physicians and staff are always available to answer questions and care for women in one of the most difficult times in their lives. The decision to have an abortion at the Abortion Cl

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@

What is a good lead in your organisation? Which leads are priority? What happens to leads? When sales and marketing give different answers to these questions, or perhaps aren't sure of the answers at all, frustrations build and opportunities are left on the table. Join us for an illuminating session with Cian McLoughlin, HubSpot Principal Customer Success Manager, as we look at that crucial piece of the customer journey in which leads are transferred from marketing to sales.

04-2024-HHUG-Sales-and-Marketing-Alignment.pptx

HampshireHUG

Handwritten Text Recognition for manuscripts and early printed texts

Maria Levchenko

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

Neo4j

A Domino Admins Adventures (Engage 2024)

Gabriella Davis

Created by Mozilla Research in 2012 and now part of Linux Foundation Europe, the Servo project is an experimental rendering engine written in Rust. It combines memory safety and concurrency to create an independent, modular, and embeddable rendering engine that adheres to web standards. Stewardship of Servo moved from Mozilla Research to the Linux Foundation in 2020, where its mission remains unchanged. After some slow years, in 2023 there has been renewed activity on the project, with a roadmap now focused on improving the engine’s CSS 2 conformance, exploring Android support, and making Servo a practical embeddable rendering engine. In this presentation, Rakhi Sharma reviews the status of the project, our recent developments in 2023, our collaboration with Tauri to make Servo an easy-to-use embeddable rendering engine, and our plans for the future to make Servo an alternative web rendering engine for the embedded devices industry. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://ossna2024.sched.com/event/1aBNF/a-year-of-servo-reboot-where-are-we-now-rakhi-sharma-igalia

A Year of the Servo Reboot: Where Are We Now?

Igalia

Real Time Object Detection Using Open CV

Khem

Enterprise Knowledge’s Urmi Majumder, Principal Data Architecture Consultant, and Fernando Aguilar Islas, Senior Data Science Consultant, presented "Driving Behavioral Change for Information Management through Data-Driven Green Strategy" on March 27, 2024 at Enterprise Data World (EDW) in Orlando, Florida. In this presentation, Urmi and Fernando discussed a case study describing how the information management division in a large supply chain organization drove user behavior change through awareness of the carbon footprint of their duplicated and near-duplicated content, identified via advanced data analytics. Check out their presentation to gain valuable perspectives on utilizing data-driven strategies to influence positive behavioral shifts and support sustainability initiatives within your organization. In this session, participants gained answers to the following questions: - What is a Green Information Management (IM) Strategy, and why should you have one? - How can Artificial Intelligence (AI) and Machine Learning (ML) support your Green IM Strategy through content deduplication? - How can an organization use insights into their data to influence employee behavior for IM? - How can you reap additional benefits from content reduction that go beyond Green IM?

Driving Behavioral Change for Information Management through Data-Driven Gree...

Enterprise Knowledge

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

Dataflow: the concurrency/parallelism architecture you need

1. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder Dataflow: Russel Winder @russel_winder http://www.russel.org.uk russel@winder.org.uk The Concurrency/Parallelism Architecture You Need

3. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder What are (in computing† ): Concurrency: Structuring solution and code such that multiple parts may execute independently and possibly even at the same time. Parallelism: Execute multiple parts of a system at the same time on different processors so as to get things working faster. † In natural language these words have very different meanings.

4. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder What is Dataflow? An architecture comprising channels allowing data to flow from one operator to another, where each operator has multiple input channels and multiple output channels, and executes code only in response to the arrival of data on the inputs.

5. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder Historically Dataflow computers: – Values flowing between… –…operators that calculate… –…new values to pass to… –…other operators. Dataflow hardware didn't take off, but the architecture works at various scales. The Manchester Prototype Dataflow Computer J R Gurd, C C Kirkham, I Watson CACM 28(1), 1985-01.

6. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder Dataflow diagrams have been an integral part of analysis and design of information systems since the 1970s T de Marco, Structured Analysis and Systems Specification, Yourdon Press, NY, 1978.

8. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder Dataflow and Functional Operators seem like they might be pure functions, but… …they are not necessarily, operators may have internal state. Operators may be referentially transparent, but they may be not. Operators may even have side effects.

15. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder Dataflow Frameworks Scala: –Future Akka: –Dataflow variables, aka Promise –Deprecated in favour of Async Java: –Pre-8, Future –8+, CompletableFuture, aka Promise

16. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder Architectural Issue Each of the aforementioned frameworks assumes that each operator creates a single value. Communication is by dataflow variables: each dataflow variable is a thread-safe single assignment variable.

19. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder GPars… Has dataflow variables (promises) and tasks and so can do everything Akka and Java can offer. Has DataflowQueue, and so can create real dataflow networks.

27. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder Summary Dataflow is an architecture: Event-driven, single-threaded operators communicating by message passing using channels. Dataflow is an easement: Synchronization is inherent in the model, and there is no shared memory, so all deadlocks are trivial.

33. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder Dataflow: Russel Winder @russel_winder http://www.russel.org.uk russel@winder.org.uk The Concurrency/Parallelism Architecture You Need

Dataflow: the concurrency/parallelism architecture you need

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Dataflow: the concurrency/parallelism architecture you need

Semelhante a Dataflow: the concurrency/parallelism architecture you need (20)

Mais de Russel Winder

Mais de Russel Winder (20)

Último

Último (20)

Dataflow: the concurrency/parallelism architecture you need