SlideShare uma empresa Scribd logo
1 de 27
Baixar para ler offline
SOLVCON: Software-
Engineering Simulations of
Conservation Laws
Yung-Yu Chen!


SOLVCON Project!
@ PyConAPAC 2014 May 17
Introduction
❖ I am: an engineer using computers to solve problems in mechanics.!
❖ I want to solve conservation laws. I want to solve for various physical
processes. I should build a framework.!
❖ Examples of the simulations:
Aerodynamics Benchmarks
Stress Waves in!
Anisotropic Solids
Supersonic Jet in Cross Flow
Goals
❖ We want multiple solvers for physical processes governed by the equations:!
!
❖ We use the space-time Conservation Element and Solution Element (CESE)
method.!
❖ We want an extensible system, modularized for various physical processes.!
❖ Then plan for interoperation.!
❖ We want parallel code (scale from 1,000 cores to more). We want hybrid
parallelism for heterogeneous architecture.!
❖ We want the code to be organized. We don’t want spaghetti code patched
from paper to paper. We want to code systematically.
It’s an Engineering Problem
❖ “Develop a software framework for the CESE method.”!
❖ Balance the following points:!
❖ Extensibility and modularity.!
❖ Parallel computing.!
❖ Maintainability.!
❖ All goals are achieved, with rough ends.
Python
❖ Python is slow, but provides a good infrastructure for working with
low-level number-crunching code.!
❖ ndarray (N-dimensional array) of NumPy (http://
www.numpy.org/).!
❖ Cython (http://cython.org/) for interfacing between Python and C.!
❖ The versatile scientific Python ecosystem simplifies coding supportive
functionalities.!
❖ Practices emerged from other fields flow into sciences:!
❖ Version control, unit tests, documenting tools, etc.
Scripting Is a Must
❖ There are too many parameters and analyses required.!
❖ No interface is sufficient for the simulations: SOLVCON
needs to be a library.!
❖ Controlling a library by using low-level languages like
C and C++ is too painful to be realistic.
Design for Productivity
❖ The outstanding capability to interface to C makes Python an
ideal manager for low-level, high-performance C code.!
❖ C++ works as well but SOLVCON uses only C to avoid the
complexity.!
❖ The hybrid approach allows us to architect the system with the
high-level Python and still have the high performance of C.!
❖ Ideas can be quickly prototyped with Python. After results
get validated, the fast C version can then developed
iteratively.
Discretize Space
❖ The conservation laws we are
solving assume continuum
model.!
❖ In computer memory nothing
is continuous.!
❖ There must be a way to
discretize the space for the
equations.!
❖ “Meshing”.
Unstructured Meshes
❖ We chose to employ the unstructured meshes of mixed
elements to model complex geometry.!
❖ In computer memory they consist of several look-up
tables.
quadrilateral triangle
hexahedron tetrahedron prism pyramid
Look-Up Tables for Meshes
❖ SOLVCON categorize the look-up tables into three: (i)
coordinate, (ii) meta-data, and (iii) connectivity.!
❖ They are either 1-D or 2-D arrays.!
❖ These arrays contain “ghost cells” for boundary-
condition treatments.!
❖ The ghost cells are outside the computing space.!
❖ The CESE method needs only one layer of ghost cells.
Domain Decomposition
❖ Millions or billions of
unstructured mesh elements
can be automatically
decomposed into multiple
blocks for parallel computing.!
❖ A graph is built for
representing the connectivity
among all the mesh elements.!
❖ The partitioned graphs are
used to decompose the
original mesh into thousands
of blocks.
Two-Level Nested Loops
❖ The time-marching nature of
the CESE method dictates a
structure of two-level nested
loops.!
❖ The outer loop is for time-
marching.!
❖ Multiple inner loops
“sweep” over the discretized
space.!
❖ All facilities can built around
this execution flow.
Module Structure
Case-Solver Relation
❖ solvcon.case.MeshCase: the outer temporal loop.!
❖ solvcon.solver.MeshSolver: house of the inner spatial loops.
In-Situ Visualization/Analysis
❖ What’s in-situ visualization/analysis (ISV)?!
❖ Perform result visualization/analysis while time-marching.!
❖ Why in-situ?!
❖ [50 million-element mesh] * [10,000 time steps] * [6 single-precision solution
fields (CFD)] mean 12 TB of data. It is one simulation case.!
❖ Segregating result analysis from simulation code is difficult.!
❖ Architecture of most research codes isn’t good enough for the segregation.!
❖ Most codes simply produce the data and use another software to post-
process.!
❖ A common practice is to reduce the resolution.!
❖ Outputting every 100 time steps reduces the output dat to 120 GB.!
❖ But it defeats the purpose of high-resolution simulations.
SOLVCON’s Segregation
❖ SOLVCON provides
companions of MeshCase and
MeshSolver: MeshHook and
MeshAnchor, respectively.!
❖ MeshCase and MeshSolver
are for equation solutions.!
❖ MeshHook and MeshAnchor
are for solution analysis.!
❖ And there’re still a lot to do.
Documenting
❖ SOLVCON documents should contain what’s in
conventional research papers.!
❖ What we want is more than conventional papers.!
❖ Bi-directional linking between the code and the
documents.!
❖ Some part of the documents should be runnable. It’s
related to validation.
Sphinx
❖ Sphinx (http://sphinx-doc.org/) is the default tool for
Python documentation.!
❖ ReadTheDocs (https://readthedocs.org/) is a service to
host Sphinx documents.!
❖ SOLVCON authors and organizes its documents with
Sphinx.!
❖ The documents are published at http://
www.solvcon.net/ (ReadTheDocs is behind the scenes).
Authoring, Editing, and Reviewing
❖ We use Mercurial for version control and BitBucket
(https://bitbucket.org/solvcon/solvcon/) to host the
project and its source code.!
❖ Reviewing the documents goes through the pull-request
flow of the code review.!
❖ We treat the reStructuredText source files of the
documents like the program source code.
Integrated Testing
❖ Code without verification can’t be taken as working code,
no matter how carefully we wrote it.!
❖ With the testing system, refactoring and quick prototyping
become possible.!
❖ Many research codes don’t have a regression testing
system, so while coding the developers can’t be sure
how the changes impact the system.!
❖ Two kinds of tests: unit tests and answer tests. Both need
automation.
Unit Tests
❖ Unit tests are short and quick, and check for simple
output of elementary API behaviors.!
❖ SOLVCON uses standard Python unittest module
(http://docs.python.org/2/library/unittest.html).!
❖ nose (https://nose.readthedocs.org/) is used as the
automatic test runner.
Answer Tests
!
❖ Answer tests check for the correctness of simulations.
Usually an answer test is a meaningful simulation, and
can serve as an example to how to use SOLVCON to
simulate other problems. !
❖ Jenkins (http://jenkins-ci.org/) is installed at http://
ci.solvcon.net/ to automatically run the answer tests for
every commit of SOLVCON in a testing farm.
Challenges (Conclusions)
❖ The simulations were done with older version of
SOLVCON; it takes a lot of efforts to upgrade (ongoing).!
❖ Unfamiliarity of the collaborators to the authoring
system (Sphinx): http://www.solvcon.net/en/latest/bulk/theory.html!
❖ Unification of code and document needs extra efforts.!
❖ Feeding back to the main framework.!
❖ Server farm for continuous integration.
Current/Future Works
❖ New physical processes:!
❖ solvcon.parcel.bulk: acoustic wave solver based on
bulk modulus.!
❖ solvcon.parcel.vewave: visco-elastic wave solver.!
❖ Document the CESE method.!
❖ Documenting with iPython in addition to Sphinx?!
❖ Get more people to play together!

Mais conteúdo relacionado

Semelhante a Software Framework for Solving Conservation Laws Using CESE Method

Microservices: Redundancy = Maintainability! (Eberhard Wolff Technology Stream)
Microservices: Redundancy = Maintainability! (Eberhard Wolff Technology Stream)Microservices: Redundancy = Maintainability! (Eberhard Wolff Technology Stream)
Microservices: Redundancy = Maintainability! (Eberhard Wolff Technology Stream)IT Arena
 
Concurrency Programming in Java - 01 - Introduction to Concurrency Programming
Concurrency Programming in Java - 01 - Introduction to Concurrency ProgrammingConcurrency Programming in Java - 01 - Introduction to Concurrency Programming
Concurrency Programming in Java - 01 - Introduction to Concurrency ProgrammingSachintha Gunasena
 
Microservices: Redundancy=Maintainability
Microservices: Redundancy=MaintainabilityMicroservices: Redundancy=Maintainability
Microservices: Redundancy=MaintainabilityEberhard Wolff
 
Developer Job in Practice
Developer Job in PracticeDeveloper Job in Practice
Developer Job in Practiceintive
 
C# Async/Await Explained
C# Async/Await ExplainedC# Async/Await Explained
C# Async/Await ExplainedJeremy Likness
 
Introduction to OpenSees by Frank McKenna
Introduction to OpenSees by Frank McKennaIntroduction to OpenSees by Frank McKenna
Introduction to OpenSees by Frank McKennaopenseesdays
 
Constrained Optimization with Genetic Algorithms and Project Bonsai
Constrained Optimization with Genetic Algorithms and Project BonsaiConstrained Optimization with Genetic Algorithms and Project Bonsai
Constrained Optimization with Genetic Algorithms and Project BonsaiIvo Andreev
 
Engineer Engineering Software
Engineer Engineering SoftwareEngineer Engineering Software
Engineer Engineering SoftwareYung-Yu Chen
 
Mk network programmability-03_en
Mk network programmability-03_enMk network programmability-03_en
Mk network programmability-03_enMiya Kohno
 
Declarative Programming and a form of SDN
Declarative Programming and a form of SDN Declarative Programming and a form of SDN
Declarative Programming and a form of SDN Miya Kohno
 
MiniOS: an instructional platform for teaching operating systems labs
MiniOS: an instructional platform for teaching operating systems labsMiniOS: an instructional platform for teaching operating systems labs
MiniOS: an instructional platform for teaching operating systems labsRafael Roman Otero
 
Meanstack Introduction by Kishore Chandra
Meanstack Introduction by Kishore ChandraMeanstack Introduction by Kishore Chandra
Meanstack Introduction by Kishore ChandraKishore Chandra
 
High Performance Machine Learning in R with H2O
High Performance Machine Learning in R with H2OHigh Performance Machine Learning in R with H2O
High Performance Machine Learning in R with H2OSri Ambati
 
Peyton jones-2011-parallel haskell-the_future
Peyton jones-2011-parallel haskell-the_futurePeyton jones-2011-parallel haskell-the_future
Peyton jones-2011-parallel haskell-the_futureTakayuki Muranushi
 
Simon Peyton Jones: Managing parallelism
Simon Peyton Jones: Managing parallelismSimon Peyton Jones: Managing parallelism
Simon Peyton Jones: Managing parallelismSkills Matter
 
[KubeCon NA 2018] Telepresence Deep Dive Session - Rafael Schloming & Luke Sh...
[KubeCon NA 2018] Telepresence Deep Dive Session - Rafael Schloming & Luke Sh...[KubeCon NA 2018] Telepresence Deep Dive Session - Rafael Schloming & Luke Sh...
[KubeCon NA 2018] Telepresence Deep Dive Session - Rafael Schloming & Luke Sh...Ambassador Labs
 
Advanced web application architecture - Talk
Advanced web application architecture - TalkAdvanced web application architecture - Talk
Advanced web application architecture - TalkMatthias Noback
 

Semelhante a Software Framework for Solving Conservation Laws Using CESE Method (20)

Microservices: Redundancy = Maintainability! (Eberhard Wolff Technology Stream)
Microservices: Redundancy = Maintainability! (Eberhard Wolff Technology Stream)Microservices: Redundancy = Maintainability! (Eberhard Wolff Technology Stream)
Microservices: Redundancy = Maintainability! (Eberhard Wolff Technology Stream)
 
Concurrency Programming in Java - 01 - Introduction to Concurrency Programming
Concurrency Programming in Java - 01 - Introduction to Concurrency ProgrammingConcurrency Programming in Java - 01 - Introduction to Concurrency Programming
Concurrency Programming in Java - 01 - Introduction to Concurrency Programming
 
Microservices: Redundancy=Maintainability
Microservices: Redundancy=MaintainabilityMicroservices: Redundancy=Maintainability
Microservices: Redundancy=Maintainability
 
Developer Job in Practice
Developer Job in PracticeDeveloper Job in Practice
Developer Job in Practice
 
C# Async/Await Explained
C# Async/Await ExplainedC# Async/Await Explained
C# Async/Await Explained
 
Introduction to OpenSees by Frank McKenna
Introduction to OpenSees by Frank McKennaIntroduction to OpenSees by Frank McKenna
Introduction to OpenSees by Frank McKenna
 
Constrained Optimization with Genetic Algorithms and Project Bonsai
Constrained Optimization with Genetic Algorithms and Project BonsaiConstrained Optimization with Genetic Algorithms and Project Bonsai
Constrained Optimization with Genetic Algorithms and Project Bonsai
 
Engineer Engineering Software
Engineer Engineering SoftwareEngineer Engineering Software
Engineer Engineering Software
 
Mk network programmability-03_en
Mk network programmability-03_enMk network programmability-03_en
Mk network programmability-03_en
 
Declarative Programming and a form of SDN
Declarative Programming and a form of SDN Declarative Programming and a form of SDN
Declarative Programming and a form of SDN
 
MiniOS: an instructional platform for teaching operating systems labs
MiniOS: an instructional platform for teaching operating systems labsMiniOS: an instructional platform for teaching operating systems labs
MiniOS: an instructional platform for teaching operating systems labs
 
Meanstack Introduction by Kishore Chandra
Meanstack Introduction by Kishore ChandraMeanstack Introduction by Kishore Chandra
Meanstack Introduction by Kishore Chandra
 
High Performance Machine Learning in R with H2O
High Performance Machine Learning in R with H2OHigh Performance Machine Learning in R with H2O
High Performance Machine Learning in R with H2O
 
Peyton jones-2011-parallel haskell-the_future
Peyton jones-2011-parallel haskell-the_futurePeyton jones-2011-parallel haskell-the_future
Peyton jones-2011-parallel haskell-the_future
 
Simon Peyton Jones: Managing parallelism
Simon Peyton Jones: Managing parallelismSimon Peyton Jones: Managing parallelism
Simon Peyton Jones: Managing parallelism
 
[KubeCon NA 2018] Telepresence Deep Dive Session - Rafael Schloming & Luke Sh...
[KubeCon NA 2018] Telepresence Deep Dive Session - Rafael Schloming & Luke Sh...[KubeCon NA 2018] Telepresence Deep Dive Session - Rafael Schloming & Luke Sh...
[KubeCon NA 2018] Telepresence Deep Dive Session - Rafael Schloming & Luke Sh...
 
Memory models
Memory modelsMemory models
Memory models
 
Advanced web application architecture - Talk
Advanced web application architecture - TalkAdvanced web application architecture - Talk
Advanced web application architecture - Talk
 
Clean architecture
Clean architectureClean architecture
Clean architecture
 
AngularJS - TechTalk 3/2/2014
AngularJS - TechTalk 3/2/2014AngularJS - TechTalk 3/2/2014
AngularJS - TechTalk 3/2/2014
 

Último

Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
Azure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAzure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAndikSusilo4
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonetsnaman860154
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024Scott Keck-Warren
 
Key Features Of Token Development (1).pptx
Key  Features Of Token  Development (1).pptxKey  Features Of Token  Development (1).pptx
Key Features Of Token Development (1).pptxLBM Solutions
 
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge GraphSIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge GraphNeo4j
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Patryk Bandurski
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 

Último (20)

Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
Azure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAzure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & Application
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024
 
Key Features Of Token Development (1).pptx
Key  Features Of Token  Development (1).pptxKey  Features Of Token  Development (1).pptx
Key Features Of Token Development (1).pptx
 
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge GraphSIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 

Software Framework for Solving Conservation Laws Using CESE Method

  • 1. SOLVCON: Software- Engineering Simulations of Conservation Laws Yung-Yu Chen! 
 SOLVCON Project! @ PyConAPAC 2014 May 17
  • 2. Introduction ❖ I am: an engineer using computers to solve problems in mechanics.! ❖ I want to solve conservation laws. I want to solve for various physical processes. I should build a framework.! ❖ Examples of the simulations: Aerodynamics Benchmarks Stress Waves in! Anisotropic Solids Supersonic Jet in Cross Flow
  • 3. Goals ❖ We want multiple solvers for physical processes governed by the equations:! ! ❖ We use the space-time Conservation Element and Solution Element (CESE) method.! ❖ We want an extensible system, modularized for various physical processes.! ❖ Then plan for interoperation.! ❖ We want parallel code (scale from 1,000 cores to more). We want hybrid parallelism for heterogeneous architecture.! ❖ We want the code to be organized. We don’t want spaghetti code patched from paper to paper. We want to code systematically.
  • 4. It’s an Engineering Problem ❖ “Develop a software framework for the CESE method.”! ❖ Balance the following points:! ❖ Extensibility and modularity.! ❖ Parallel computing.! ❖ Maintainability.! ❖ All goals are achieved, with rough ends.
  • 5. Python ❖ Python is slow, but provides a good infrastructure for working with low-level number-crunching code.! ❖ ndarray (N-dimensional array) of NumPy (http:// www.numpy.org/).! ❖ Cython (http://cython.org/) for interfacing between Python and C.! ❖ The versatile scientific Python ecosystem simplifies coding supportive functionalities.! ❖ Practices emerged from other fields flow into sciences:! ❖ Version control, unit tests, documenting tools, etc.
  • 6. Scripting Is a Must ❖ There are too many parameters and analyses required.! ❖ No interface is sufficient for the simulations: SOLVCON needs to be a library.! ❖ Controlling a library by using low-level languages like C and C++ is too painful to be realistic.
  • 7. Design for Productivity ❖ The outstanding capability to interface to C makes Python an ideal manager for low-level, high-performance C code.! ❖ C++ works as well but SOLVCON uses only C to avoid the complexity.! ❖ The hybrid approach allows us to architect the system with the high-level Python and still have the high performance of C.! ❖ Ideas can be quickly prototyped with Python. After results get validated, the fast C version can then developed iteratively.
  • 8. Discretize Space ❖ The conservation laws we are solving assume continuum model.! ❖ In computer memory nothing is continuous.! ❖ There must be a way to discretize the space for the equations.! ❖ “Meshing”.
  • 9. Unstructured Meshes ❖ We chose to employ the unstructured meshes of mixed elements to model complex geometry.! ❖ In computer memory they consist of several look-up tables. quadrilateral triangle hexahedron tetrahedron prism pyramid
  • 10. Look-Up Tables for Meshes ❖ SOLVCON categorize the look-up tables into three: (i) coordinate, (ii) meta-data, and (iii) connectivity.! ❖ They are either 1-D or 2-D arrays.! ❖ These arrays contain “ghost cells” for boundary- condition treatments.! ❖ The ghost cells are outside the computing space.! ❖ The CESE method needs only one layer of ghost cells.
  • 11. Domain Decomposition ❖ Millions or billions of unstructured mesh elements can be automatically decomposed into multiple blocks for parallel computing.! ❖ A graph is built for representing the connectivity among all the mesh elements.! ❖ The partitioned graphs are used to decompose the original mesh into thousands of blocks.
  • 12. Two-Level Nested Loops ❖ The time-marching nature of the CESE method dictates a structure of two-level nested loops.! ❖ The outer loop is for time- marching.! ❖ Multiple inner loops “sweep” over the discretized space.! ❖ All facilities can built around this execution flow.
  • 14. Case-Solver Relation ❖ solvcon.case.MeshCase: the outer temporal loop.! ❖ solvcon.solver.MeshSolver: house of the inner spatial loops.
  • 15. In-Situ Visualization/Analysis ❖ What’s in-situ visualization/analysis (ISV)?! ❖ Perform result visualization/analysis while time-marching.! ❖ Why in-situ?! ❖ [50 million-element mesh] * [10,000 time steps] * [6 single-precision solution fields (CFD)] mean 12 TB of data. It is one simulation case.! ❖ Segregating result analysis from simulation code is difficult.! ❖ Architecture of most research codes isn’t good enough for the segregation.! ❖ Most codes simply produce the data and use another software to post- process.! ❖ A common practice is to reduce the resolution.! ❖ Outputting every 100 time steps reduces the output dat to 120 GB.! ❖ But it defeats the purpose of high-resolution simulations.
  • 16. SOLVCON’s Segregation ❖ SOLVCON provides companions of MeshCase and MeshSolver: MeshHook and MeshAnchor, respectively.! ❖ MeshCase and MeshSolver are for equation solutions.! ❖ MeshHook and MeshAnchor are for solution analysis.! ❖ And there’re still a lot to do.
  • 17. Documenting ❖ SOLVCON documents should contain what’s in conventional research papers.! ❖ What we want is more than conventional papers.! ❖ Bi-directional linking between the code and the documents.! ❖ Some part of the documents should be runnable. It’s related to validation.
  • 18. Sphinx ❖ Sphinx (http://sphinx-doc.org/) is the default tool for Python documentation.! ❖ ReadTheDocs (https://readthedocs.org/) is a service to host Sphinx documents.! ❖ SOLVCON authors and organizes its documents with Sphinx.! ❖ The documents are published at http:// www.solvcon.net/ (ReadTheDocs is behind the scenes).
  • 19.
  • 20. Authoring, Editing, and Reviewing ❖ We use Mercurial for version control and BitBucket (https://bitbucket.org/solvcon/solvcon/) to host the project and its source code.! ❖ Reviewing the documents goes through the pull-request flow of the code review.! ❖ We treat the reStructuredText source files of the documents like the program source code.
  • 21. Integrated Testing ❖ Code without verification can’t be taken as working code, no matter how carefully we wrote it.! ❖ With the testing system, refactoring and quick prototyping become possible.! ❖ Many research codes don’t have a regression testing system, so while coding the developers can’t be sure how the changes impact the system.! ❖ Two kinds of tests: unit tests and answer tests. Both need automation.
  • 22. Unit Tests ❖ Unit tests are short and quick, and check for simple output of elementary API behaviors.! ❖ SOLVCON uses standard Python unittest module (http://docs.python.org/2/library/unittest.html).! ❖ nose (https://nose.readthedocs.org/) is used as the automatic test runner.
  • 23.
  • 24. Answer Tests ! ❖ Answer tests check for the correctness of simulations. Usually an answer test is a meaningful simulation, and can serve as an example to how to use SOLVCON to simulate other problems. ! ❖ Jenkins (http://jenkins-ci.org/) is installed at http:// ci.solvcon.net/ to automatically run the answer tests for every commit of SOLVCON in a testing farm.
  • 25.
  • 26. Challenges (Conclusions) ❖ The simulations were done with older version of SOLVCON; it takes a lot of efforts to upgrade (ongoing).! ❖ Unfamiliarity of the collaborators to the authoring system (Sphinx): http://www.solvcon.net/en/latest/bulk/theory.html! ❖ Unification of code and document needs extra efforts.! ❖ Feeding back to the main framework.! ❖ Server farm for continuous integration.
  • 27. Current/Future Works ❖ New physical processes:! ❖ solvcon.parcel.bulk: acoustic wave solver based on bulk modulus.! ❖ solvcon.parcel.vewave: visco-elastic wave solver.! ❖ Document the CESE method.! ❖ Documenting with iPython in addition to Sphinx?! ❖ Get more people to play together!