We present VIRTUS, a system for running blood flow simulations in the cloud. Thanks to our mesher one can directly operate on CT/MRI images, generate the mesh, define the boundary conditions and finally solve and visualize the results. In addition, thanks to GPU time-dependent flow simulations can be solved up to x3 faster than on multi-core CPU.

1. VITRTUS
CFD OF THE BLOOD FLOW IN THE CLOUD
JAKUB POLA
SOFTWARE DEVELOPER, VRATIS LTD.

2. HEART DIAGNOSIS
 Non-invasive:
‒ X-Ray Computer Tomography.
‒ Magnetic Resonance Imaging.
‒ Electrocardiography.
 Invasive:
‒ Angiography.
‒ Fractional Flow Reserve.
2 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

3. HEART DIAGNOSIS
FRACTIONAL FLOW RESERVE EXAMPLE
 COURAGE, FAME2:
FFR VALUE
Stent
implantation
Drug
therapy
 DeFACTO (2012):
FFR
3 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
FFR CT

4. COMPUTATIONAL FLUID DYNAMICS
FRAMEWORK
 Solve Navier – Stokes equations
 Procedure:
‒ Preprocessing:
‒
‒
‒
‒
Definition of the geometry.
Generation of volumetric mesh.
Definition of physical model.
Definition of boundary conditions.
‒ Simulation: iterative solvers.
‒ Post-processing:
‒ Analysis
‒ Visualisation
 How to obtain geometry and create a mesh?
 How to solve the problem fast and efficiently?
4 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

5. CFD IN MEDICINE
 Stent graft design:
‒ Determination of proper shape of the stent graft
 Design of heart valves:
‒ Determination of shape and biological and mechanical
properties
 Drug delivery:
‒ Determination of the delivery device
‒ Determination of the drug state
5 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

6. VIRTUS: OVERVIEW
Mesh storage format
User Interface
Database
PACS
Software as a Service
Active Mesh
6 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
Calculations

7. VIRTUS: WORKFLOW
CT / MRI Scanning
Segmentation
CFD Simulation
7 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
Meshing
Post-processing / Visualisation

8. VIRTUS: ARCHITECTURE
STL
Segmentation
CT/MRI
V Mesh
Surface representation of
volumetric mesh
Simulation
4 txt config files
Set boundary conditions
Visualization
8 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
Generation of Volumetric
Mesh
(NETGEN)
surface & U, p, WSS
Simulation

9. VIRTUS: GRAPHIC USER INTERFACE
9 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

10. VIRTUS: ARCHITECTURE
STL
Segmentation
CT/MRI
V Mesh
Surface representation of
volumetric mesh
Simulation
4 txt config files
Set boundary conditions
Visualization
10 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
Generation of Volumetric
Mesh
(NETGEN)
surface & U, p, WSS
Simulation

11. VIRTUS: SEGMENTATION
 Create 3D model from series of images.
 Very time consuming task.
 Require mesh modeling to:
‒ fix the errors in the mesh
‒ prepare mesh to generate volumetric mesh
11 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

12. VIRTUS: SEGMENTATION
ACTIVE MESH
 Input:
‒ CT or MRI scans in DICOM format.
 Outputs:
‒ Surface mesh in STL format.
‒ Volumetric mesh in OpenFOAM format.
 Real-time procedure.
 Fasat algorithm.
 Many mesh editing tools:
‒ Global/local mesh smoothing.
‒ Cut Mesh.
‒ Freeze Mesh.
‒ Push Mesh.
‒ Define Flow Inlets and Outlets
‒ Automated determination of artery centerline(s)
12 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

13. VIRTUS: MESHING MODULE
13 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

14. VIRTUS: ARCHITECTURE
STL
Segmentation
CT/MRI
V Mesh
Surface representation of
volumetric mesh
Simulation
4 txt config files
Set boundary conditions
Visualization
14 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
Generation of Volumetric
Mesh
(NETGEN)
surface & U, p, WSS
Simulation

15. VIRTUS: SIMULATION MODULE
USER SIDE
 Just two steps:
‒ Type of simulation:
‒ Transient flows.
‒ Steady-state flows.
‒ Inlet boundary condition:
‒ Inlet velocity.
‒ Inlet pressure.
‒ Point and click procedure
 Configuration
‒ Just four txt files
15 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

16. VIRTUS: SIMULATION MODULE
USER SIDE
16 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

17. VIRTUS: SIMULATION MODULE
SERVER SIDE
 Three ways to execute simulation:
‒ Using CPU in parallel mode.
‒ Using CPU + GPU in parallel mode.
‒ Using GPU only.
17 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

18. VIRTUS: SIMULATION MODULE
SERVER SIDE: CPU
CPU
Read mesh
Assembly matrices
Ax=b
~33%
No
Conv
erged
?
Yes
Finalize
18 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
Solve
Ax=b
~66%

19. VIRTUS: SIMULATION MODULE
SERVER SIDE: CPU + GPU
CPU
PCI
GPU
Read mesh
Assembly matrices
Ax=b
No
Conv
erged
?
Yes
Finalize
19 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
Ax=b
x
Solve
Ax=b

20. VIRTUS: SPEEDIT TOOLKIT
 Solvers:
‒ Conjugate Gradient.
‒ Bi-Conjugate Gradient.
 Preconditioners:
‒ Diagonal.
‒ Approximate Inverse.
‒ Algebraic Multigrid with Smoothed
Aggregation (CUSP).
 Support for Multi-GPU.
 Platforms:
‒ OpenCL.
‒ CUDA.
20 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

21. VIRTUS: SIMULATION MODULE
SERVER SIDE: CPU + GPU
CPU
PCI
GPU
Read mesh
Assembly matrices
Ax=b
No
Conv
erged
?
Yes
Finalize
21 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
Ax=b
x
Solve
Ax=b

22. VIRTUS: SIMULATION MODULE
SERVER SIDE: GPU
CPU
Read mesh
PCI
GPU
Mesh
Assembly matrices
Ax=b
No
Finalize
22 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
x
Solve
Ax=b
Conv
erged
?
Yes

23. VIRTUS: SPEEDIT FLOW
 Full GPU implementation of:
‒ PISO (Pressure Implicit with Split Operator) – transient solver for incompressible flows.
‒ SIMPLE (Semi-implicit Method for Pressure Linked Equations): Steady-state solver for incompressible
flows.
 Boundary Conditions:
‒ Zero Gradient
‒ Time dependent and Fixed value.
 Adjustable time step.
 Roadmap:
‒ Support for OpenCL
‒ Turbulence (RANS, kOmegaSST model)
‒ Support for Multi-GPU
23 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

24. SPEEDIT FLOW
TEST CASES
 U-shaped pipe
‒ Radius – 1in
‒ 4M hex cells
‒ Transient flow simulation:
‒ U(t) = Asin(ωt) + B
‒ Steady-state simulation
‒ Re = 100
‒ Re = 1000
 Basilar artery*:
‒ 2M hex cells
‒ Transient flow.
‒ Two heart cycles.
* Geometry obtained from AneuriskWeb project. Emory University, Department of Math&CS, 2012
24 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

25. SPEEDIT FLOW: U-SHAPED PIPE
SIMULATION EXECUTION TIME
25 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

26. SPEEDIT FLOW: U-SHAPED PIPE
ACCELERATION RATIO
26 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

27. SPEEDIT FLOW: BASILAR ARTERY
SIMULATION EXECUTION TIME
27 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

28. SPEEDIT FLOW: BASILAR ARTERY
ACCELERATION RATIO
28 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

29. VIRTUS: ARCHITECTURE
STL
Segmentation
CT/MRI
V Mesh
Surface representation of
volumetric mesh
Simulation
4 txt config files
Set boundary conditions
Visualization
29 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
Generation of Volumetric
Mesh
(NETGEN)
surface & U, p, WSS
Simulation

30. VRATIS: VISUALIZATION MODULE
30 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

31. SUMMARY
 VIRTUS is a platform for personalized medicine and CFD based diagnosis.
 Cloud-based approach simplifies the usage.
 GPU reduces time-to-solution.
31 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

32. Questions?
Comments?
Jakub Pola
jakub.pola@vratis.com
virtus.vratis.com
Acknowledemnents:
Vratis: Wojciech Tarnawski, Lukasz Miroslaw, Pawel Czubinski
Wroclaw Uni.: Zbigniew Koza, Piotr Olkiewicz
Wroclaw Uni. of Technology: Tadeusz Tomczak, Andrzej Kosior
32 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL

33. DISCLAIMER & ATTRIBUTION
The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors.
The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap
changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, software
changes, BIOS flashes, firmware upgrades, or the like. AMD assumes no obligation to update or otherwise correct or revise this information. However, AMD
reserves the right to revise this information and to make changes from time to time to the content hereof without obligation of AMD to notify any person of
such revisions or changes.
AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY
INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION.
AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE
LIABLE TO ANY PERSON FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION
CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
ATTRIBUTION
© 2013 Advanced Micro Devices, Inc. All rights reserved. AMD, the AMD Arrow logo and combinations thereof are trademarks of Advanced Micro Devices,
Inc. in the United States and/or other jurisdictions. SPEC is a registered trademark of the Standard Performance Evaluation Corporation (SPEC). Other names
are for informational purposes only and may be trademarks of their respective owners.
33 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL