Japan Virtual Open House Feb 2021OpenCL Overview More details here https://khr.io/japanvirtualopenhouse

1. © The Khronos® Group Inc. 2021 - Page 1
This work is licensed under a Creative Commons Attribution 4.0 International License
OpenCL Overview
Heterogeneous Parallel Computation
Neil Trevett
Khronos President and OpenCL Chair
VP Developer Ecosystems, NVIDIA
ntrevett@nvidia.com|@neilt3d
January 2021

2. © The Khronos® Group Inc. 2021 - Page 2
This work is licensed under a Creative Commons Attribution 4.0 International License
Khronos Compute Acceleration Standards
GPU
GPU rendering +
compute
acceleration
Heterogeneous
compute
acceleration
Single source C++ programming
with compute acceleration
Graph-based vision and
inferencing acceleration
Lower-level APIs
Direct Hardware Control
Intermediate
Representation
(IR) supporting
parallel execution
and graphics
Higher-level Languages and APIs
Streamlined development and
performance portability
GPU
FPGA DSP
Custom Hardware
GPU
CPU
CPU
CPU
AI/Tensor HW
Increasing industry interest in
parallel compute acceleration to
combat the ‘End of Moore’s Law’
SYCL and SPIR were
originally OpenCL
Subgroups

3. © The Khronos® Group Inc. 2021 - Page 3
This work is licensed under a Creative Commons Attribution 4.0 International License
OpenCL – Low-level Parallel Programing
Complements GPU-only APIs
Simpler programming model
Relatively lightweight run-time
More language flexibility, e.g., pointers
Rigorously defined numeric precision
OpenCL
Kernel
Code
OpenCL
Kernel
Code
OpenCL
Kernel
Code
OpenCL C
Kernel
Code
GPU
DSP
CPU
CPU
FPGA
OpenCL
Devices
Host
CPU
NN HW
Runtime OpenCL API to
compile, load and execute
kernels across devices
Programming and Runtime Framework
for Application Acceleration
Offload compute-intensive kernels onto parallel
heterogeneous processors
CPUs, GPUs, DSPs, FPGAs, Tensor Processors
OpenCL C or C++ kernel languages
Platform Layer API
Query, select and initialize compute devices
Runtime API
Build and execute kernels programs on multiple devices
Explicit Application Control
Which programs execute on what device
Where data is stored in memories in the system
When programs are run, and what operations are
dependent on earlier operations

4. © The Khronos® Group Inc. 2021 - Page 4
This work is licensed under a Creative Commons Attribution 4.0 International License
OpenCL is Widely Deployed and Used
Accelerated Implementations
Modo
Desktop Creative Apps
CLBlast
SYCL-BLAS
Linear Algebra
Libraries
Parallel
Languages
Math and Physics
Libraries
Vision, Imaging
and Video Libraries
The industry’s most pervasive, cross-vendor, open standard
for low-level heterogeneous parallel programming
Arm Compute Library
SYCL-DNN
Machine Learning
Libraries and Frameworks
TI DL Library (TIDL)
VeriSilicon
Xiaomi
clDNN
Intel
Intel
Synopsis
MetaWare EV
NNAPI
https://en.wikipedia.org/wiki/List_of_OpenCL_applications
Vegas Pro
ForceBalance
Molecular Modelling Libraries
Machine Learning
Compilers

5. © The Khronos® Group Inc. 2021 - Page 5
This work is licensed under a Creative Commons Attribution 4.0 International License
OpenCL Open-Source Ecosystem Momentum
Tripling in under four years
April 2020, Folding@Home hit a new record of 2.4 exaflops,
faster than the top 500 traditional supercomputers combined,
thanks to almost 1 million new members of the network.
Folding@Home uses OpenCL to offload computations onto the
GPUs contained in the networked home PCs
October 2020, SARS-CoV-2 Simulations Go Exascale to Capture Spike Opening
with over a million citizen scientists banding together through the
Folding@home distributed computing project to create the first Exascale
computer and simulate an unprecedented 0.1 seconds of the viral proteome

6. © The Khronos® Group Inc. 2021 - Page 6
This work is licensed under a Creative Commons Attribution 4.0 International License
OpenCL 3.0
OpenCL C:
- kernels,
- address spaces,
- special types,
...
Most of C++17:
- inheritance,
- templates,
- type deduction,
...
C++ for OpenCL
Increased Ecosystem Flexibility
All functionality beyond OpenCL 1.2 queryable plus
macros for optional OpenCL C language features
New extensions that become widely adopted will be
integrated into new OpenCL core specifications
OpenCL C++ for OpenCL
Open-source C++ for OpenCL front end compiler
combines OpenCL C and C++17 replacing
OpenCL C++ language specification
Unified Specification
All versions of OpenCL in one specification for easier
maintenance, evolution and accessibility
Source on Khronos GitHub for community feedback,
functionality requests and bug fixes
Moving Applications to OpenCL 3.0
OpenCL 1.2 applications – no change
OpenCL 2.X applications - no code changes if all used
functionality is present
Queries recommended for future portability
C++ for OpenCL
Supported by Clang and uses the LLVM
compiler infrastructure
OpenCL C code is valid and fully compatible
Supports most C++17 features
Generates SPIR-V kernels

7. © The Khronos® Group Inc. 2021 - Page 7
This work is licensed under a Creative Commons Attribution 4.0 International License
Asynchronous DMA Extensions
OpenCL embraces a new class of Embedded Processors
Many DSP-like devices have Direct Memory Access hardware
Transfer data between global and local memories via DMA transactions
Transactions run asynchronously in parallel to device compute enabling wait for transactions to complete
Multiple transactions can be queued to run concurrently or in order via fences
OpenCL abstracts DMA capabilities via extended asynchronous workgroup copy built-ins
(New!) 2- and 3-dimensional async workgroup copy extensions support complex memory transfers
(New!) async workgroup fence built-in controls execution order of dependent transactions
New extensions complement the existing 1-dimensional async workgroup copy built-ins
Async Fence controls order of dependent transactions
All transactions prior to async_fence must complete
before any new transaction starts, without a
synchronous wait
async_copy1
async_copy2
async_fence
async_copy3
Async 3D-3D Copy Transaction
Copy
Transaction
Reshaping possible
Vglobal = Vlocal
Volume
global
Volume
local
The first of significant upcoming advances in OpenCL to
enhance support for embedded processors

8. © The Khronos® Group Inc. 2021 - Page 8
This work is licensed under a Creative Commons Attribution 4.0 International License
Roadmap: External Memory Sharing
• Generic extension to import external memory and semaphores exported by other APIs
- Explicitly hand-off memory ownership with OpenCL
- Wait and signal imported external semaphores
• Layer with API-specific interop extensions
- Vulkan interop first
- DX12 and other APIs in the future
• Improved flexibility over previous interop APIs using implicit resources
- As were used for DX9-11 and OpenGL
Import handles to memory
and semaphores
Synchronize memory
access and ownership
Vulkan
OpenCL
Interop

9. © The Khronos® Group Inc. 2021 - Page 9
This work is licensed under a Creative Commons Attribution 4.0 International License
Google Ports TensorFlow Lite to OpenCL
OpenCL providing ~2x inferencing
speedup over OpenGL ES
acceleration
TensorFlow Lite uses OpenGL ES as
a backup if OpenCL not available …
…but most mobile GPU vendors
provide OpenCL drivers - even if
not exposed directly to Android
developers
OpenCL is increasingly used as
acceleration target for higher-level
framework and compilers

10. © The Khronos® Group Inc. 2021 - Page 10
This work is licensed under a Creative Commons Attribution 4.0 International License
ML Compiler Steps
1.Import Trained
Network Description
2. Apply graph-level
optimizations e.g., node fusion,
node lowering and memory tiling
3. Decompose to primitive
instructions and emit programs
for accelerated run-times
Consistent Steps
Fast progress but still area of intense research
If compiler optimizations are effective - hardware accelerator APIs can stay ‘simple’ and
won’t need complex metacommands (e.g., combined primitive commands like DirectML)
Embedded NN Compilers
CEVA Deep Neural Network (CDNN)
Cadence Xtensa Neural Network Compiler (XNNC)

11. © The Khronos® Group Inc. 2021 - Page 11
This work is licensed under a Creative Commons Attribution 4.0 International License
SPIR-V Language Ecosystem
OpenCL C
C++ for OpenCL
clspv
triSYCL
Intel DPC++
Codeplay
ComputeCpp
LLVM
Clang
SYCL
SPIR-V LLVM
IR Translator
Khronos Open Source
3rd Party Open Source
Language Definitions
Closed Source
Environment Specs
OpenCL Vulkan
OpenCLon12
Inc. Mesa SPIR-V to
DXIL
SPIRV-Cross
GLSL
HLSL
Metal
Shading
Language
glslang
GLSL
HLSL DXC
DXIL
SPIR-V Tools
(Dis)Assembler
Validator
Optimize/Remap
Fuzzer
Reducer
OpenCL C
Online
Compilation
SPIR-V enables a rich ecosystem of languages and compilers to
target low-level APIs such as Vulkan and OpenCL, including
deployment flexibility: e.g., running OpenCL C kernels on Vulkan
IREE

12. © The Khronos® Group Inc. 2021 - Page 12
This work is licensed under a Creative Commons Attribution 4.0 International License
Layered OpenCL over Vulkan
• Clspv – Google’s open-source OpenCL kernel to Vulkan SPIR-V compiler
- Tracks top-of-tree LLVM and Clang, not a fork
• Clvk – prototype open-source OpenCL to Vulkan run-time API translator
• Used for shipping production apps and engines on Android
- Adobe Premiere Rush video editor – 200K lines of OpenCL C kernel code
- Butterfly Network iQ Ultrasound on Android
- Experimenting with Xiaomi MACE inferencing engine
Clang+Clspv
Compiler
OpenCL C or
C++ for OpenCL
Kernel Sources
OpenCL
Application
Host Code
Clvk run-time
API Translator
https://github.com/kpet/clvk
https://github.com/google/clspv
Vulkan
Runtime

13. © The Khronos® Group Inc. 2021 - Page 13
This work is licensed under a Creative Commons Attribution 4.0 International License
Layered OpenCL over DirectX12
• GPU-accelerated OpenCL on any system with DX12
- PC (x86 or Arm) and Cloud
• OpenCLOn12 - Microsoft and COLLABORA leveraging Clang/LLVM and MESA
- OpenCL 1.2 over DX12 is in development
- Also, OpenGLOn12 – OpenGL 3.3 over DX12
- https://devblogs.microsoft.com/directx/in-the-works-opencl-and-opengl-mapping-layers-to-directx/
DX12
Runtime
Clang+LLVM+
SPIR-V LLVM
OpenCL C or
C++ for OpenCL
Kernel Sources
OpenCL
Application
Host Code
CLOn12 Run-time
API Translator
Mesa SPIR-V
to DXIL
DXIL
Translates through
MESA’s NIR Intermediate
Representation

14. © The Khronos® Group Inc. 2021 - Page 14
This work is licensed under a Creative Commons Attribution 4.0 International License
Get Involved!
• OpenCL 3.0 increases deployment flexibility and
sets the stage for raising the bar on pervasively available functionality
- https://www.khronos.org/registry/OpenCL/
• OpenCL specification feedback on GitHub
- https://github.com/KhronosGroup/OpenCL-Docs/issues
• We want to know what you need next from OpenCL on the Khronos Forums!
- https://community.khronos.org/c/opencl
• Engage with Khronos and help OpenCL evolve
- Join as a Khronos member for a voice and a vote in any Khronos standard
- Or request an invite to the OpenCL Advisory Panel
- https://www.khronos.org/members/
• Neil Trevett
- ntrevett@nvidia.com
- @neilt3d