2. Neuronics
• Developing projects in
• Computer vision & AI
• 3D generation and simulation
• Nvidia Edge and server platforms and SDKs
• Defense / Military
• UAV / Autonomous vehicles
• Autonomous vassals
• robots
• Medical
COMPUTER
VISION & AI
NVIDIA
PLATFORMS
3D
GENERATION
&
SIMULATION
6. Noise
•DSNU - Dark Signal Non-Uniformity ~1% bias
•PRNU - Photo-Response Non-Uniformity ~1% gain
variability
•Thermal noise
•Sensor
•Gain
•Temp
7. Data problems
•Capture & Label Data – expensive / not possible
•Augmentation - Sensitive
•Just changing brightness like in RGB images wont work!
•Simulation
•3D World simulation (Real, Blender, Omniverse, Unity)
•Thermal assignment (simple)
• Glitter simulation / Ray tracing
•Atmospheric simulation (Frequency, distance, angle)
8. Simulation – Basic
3D World
simulation
Temperature
Map
Sensor
Simulation
Distance
Map
Glitter Map
Shared
config
9. Sensor Simulation – Advance
Temperature
to electrons
Lens Shading
PSF
) Point spread function(
Motion Blur
PRNU /
DSNU
NUC Simulation
Shot Noise
Electrons to
sensor out
Temperature
map
Distance Map
Glitter Map
Per frame
exposure Distance Map
Wave Gliter
to sensor
out
The simulation is not standalone since exposure is
dynamic and changed by the processing pipe
12. IR Recording and transfer
•We DON’T compress IR sensor output and pixel targets
•Store in raw format 8/10/12 bit!!!
•Sometimes we have problem in streaming it
•Lossless JPEG, Lossless AV1
•Process at the edge - > Jetsons!!!
•Stream Raw over fiber using NVIDIA Rivermax
14. NVIDIA Video Analytics flow
Sensors Actions
Capture
and Decode
Pre-processing &
Batching
AI Inference Tracking Composition
Business Rules
and Analytics
…
…
H.265
15. Processing Pipe requirements
• Process stream/s of video (and audio/other data synchronously)
• Block based for each unit/function
• Easy connection between camera / socket / encoder / decoder / file / parser
• Ability to reuse and enhance existing modules (“Inheritance” + open source)
• Fix all the threads / buffers on its own
• Runs on Linux
16. Edge Processing Architecture
•DeepStream Nvidia Standard processing architecture – Not used
•Python CPP Processing
•Basic filtering using OpenCV/CUDA , RAPIDs
• Double Filter – DoG
• Inverse PSF
• Neural Networks
• Segmentation
• Binary Operations
•Tracking – MTT with Kalman/EKF/UKF