Ramon Fried covers the following topics:
* What DMA is.
* DMA Buffer Allocations and Management.
* Cache Coherency.
* PCI and DMA.
* dmaengine Framework.
Ramon is an Embedded Linux team leader in TandemG, leading various cutting edge projects in the Linux kernel.
He has years of experience in embedded systems, operating systems and Linux kernel.
2. Property of Tandem Group
Whenwe’redone, it’sdone!
Ramon Fried
Developer and hacker since 1983
B.sc in computer science
Expertise in Embedded systems and Linux
system and kernel development.
Currently Embedded Linux Team Leader in
TandemG
3. Property of Tandem Group
Whenwe’redone, it’sdone!
TandemG
TandemG is Israel’s leading Software, Hardware and Systems R&D
center, acting as a one-stop-shop for our range of partners, from
prominent start-ups through to market leaders
In the embedded domain, TandemG tailors solutions spanning
across RTOS, Embedded Linux, low-level Android and DSP.
For the second year in a row, TandemG has been selected to
Delloite’s Israel Technology Fast50 list in the 23rd place
Visit us at www.tandemG.com, for additional details.
4. Property of Tandem Group
Whenwe’redone, it’sdone!
Agenda
What is DMA
DMA Buffer allocation
— Coherent
— Streaming
Scatter Gather mapping
DMA pools
DMA Triggering
— PCI
— dmaengine
5. Property of Tandem Group
Whenwe’redone, it’sdone!
What is DMA
Direct memory access
Feature of computer systems that allows
certain hardware subsystems to access main
system memory (RAM), independent of the
central processing unit (CPU).
CPU can be notified on the end of operation by
IRQ.
6. Property of Tandem Group
Whenwe’redone, it’sdone!
What is DMA
Photo from: http://encyclopedia2.thefreedictionary.com/DMA
7. Property of Tandem Group
Whenwe’redone, it’sdone!
DMA Buffer allocation
DMA controller works on physical addresses
Physical memory needs to be accessible by
DMA controller
Memory must be continuous
8. Property of Tandem Group
Whenwe’redone, it’sdone!
DMA Buffer allocation
Coherent DMA mapping
— Usually long lasting.
— Can be accessed by both ends.
— No-caching *
— At least page sized.
Streaming DMA mapping
— Usually singly used and freed.
— Architecture/Platform optimized.
— direction must be defined explicitly.
9. Property of Tandem Group
Whenwe’redone, it’sdone!
DMA access mask
#include <linux/dma-mapping.h>
int dma_set_mask_and_coherent(struct device
*dev, u64 mask);
int dma_set_mask (struct device *dev, u64
mask);
int dma_set_coherent_mask(struct device *dev,
u64 mask);
12. Property of Tandem Group
Whenwe’redone, it’sdone!
Streaming DMA mapping (cont’d)
enum dma_data_direction
— DMA_TO_DEVICE
— DMA_FROM_DEVICE
— DMA_BIDIRECTIONAL
13. Property of Tandem Group
Whenwe’redone, it’sdone!
Streaming DMA usage
Buffer Ownership
— Buffer is owned by the device.
— Altering the buffer can be done only after acquiring
ownership
dma_sync_single_for_cpu()
— After altering the buffer, the ownership needs to be
returned to the device.
dma_sync_single_for_device()
14. Property of Tandem Group
Whenwe’redone, it’sdone!
Scatter gather buffers
Special type of streaming DMA
writev, readv, clustered buffers (YUV plannar,
non continuous memory, etc.)
15. Property of Tandem Group
Whenwe’redone, it’sdone!
Scatter/Gather API
#include linux/dma-mapping.h
int dma_map_sg( struct device *dev,
struct scatterlist *sg,
int nents,
enum dma_data_direction dir);
void dma_unmap_sg(struct device *dev,
struct scatterlist *list,
int nents,
enum dma_data_direction dir);
16. Property of Tandem Group
Whenwe’redone, it’sdone!
Don’t forget to sync
#include linux/dma-mapping.h
void dma_sync_sg_for_cpu(struct device *dev,
struct scatterlist *sg,
int nelems,
enum dma_data_direction dir);
void dma_sync_sg_for_device(struct device *dev,
struct scatterlist *sg,
int nelems,
enum dma_data_direction dir);
17. Property of Tandem Group
Whenwe’redone, it’sdone!
DMA Pools
DMA Pools
— Coherent
— Used to allocate buffers smaller than a page.
— dma_pool_create()
— dma_pool_destroy()
— dma_pool_alloc()
— dma_pool_free()
18. Property of Tandem Group
Whenwe’redone, it’sdone!
Start the DMA operation
19. Property of Tandem Group
Whenwe’redone, it’sdone!
Triggering the DMA operation
PCI
dmaengine
20. Property of Tandem Group
Whenwe’redone, it’sdone!
PCI Wrappers
PCI wrappers
— pci_alloc_consistent()
— pci_free_consistent()
— pci_set_dma_mask()
— pci_pool_create()
— …
21. Property of Tandem Group
Whenwe’redone, it’sdone!
PCI DMA transaction example
dma_addr_t bus_addr;
bus_addr = dma_map_single(&dev->pci_dev->dev, buffer, count, DMA_TO_DEVICE);
writeb(dev->registers.command, DAD_CMD_DISABLEDMA);
writeb(dev->registers.command, DAD_CMD_WR);
writel(dev->registers.addr, cpu_to_le32(bus_addr));
writel(dev->registers.len, cpu_to_le32(count));
writeb(dev->registers.command, DAD_CMD_ENABLEDMA);
22. Property of Tandem Group
Whenwe’redone, it’sdone!
DMA Engine
Subsystem to handle memory-to-device
transfers
Exists since 2.6.18 (2006)
Code in “drivers/dma”
Documentation in “dmaengine/*”
Poorly documented
DMA Controller work with physical memory. It doesn’t understands CPU’s virtual.
It can access only the physical memory that is actually connected to it in the interconnect,.
This memory is called bus addresses and usually is the same as CPU’s physical memory.
-----
Virtual memory is not always continuous. The underling physical memory must be continuous.
Basically kmalloc returns continuous, vmalloc not.
Coherent examples: ring buffers, messages queues, mailboxes, etc.
However, Reordering can occur. * We’ll see soon a way of allocating smaller chunks.
Streaming examples: file systems buffers, network buffers
Returns 0 on fail.
Dma_handle is output param
The buffer is given in ptr.
Mapping can fail of course. Check return value using: dma_mapping_error()