SPDK, PMDK & VtuneTM Summit Agenda

Andy Rudoff (Intel Data Center Group)
Pawel Skowron (Intel Data Center Group)
April 16th, 2019

SPDK, PMDK & Vtune™ Summit
Agenda(TwoSessions)
 First Half
– Persistent Memory Concepts
– Operating System Essentials
– The PMDK Libraries
 Second Half
– Flushing, Transactions, Allocation
– Language Support
– Comparing High and Low Level Languages
 Q & A (but don’t wait until here to ask!)
2

SPDK, PMDK & Vtune™ Summit 4
TheStorageStack(50,000ftview…)
User
Space
Kernel
Space
Standard
File API
Driver
Application
File System
Application
Standard
Raw Device
Access
Management Library
Management UI
Storage

A Programmer’s View
(not just C programmers!)
fd = open(“/my/file”, O_RDWR);
…
count = read(fd, buf, bufsize);
…
count = write(fd, buf, bufsize);
…
close(fd);
“Buffer-Based”

A Programmer’s View (mapped files)
fd = open(“/my/file”, O_RDWR);
…
base = mmap(NULL, filesize, PROT_READ|PROT_WRITE,
MAP_SHARED, fd, 0);
close(fd);
…
base[100] = ‘X’;
strcpy(base, “hello there”);
*structp = *base_structp;
…
“Load/Store”

Memory-MappedFiles
What are memory-mapped files really?
 Direct access to the page cache
 Storage only supports block access (paging)
With load/store access, when does I/O happen?
 Read faults/Write faults
 Flush to persistence
Not that commonly used or understood
 Quite powerful
 Sometimes used without realizing it Good reference: http://nommu.org/memory-
faq.txt

OSPaging
User
Space
Kernel
Space
Application ApplicationApplication
NVDIMM
NVDIMM
DRAM
… …
load/store
access
page fault
access

NVDIMM-N
Source: SNIA

Direct Load/Store Access
128, 256, 512GB
DDR4 Pin Compatible
Native Persistence
CPU
core
L1 Cache
L2 Cache
L3 Cache
Memory
Controller
DRAM
Optane
Controller
…
Firmware
• BIOS
• Operating System
• SNIA NVM programming Model
• Application

MotivationforthePMProgrammingModel?
0
25
50
75
100
Idle Average Random Read
Latency1
Storage
With
NAND
SSD
Storage with
Intel® Optane™
SSD
Hardware Latency
Software Latency
Idle Avg. is About
10µs
for 4kB
storage
Idle Avg. is About
80µs
for 4kB
NAND SSD latency
dominated by media
latency
Optane SSD latency
balanced between
SSD and System
1 Source–Intel-tested:Averagereadlatencymeasuredatqueuedepth1during4krandomwriteworkload.MeasuredusingFIO3.1.CommonConfiguration-Intel2UServerSystem,OSCentOS7.5,kernel4.17.6-1.el7.x86_64,CPU2xIntel®Xeon®6154Gold@3.0GHz(18cores),RAM256GB
DDR4@2666MHz.Configuration–Intel®Optane™SSDDCP4800X375GBandIntel®SSDDCP46001.6TB. Latency– AveragereadlatencymeasuredatQD1during4KRandomWriteoperationsusingFIO3.1.IntelMicrocode:0x2000043;SystemBIOS:00.01.0013;MEFirmware:
04.00.04.294;BMCFirmware:1.43.91f76955;FRUSDR:1.43.SSDstestedwerecommerciallyavailableattimeoftest.Thebenchmarkresultsmayneedtoberevisedasadditionaltestingis conducted.PerformanceresultsarebasedontestingasofJuly24,2018andmaynotreflectallpublicly
availablesecurityupdates.Seeconfigurationdisclosurefordetails.No productcanbeabsolutelysecure.SoftwareandworkloadsusedinperformancetestsmayhavebeenoptimizedforperformanceonlyonIntelmicroprocessors.Performancetests,suchasSYSmarkandMobileMark,are
measuredusingspecificcomputersystems,components,software,operationsandfunctions.Any changetoanyofthosefactorsmaycausetheresultstovary.Youshouldconsultotherinformationandperformanceteststoassistyouin fullyevaluatingyourcontemplatedpurchases,including
theperformanceofthatproductwhencombinedwithotherproducts.For morecompleteinformationvisitwww.intel.com/benchmarks.

MotivationforthePMProgrammingModel?
0
25
50
75
100
Latency1
Storage
With
NAND
SSD
Storage with
Intel® Optane™
SSD
Hardware Latency
Software Latency
Idle Avg. is About
10µs
for 4kB
storage
Idle Avg. is About
80µs
for 4kB
1 Source–Intel-tested:Averagereadlatencymeasuredatqueuedepth1during4krandomwriteworkload.MeasuredusingFIO3.1.CommonConfiguration-Intel2UServerSystem,OSCentOS7.5,kernel4.17.6-1.el7.x86_64,CPU2xIntel®Xeon®6154Gold@3.0GHz(18cores),RAM256GB
DDR4@2666MHz.Configuration–Intel®Optane™SSDDCP4800X375GBandIntel®SSDDCP46001.6TB. Latency– AveragereadlatencymeasuredatQD1during4KRandomWriteoperationsusingFIO3.1.IntelMicrocode:0x2000043;SystemBIOS:00.01.0013;MEFirmware:
04.00.04.294;BMCFirmware:1.43.91f76955;FRUSDR:1.43.SSDstestedwerecommerciallyavailableattimeoftest.Thebenchmarkresultsmayneedtoberevisedasadditionaltestingis conducted.PerformanceresultsarebasedontestingasofJuly24,2018andmaynotreflectallpublicly
availablesecurityupdates.Seeconfigurationdisclosurefordetails.No productcanbeabsolutelysecure.SoftwareandworkloadsusedinperformancetestsmayhavebeenoptimizedforperformanceonlyonIntelmicroprocessors.Performancetests,suchasSYSmarkandMobileMark,are
measuredusingspecificcomputersystems,components,software,operationsandfunctions.Any changetoanyofthosefactorsmaycausetheresultstovary.Youshouldconsultotherinformationandperformanceteststoassistyouin fullyevaluatingyourcontemplatedpurchases,including
theperformanceofthatproductwhencombinedwithotherproducts.For morecompleteinformationvisitwww.intel.com/benchmarks.
Next logical
improvement:
remove the SW stack.

0
25
50
75
100
Latency1
Storage With
NAND SSD
Storage with
Intel® Optane™ SSD
Hardware Latency
Software Latency
Memory
Subsystem with
Intel® Optane™
DC Persistent
memory
Storage
Idle Avg. is About
10µs
for 4kB
Memory Subsystem
Idle Avg. is About
~100ns to ~350ns
for 64B2
1 Source: Intel-tested: Average read latency measured at queue depth 1 during 4k random write workload. Measured using FIO 3.1. comparing Intel Reference platform with Optane™ SSD DC P4800X 375GB and Intel® SSD DC P4600 1.6TB compared to
SSDs commercially available as of July 1, 2018. Performance results are based on testing as of July 24, 2018 and may not reflect all publicly available security updates. See configuration disclosure for details. No product can be absolutely secure. For
more complete information about performance and benchmark results, visit www.intel.com/benchmarks.
2 App Direct Mode , NeonCity, LBG B1 chipset , CLX B0 28 Core (QDF QQYZ), Memory Conf 192GB DDR4 (per socket) DDR 2666 MT/s, Optane DCPMM 128GB, BIOS 561.D09, BKC version WW48.5 BKC, Linux OS 4.18.8-100.fc27, Spectre/Meltdown
Patched (1,2,3, 3a)

TheValueofPersistentMemory
Data sets addressable with no DRAM footprint
 At least, up to application if data copied to DRAM
Typically DMA (and RDMA) to PM works as expected
 RDMA directly to persistence – no buffer copy required!
The “Warm Cache” effect
 No time spend loading up memory
Byte addressable
Direct user-mode access
 No kernel code in data path
14

TheSNIANVMProgrammingModel
Persistent Memory
User
Space
Kernel
Space
Standard
File API
NVDIMM Driver
Application
File System
ApplicationApplication
Standard
Raw Device
Access
Storage File Memory
Load/Store
Management Library
Management UI
Standard
File API
Mgmt.
PM-Aware
File System
MMU
Mappings

TheProgrammingModelBuildsontheStorageAPIs
Persistent Memory
User
Space
Kernel
Space
Standard
File API
NVDIMM Driver
Application
File System
Standard
Raw Device
Access
Storage File Memory
Load/Store
Management Library
Management UI
Standard
File API
Mgmt.
PM-Aware
File System
MMU
Mappings
Use PM
Like an SSD

TheProgrammingModelBuildsontheStorageAPIs
Persistent Memory
User
Space
Kernel
Space
Standard
File API
NVDIMM Driver
Application
File System
Standard
Raw Device
Access
Storage File Memory
Load/Store
Management Library
Management UI
Standard
File API
Mgmt.
PM-Aware
File System
MMU
Mappings
Use PM
Like an SSD
Use PM
Like an SSD
(no page cache)
“DAX”

OptimizedFlushisthePrimaryNewAPI
Persistent Memory
User
Space
Kernel
Space
Standard
File API
NVDIMM Driver
Application
File System
Standard
Raw Device
Access
Storage File Memory
Load/Store
Management Library
Management UI
Standard
File API
Mgmt.
PM-Aware
File System
MMU
Mappings
Use PM
Like an SSD
Use PM
Like an SSD
(no page cache)
“DAX”
Optimized flush

User
Space
Kernel
Space
Application
RAM
• Well-worn interface, around for decades
• Memory is gone when application exits
– Or machine goes down
RAM
RAM RAM
Memory
Management
ptr = malloc(len)
ApplicationMemoryAllocation

User
Space
Kernel
Space
Application
NVM
• Simple, familiar interface, but then what?
– Persistent, so apps want to “attach” to regions
– Need to manage permissions for regions
– Need to resize, remove, …, backup the data
NVM
NVM NVM
Memory
Management
ptr = pm_malloc(len)
ApplicationNVMAllocation

It has always been thus:
 open()
 mmap()
 store...
 msync()
pmem just follows this decades-old model
 But the stores are cached in a different spot
visible
persistent
Visibilityversuspersistence

HowtheHWworks
WPQ
ADR
-or-
WPQ Flush (kernel only)
Core
L1 L1
L2
L3
WPQ
MOV
DIMM
CPUCACHES
CLWB + fence
-or-
CLFLUSHOPT + fence
-or-
CLFLUSH
-or-
NT stores + fence
-or-
WBINVD (kernel only)
Minimum Required
Power fail protected domain:
Memory subsystem
Custom
Power fail protected domain
indicated by ACPI property:
CPU Cache Hierarchy

AppResponsibilities
DAX mapped file?
(OS provides info)
CPU caches
considered
persistent?
(ACPI provides info)
CLWB?
(CPU_ID provides info)
CLFLUSHOPT?
(CPU_ID provides info)
Program Initialization
Use standard API for flushing
(msync/fsync or FlushFileBuffers)
Use CLFLUSH for flushing
Use CLFLUSHOPT+SFENCE
for flushing
Use CLWB+SFENCE
for flushing
Stores considered persistent
when globally-visible
no yes
yes
yes
yes
no
no
no

AppResponsibilities
(Recovery)
Dirty Shutdown?
Known Poison Blocks
Program Initialization
Data set is potentially inconsistent.
Recover.
Repair data set Normal Operation
yes no
nono

Creatingaprogrammingenvironment
NVDIMM
Kernel
Space
Application
Load/Store
Standard
File API
PM-Aware
File System
MMU
Mappings
Language Runtime
Libraries
Tools
Tools for correctness
and performance
Language support
Optimized allocators,
transactions
Result:
Safer, less error-prone

EnablingintheEcosystem
● Linux kernel version 4.19 (ext4, xfs)
● Windows Server 2019 (NTFS)
● VMware vSphere 6.7
● RHEL 7.5
● SLES 15 and SLES 12 SP4
● Ubuntu 18.*
● Java JDK 12
● Kubernetes 1.13
● OpenStack ‘Stein’
Be sure to see Steve Scargall’s talk: Persistent Memory Provisioning/Configuration tools

ProgrammingwithOptimizedFlush
• Use Standard unless OS says it is safe to use Optimized Flush
• On Windows
• When you successfully memory map a DAX file:
• Optimized Flush is safe
• On Linux
• When you successfully memory map a DAX file with MAP_SYNC:
• Optimized Flush is safe
• MAP_SYNC flag to mmap() is new

PMDKLibraries
Support for
volatile
memory usage
Low level support for
local persistent
memory
libpmem
remote access to
persistent memory
librpmem
NVDIMM
User
Space
Kernel
Space
Application
Load/Store
Standard
File API
pmem-Aware
File System
MMU
Mappings
PMDK
Interface to create arrays of
pmem-resident blocks, of
same size, atomically
updated
Interface for persistent memory
allocation, transactions and
general facilities
Interface to create a
persistent memory
resident log file
libpmemblklibpmemlog libpmemobj
Transaction
Support
C++ C
PCJ /
LLPL
Python
Low-level support
PCJ – Persistent
Collection for
Java
memkind
pmemkv
vmemcache
http://pmem.io
https://github.com/pmem/pmdk
Experimental
C++
Persistent
Containers
Language bindings
High Level Interfaces
( in development)

PMDKLibraries
Support for
volatile
memory usage
local persistent
memory
libpmem
remote access to
persistent memory
librpmem
NVDIMM
User
Space
Kernel
Space
Application
Load/Store
Standard
File API
pmem-Aware
File System
MMU
Mappings
PMDK
updated
general facilities
persistent memory
resident log file
Transaction
Support
C++ C
PCJ /
LLPL
Python
Low-level support
PCJ – Persistent
Collection for
Java
memkind
pmemkv
vmemcache
http://pmem.io
Experiment
al C++
Persistent
Containers
Language bindings
See Rob Dickinson’s talk on pmemkv
( in development)

PMDKLibraries
Support for
volatile
memory usage
local persistent
memory
libpmem
remote access to
persistent memory
librpmem
NVDIMM
User
Space
Kernel
Space
Application
Load/Store
Standard
File API
pmem-Aware
File System
MMU
Mappings
PMDK
updated
general facilities
persistent memory
resident log file
Transaction
Support
C++ C
PCJ /
LLPL
Python
Low-level support
PCJ – Persistent
Collection for
Java
memkind
pmemkv
vmemcache
http://pmem.io
Experiment
al C++
Persistent
Containers
Language bindings
See Usha and Piotr’s talk on vmemcache
( in development)

BARRIER TO ADOPTION
GAIN
PMEM as less
expensive DRAM
Volatile tiered
memory
Volatile object
cache
Persistent
key-value store
High-level
persistent
application
Low-level
persistent
application
Differentwaystousepersistentmemory

BARRIER TO ADOPTION
GAIN
PMEM as less
expensive DRAM
Volatile tiered
memory
Volatile object
cache
Persistent
key-value store
High-level
persistent
application
Low-level
persistent
application
Memory Mode

MemoryMode
 Not really a part of PMDK…
 … but it’s the easiest way to take advantage of Persistent Memory
 Memory is automatically placed in PMEM, with caching in DRAM
char *memory = malloc(sizeof(struct my_object));
strcpy(memory, “Hello World”);
When To Use
 modifying applications is not feasible
 massive amounts of memory is required (more TB)
 CPU utilization is low in shared environment (more VMs)

BARRIER TO ADOPTION
GAIN
PMEM as less
expensive DRAM
Volatile tiered
memory
Volatile object
cache
Persistent
key-value store
High-level
persistent
application
Low-level
persistent
application
libmemkind

libmemkind
 Explicitly manage allocations from App Direct, allowing for fine-grained
control of DRAM/PMEM
 The application can decide what type of memory to use for objects
struct memkind *pmem_kind = NULL;
size_t max_size = 1 << 30; /* gigabyte */
/* Create PMEM partition with specific size */
memkind_create_pmem(PMEM_DIR, max_size, &pmem_kind);
/* allocate 512 bytes from 1 GB available */
char *pmem_string = (char *)memkind_malloc(pmem_kind, 512);
/* deallocate the pmem object */
memkind_free(pmem_kind, pmem_string);
When To Use
 application can be modified
 different tiers of objects (hot, warm) can be identified
 persistence is not required

BARRIER TO ADOPTION
GAIN
PMEM as less
expensive DRAM
Volatile tiered
memory
Volatile object
cache
Persistent
key-value store
High-level
persistent
application
Low-level
persistent
application
libvmemcache

libvmemcache
 Seamless and easy-to-use LRU caching solution for persistent memory
Keys reside in DRAM, values reside in PMEM
 Designed for easy integration with existing systems
VMEMcache *cache = vmemcache_new();
vmemcache_add(cache, "/tmp");
const char *key = "foo";
vmemcache_put(cache, key, strlen(key), "bar", sizeof("bar"));
char buf[128];
ssize_t len = vmemcache_get(cache, key, strlen(key),
buf, sizeof(buf), 0, NULL);
vmemcache_delete(cache);
When To Use
 caching large quantities of data
 low latency of operations is needed
 persistence is not required

BARRIER TO ADOPTION
GAIN
PMEM as less
expensive DRAM
Volatile tiered
memory
Volatile object
cache
Persistent
key-value store
High-level
persistent
application
Low-level
persistent
application
libpmemkv

libpmemkv
 Local/embedded key-value datastore optimized for persistent memory.
Provides different language bindings and storage engines.
 High-level storage layer optimized for PMEM
const pmemkv = require('pmemkv');
const kv = new KVEngine('vsmap', '{"path":"/dev/shm/"}');
kv.put('key1', 'value1');
assert(kv.count === 1);
assert(kv.get('key1') === 'value1');
kv.all((k) => console.log(` visited: ${k}`));
kv.remove('key1');
kv.stop();
When To Use
 storing large quantities of data
 low latency of operations is needed
 persistence is required

BARRIER TO ADOPTION
GAIN
PMEM as less
expensive DRAM
Volatile tiered
memory
Volatile object
cache
Persistent
key-value store
High-level
persistent
application
Low-level
persistent
application
libpmemobj

libpmemobj
 Transactional object store, providing memory allocation, transactions, and
general facilities for persistent memory programming.
 Flexible and relatively easy way to leverage PMEM
When To Use
 direct byte-level access to objects is needed
 using custom storage-layer algorithms
 persistence is required
typedef struct foo {
PMEMoid bar; // persistent pointer
int value;
} foo;
int main() {
PMEMobjpool *pop = pmemobj_open (...);
TX_BEGIN(pop) {
TOID(foo) root = POBJ_ROOT(foo);
D_RW(root)->value = 5;
} TX_END;
}

BARRIER TO ADOPTION
GAIN
PMEM as less
expensive DRAM
Volatile tiered
memory
Volatile object
cache
Persistent
key-value store
High-level
persistent
application
Low-level
persistent
application
libpmem

libpmem
 Low-level library that provides basic primitives needed for persistent
memory programming and optimized memcpy/memmove/memset
 The very basics needed for PMEM programming
When To Use
 modifying application that already uses memory mapped I/O
 other libraries are too high-level
 only need low-level PMEM-optimized primitives (memcpy etc)
void *pmemaddr = pmem_map_file("/mnt/pmem/data", BUF_LEN,
PMEM_FILE_CREATE|PMEM_FILE_EXCL,
0666, &mapped_len, &is_pmem));
const char *data = "foo";
if (is_pmem) {
pmem_memcpy_persist(pmemaddr, data, strlen(data));
} else {
memcpy(pmemaddr, data, strlen(data));
pmem_msync(pmemaddr, strlen(data));
}
close(srcfd);
pmem_unmap(pmemaddr, mapped_len);

BARRIER TO ADOPTION
GAIN
PMEM as less
expensive DRAM
Volatile tiered
memory
Volatile object
cache
Persistent
key-value store
High-level
persistent
application
Low-level
persistent
application
libpmem
libpmemobj
libpmemkvlibvmemcache
libmemkind
Memory Mode

Persistent Memory
NVDIMMs
User
Space
Kernel
Space
Standard
File API
NVDIMM Driver
Application
File System
Standard
Raw Device
Access
mmap
Load/Store
Management Library
Management UI
Standard
File API
pmem-Aware
File System
MMU
Mappings
Hardware
CPU DDR
Block
PMDK
47
ProgrammingModelTools
pmempool
pmemcheck
daxio
daxctl
Persistence Inspector
VTune Amplifier
Valgrind
VTune Platform Profiler
FIO
MLC
pmembench
PMEMOBJ_LOG_LEVEL
Administration, Benchmark, Debug, Performance
ixpdimm-cli
ndctl

Flushing,Transactions,Allocation
48

Libpmemexamples
Source: https://github.com/pmem/pmdk/tree/master/src/examples/libpmem
/*
* simple_copy.c -- show how to use pmem_memcpy_persist()
*
* usage: simple_copy src-file dst-file
*
* Reads 4k from src-file and writes it to dst-file.
*/
…
/* create a pmem file and memory map it */
if ((pmemaddr = pmem_map_file(argv[2], BUF_LEN, PMEM_FILE_CREATE|PMEM_FILE_EXCL,
0666, &mapped_len, &is_pmem)) == NULL) {
perror("pmem_map_file");
exit(1);
}

Usingis_pmem
if (is_pmem) {
pmem_memcpy_persist(pmemaddr, buf, cc);
} else {
memcpy(pmemaddr, buf, cc);
pmem_msync(pmemaddr, cc);
}

POSIXLoad/StorePersistence
51
open(…);
mmap(…);
strcpy(pmem, "moose");
msync(pmem, 6, MS_SYNC);

Optimizedflush(useonlywhensafe!)
52
open(…);
mmap(…);
strcpy(pmem, "moose");
pmem_persist(pmem, 6);
Application
libpmem
memory-mapped pmem

Flushesarenotatomicoperations
open(…);
mmap(…);
strcpy(pmem, "Hello, World!");
pmem_persist(pmem, 14); Crash
1. "0000000000..."
2. "Hello, W000000..."
3. "00000000orld!0"
4. "Hello, 00000000"
5. "Hello, World!0"
Result

libpmemobj
“transactionalobjectstore”
Application
libpmem
Load/Store
memory-mapped pmem
libpmemobj
transactionsatomic
operations
allocatorlists
locks
API API API

TransactionalAPI
 Need for Transactions
• To enable updates and persist larger & non-contiguous portions of data atomically x86
CPUs guarantee atomicity only for 8 byte stores
 libpmemobj provides ACID transactions for persistent memory
• Atomicity means that a transaction either succeeds or fails completely
• Consistency means that the transaction transforms PMEMobjpool from one consistent state
to another. This means that a pool won’t get corrupted by a transaction.
• Isolation means that transactions can be executed as if the operations were executed
serially on the pool. This is optional, and requires user-provided locks.
• Durability means that once a transaction is committed, it remains committed even in the
case of system failures

TransactionalAPI
 Inside of a transaction the application can:
• Allocate new objects
• Free existing objects
• Modify existing objects
• Isolate objects
TX_BEGIN_PARAM(pool, TX_PARAM_MUTEX, &root->lock, TX_PARAM_NONE) {
pmemobj_tx_add_range_direct(root, sizeof(*root));
root->objA = pmemobj_tx_alloc(sizeof(struct objectA), type_num);
pmemobj_tx_free(root->objB):
root->objB = OID_NULL;
} TX_END http://pmem.io/pmdk/manpages/linux/master/libpmemobj/pmemobj_tx_begin.3

AllocatorDesignDecisions
 Inspired by Hoard allocator
 For faster access, allocator run-time metadata is stored in DRAM
 Operates on Persistent Heap
• Persistent Memory Pools
 Dynamic growing of the heap at runtime not supported
 Redo log and Undo log for Recovery
57

AtomicAPIs
 Memory allocation has at least two steps:
1. Selection of the memory block to allocate
2. Assignment of the resulting pointer to some destination pointer
 If the application is interrupted in between these steps
• On DRAM, nothing happens, because all memory allocations vanish
• On PMEM, memory is leaked, because the allocated object is unreachable
root->objA = pmalloc(pool, sizeof(struct objectA));
Step 2
Step 1

AtomicAPIs
• In libpmemobj atomic API these two steps are merged into one. The object is fail-safe
atomically allocated and assigned to the destination pointer.
• This API also introduces a type numbers and cunstructors
• Type number is an 8 byte embedded metadata field which identifies the object in
the pool. Can be used to recover data if objects become unreachable.
• Constructors are used to initialize objects with data. Once an object is allocated, the
constructor was ran successfully.
pmemobj_alloc(pool, &root->objA, sizeof(struct objectA),
type_num, constr, constr_arg);
http://pmem.io/pmdk/manpages/linux/master/libpmemobj/pmemobj_alloc.3

HowtoChooseyourAPI/Language
Start from the highest-level interface that works for you
 Example: libpmemkv
 If that doesn’t meet your needs, try a Java or C++ container
 Only if that doesn’t meet your needs, build your own data structures
– In Java, C++, C (more later)

From C
 Fully validated, delivered on Linux, early
access on Windows
 Can stick to pure API calls, but macros
add:
– Compile-time type safety
– Transaction syntax, similar to try/catch
From Java
 Persistent Containers for Java
(Experimental)
From C++
 Fully validated, delivered on Linux, early
access on Windows
 Use C++ type system & syntax: much
cleaner, less error-prone
 Experimental persistent STL like
containers ( work in progress )
From Python
 PyNVM (Experimental)
Other work
 pmemcheck, pmreorder, Persistence
Inspector
Builtonlibpmemobj…

Otherlanguagesupport
libpmemkv (see Rob Dickinson’s talk later today)
 Building up quite a long list of supported languages
 Not just ”call C from language X” – idiomatic support
Research underway for pmem support in other languages
 Don’t be shy about letting us know your needs
– Example: (FORTRAN support for memkind)

ComparingHighandLowLevelLanguages
64

CProgrammingwithlibpmemobj
65

TransactionSyntax
TX_BEGIN(Pop) {
/* the actual transaction code goes here... */
} TX_ONCOMMIT {
/*
* optional − executed only if the above block
* successfully completes
*/
} TX_ONABORT {
/*
* optional − executed if starting the transaction fails
* or if transaction is aborted by an error or a call to
* pmemobj_tx_abort()
*/
} TX_FINALLY {
/*
* optional − if exists, it is executed after
* TX_ONCOMMIT or TX_ONABORT block
*/
} TX_END /* mandatory */

PropertiesofTransactions
TX_BEGIN_PARAM(Pop, TX_PARAM_MUTEX, &D_RW(ep)->mtx, TX_PARAM_NONE) {
TX_ADD(ep);
D_RW(ep)->count++;
} TX_END
Powerfail
Atomicity
Multi-Thread
Atomicity
Caller must
instrument code
for undo logging

PersistentMemoryLocks
 Want locks to live near the data they protect (i.e. inside structs)
 Does the state of locks get stored persistently?
– Would have to flush to persistence when used
– Would have to recover locked locks on start-up
– Might be a different program accessing the file
– Would run at pmem speeds
 PMEMmutex
– Runs at DRAM speeds
– Automatically initialized on pool open

C++Programmingwithlibpmemobj
69

C++QueueExample:Declarations
/* entry in the queue */
struct pmem_entry {
persistent_ptr<pmem_entry> next;
p<uint64_t> value;
};
persistent_ptr<T>
Pointer is really a position-independent
Object ID in pmem.
Gets rid of need to use C macros like
D_RW()
p<T>
Field is pmem-resident and needs to be
maintained persistently.
Gets rid of need to use C macros like
TX_ADD()

C++QueueExample:Transaction
void push(pool_base &pop, uint64_t value) {
transaction::exec_tx(pop, [&] {
auto n = make_persistent<pmem_entry>();
n->value = value;
n->next = nullptr;
if (head == nullptr) {
head = tail = n;
} else {
tail->next = n;
tail = n;
}
});
}
Transactional
(including allocations &
frees)

LinkstoMoreinformation
Find the PMDK (Persistent Memory Development Kit) at http://pmem.io/pmdk/
Getting Started
 Intel IDZ persistent memory- https://software.intel.com/en-us/persistent-memory
 Entry into overall architecture - http://pmem.io/2014/08/27/crawl-walk-run.html
 Emulate persistent memory - http://pmem.io/2016/02/22/pm-emulation.html
Linux Resources
 Linux Community Pmem Wiki - https://nvdimm.wiki.kernel.org/
 Pmem enabling in SUSE Linux Enterprise 12 SP2 - https://www.suse.com/communities/blog/nvdimm-enabling-
suse-linux-enterprise-12-service-pack-2/
Windows Resources
 Using Byte-Addressable Storage in Windows Server 2016 -https://channel9.msdn.com/Events/Build/2016/P470
 Accelerating SQL Server 2016 using Pmem - https://channel9.msdn.com/Shows/Data-Exposed/SQL-Server-2016-
and-Windows-Server-2016-SCM--FAST
Other Resources
 SNIA Persistent Memory Summit 2018 - https://www.snia.org/pm-summit
 Intel manageability tools for Pmem - https://01.org/ixpdimm-sw/

SPDK, PMDK & VtuneTM Summit Agenda

SPDK, PMDK & VtuneTM Summit Agenda

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (17)

Semelhante a SPDK, PMDK & VtuneTM Summit Agenda

Semelhante a SPDK, PMDK & VtuneTM Summit Agenda (20)

Mais de Intel® Software

Mais de Intel® Software (20)

Último

Último (20)

SPDK, PMDK & VtuneTM Summit Agenda