2.  Overview
 Sub-systems
 Kernel API & EBA subsystem
 Wear-leveling subsystem
 Scanning subsystem
2

3.  UBI – Unsorted Block Images
 A volume management system
 Provides static and dynamic volumes
 Wear-leveling across whole flash device
 Transparent bad blocks management
 Read disturbance handling
 Merged in the mainline Linux kernel since
v2.6.22
3

4. Bootloader Kernel Image Root Filesystem (UBIFS)
Static UBI Volume Dynamic UBI Volume
(0,0) (0,1) ... (0,P) (1,0) (1,1) (1,2) (1,3) (1,4) (1,5) ... (1,Q)
0 1 2 3 4 5 6 7 8 9 ... N
MTD Partition MTD Partition (UBI Device)
MTD Device
4

5. UBI Kernel API
UBI Initialization UBI Erase Block Association Subsystem
UBI Scanning Subsystem UBI Wear-leveling Subsystem
UBI I/O Subsystem
MTD Layer
5

6.  Read from an unmapped LEB
UBI WL UBI IO
 Read from a mapped LEB
ubi_wl_get_peb() ubi_io_read()
 Write to a mapped LEB
 Write to an unmapped LEB ubi_wl_put_peb() ubi_io_write()
 Map a LEB ubi_wl_scrub_peb() ubi_io_sync_erase()
 Unmap a LEB
ubi_wl_flush() ubi_io_mark_bad()
 Erase a LEB
ubi_io_read_data()
Filesystem UBI KAPI UBI EBA
fs_read() ubi_leb_read() ubi_eba_read_leb() ubi_io_write_data()
fs_write() ubi_leb_write() ubi_eba_write_leb() ubi_io_read_ec_hdr()
ubi_leb_map() ubi_eba_map_leb() ubi_io_write_ec_hdr()
ubi_leb_erase() ubi_eba_copy_leb() ubi_io_read_vid_hdr()
ubi_leb_unmap() ubi_eba_unmap_leb() ubi_io_write_vid_hdr()
6

8.  Responsible for
 Management of PEBs
 Wear-leveling
 Scrubbing (read disturbance)
 Works in terms of PEBs and erase counters
 Knows nothing about LEBs, volumes, etc
 Internal data structures
 Four RB-trees and one queue
 External interfaces
 ubi_wl_get_peb()
 ubi_wl_put_peb()
 ubi_wl_scrub_peb()
 ubi_wl_flush()
8

9. All good PEBs are managed with four RB-trees, and one queue
drivers/mtd/ubi/ubi.h
struct ubi_device {
...
struct rb_root used;
struct rb_root erroneous;
struct rb_root free;
struct rb_root scrub;
struct list_head pq[UBI_PROT_QUEUE_LEN];
...
}
Free PEBs free 1,1 1,7 2,5 3,2 7,8
Good PEBs pq 3,4
used 3,9 6,6 8,3
In-used PEBs
scrub
erroneous
Note: These RB-trees use (ec, pnum) pairs as keys
9

10. drivers/mtd/ubi/wl.c
int ubi_wl_get_peb(struct ubi_device *ubi, int dtype)
int ubi_wl_put_peb(struct ubi_device *ubi, int pnum, int torture)
int ubi_wl_scrub_peb(struct ubi_device *ubi, int pnum)
int ubi_wl_flush(struct ubi_device *ubi)
free 1,1 1,7 2,5 3,2 7,8
ubi_wl_get_peb()
pq 3,4
ubi_wl_put_peb()
used 3,9 6,6 8,3
ubi_wl_scrub_peb() scrub
erroneous
ubi_wl_flush()
ubi_thread
10

11. drivers/mtd/ubi/wl.c
int ubi_wl_get_peb(struct ubi_device *ubi, int dtype)
shortterm unknown longterm
1. Pick a PEB from the free RB-tree
according to the hint @dtype
free 1,1 1,7 2,5 3,2 7,8
• longterm
• shortterm
• unknown pq 3,4
2. Move the picked PEB to the pq queue used 3,9 6,6 8,3
• why pq? why not used? scrub
erroneous
Keep newly allocated PEBs from
being moved due to wear-leveling.
ubi_thread
11

12. drivers/mtd/ubi/wl.c
int ubi_wl_scrub_peb(struct ubi_device *ubi, int pnum)
1. Move the PEB @pnum from pq/used to
scrub
free 1,7 2,5 3,2 7,8
2. Schedule a wear-leveling request
pq 3,4 1,1
used 3,9 6,6 8,3
scrub
erroneous
Besides wear-leveling, I also take
care of scrubbing.
ubi_thread
12

13. drivers/mtd/ubi/wl.c
int ubi_wl_put_peb(struct ubi_device *ubi, int pnum, int torture)
1. Remove the PEB @pnum from one of
the in-used RB-trees or pq.
free 1,7 2,5 3,2 7,8 6,6
2. Schedule the PEB @pnum for erasure.
3. When the erasure is done without any pq 3,4 1,1
error, the PEB will be put back to the free
RB-tree. used 3,9 6,6
scrub 8,3
erroneous
Again, the erasure will be delagated
to me.
ubi_thread
13

14. drivers/mtd/ubi/wl.c
int ubi_wl_flush(struct ubi_device *ubi)
drivers/mtd/ubi/wl.c
struct ubi_work {
struct list_head list;
int (*func)(struct ubi_device *ubi, struct ubi_work *wrk, int cancel);
/* The below fields are only relevant to erasure works */
struct ubi_wl_entry *e;
int torture;
};
1. Flush all pending works erase_worker() wear_leveling_worker()
ubi_work ubi_work ubi_work ubi_work
ubi_thread
14

15. drivers/mtd/ubi/wl.c
static int wear_leveling_worker(struct ubi_device *ubi,
struct ubi_work *wrk, int cancel)
static struct ubi_wl_entry *find_wl_entry(struct rb_root *root, int max)
if (!free || (!scrub && !used))
return
if (scrub) {
e1 = pick the least worn out PEB from the @scrub
e2 = find_wl_entry(free, WL_FREE_MAX_DIFF) free 1,7 2,5 3,2 6,6 7,8
}
else {
e1 = pick the least worn out PEB from the @used
e2 = find_wl_entry(free, WL_FREE_MAX_DIFF) pq 3,4 1,1
if ((e2->ec – e1->ec)<UBI_WL_THRESHOLD) used 3,9
return;
} scrub 8,3
ubi_eba_copy_leb(ubi, e1->pnum, e2->pnum, vid_hdr)
erroneous
15

16. drivers/mtd/ubi/wl.c
static int erase_worker(struct ubi_device *ubi, struct ubi_work *wrk, int cancel)
err = sync_erase(ubi, e, wl_wrk->torture);
if (!err) {
wl_tree_add(e, &ubi->free);
serve_prot_queue(ubi);
return ensure_wear_leveling(ubi);
}
if (err == -EINTR || err == -ENOMEM || err == -EAGAIN || err == -EBUSY)
return schedule_erase(ubi, e, 0)
else if (err != -EIO)
goto out_ro; free 1,7 2,5 3,2 6,6 7,8
/* It is %-EIO, the PEB went bad */
if (!ubi->bad_allowed)
goto out_ro; pq 3,4 1,1
if (ubi->beb_rsvd_pebs == 0) used 3,9
goto out_ro;
scrub 8,3
err = ubi_io_mark_bad(ubi, pnum);
return err;
erroneous
out_ro:
ubi_ro_mode(ubi) /* switch to read-only mode */
return err;
16

18.  Responsible for
 Scanning the flash media
 Checking UBI headers
 Providing complete information about the UBI flash image
 UBI on-flash data structures
 Erase Counter Header
 Volume Identifier Header
 Volume Table
 Temporary data structures during scanning process
 Scan Info
 Scan Volume
 Scan Erase Block
 Four lists: free, erase, corr, alien
 Unclean reboot
18

19.  Every good PEB has a 64-byte Erase Counter Header
 Every good mapped PEB has a 64-byte Volume Identifier Header
 A “layout volume” contains two copies of the Volume Table
…
LEBs 0,0 0,1 ... 0,P 1,0 1,1 2,0 2,1 2,2 ... 2,Q
PEBs 0 1 2 3 4 5 6 7 8 ... N
19

20.  Every good PEB has a 64-byte Erase Counter Header
drivers/mtd/ubi/ubi-media.h
struct ubi_ec_hdr {
__be32 magic; /* EC header magic number (%UBI_EC_HDR_MAGIC) */
__u8 version; /* version of UBI implementation */
__u8 padding1[3]; /* reserved for future, zeroes */
__be64 ec; /* the erase counter */
__be32 vid_hdr_offset; /* where the VID header starts */
__be32 data_offset; /* where the user data start */
__be32 image_seq; /* image sequence number */
__u8 padding2[32]; /* reserved for future, zeroes */
__be32 hdr_crc; /* erase counter header CRC checksum */
} __attribute__ ((packed)); …
LEBs 0,0 0,1 ... 0,P 1,0 1,1 2,0 2,1 2,2 ... 2,Q
PEBs 0 1 2 3 4 5 6 7 8 ... N
20

21.  Every good mapped PEB has a 64-byte Volume Identifier Header
drivers/mtd/ubi/ubi-media.h
struct ubi_vid_hdr {
__be32 magic; /* VID magic number (%UBI_VID_HDR_MAGIC)*/
__u8 version; /* version of UBI implementation */
__u8 vol_type; /* volume type (%UBI_VID_DYNAMIC or %UBI_VID_STATIC) */
__u8 copy_flag; /* for wear-leveling reasons */
__u8 compat; /* compatibility of this volume */
__be32 vol_id; /* ID of this volume */
__be32 lnum; /* LEB number */
__u8 padding1[4]; /* reserved for future, zeroes */
__be32 data_size; /* bytes of data this LEB contains */
__be32 used_ebs; /* total number of used LEBs in this volume */
__be32 data_pad; /* padded bytes at the end of this PEB */
…
__be32 data_crc; /* CRC of the data stored in this LEB */
__u8 padding2[4]; /* reserved for future, zeroes */
__be64 sqnum; /* sequence number */
__u8 padding3[12]; /* reserved for future, zeroes */
__be32 hdr_crc; /* VID header CRC checksum */
} __attribute__ ((packed));
LEBs 0,0 0,1 ... 0,P 1,0 1,1 2,0 2,1 2,2 ... 2,Q
PEBs 0 1 2 3 4 5 6 7 8 ... N
21

22.  A “layout volume” contains two copies of the Volume Table
drivers/mtd/ubi/ubi-media.h
struct ubi_vtbl_record {
__be32 reserved_pebs; /* physical eraseblocks reserved for this volume */
__be32 alignment; /* volume alignment */
__be32 data_pad; /* padded bytes for the requested alignment */
__u8 vol_type; /* %UBI_VID_DYNAMIC or %UBI_VID_STATIC */
__u8 upd_marker; /* if volume update was started but not finished */
__be16 name_len; /* volume name length */
__u8 name[UBI_VOL_NAME_MAX+1]; /* volume name */
__u8 flags; /* volume flags (%UBI_VTBL_AUTORESIZE_FLG) */
…
__u8 padding[23]; /* reserved for future, zeroes */
__be32 crc; /* CRC32 checksum of the record */
} __attribute__ ((packed));
LEBs 0,0 0,1 ... 0,P 1,0 1,1 2,0 2,1 2,2 ... 2,Q
PEBs 0 1 2 3 4 5 6 7 8 ... N
22

23. Volumes
... ...
SEB corr …
Scan Erase Block 100
…
SEB free
...
205
…
erase
SEB
Scan Volume … …
0 …
alien
Scan Info
SEB
522
SEB
1
“layout volume” … …
(internal) SEB
0
SEB SEB
…
2 101
PEBs ... 23

24.  EC hdr is written to a PEB right after the PEB is erased
drivers/mtd/ubi/wl.c
static int sync_erase(struct ubi_device *ubi, struct ubi_wl_entry *e, int torture)
{
unsigned long long ec = e->ec;
[... Deleted ...]
err = ubi_io_sync_erase(ubi, e->pnum, torture);
if (err < 0)
goto out_free;
ec += err;
if (ec > UBI_MAX_ERASECOUNTER) {
/*
* Erase counter overflow. Upgrade UBI and use 64-bit
* erase counters internally.
*/
ubi_err("erase counter overflow at PEB %d, EC %llu", e->pnum, ec);
err = -EINVAL;
goto out_free;
}
dbg_wl("erased PEB %d, new EC %llu", e->pnum, ec);
ec_hdr->ec = cpu_to_be64(ec);
err = ubi_io_write_ec_hdr(ubi, e->pnum, ec_hdr);
[... Deleted ...]
}
24

25.  Map a LEB L to PEB P
 Write VID header (with lnum L) to P
 Unmap a LEB L to PEB P
 Schedule P for erasure
 Remap a LEB L from PEB P0 to PEB P1
 Schedule P0 for erasure
 Write VID header (with lnum L) to P1
 Copy a PEB P0 which is mapped to L to PEB P1
 Write VID header (with lnum L) to P1
 Copy contents of P0 to P1
 Schedule P0 for erasure
25

26.  Whenever the volume table needs update
 (The following speaks in the context of “layout volume”)
 Unmap LEB 0
 Write updated table to LEB 0
 Unmap LEB 1
 Write updated table to LEB 1
drivers/mtd/ubi/vtbl.c
int ubi_change_vtbl_record(struct ubi_device *ubi, int idx,
struct ubi_vtbl_record *vtbl_rec)
{
[... Deleted ...]
layout_vol = ubi->volumes[vol_id2idx(ubi, UBI_LAYOUT_VOLUME_ID)];
[... Deleted ...]
memcpy(&ubi->vtbl[idx], vtbl_rec, sizeof(struct ubi_vtbl_record));
for (i = 0; i < UBI_LAYOUT_VOLUME_EBS; i++) {
err = ubi_eba_unmap_leb(ubi, layout_vol, i);
if (err)
return err;
err = ubi_eba_write_leb(ubi, layout_vol, i, ubi->vtbl, 0,
ubi->vtbl_size, UBI_LONGTERM);
if (err)
return err;
return 0;
}
} 26

27.  Every piece about MTD and UBI can be found
on the MTD website
 http://www.linux-mtd.infradead.org/