Sketch of the ZXFS

Sketch of the ZXFS
By @ZxMYS
http://www.slideshare.net/ZxMYS

What’s it
• ZXFS - Zhu Xiao’s File System
• A file system that satisfy all Ben’s
requirements
• Similar to the design of the UNIX file
system
• Implementable, Efficient and Simple

Layers in ZXFS
Layer Propose

Symbolic link layer Integrate multiple file systems
with symbolic links.

High level scope (HLS) layer Provide a root for the naming
user-oriented names
hierarchies.

Scope layer Organize files into naming
hierarchies.
Tag layer Provide human-oriented tags for
machine-user interface
files.
Inode Number layer Provide machine-oriented names
for files.

machine-oriented names
File layer Organize blocks into files.

Block layer Identify disk blocks.

Layers in ZXFS
Layer Propose
Due to compatible design in lower layers, we can take this layer directly.

user-oriented names
hierarchies.

hierarchies.
files.
for files.



Layers in ZXFS
Layer Propose

Mainly use these three layers to achieve the goal .
user-oriented names
hierarchies.

hierarchies.
files.
for files.



Layers in ZXFS
Layer Propose


user-oriented names
hierarchies.

hierarchies.
Need some modification in implementation machine-user interface
files.
for files.



Layers in ZXFS
Layer Propose


user-oriented names
hierarchies.

hierarchies.
files.
for files.
Take them directly


Layers in ZXFS - the Tag layer
• Definition:
Tag: Some (name, value) pairs attached to files
• All files are located (in the final step) by tags,
thus the tag layer has similar functionality like
old file name layer.
• A file can have multi-tags, but must has at
least one tag, otherwise it should be deleted.
• Some traditional metadata, such as ctime,
mime and atime, are dealt as file.

Layers in ZXFS - the Scope Layer
• Definition:
Scope: A set of something. In this layer we discuss
‘scope for file’, which is a set of files, and ‘scope for
tag’, a set of files tagged with a certain tag.
• Each tag has it’s scope.
• Use scopes to string (group) files, like directory.
• Scope can be static or dynamic. Can present them
in memory.
• Device store scopes for all tags on that device.

Layers in ZXFS - the High Level Scope
(HLS) Layer
• Definition:
HLS: ‘scope for scope’
• A set of scopes, providing root for scopes
• Like ‘/’ in traditional UNIX FS
• Each device has it’s scope. When system start
up, system merge them into memory and get
an overall HLS

Layers in ZXFS - Path Name Stimulating
with these 3 layers
• To be more compatible with traditional UNIX
FS and enable symbolic links.
• Consider following path:
/(mount point of ZXFS)/tag_expression1/tag_
expression2/...../tag_expressionN
• Tag Expression is some expression used to
locate file using tag, like SQL or XPath
• latter N-1 ‘/’ perform AND logic here

Layers in ZXFS - Path Name Stimulating
with these 3 layers
• Example
/mnt/zxfs1/`ctime` greater than #2011-4-19#/`atime`
equal to #2011-4-20#/
• Looks strange but can work
• Resulting in a scope contains all files have tag ctime
and atime satisfying the expression
• If this scope has multiple files, return a directory (or
something like it)
• Else, return the file or null
• NOTE: THE DESIGN OF TAG EXPRESSION IS NOT A PART
OF THE DESIGN OF ZXFS. SO ONLY MENTION IT HERE.

EXAMPLE LAYER IMPLEMENTATION
LET’S CALL IT ZXFS 0.1

Sketch

Stored in disk
exactly like file data

Basic Knowledge – B+ tree
• a B+ tree or B plus tree is a type of tree which
represents sorted data in a way that allows for
efficient insertion, retrieval and removal of
records
• The primary value of a B+ tree is in storing data
for efficient retrieval in a block-oriented storage
context—in particular, file systems. Because it can
reduce the number of I/O operations required to
find an element in the tree.

Basic Data Structure - inode
struct inode
integer type, refcnt, filesize, userid, groupid, mode //type = FILE_TYPE or SCOPE_TYPE
integer block_numbers_for_file[N] //blocks used to save file
union
struct
integer tagsize //size of the tags
integer block_numbers_for_tags[M-1] //if type==FILE_TYPE. blocks used to
//save tags for this file
integer rest_block_numbers_for_file[M] //if type== SCOPE_TYPE or other type that have no
//tags, the rest space of this inode block can be
//used to store more inode number for files

Change inode structure into this, to implement tags and scopes

Basic Data Structure – tag_data

struct tag_data
integer size //indicate the size of this tag_data
integer inode_number_for_scope //pointer to the scope fot this tag
string tag_value

Every file has a B+ tree, in which
(key, value)=(tag name, tag_data)
And this B+ tree is called tag data of this file

Basic Data Structure – scope_data

struct scope_data
string tag_name //tag name of this scope
B_plus_tree files //B+ tree which store info about file tagged with this tag

The scope_data represent scope.
B+ tree here has
(key, value) = (tag value, file inode number)
(in device)
or
(key, value) = (tag value, (deviceID, file inode number))
(in memory)

Basic Data Structure
HLS is a B+ tree with
(key, value) = (tag name, inode of scope for that tag)
(in device)
Or
(key, value) = (tag name, (deviceID, inode in that device of
scope for that tag))
(in memory)

THE ZXFS API

VERY EASY IF YOU HAVE A NICE LAYER DESIGN

Outline Of API
• Most UNIX API can be implemented on ZXFS
• ALL Ben’s API can be implemented on ZXFS

Part of API description table
API description
search(device ID, tag name list, destination Search for files that have tagged with given tag. Device ID=0 indicates
scope ID) overall HLS of system
search(device ID or scope ID, tag expression Search for files satisfying given tag expressions. ID=0 indicates overall
list, destination scope ID) HLS of system
create(device ID) Return a file ID for a new file on given device.
delete(device ID, file ID) Remove all tags on that file, and delete that file.
read/write (device ID, file ID, offset, buf, Read/Write file
length)
list(scope ID) Returns a list of (device ID, file ID) tuples for files in that scope.
Implement by Iterating B+ tree.
mkscope() Creates an empty scope and return it’s ID. Only for scopes in memory.
merge_scope(source scope ID, destination Merge two scopes. Destination scope can only be in memory
scope ID)
tag_add/tag_remove (device ID, file ID, tag Add/Remove tag on a file.
name)
tag_get(device ID, file ID) List all tags on that file. Implement by reading tagdata of that file.
get_ HLS (device ID) Get HLS. When the device ID is 0, get the system (overall) HLS
device_list() Returns the list of currently plugged-in devices

Some Of API Implementation under
ZXFS 0.1
• SEARCH (TAG NAME LIST)
Just do search on B+ tree of HLS. If user gives a
device ID, use HLS of that tag

• SEARCH (TAG EXPRESSION)
Use a tag expression interpreter to interpret the
expression first, and then do search on HLS, find
corresponding scopes for tags, do another search
on them. Still, the search is performed on B+ tree
(thus very quick)

ZXFS 0.1
• CREATE
Just find an empty inode and tag it with basic
tags like atime/ctime and mtime, update
corresponding scopes

• DELETE
Call tag_remove to remove all tags on that file.
The last remove action will cause the file be
removed.

ZXFS 0.1
• TAG_ADD/TAG_REMOVE

By changing tag data of that file and update
corresponding scope (including HLS if necessary).
If a new tag name is introduced, make a new
scope for it on the device. If all tag of a file is
removed, remove that file.

ANALYSIS

THE TIME TO CHECK UP

Performance?
• The performance can’t be very bad since we
use B+ tree for searching, creating or
removing tag.
• Tag data has pointers back to scope, so
updating tag_value or the removal of a tag is
fast.

A Trade-off
• If we want to change a tag’s name, all file
tagged with that must be modified as well as
some coherence should be considered.
• However, put tag name inside tag data enable
us to find a certain tag quickly.
• We assume that finding a tag for a file
happens more often than changing name of a
tag.

Another Trade-off
• Storing tag name in every tag data require large
space.
– Assume user will NOT use long tag name for many
files.
• B+ tree is very large, comparing to other data
structure, in memory.
• Duplicate pointers to file in scopes for
ctime/mtime/atime, etc.

SOLUTION: ZXFS prefer time to space!

OTHER THINGS

MAYBE WE HAVE FORGOTTEN SOMETHING?

The practicability of tag-based FS
• If the user is hardworking in adding tags for
files, It’s OK.
• What if user is lazy?
– We need advanced AI to help, or do it by itself, to
add tags for files, according to the content of that
file.
– In near future? Maybe.

THE END

DO YOU HAVE QUESTIONS?

Sketch of the ZXFS

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (18)

Destaque

Destaque (9)

Semelhante a Sketch of the ZXFS

Semelhante a Sketch of the ZXFS (20)

Mais de Zx MYS

Mais de Zx MYS (7)

Último

Último (20)

Sketch of the ZXFS