Masterizing php data structure 102

Masterizing PHP Data Structure 102
Patrick Allaert

PHPBenelux Conference Antwerp 2012

About me
● Patrick Allaert
● Founder of Libereco
● Playing with PHP/Linux for +10 years
● eZ Publish core developer
● Author of the APM PHP extension
● @patrick_allaert
● patrickallaert@php.net
● http://github.com/patrickallaert/
● http://patrickallaert.blogspot.com/

Masterizing =
Mastering +
Rising

PHP native datatypes
● NULL (IS_NULL)
● Booleans (IS_BOOL)
● Integers (IS_LONG)
● Floating point numbers
(IS_DOUBLE)
● Strings (IS_STRING)
● Arrays (IS_ARRAY,
IS_CONSTANT_ARRAY)
● Objects (IS_OBJECT)
● Resources (IS_RESOURCE)
● Callable (IS_CALLABLE)

Wikipedia datatypes
●
2-3-4 tree ● Bit field ●
Directed acyclic ● Hashed array tree Map/Associative
● ●
Rapidly-exploring ●
Suffix tree
graph array/Dictionary random tree Symbol table
●
2-3 heap ●
Bitmap Hash list ●
●
Directed graph Matrix Record (also called Syntax tree
2-3 tree BK-tree ● ● ●

Hash table
●
●
●
Disjoint-set tuple or struct) Tagged union (variant
●
Metric tree
AA tree
●

Bloom filter record, discriminated
● ●
●
Distributed hash Hash tree Red-black tree
●
● Minimax tree ●
union, disjoint union)
Abstract syntax tree ● ●
●
● Boolean table Hash trie Min/max kd-tree ●
Rope Tango tree
(a,b)-tree
● ● ●
●
Bounding interval Double Routing table Ternary heap
● ●
Heap M-tree ●

Adaptive k-d tree
●

hierarchy ● ●
● Doubly connected R-tree Ternary search tree
●
Heightmap Multigraph ● ●

●
Adjacency list B sharp tree edge list ●
●
R* tree Threaded binary tree
●
Multimap ● ●

●
Adjacency matrix BSP tree ●
Doubly linked list ● Hilbert R-tree ●
R+ tree Top tree
Multiset
●
● ●

AF-heap Dynamic array Treap
Hypergraph
●

B-tree Scapegoat tree ●

Octree
● ● ● ●
●
Tree
Alternating decision Enfilade ●
Scene graph
Iliffe vector
●
●

tree ●
B*-tree ●
● Pagoda ●
Trees
Enumerated type ●
Segment tree
●

B+ tree
●
Image Pairing heap ● Trie
And-inverter graph ●
Expectiminimax tree
● ● ●
●
Self-balancing T-tree
Implicit kd-tree Parallel array
●

And–or tree ●
B-trie Exponential tree ● ●
●

binary search tree
●

●
UB-tree
Parse tree
●

●
Array ●
Bx-tree Fenwick tree ● Interval tree ●
●
Self-organizing list
●

Union
Plain old data
●
●

AVL tree Cartesian tree Fibonacci heap Int ●
Set Unrolled linked list
● ● ● ●
structure ● ●

●
Beap ●
Char ●
Finger tree ● Judy array Prefix hash tree ●
Skew heap ●
Van Emde Boas tree
●
Skip list Variable-length array
●
Bidirectional map ●
Circular buffer ●
Float Kdb tree Priority queue ●
●

● ●
VList
Bin Compressed suffix FM-index Soft heap ●
● ● ●
Kd-tree ●
Propositional ●
VP-tree
●
Binary decision array Fusion tree directed acyclic Sorted array ●

● ●
Koorde
●
Weight-balanced tree
diagram Gap buffer graph Spaghetti stack
●

●
Container ●
●
●
Winged edge
Binary heap Generalised suffix Leftist heap Quad-edge Sparse array
●

●
● Control table ●
● ●
● X-fast trie
Quadtree
●

Binary search tree tree Lightmap Sparse matrix Xor linked list
Cover tree
●
● ● ● ●

Binary tree
●
Graph Queap Splay tree X-tree
●
Ctrie
●
● Linear octree ●
● ●

Binomial heap
●
Graph-structured Queue SPQR-tree Y-fast trie
●
Dancing tree
●

stack Link/cut tree ●
●
●
●
Radix tree Zero suppressed
Stack
●
Bit array ● ●
●
D-ary heap Hash ● Linked list Randomized binary
●
decision diagram
Bitboard
● ●
● String Zipper
Hash array mapped Lookup table search tree
● ●
Decision tree ● ●
Suffix array
●
●
trie Z-order
Deque ●
Range tree ● ●

●

Game:
Can you recognize some structures?

Array: PHP's untruthfulness
PHP “Arrays” are not true Arrays!

PHP “Arrays” are not true Arrays!
An array is typically implemented like this:

Data Data Data Data Data Data

PHP “Arrays” can be iterated both directions (reset(),
next(), prev(), end()), exclusively with O(1) operations.

PHP “Arrays” can be iterated both directions (reset(),
next(), prev(), end()), exclusively with O(1) operations.
Implementation based on a Doubly Linked List (DLL):
Head Tail

Data Data Data Data Data

Enables List, Deque, Queue and Stack
implementations

PHP “Arrays” elements are always accessible using a
key (index).

PHP “Arrays” elements are always accessible using a
key (index).
Implementation based on a Hash Table:
Head Bucket pointers array Tail
0 1 2 3 4 5 ... nTableSize -1
Bucket * Bucket * Bucket * Bucket * Bucket * Bucket * Bucket *

Bucket Bucket Bucket Bucket Bucket

Data Data Data Data Data

http://php.net/manual/en/language.types.array.php:

“This type is optimized for several
different uses; it can be treated as an
array, list (vector), hash table (an
implementation of a map),
dictionary, collection, stack, queue,
and probably more.”

Optimized for anything ≈ Optimized for nothing!

● In C: 100 000 integers (using long on 64bits => 8
bytes) can be stored in 0.76 Mb.
● In PHP: it will take ≅ 13.97 Mb!
● A PHP variable (containing an integer) takes 48
bytes.
● The overhead of buckets for every “array” entries is
about 96 bytes.
● More details:
http://nikic.github.com/2011/12/12/How-big-are-PHP-arrays-really-Hint-BIG.html

Structs (or records, tuples,...)
● A struct is a value containing other values which
are typically accessed using a name.
● Example:
Person => firstName / lastName
ComplexNumber => realPart / imaginaryPart

Structs – Using array
$person = array(
"firstName" => "Patrick",
"lastName" => "Allaert"
);

Structs – Using a class
$person = new PersonStruct(
"Patrick", "Allaert"
);

(Implementation)
class PersonStruct
{
public $firstName;
public $lastName;

public function __construct($firstName, $lastName)
{
$this->firstName = $firstName;
$this->lastName = $lastName;
}
}

(Implementation)
class PersonStruct
{
public $firstName;
public $lastName;

public function __construct($firstName, $lastName)
{
$this->firstName = $firstName;
$this->lastName = $lastName;
}

public function __set($key, $value)
{
// a. Do nothing
// b. trigger_error()
// c. Throws an exception
}
}

Structs – Pros and Cons
Array Class
+ Uses less memory (PHP < 5.4) - Uses more memory (PHP < 5.4)
- Uses more memory (PHP = 5.4) + Uses less memory (PHP = 5.4)
- No type hinting + Type hinting possible
- Flexible structure + Rigid structure
+|- Less OO +|- More OO
+ Slightly faster - Slightly slower

“true” Arrays
● An array is a fixed size collection where elements
are each identified by a numeric index.

“true” Arrays
● An array is a fixed size collection where elements
are each identified by a numeric index.

0 1 2 3 4 5


“true” Arrays – Using
SplFixedArray
$array = new SplFixedArray(3);
$array[0] = 1; // or $array->offsetSet()
$array[0]; // gives 1

“true” Arrays – Pros and Cons
Array SplFixedArray
- Uses more memory + Uses less memory
+ Slightly faster - Slightly slower

Queues
● A queue is an ordered collection respecting First
In, First Out (FIFO) order.
● Elements are inserted at one end and removed at
the other.

Queues
● A queue is an ordered collection respecting First
In, First Out (FIFO) order.
● Elements are inserted at one end and removed at
the other.

Data

Dequeue
Enqueue

Data

Queues – Using array
$queue = array();
$queue[] = 1; // or array_push()
array_shift($queue); // gives 1

Queues – Using SplQueue
$queue = new SplQueue();
$queue[] = 1; // or $queue->enqueue()
$queue->dequeue(); // gives 1

Queues – Pros and Cons
Array SplQueue
(overhead / entry: 96 bytes) (overhead / entry: 48 bytes)

Stacks
● A stack is an ordered collection respecting Last In,
First Out (LIFO) order.
● Elements are inserted and removed on the same
end.

Stacks
● A stack is an ordered collection respecting Last In,
First Out (LIFO) order.
● Elements are inserted and removed on the same
end.

Data

Push
Pop

Data

Stacks – Using array
$stack = array();
$stack[] = 1; // or array_push()
array_pop($stack); // gives 3

Stacks – Using SplStack
$stack = new SplStack();
$stack[] = 1; // or $stack->push()
$stack->pop(); // gives 3

Stacks – Pros and Cons
Array Class
(overhead / entry: 96 bytes) (overhead / entry: 48 bytes)

Sets
● A set is a collection with no particular ordering
especially suited for testing the membership of a
value against a collection or to perform
union/intersection/complement operations
between them.

Sets
● A set is a collection with no particular ordering
especially suited for testing the membership of a
value against a collection or to perform
union/intersection/complement operations
between them.
Data

Data
Data

Data
Data

Sets – Using array
$set = array();
$set[] = 1;
$set[] = 2;
$set[] = 3;

in_array(2, $set); // true
in_array(5, $set); // false

array_merge($set1, $set2); // union
array_intersect($set1, $set2); // intersection
array_diff($set1, $set2); // complement

Sets – Using array
$set = array();
$set[] = 1;
$set[] = 2;
$set[] = 3;

in_array(2, $set); // true True
in_array(5, $set); // false
performance
killers!
array_merge($set1, $set2); // union
array_intersect($set1, $set2); // intersection
array_diff($set1, $set2); // complement

Sets – Using array (simple types)
$set = array();
$set[1] = true; // Any dummy value
$set[2] = true; // is good but NULL!
$set[3] = true;

isset($set[2]); // true
isset($set[5]); // false

$set1 + $set2; // union
array_intersect_key($set1, $set2); // intersection
array_diff_key($set1, $set2); // complement

Sets – Using array (simple types)
$set = array();
$set[1] = true; // Any dummy value
$set[2] = true; // is good but NULL!
$set[3] = true;

isset($set[2]); // true
isset($set[5]); // false


● Remember that PHP Array keys can be integers or
strings only!

Sets – Using array (objects)
$set = array();
$set[spl_object_hash($object1)] = $object1;

isset($set[spl_object_hash($object2)]); // true
isset($set[spl_object_hash($object5)]); // false


Sets – Using SplObjectStorage
(objects)
$set = new SplObjectStorage();
$set->attach($object1); // or $set[$object1] = null;

isset($set[$object2]); // true
isset($set[$object2]); // false

$set1->addAll($set2); // union
$set1->removeAllExcept($set2); // intersection
$set1->removeAll($set2); // complement

Sets – Using QuickHash (int)
$set = new QuickHashIntSet(64,
QuickHashIntSet::CHECK_FOR_DUPES);
$set->add(1);
$set->add(2);
$set->add(3);

$set->exists(2); // true
$set->exists(5); // false

● No union/intersection/complement operations
(yet?)
● Yummy features like (loadFrom|saveTo)(String|File)

Sets – With finite possible values
define("E_ERROR", 1); // or 1<<0
define("E_WARNING", 2); // or 1<<1
define("E_PARSE", 4); // or 1<<2
define("E_NOTICE", 8); // or 1<<3

$set = 0;
$set |= E_ERROR;
$set |= E_WARNING;
$set |= E_PARSE;

$set & E_ERROR; // true
$set & E_NOTICE; // false

$set1 | $set2; // union
$set1 & $set2; // intersection
$set1 ^ $set2; // complement

(function features)
Instead of:
function remove($path, $files = true, $directories = true, $links = true,
$executable = true)
{
if (!$files && is_file($path))
return false;

if (!$directories && is_dir($path))
return false;

if (!$links && is_link($path))
return false;

if (!$executable && is_executable($path))
return false;

// ...
}

remove("/tmp/removeMe", true, false, true, false); // WTF ?!

(function features)
Instead of:
define("REMOVE_FILES", 1 << 0);
define("REMOVE_DIRS", 1 << 1);
define("REMOVE_LINKS", 1 << 2);
define("REMOVE_EXEC", 1 << 3);
define("REMOVE_ALL", ~0); // Setting all bits

function remove($path, $options = REMOVE_ALL)
{
if (~$options & REMOVE_FILES && is_file($path))
return false;

if (~$options & REMOVE_DIRS && is_dir($path))
return false;

if (~$options & REMOVE_LINKS && is_link($path))
return false;

if (~$options & REMOVE_EXEC && is_executable($path))
return false;

// ...
}

remove("/tmp/removeMe", REMOVE_FILES | REMOVE_LINKS); // Much better :)

Sets: Conclusions
● Use the key and not the value when using PHP
Arrays.
● Use QuickHash for set of integers if possible.
● Use SplObjectStorage as soon as you are playing
with objects.
● Don't use array_unique() when you need a set!

Bloom filters
● A bloom filter is a space-efficient probabilistic data
structure used to test whether an element is
member of a set.
● False positives are possible, but false negatives are
not!

Bloom filters – Using bloomy
// BloomFilter::__construct(int capacity [, double
error_rate [, int random_seed ] ])
$bloomFilter = new BloomFilter(10000, 0.001);

$bloomFilter->add("An element");

$bloomFilter->has("An element"); // true for sure
$bloomFilter->has("Foo"); // false, most probably

Maps
● A map is a collection of key/value pairs where all
keys are unique.

Maps – Using array
$map = array();
$map["ONE"] = 1;
$map["TWO"] = 2;
$map["THREE"] = 3;

// Merging maps:
array_merge($map1, $map2); // SLOW!
$map2 + $map1; // Fast :)
● Don't use array_merge() on maps.

Multikey Maps – Using array
$map = array();
$map["ONE"] = 1;
$map["UN"] =& $map["ONE"];
$map["UNO"] =& $map["ONE"];
$map["TWO"] = 2;
$map["DEUX"] =& $map["TWO"];
$map["DUE"] =& $map["TWO"];

$map["UNO"] = "once";
$map["DEUX"] = "twice";

var_dump($map);
/*
array(6) {
["ONE"] => &string(4) "once"
● Don't use array_merge() on maps.
["UN"] => &string(4) "once"
["UNO"] => &string(4) "once"
["TWO"] => &string(5) "twice"
["DEUX"] => &string(5) "twice"
["DUE"] => &string(5) "twice"
}
*/

Heap
● A heap is a tree-based structure in which all
elements are ordered with largest key at the top,
and the smallest one as leafs.

Heap – Using array
$heap = array();
$heap[] = 3;
sort($heap);
$heap[] = 1;
sort($heap);
$heap[] = 2;
sort($heap);

Heap – Using Spl(Min|Max)Heap
$heap = new SplMinHeap;
$heap->insert(3);
$heap->insert(1);
$heap->insert(2);

Heaps: Conclusions
● MUCH faster than having to re-sort() an array at
every insertion.
● If you don't require a collection to be sorted at
every single step and can insert all data at once
and then sort(). Array is a much better/faster
approach.
● SplPriorityQueue is very similar, consider it is the
same as SplHeap but where the sorting is made on
the key rather than the value.

Other related projects
● SPL Types: Various types implemented as object:
SplInt, SplFloat, SplEnum, SplBool and SplString
http://pecl.php.net/package/SPL_Types

● Judy: Sparse dynamic arrays implementation
http://pecl.php.net/package/Judy

● Judy: Sparse dynamic arrays implementation
http://pecl.php.net/package/Judy
● Weakref: Weak references implementation.
Provides a gateway to an object without
preventing that object from being collected by the
garbage collector.

Conclusions
● Use appropriate data structure. It will keep your
code clean and fast.

Conclusions
● Think about the time and space complexity
involved by your algorithms.

Conclusions
● Think about the time and space complexity
involved by your algorithms.
● Name your variables accordingly: use “Map”, “Set”,
“List”, “Queue”,... to describe them instead of using
something like: $ordersArray.

Thanks
● Don't forget to rate this talk on https://joind.in/4753

Photo Credits
● Northstar Ski Jump:
http://www.flickr.com/photos/renotahoe/5593248965
● Tuned car:
http://www.flickr.com/photos/gioxxswall/5783867752
● London Eye Structure:
http://www.flickr.com/photos/photographygal123/4883546484
● Cigarette:
http://www.flickr.com/photos/superfantastic/166215927
● Heap structure:
http://en.wikipedia.org/wiki/File:Max-Heap.svg

Masterizing php data structure 102

Recomendados

Recomendados

Mais conteúdo relacionado

Destaque

Destaque (19)

Mais de Patrick Allaert

Mais de Patrick Allaert (8)

Último

Último (20)

Masterizing php data structure 102