SlideShare uma empresa Scribd logo
1 de 97
Thinking Distributed: The Hazelcast
Way
RAHUL GUPTA
SR SOLUTIONS ARCHITECT
About Me
Senior Solutions Architect
Worked with Terracotta
In-memory Distributed Systems since 2009
Java Programmer since 1998
rahul@hazelcast.com
Rahul Gupta
follow me @wildnez
© 2017 Hazelcast Inc. Confidential & Proprietary 3
Hazelcast Reborn
1010110010000101001
1010110010001110101001
10111001001
10101100100001111010100
Challenges of Big
Data Management
Why Hazelcast?
• Scale-out Computing enables cluster capacity to be increased
or decreased on-demand
• Resilience with automatic recovery from member failures
without losing data while minimizing performance impact on
running applications
• Programming Model provides a way for developers to easily
program a cluster application as if it is a single process
• Fast Application Performance enables very large data sets to be
held in main memory for real-time performance
© 2017 Hazelcast Inc. Confidential & Proprietary 6
Low-Level Services API
Node Engine
(Threads, Instances, Eventing, Wait/Notify, Invocation)
Partition Management
(Members, Lite Members, Master Partition, Replicas, Migrations, Partition Groups, Partition Aware)
Cluster Management with Cloud Discovery SPI
(AWS, Azure, Consul, etcd, Heroku, IP List, Apache jclouds, Kubernetes, Multicast, Zookeeper)
Networking
(IPv4, IPv6)
On-Heap Store High-Density Memory Store
(Intel, Sparc)
Serialization
(Serializable, Externalizable, DataSerializable, IdentifiedDataSerializable, Portable, Custom)
Near Cache HD
C++ C#/.NET Python Node.js Scala
Memcached Clojure Open Client Network Protocol
(Backward & Forward Compatibility, Binary Protocol)
REST
Java
JVM
(JDK: 6, 7, 8, Vendors: Oracle JDK, Open JDK, IBM JDK, Azul Zing & Zulu)
Hardware/VM/Container
(On Premise, AWS, Azure, Docker, Pivotal Cloud Foundry, OpenShift, VMware)
Executor
Service
SQL Query MapReduce AggregationPredicate Entry Processor
Map MultiMap
Replicated
Map
Web Sessions
(Tomcat/Jetty/Generic)
Continuous
QueryHD
HD
Open Source Enterprise Edition
Hazelcast IMDG
Enterprise HD Edition HD-Enabled Feature
WAN
(Socket,SolaceSystems:One-way,Multiway,InitnewDataCenter,DRDataCenterRecovery)
ManagementCenter
(JMX/REST)
SecuritySuite
(Connection,Encryption,Authentication,Authorization,JAASLoginModule,SocketInterceptor)
Engine
Clients
HD
HD
HD
Queue Topic
Ring
Buffer
Atomic
Long
Hibernate 2nd Level
Cache HD
JCache
Set
Lock/
Sem.
HD
java.util.concurrent
UUID
Hot Restart Store
(SSD, HDD) HD
Storage
Operating System
(Linux, Oracle Solaris, Windows, AIX, Unix)
Operating
Env
APIs
Hyper
LogLog
RollingUpgrades
(RollingClientUpgrades,RollingMemberUpgrades,NoDowntime)
HazelcastStriimHotCache
(SyncUpdatesfromOracleDB,MSSQLServer,MySQLandNonStopDB)
Operations
Hazelcast Solution
Ecosystem Traction
Dozens of Commercial and Open Source Projects Embed Hazelcast
01001
10101
01010
In Memory
Data Computing
In Memory
Data Messaging++In Memory
Data Storage
In Memory Data Grid
Business Systems
A B C
RDBMS Mainframe
MongoDB
NoSQL
REST
Scale
Hazelcast
In-Memory Caching
java.util.concurrent.ConcurrentMap
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
public static void main(String[] args) {
ConcurrentMap<Integer, String> map = new ConcurrentHashMap<>();
map.put(1, "Paris");
map.put(2, "London");
map.put(3, "San Francisco");
String oldValue = map.remove(2);
}
Distributed Map
import java.util.concurrent.ConcurrentMap;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
public static void main(String[] args) {
HazelcastInstance h = Hazelcast.newHazelcastInstance();
ConcurrentMap<Integer, String> map = h.getMap("myMap");
map.put(1, "Paris");
map.put(2, "London");
map.put(3, "San Francisco");
String oldValue = map.remove(2);
}
DEMO
Persistence API
public class MapStorage
implements MapStore<String, User>, MapLoader<String, User> {
// Some methods missing ...
@Override public User load(String key) { return loadValueDB(key); }
@Override public Set<String> loadAllKeys() { return loadKeysDB(); }
@Override public void delete(String key) { deleteDB(key); }
@Override public void store(String key, User value) {
storeToDatabase(key, value);
}
}
<map name="users">
<map-store enabled="true">
<class-name>com.hazelcast.example.MapStorage</class-name>
<write-delay-seconds>0</write-delay-seconds>
</map-store>
</map>
JCache API
// Retrieve the CachingProvider which is automatically baced by
// the chosen Hazelcast server or client provider
CachingProvider cachingProvider = Caching.getCachingProvider();
// Create a CacheManager
CacheManager cacheManager = cachingProvider.getCacheManager();
// Cache<String, String> cache = cacheManager
// .getCache( name, String.class, String.class );
// Create a simple but typesafe configuration for the cache
CompleteConfiguration<String, String> config =
new MutableConfiguration<String, String>()
.setTypes( String.class, String.class );
JCache API
// Create and get the cache
Cache<String, String> cache = cacheManager
.createCache( "example", config );
// Alternatively to request an already existing cache
Cache<String, String> cache = cacheManager
.getCache( name, String.class, String.class );
// Put a value into the cache
cache.put( "world", "Hello World" );
// Retrieve the value again from the cache
String value = cache.get( "world" );
System.out.println( value );
Eviction
• Unless deleted, entries remain in the map.
• Use eviction policies to prevent OOM situations.
• Eviction Triggers run LFU or LRU.
time-to-live
max-idle-seconds
max-size (PER_NODE(number of entries),
PER_PARTITION(number of entries),
USED_HEAP_SIZE(mb),
USED_HEAP_PERCENTAGE(mb))
• Setting eviction-percentage removes that % of entries when eviction is
triggered.
Features Description
MultiMap Store multiple values against one Key
Replicated Map Cluster wide replication, all entries on all nodes. Good for read heavy use
cases
Near Cache Map Entries on Client/Application local memory
RingBuffer Stores data in a ring-like structure, like a circular array with given capacity
More Distributed Structures
Java Collection API: Map, List, Set, Queue
Jcache
High Density Memory Store
Hibernate 2nd Level Cache
Web Session Replication: Tomcat, Jetty
Predicate API: Indexes, SQL Query
Persistence: Map/Queue Store & Loader. Write Behind/Through
Spring Compliance
Transactions: Local & XA
WAN & DR Replication
IM Data Store (Caching) Features
Java: Will it make the cut?
Garbage Collection limits heap usage. G1 and
Balanced aim for <100ms at 10GB.
Unused
Memory
64GB
4GB 4s
Heap
Java Apps Memory Bound
GC Pause
Time
Available
Memory
GC
Off-Heap Storage
No low-level CPU access
Java is challenged as an infrastructure language
despite its newly popular usage for this
Heap
GC Pause
64 Gb
JVM
4 Gb JVM 1
4 Gb JVM 2
4 Gb JVM 3
4 Gb JVM 4
4 Gb JVM 5
4 Gb JVM 16
GC Pause
0 GB
64 GB
64 Gb in-memory data
time
Standard Impediments of Caching
GC Pause
64 Gb
JVM
4 Gb JVM 1
4 Gb JVM 2
4 Gb JVM 3
4 Gb JVM 4
4 Gb JVM 5
4 Gb JVM 16
GC Pause
0 Gb
64 Gb
64 Gb in-memory data
time
64 Gb
HD
2 Gb JVM
GC Pause
HD Memory
Caching with HD Memory
On-Heap Memory Store
(Objects Stored as Objects)
High-Density Memory Store
(Objects Serialized and Stored as
Bytes)
s.m.Unsafe s.m.Unsafe
2-4GB
(Limited by Garbage
Collection)
0-1TB
(Limited by
Machine RAM)
Memory Stores
•Member
•Client
(Near Cache)
RAM in JVM Process
APIs
JCache
(ICache)
Map
(IMap)
HD Memory
On Heap Vs. High-Density
Memory Management
On Heap Memory HD Memory
0 MB HD 3.3 GB
3.9 GB Heap Storage 0.6 GB
9 (4900 ms) Major GC 0 (0 ms)
31 (4200 ms) Minor GC 356 (349 ms)
Node Used
Heap
Total
Heap
Max.
Heap
Heap Usage
Percentage
Used Heap: 0.2 GB
192.168.1.10:5701 57 MB 229 MB 910 MB 6.28%
Memory Utilization
Home Offheap-test
Node Used Heap: 3.9 GB
192.168.1.10:5701 3933 MB 4658MB 4653MB 84.45%
Memory Utilization
Home
Used
Heap
Total
Heap
Max.
Heap
Heap Usage
Percentage
Example: On Heap Memory Example: HD Memory
Hazelcast Servers
Hazelcast Server
JVM [Memory]
A B C
Business Logic
Data Data Data
CE = Compute Engine
Result
Business / Processing Logic
Result
TCP / IP
Client Client
Distributed Computing
IM Distributed Computing Feature
HD
Cache
Dist.
Compute
Dist.
Message
Java Concurrency API
(Lock, Semaphore, AtomicLong, AtomicReference, Executor Service, Blocking Queue)
Entry and Item Listeners
Entry Processor
Aggregators
Map/Reduce
Data Affinity
Continues Query
Map Interceptors
Delta Update
Executor Service API
public interface com.hazelcast.core.IExecutorService
extends java.util.concurrent.ExecutorService
HazelcastInstance hz = getHazelcastInstance();
//java.util.concurrent.ExecutorService implementation
IExecutorService es = hz.getExecutorService("name");
es.executeOnAllMembers(buildRunnable());
es.executeOnKeyOwner(buildRunnable(), "Peter");
es.execute(buildRunnable());
Map<..> futures = es.submitToAllMembers(buildCallable());
Future<..> future = es.submitToKeyOwner(buildCallable(), "Peter");
es.submitToAllMembers(buildCallable(), buildCallback());
es.submitToKeyOwner(buildCallable(), "Peter", buildCallback());
EntryProcessor API
Lock API
HazelcastInstance hz = getHazelcastInstance();
// Distributed Reentrant
L​ock l​ock = hz.getLock("myLock");
l​ock.lock();
try {
// Do something
} finally {
l​ock.unlock();
}
Distributed Lock
Lock API
Pessimistic Locking (IMap)
Lock API
Optimistic Locking
Map/Reduce API
HazelcastInstance hz = getHazelcastInstance();
Map users = hz.getMap("users");
JobTracker tracker = hz.getJobTracker("default");
KeyValueSource source = KeyValueSource.fromMap(users);
Job job = tracker.newJob(source);
ICompleteFuture future = job.mapper(new MyMapper())
.reducer(new MyReducer())
.submit();
Map result = future.get();
Aggregations API
HazelcastInstance hz = getHazelcastInstance();
Map users = hz.getMap("users");
int sum = users.aggregate(
Supplier.all((user) -> user.getSalary()),
Aggregations.longSum()
);
Hazelcast Distributed
Topic Bus
Hazelcast
Topic
Hazelcast
Node 1
Hazelcast
Node 2
Hazelcast
Node 3
MSG
Subscribes
Delivers
Subscribes
Delivers
Distributed Messaging
Queue
Topic (Pub/Sub)
Event Listeners
Ring Buffers
IM Distributed Messaging Features
HD
Cache
Dist.
Compute
Dist.
Message
Queue API
interface com.hazelcast.core.IQueue<E>
extends java.util.concurrent.BlockingQueue
HazelcastInstance hz = getHazelcastInstance();
//java.util.concurrent.BlockingQueue implementation
IQueue<Task> queue = hz.getQueue("tasks");
queue.offer(newTask());
queue.offer(newTask(), 500, TimeUnit.MILLISECONDS);
Task task = queue.poll();
Task task = queue.poll(100, TimeUnit.MILLISECONDS);
Task task = queue.take();
Topic API
public class Example implements MessageListener<String> {
public void sendMessage {
HazelcastInstance hz = getHazelcastInstance();
ITopic<String> topic = hz.getTopic("topic");
topic.addMessageListener(this);
topic.publish("Hello World");
}
@Override
public void onMessage(Message<String> message) {
System.out.println("Got message: " + message.getMessageObject());
}
}
Data Distribution and Resilience
38
Distributed Maps
Fixed number of partitions (default 271)
Each key falls into a partition
partitionId = hash(keyData)%PARTITION_COUNT
Partition ownerships are reassigned upon membership change
A B C
New Node Added
DA B C
Migration
DA B C
Migration
DA B C
Migration
DA B C
Migration
DA B C
Migration
DA B C
Migration
DA B C
Migration Complete
DA B C
Data Safety on Node Failure
48
Node Crashes
DA B C
Backups Are Restored
DA B C
Backups Are Restored
DA B C
Backups Are Restored
DA B C
Backups Are Restored
DA B C
Backups Are Restored
DA B C
Backups Are Restored
DA B C
Backups Are Restored
DA B C
Backups Are Restored
DA B C
Recovery Is Complete
DA C
© 2017 Hazelcast Inc. Confidential & Proprietary 59
Topologies
Deployment
Architectures
© 2017 Hazelcast Inc. Confidential & Proprietary 60
Embedded Mode
Great for microservices,
OEM and ops simplification
IMDG
Application
Java API
IMDG
Application
Java API
IMDG
Application
Java API
© 2017 Hazelcast Inc. Confidential & Proprietary 61
Embedded Mode
Embedded model, for example in a J2EE container
Service and storage in the one JVM
Easy for getting started
With three nodes, 1/3rd of the data is local, good read performance
-> This benefit diminishes as you scale up
-> With ten nodes, only 1/10th of the data is local
If the application needs to go offline, so does the data
© 2017 Hazelcast Inc. Confidential & Proprietary 62
Client-Server Mode
Great for scale-up or scale-out deployments with cluster
lifecycle decoupled from app servers
Use for non-Java clients
Node.js API
Application
Node.js API
Application
Node.js API
Application
Node.js API
ApplicationJava API
Application
.Net API
Application
C++ API
Application
C++ API
Application
C++ API
Application
C++ API
Application
.Net API
Application
.Net API
Application
.Net API
ApplicationJava API
Application
Java API
Application
IMDG
IMDG IMDG
© 2017 Hazelcast Inc. Confidential & Proprietary 63
Client-Server Mode
Follows the traditional Client-Server approach
Clients do not change cluster topology
Allows you to segregate service from storage
Smart clients connect to all cluster members. Operations are routed directly to a
member holding the necessary data.
-> One network hop
Other clients connect to a single cluster member and use it as a proxy to issue
requests.
-> Two network hops if the proxy doesn’t have the data needed
© 2017 Hazelcast Inc. Confidential & Proprietary 64
Pushing Into New Language Ecosystems
Hazelcast.org/Plugins
• Fully featured
• Hazelcast supported
• Open Source
• Common Hazelcast Feature Set
• Hazelcast supported
• Open Source
• High Performance
• Hazelcast supported
• Open Source
• Client and Member APIs
• Scala 2.11 support
• Hazelcast Supported
• Open Source
• Community supported
• Open Source
• Fully Featured
• Hazelcast supported
• Open Source
• Industry Leading Performance
• High-Density Memory Store Near Cache
• Hazelcast Supported
• Open Source
Feature Matrix: https://hazelcast.org/clients-languages
© 2017 Hazelcast Inc. Confidential & Proprietary 65
Hazelcast IMDG Microservices
Deployment Options
Microservices deployed embedded
with isolated Hazelcast IMDG clusters
Maximum service isolation
Microservices deployed client server
with a shared Hazelcast IMDG cluster
Transition to microservices
MS 1
IMDG IMDG IMDG
MS 1 MS 1
.NET
Client
MS 1
IMDG IMDG IMDG IMDG
.NET
Client
MS 1
Node.js
Client
MS 2
Node.js
Client
MS 2
C++
Client
MS 3
Java
Client
MS 4
Java
Client
MS 4
MS 2
IMDG IMDG IMDG
MS 2 MS 2
MS 3
IMDG IMDG IMDG
MS 3 MS 3
© 2017 Hazelcast Inc. Confidential & Proprietary 66
Hazelcast IMDG Microservices
Deployment Options
Microservices deployed client server with isolated Hazelcast cluster per service
Maximum service isolation and non-Java clients
.NET
Client
MS 1
IMDG IMDG IMDG
.NET
Client
MS 1
.NET
Client
MS 1
.NET
Client
MS 1
.NET
Client
MS 1
Python
Client
MS 2
IMDG IMDG IMDG
Python
Client
MS 2
Python
Client
MS 2
Python
Client
MS 2
Python
Client
MS 2
Same Features – Embedded vs Client-Server
// Creating a new Hazelcast node
HazelcastInstance hz = Hazelcast.newHazelcastInstance();
// Getting a Map, Queue, Topic, ...
Map map = hz.getMap("my-map");
Queue queue = hz.getQueue("my-queue");
ITopic topic = hz.getTopic("my-topic");
//Creating a Hazelcast Client
HazelcastInstance client = HazelcastClient.newHazelcastClient();
// Shutting down the node
hz.shutdown();
© 2017 Hazelcast Inc. Confidential & Proprietary 68
Hazelcast IMDG – Any Cloud, Anytime
Apache jClouds
PaaS Environment Integrations
Cloud & Cloud Discovery Plugins
Deployment Options
YARN
© 2017 Hazelcast Inc. Confidential & Proprietary 69
Hazelcast IMDG WAN
Replication Deployment Options
Independent clusters, sharing selected
data to maintain alignment
One-way or two-way
Choose which data to share, select by type,
can de-select records by attributes
One-way  keep DR site populated
and ready-to-go
Two-way  worldwide operations,
access your nearest data copy
API
Applications
API
Applications
API
Applications
IMDG
IMDG IMDG
IMDG IMDG
IMDG IMDG
70
Hazelcast IMDG Roadmap
© 2017 Hazelcast Inc. Confidential & Proprietary 71
Hazelcast IMDG High Level Roadmap
Hi-Density Caching
In-Memory Data Grid
2015 2016
JCache | Scalability | HD Memory | Hot Restart | Pre-Load Clients
Cloud Discovery SPI | Azure Cloud Foundry | AWS
j.u.c. | Low-Latency | Elasticity | Resiliency | Performance | Rolling Upgrades
Open Cloud Platform
2017
j.u.s | Jet | Cascading | Streaming | Mesos | Reactive
Expanded In-memory
Computing
2018
© 2017 Hazelcast Inc. Confidential & Proprietary 72
Recent Releases & Roadmap
Release Date
Hazelcast IMDG 3.7 August 2016 (Released)
Hazelcast IMDG 3.8 February 2017 (Released)
Hazelcast IMDG 3.9 August 2017
Hazelcast IMDG 4.0 September 2017
© 2017 Hazelcast Inc. Confidential & Proprietary 73
New Hazelcast IMDG 3.8 Features
Features Description
Fast Aggregations
Based on Queries
Make aggregations significantly faster by moving in-built aggregations to be on top of the
query infrastructure of Hazelcast IMDG. Another goal is to expose a simplified
Aggregations API.
User Code Deployment Ability to access and use user classes without having them on the classpath.
Projection for Queries
Provide projection in queries to allow returning specific fields of an entry, not the whole
entry. This will cut the network and serialisation overhead and increase the throughput.
RingBuffer Store
Implementation of the storage mechanism for the RingBuffer, analogous to the existing
QueueStore mechanism. It will also allow users of the ReliableTopic to replay messages
older than the messages in the underlying RingBuffer.
Continuous Query
The previously Enterprise feature Continuous Query Cache becomes open source with
3.8.
© 2017 Hazelcast Inc. Confidential & Proprietary 74
New Hazelcast IMDG 3.8 Features
Features Description
Unified Near Cache
Implementation
for IMap and JCache
A technical debt to have just 2 Near-Cache implementations: OnHeapNearCache and
HDNearCache. These 2 implementations should be used by both IMap and JCache as
well as member and client.
High Performance Near Cache Several Improvements on Near Cache including pre-loading near cache.
Eventually Consistent
Near Cache
A guarantee of delivery of cache invalidations to the near cache.
HyperLogLog
Implementation of the HyperLogLog data structure on top of Hazelcast IMDG. It is a
probabilistic structure used to estimate cardinality.
Scheduled Executor Service Distributed version of java.util.concurrent.ScheduledExecutorService.
Cluster Quorum
for Queue and Lock
Quorum support for Hazelcast IMDG Queue and Lock.
© 2017 Hazelcast Inc. Confidential & Proprietary 75
New Hazelcast IMDG 3.8 Enterprise Features
Features Description
Rolling Member Upgrades
for Minor Releases
Extending current rolling upgrade capability of Hazelcast IMDG (client rolling
upgradeability, patch version rolling member upgrades) to allow heterogeneous versions of
Hazelcast IMDG members to be rolling upgraded without service interruption for minor
version changes.
Copying HRS Data
from a Source to a Target Cluster
Ability to copy the Hot Restart store files from a source environment to a target
environment (for example, product to test environment). This feature also applies to
running sources.
One-Off WAN Sync with
Dynamically Added Endpoint
Ability to copy all Maps’ content to a target cluster dynamically in runtime.
SSL Performance Improvements
(Java)
Large performance increase in SSL when using the Java client.
LDAP Integration for
Management Center
Authentication and authorizarion with LDAP in Management Center.
Many Management Center
Enhancements
More metrics, stats, and improved usability and monitoring.
© 2017 Hazelcast Inc. Confidential & Proprietary 76
New Hazelcast IMDG 3.8 Clients Features
Features Description
Mutual Auth with TLS
Certificates
Allow using client certificates to identify the client to the server.
Simulator Integration
for Hazelcast .NET Client
Integration of the .NET client into Hazelcast Simulator.
Node.js Client Enhancements RingBuffer, Topic, and Near Cache Support for Node.js client.
SSL Support for Node.js Client
SSL support for Hazelcast Node.js Client to secure the connection between the client and
the members.
Near Cache Support
for C++ Client
Configurable near cache support for IMap in the Hazelcast C++ client.
AWS Discovery for C++ Client AWS cluster discovery implementation for Hazelcast C++ Client.
© 2017 Hazelcast Inc. Confidential & Proprietary 77
New Hazelcast IMDG 3.8 Cloud & Plugins Features
Features Description
Pivotal® CloudFoundry
Integration
Hazelcast IMDG on Pivotal CloudFoundry as a Tile.
IBM DynaCache API Support
A WebSphere DynamicCache provider, which could be used either by traditional
WebSphere or WebSphere Liberty Profile applications.
Discovery SPI based AWS
Module
Discovery SPI compatible hazelcast-aws module. AWS users would take advantage of
ZONE_AWARE (availability zone aware partition groups) without using Apache jclouds.
Striim Hot Cache Adapter
HotCache functionality with Striim based on Change Data Capture ("CDC") of underlying
database updates.
Hibernate 5.2 Support Improvements in distributed L2 cache for Hibernate to support Hibernate 5.2.
© 2017 Hazelcast Inc. Confidential & Proprietary 78
Hazelcast 3.9 Roadmap
August 2017
© 2017 Hazelcast Inc. Confidential & Proprietary 79
3.9 Features
Features Description
Near Cache: Key Store-by-
Reference Option
Key store-by-reference option for IMap and Hazelcast JCache Near Cache.
Dynamic Creation of Specificly
Configured Distributed Objects
Ability to define new data structures with custom configuration on the fly.
Gigantic Cache Migration
Enhancements
Improvements in partition migration process to allow better gigantic cache migrations.
Lazy Initiation for Java Client An async client factory to connect lazily and in a non-blocking manner.
Client Support for User Code
Deployment
Dynamic Classloading from clients for User Code Deployment.
Query Async API Async query methods on the IMap
© 2017 Hazelcast Inc. Confidential & Proprietary 80
3.9 – Security Features
Features Description
Mutual TLS Auth Java Client Allow a member to authenticate a client using a certificate installed in the client
Mutual TLS Auth C# Client Allow a member to authenticate a client using a certificate installed in the client
Management Center TLS TLS from members to Man Center. TLS from browser. TLS for REST endpoint
TLS for REST endpoint on
Members
General Security Hardening
Implement things like password blacklisting, password complexity requirements, toggling off
of non-secure mechanisms when secure version is enabled and so on
Fast SSL/TLS Make SSL/TLS almost as fast as unencrypted.
© 2017 Hazelcast Inc. Confidential & Proprietary 81
Fast SSL/TLS
© 2017 Hazelcast Inc. Confidential & Proprietary 82
New Hazelcast IMDG 3.9 Enterprise Features
Features Description
Dynamic Addition of WAN
Endpoints
Ability to add WAN Targets to a running cluster.
NearCache Stats in Management
Center
Provision of showing data inside Near-Cache of client or member, similar to IMap/Cache in
Management Center.
Search Indexes in HD Memory Support for sorted and unsorted indexes in Hazelcast HD Memory Store.
© 2017 Hazelcast Inc. Confidential & Proprietary 83
New Hazelcast IMDG 3.9 Clients Features
Features Description
Simulator Integrations of
C++, Python, and Node.js Clients
Integration of C++, Python, and Node.js clients into Hazelcast Simulator.
SSL Performance Improvements
for .NET Client
Large performance increase in SSL when using the .NET Client.
Projections for .NET Client Add projections (returning attributes rather than full values) to .NET.
Paging Predicate for .NET Client Support for paging predicates in Hazelcast .NET Client.
ExecutorService for C++ Client Ability to use the distributed ExecutorService from Hazelcast C++ Client.
Near Cache for C++ Client Add near cache to C++ client
TLS for C++ Client Add TLS encryption for C++ Clients communicating with members
© 2017 Hazelcast Inc. Confidential & Proprietary 84
New Hazelcast IMDG 3.9 Cloud & Plugins Features
Features Description
Google Cloud Platform Integration SPI Plugin for GCP for Cloud Discovery with a support for partition groups.
AWS support for C++ Client We have comprehensive cloud support for Java. Adding AWS to C++.
Stats for Hibernate
LocalRegionCache
Ability to provide statistics for Hibernate LocalRegionCache.
Web Sessions for WebSphere
Liberty
Web sessions replication plugin for WebSphere Liberty.
© 2017 Hazelcast Inc. Confidential & Proprietary 85
New Hazelcast IMDG 4.0 Features
Features Description
Java 8 Language Level
Lift the language level for Hazelcast members and their API to Java 8. Add a new Java 8
client. Continue to maintain the Java 6 compatible client. This will allow us to take advantage
of Java 8 language features.
Java 9 Modularisation
Java 9 modularisation for Hazelcast IMDG, which introduces Jigsaw - a new modular
systems with a strict enforcement of module boundaries.
Proper Separation of Public and
Private APIs
Re-design of the whole private/public API to keep as much private as possible.
Public and Stable SPI
Make SPI a second public and stable API. This work includes many refactorings, client
protocol enhancements, documentation efforts, and reference implementations.
EntryProcessor Contract
Formalisation
Optimization of the EntryProcessor which would bring the IMap behaviour inline with the
JCache behaviour.
Unification of Client / Member
Behaviour in Exceptional States
Equalization of exceptions in client-side and member-side so that clients can throw the same
exceptions as server whenever possible. This work also includes a documentation effort on
exceptional cases for each method.
What’s Hazelcast Jet?
• General purpose distributed data processing
framework
• Based on Direct Acyclic Graph to model data
flow
• Built on top of Hazelcast
• Comparable to Apache Spark or Apache Flink
8
7
DAG
8
8
Job Execution
8
9
9
0
Hazelcast Services
Service Offerings
Hazelcast (Apache Licensed)
• Professional Subscription – 24x7 support*
Hazelcast Enterprise Support
• Available with Hazelcast Enterprise software subscription - 24x7 support*
Additional Services
• Development Support Subscription – 8x5 support*
• Simulator TCK
• Training
• Expert Consulting
• Development Partner Program
* All subscriptions include Management Center
© 2016 Hazelcast Inc. Confidential & Proprietary 92
Support Subscriptions
What’s Included
100% SUCCESS RATE ON CUSTOMER ISSUES:
“As usual, the response was timely beyond
expectations, and very good technical content
returned. Exemplary support, hard to find in
any company…”
- Fortune 100 Financial Services Customer
ENTERPRISE HD ENTERPRISE PROFESSIONAL OPEN SOURCE
SUPPORT WINDOW 24/7 24/7 24/7
RESPONSE TIME FOR CRITIAL ISSUES 1 Hour 1 Hour 2 Hours
SUPPORTED SOFTWARE
Hazelcast &
Hazelcast Enterprise
Hazelcast &
Hazelcast Enterprise
Hazelcast
SUPPORT CONTACTS 4 4 2
SUPPORT CHANNELS Email, IM & Phone Email, IM & Phone Email, IM & Phone
PATCH LEVEL FIXES
REMOTE MEETINGS (via GoToMeeting)
CODE REVIEW (with a Senior Solutions Architect) 2 Hours 2 Hours 2 Hours
QUARTERLY REVIEW OF FEATURE REQUES*
QUARTERLY REVIEW OF HAZELCAST ROADMAP*
© 2016 Hazelcast Inc. Confidential & Proprietary
9
3
Best In Class Support
Support from the Engineers who wrote
the code
SLA Driven – 100% attainment of
support response time
Follow the Sun
Portal, Email and Phone access
Go Red, Go Green. Reproduction of
issues on Simulator. Proof of fix on
Simulator.
Periodic Technical Reviews
Meet your production schedule and
corporate compliance requirements
Ensure the success of your
development team with training and
best practices
9
4
Hazelcast Support Coverage
India
Turkey
London
U.S.
© 2016 Hazelcast Inc. Confidential & Proprietary 95
Support Testimonials
© 2016 Hazelcast Inc. Confidential & Proprietary 96
Release Lifecycle
• Regular Feature release each 4-5 months, e.g. 3.3, 3.4, 3.5
• Maintenance release approximately each month with bug fixes based
on the current feature release, e.g. 3.4.1
• For older versions, patch releases made available to fix issues
• Release End of Life per support contract
Thank you

Mais conteúdo relacionado

Mais procurados

Geek Nights Hong Kong
Geek Nights Hong KongGeek Nights Hong Kong
Geek Nights Hong KongRahul Gupta
 
Machine Learning for Capacity Management
 Machine Learning for Capacity Management Machine Learning for Capacity Management
Machine Learning for Capacity ManagementEDB
 
Pig on Tez: Low Latency Data Processing with Big Data
Pig on Tez: Low Latency Data Processing with Big DataPig on Tez: Low Latency Data Processing with Big Data
Pig on Tez: Low Latency Data Processing with Big DataDataWorks Summit
 
Tuning up with Apache Tez
Tuning up with Apache TezTuning up with Apache Tez
Tuning up with Apache TezGal Vinograd
 
What's the Hadoop-la about Kubernetes?
What's the Hadoop-la about Kubernetes?What's the Hadoop-la about Kubernetes?
What's the Hadoop-la about Kubernetes?DataWorks Summit
 
Red Hat Ceph Storage: Past, Present and Future
Red Hat Ceph Storage: Past, Present and FutureRed Hat Ceph Storage: Past, Present and Future
Red Hat Ceph Storage: Past, Present and FutureRed_Hat_Storage
 
Tez big datacamp-la-bikas_saha
Tez big datacamp-la-bikas_sahaTez big datacamp-la-bikas_saha
Tez big datacamp-la-bikas_sahaData Con LA
 
Apache Tez - Accelerating Hadoop Data Processing
Apache Tez - Accelerating Hadoop Data ProcessingApache Tez - Accelerating Hadoop Data Processing
Apache Tez - Accelerating Hadoop Data Processinghitesh1892
 
20150314 sahara intro and the future plan for open stack meetup
20150314 sahara intro and the future plan for open stack meetup20150314 sahara intro and the future plan for open stack meetup
20150314 sahara intro and the future plan for open stack meetupWei Ting Chen
 
YARN Containerized Services: Fading The Lines Between On-Prem And Cloud
YARN Containerized Services: Fading The Lines Between On-Prem And CloudYARN Containerized Services: Fading The Lines Between On-Prem And Cloud
YARN Containerized Services: Fading The Lines Between On-Prem And CloudDataWorks Summit
 
Real-world Cloud HPC at Scale, for Production Workloads (BDT212) | AWS re:Inv...
Real-world Cloud HPC at Scale, for Production Workloads (BDT212) | AWS re:Inv...Real-world Cloud HPC at Scale, for Production Workloads (BDT212) | AWS re:Inv...
Real-world Cloud HPC at Scale, for Production Workloads (BDT212) | AWS re:Inv...Amazon Web Services
 
Hazelcast 3.6 Roadmap Preview
Hazelcast 3.6 Roadmap PreviewHazelcast 3.6 Roadmap Preview
Hazelcast 3.6 Roadmap PreviewHazelcast
 
OLTP+OLAP=HTAP
 OLTP+OLAP=HTAP OLTP+OLAP=HTAP
OLTP+OLAP=HTAPEDB
 
Hazelcast Deep Dive (Paris JUG-2)
Hazelcast Deep Dive (Paris JUG-2)Hazelcast Deep Dive (Paris JUG-2)
Hazelcast Deep Dive (Paris JUG-2)Emrah Kocaman
 
20151027 sahara + manila final
20151027 sahara + manila final20151027 sahara + manila final
20151027 sahara + manila finalWei Ting Chen
 
The Future of Computing is Distributed
The Future of Computing is DistributedThe Future of Computing is Distributed
The Future of Computing is DistributedAlluxio, Inc.
 
Apache Tez : Accelerating Hadoop Query Processing
Apache Tez : Accelerating Hadoop Query ProcessingApache Tez : Accelerating Hadoop Query Processing
Apache Tez : Accelerating Hadoop Query ProcessingBikas Saha
 
Hive+Tez: A performance deep dive
Hive+Tez: A performance deep diveHive+Tez: A performance deep dive
Hive+Tez: A performance deep divet3rmin4t0r
 
Sahara presentation latest - Codemotion Rome 2015
Sahara presentation latest - Codemotion Rome 2015Sahara presentation latest - Codemotion Rome 2015
Sahara presentation latest - Codemotion Rome 2015Codemotion
 
Hazelcast Essentials
Hazelcast EssentialsHazelcast Essentials
Hazelcast EssentialsRahul Gupta
 

Mais procurados (20)

Geek Nights Hong Kong
Geek Nights Hong KongGeek Nights Hong Kong
Geek Nights Hong Kong
 
Machine Learning for Capacity Management
 Machine Learning for Capacity Management Machine Learning for Capacity Management
Machine Learning for Capacity Management
 
Pig on Tez: Low Latency Data Processing with Big Data
Pig on Tez: Low Latency Data Processing with Big DataPig on Tez: Low Latency Data Processing with Big Data
Pig on Tez: Low Latency Data Processing with Big Data
 
Tuning up with Apache Tez
Tuning up with Apache TezTuning up with Apache Tez
Tuning up with Apache Tez
 
What's the Hadoop-la about Kubernetes?
What's the Hadoop-la about Kubernetes?What's the Hadoop-la about Kubernetes?
What's the Hadoop-la about Kubernetes?
 
Red Hat Ceph Storage: Past, Present and Future
Red Hat Ceph Storage: Past, Present and FutureRed Hat Ceph Storage: Past, Present and Future
Red Hat Ceph Storage: Past, Present and Future
 
Tez big datacamp-la-bikas_saha
Tez big datacamp-la-bikas_sahaTez big datacamp-la-bikas_saha
Tez big datacamp-la-bikas_saha
 
Apache Tez - Accelerating Hadoop Data Processing
Apache Tez - Accelerating Hadoop Data ProcessingApache Tez - Accelerating Hadoop Data Processing
Apache Tez - Accelerating Hadoop Data Processing
 
20150314 sahara intro and the future plan for open stack meetup
20150314 sahara intro and the future plan for open stack meetup20150314 sahara intro and the future plan for open stack meetup
20150314 sahara intro and the future plan for open stack meetup
 
YARN Containerized Services: Fading The Lines Between On-Prem And Cloud
YARN Containerized Services: Fading The Lines Between On-Prem And CloudYARN Containerized Services: Fading The Lines Between On-Prem And Cloud
YARN Containerized Services: Fading The Lines Between On-Prem And Cloud
 
Real-world Cloud HPC at Scale, for Production Workloads (BDT212) | AWS re:Inv...
Real-world Cloud HPC at Scale, for Production Workloads (BDT212) | AWS re:Inv...Real-world Cloud HPC at Scale, for Production Workloads (BDT212) | AWS re:Inv...
Real-world Cloud HPC at Scale, for Production Workloads (BDT212) | AWS re:Inv...
 
Hazelcast 3.6 Roadmap Preview
Hazelcast 3.6 Roadmap PreviewHazelcast 3.6 Roadmap Preview
Hazelcast 3.6 Roadmap Preview
 
OLTP+OLAP=HTAP
 OLTP+OLAP=HTAP OLTP+OLAP=HTAP
OLTP+OLAP=HTAP
 
Hazelcast Deep Dive (Paris JUG-2)
Hazelcast Deep Dive (Paris JUG-2)Hazelcast Deep Dive (Paris JUG-2)
Hazelcast Deep Dive (Paris JUG-2)
 
20151027 sahara + manila final
20151027 sahara + manila final20151027 sahara + manila final
20151027 sahara + manila final
 
The Future of Computing is Distributed
The Future of Computing is DistributedThe Future of Computing is Distributed
The Future of Computing is Distributed
 
Apache Tez : Accelerating Hadoop Query Processing
Apache Tez : Accelerating Hadoop Query ProcessingApache Tez : Accelerating Hadoop Query Processing
Apache Tez : Accelerating Hadoop Query Processing
 
Hive+Tez: A performance deep dive
Hive+Tez: A performance deep diveHive+Tez: A performance deep dive
Hive+Tez: A performance deep dive
 
Sahara presentation latest - Codemotion Rome 2015
Sahara presentation latest - Codemotion Rome 2015Sahara presentation latest - Codemotion Rome 2015
Sahara presentation latest - Codemotion Rome 2015
 
Hazelcast Essentials
Hazelcast EssentialsHazelcast Essentials
Hazelcast Essentials
 

Semelhante a Distributed caching-computing v3.8

Think Distributed: The Hazelcast Way
Think Distributed: The Hazelcast WayThink Distributed: The Hazelcast Way
Think Distributed: The Hazelcast WayRahul Gupta
 
Distributed caching and computing v3.7
Distributed caching and computing v3.7Distributed caching and computing v3.7
Distributed caching and computing v3.7Rahul Gupta
 
Strata Singapore: Gearpump Real time DAG-Processing with Akka at Scale
Strata Singapore: GearpumpReal time DAG-Processing with Akka at ScaleStrata Singapore: GearpumpReal time DAG-Processing with Akka at Scale
Strata Singapore: Gearpump Real time DAG-Processing with Akka at ScaleSean Zhong
 
Web session replication with Hazelcast
Web session replication with HazelcastWeb session replication with Hazelcast
Web session replication with HazelcastEmrah Kocaman
 
Intro to SnappyData Webinar
Intro to SnappyData WebinarIntro to SnappyData Webinar
Intro to SnappyData WebinarSnappyData
 
Embrace Sparsity At Web Scale: Apache Spark MLlib Algorithms Optimization For...
Embrace Sparsity At Web Scale: Apache Spark MLlib Algorithms Optimization For...Embrace Sparsity At Web Scale: Apache Spark MLlib Algorithms Optimization For...
Embrace Sparsity At Web Scale: Apache Spark MLlib Algorithms Optimization For...Jen Aman
 
Explore big data at speed of thought with Spark 2.0 and Snappydata
Explore big data at speed of thought with Spark 2.0 and SnappydataExplore big data at speed of thought with Spark 2.0 and Snappydata
Explore big data at speed of thought with Spark 2.0 and SnappydataData Con LA
 
SnappyData at Spark Summit 2017
SnappyData at Spark Summit 2017SnappyData at Spark Summit 2017
SnappyData at Spark Summit 2017Jags Ramnarayan
 
SnappyData, the Spark Database. A unified cluster for streaming, transactions...
SnappyData, the Spark Database. A unified cluster for streaming, transactions...SnappyData, the Spark Database. A unified cluster for streaming, transactions...
SnappyData, the Spark Database. A unified cluster for streaming, transactions...SnappyData
 
End to End Processing of 3.7 Million Telemetry Events per Second using Lambda...
End to End Processing of 3.7 Million Telemetry Events per Second using Lambda...End to End Processing of 3.7 Million Telemetry Events per Second using Lambda...
End to End Processing of 3.7 Million Telemetry Events per Second using Lambda...DataWorks Summit/Hadoop Summit
 
Big data on Azure for Architects
Big data on Azure for ArchitectsBig data on Azure for Architects
Big data on Azure for ArchitectsTomasz Kopacz
 
Tajo_Meetup_20141120
Tajo_Meetup_20141120Tajo_Meetup_20141120
Tajo_Meetup_20141120Hyoungjun Kim
 
Deep Dive on Amazon EC2 instances
Deep Dive on Amazon EC2 instancesDeep Dive on Amazon EC2 instances
Deep Dive on Amazon EC2 instancesAmazon Web Services
 
Big Data LDN 2016: Kick Start your Big Data project with Hyperconverged Infra...
Big Data LDN 2016: Kick Start your Big Data project with Hyperconverged Infra...Big Data LDN 2016: Kick Start your Big Data project with Hyperconverged Infra...
Big Data LDN 2016: Kick Start your Big Data project with Hyperconverged Infra...Matt Stubbs
 
Ibm power ha v7 technical deep dive workshop
Ibm power ha v7 technical deep dive workshopIbm power ha v7 technical deep dive workshop
Ibm power ha v7 technical deep dive workshopsolarisyougood
 
An Engineer's Intro to Oracle Coherence
An Engineer's Intro to Oracle CoherenceAn Engineer's Intro to Oracle Coherence
An Engineer's Intro to Oracle CoherenceOracle
 

Semelhante a Distributed caching-computing v3.8 (20)

Think Distributed: The Hazelcast Way
Think Distributed: The Hazelcast WayThink Distributed: The Hazelcast Way
Think Distributed: The Hazelcast Way
 
Distributed caching and computing v3.7
Distributed caching and computing v3.7Distributed caching and computing v3.7
Distributed caching and computing v3.7
 
Strata Singapore: Gearpump Real time DAG-Processing with Akka at Scale
Strata Singapore: GearpumpReal time DAG-Processing with Akka at ScaleStrata Singapore: GearpumpReal time DAG-Processing with Akka at Scale
Strata Singapore: Gearpump Real time DAG-Processing with Akka at Scale
 
Clustering van IT-componenten
Clustering van IT-componentenClustering van IT-componenten
Clustering van IT-componenten
 
Web session replication with Hazelcast
Web session replication with HazelcastWeb session replication with Hazelcast
Web session replication with Hazelcast
 
Intro to SnappyData Webinar
Intro to SnappyData WebinarIntro to SnappyData Webinar
Intro to SnappyData Webinar
 
Embrace Sparsity At Web Scale: Apache Spark MLlib Algorithms Optimization For...
Embrace Sparsity At Web Scale: Apache Spark MLlib Algorithms Optimization For...Embrace Sparsity At Web Scale: Apache Spark MLlib Algorithms Optimization For...
Embrace Sparsity At Web Scale: Apache Spark MLlib Algorithms Optimization For...
 
11g R2
11g R211g R2
11g R2
 
Explore big data at speed of thought with Spark 2.0 and Snappydata
Explore big data at speed of thought with Spark 2.0 and SnappydataExplore big data at speed of thought with Spark 2.0 and Snappydata
Explore big data at speed of thought with Spark 2.0 and Snappydata
 
EVCache & Moneta (GoSF)
EVCache & Moneta (GoSF)EVCache & Moneta (GoSF)
EVCache & Moneta (GoSF)
 
SnappyData at Spark Summit 2017
SnappyData at Spark Summit 2017SnappyData at Spark Summit 2017
SnappyData at Spark Summit 2017
 
SnappyData, the Spark Database. A unified cluster for streaming, transactions...
SnappyData, the Spark Database. A unified cluster for streaming, transactions...SnappyData, the Spark Database. A unified cluster for streaming, transactions...
SnappyData, the Spark Database. A unified cluster for streaming, transactions...
 
End to End Processing of 3.7 Million Telemetry Events per Second using Lambda...
End to End Processing of 3.7 Million Telemetry Events per Second using Lambda...End to End Processing of 3.7 Million Telemetry Events per Second using Lambda...
End to End Processing of 3.7 Million Telemetry Events per Second using Lambda...
 
Big data on Azure for Architects
Big data on Azure for ArchitectsBig data on Azure for Architects
Big data on Azure for Architects
 
Hadoop tutorial
Hadoop tutorialHadoop tutorial
Hadoop tutorial
 
Tajo_Meetup_20141120
Tajo_Meetup_20141120Tajo_Meetup_20141120
Tajo_Meetup_20141120
 
Deep Dive on Amazon EC2 instances
Deep Dive on Amazon EC2 instancesDeep Dive on Amazon EC2 instances
Deep Dive on Amazon EC2 instances
 
Big Data LDN 2016: Kick Start your Big Data project with Hyperconverged Infra...
Big Data LDN 2016: Kick Start your Big Data project with Hyperconverged Infra...Big Data LDN 2016: Kick Start your Big Data project with Hyperconverged Infra...
Big Data LDN 2016: Kick Start your Big Data project with Hyperconverged Infra...
 
Ibm power ha v7 technical deep dive workshop
Ibm power ha v7 technical deep dive workshopIbm power ha v7 technical deep dive workshop
Ibm power ha v7 technical deep dive workshop
 
An Engineer's Intro to Oracle Coherence
An Engineer's Intro to Oracle CoherenceAn Engineer's Intro to Oracle Coherence
An Engineer's Intro to Oracle Coherence
 

Último

From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionDilum Bandara
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfRankYa
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DaySri Ambati
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 

Último (20)

From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 

Distributed caching-computing v3.8

  • 1. Thinking Distributed: The Hazelcast Way RAHUL GUPTA SR SOLUTIONS ARCHITECT
  • 2. About Me Senior Solutions Architect Worked with Terracotta In-memory Distributed Systems since 2009 Java Programmer since 1998 rahul@hazelcast.com Rahul Gupta follow me @wildnez
  • 3. © 2017 Hazelcast Inc. Confidential & Proprietary 3 Hazelcast Reborn
  • 5. Why Hazelcast? • Scale-out Computing enables cluster capacity to be increased or decreased on-demand • Resilience with automatic recovery from member failures without losing data while minimizing performance impact on running applications • Programming Model provides a way for developers to easily program a cluster application as if it is a single process • Fast Application Performance enables very large data sets to be held in main memory for real-time performance
  • 6. © 2017 Hazelcast Inc. Confidential & Proprietary 6 Low-Level Services API Node Engine (Threads, Instances, Eventing, Wait/Notify, Invocation) Partition Management (Members, Lite Members, Master Partition, Replicas, Migrations, Partition Groups, Partition Aware) Cluster Management with Cloud Discovery SPI (AWS, Azure, Consul, etcd, Heroku, IP List, Apache jclouds, Kubernetes, Multicast, Zookeeper) Networking (IPv4, IPv6) On-Heap Store High-Density Memory Store (Intel, Sparc) Serialization (Serializable, Externalizable, DataSerializable, IdentifiedDataSerializable, Portable, Custom) Near Cache HD C++ C#/.NET Python Node.js Scala Memcached Clojure Open Client Network Protocol (Backward & Forward Compatibility, Binary Protocol) REST Java JVM (JDK: 6, 7, 8, Vendors: Oracle JDK, Open JDK, IBM JDK, Azul Zing & Zulu) Hardware/VM/Container (On Premise, AWS, Azure, Docker, Pivotal Cloud Foundry, OpenShift, VMware) Executor Service SQL Query MapReduce AggregationPredicate Entry Processor Map MultiMap Replicated Map Web Sessions (Tomcat/Jetty/Generic) Continuous QueryHD HD Open Source Enterprise Edition Hazelcast IMDG Enterprise HD Edition HD-Enabled Feature WAN (Socket,SolaceSystems:One-way,Multiway,InitnewDataCenter,DRDataCenterRecovery) ManagementCenter (JMX/REST) SecuritySuite (Connection,Encryption,Authentication,Authorization,JAASLoginModule,SocketInterceptor) Engine Clients HD HD HD Queue Topic Ring Buffer Atomic Long Hibernate 2nd Level Cache HD JCache Set Lock/ Sem. HD java.util.concurrent UUID Hot Restart Store (SSD, HDD) HD Storage Operating System (Linux, Oracle Solaris, Windows, AIX, Unix) Operating Env APIs Hyper LogLog RollingUpgrades (RollingClientUpgrades,RollingMemberUpgrades,NoDowntime) HazelcastStriimHotCache (SyncUpdatesfromOracleDB,MSSQLServer,MySQLandNonStopDB) Operations Hazelcast Solution
  • 7. Ecosystem Traction Dozens of Commercial and Open Source Projects Embed Hazelcast
  • 8. 01001 10101 01010 In Memory Data Computing In Memory Data Messaging++In Memory Data Storage In Memory Data Grid
  • 9. Business Systems A B C RDBMS Mainframe MongoDB NoSQL REST Scale Hazelcast In-Memory Caching
  • 10. java.util.concurrent.ConcurrentMap import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentMap; public static void main(String[] args) { ConcurrentMap<Integer, String> map = new ConcurrentHashMap<>(); map.put(1, "Paris"); map.put(2, "London"); map.put(3, "San Francisco"); String oldValue = map.remove(2); }
  • 11. Distributed Map import java.util.concurrent.ConcurrentMap; import com.hazelcast.core.Hazelcast; import com.hazelcast.core.HazelcastInstance; public static void main(String[] args) { HazelcastInstance h = Hazelcast.newHazelcastInstance(); ConcurrentMap<Integer, String> map = h.getMap("myMap"); map.put(1, "Paris"); map.put(2, "London"); map.put(3, "San Francisco"); String oldValue = map.remove(2); }
  • 12. DEMO
  • 13. Persistence API public class MapStorage implements MapStore<String, User>, MapLoader<String, User> { // Some methods missing ... @Override public User load(String key) { return loadValueDB(key); } @Override public Set<String> loadAllKeys() { return loadKeysDB(); } @Override public void delete(String key) { deleteDB(key); } @Override public void store(String key, User value) { storeToDatabase(key, value); } } <map name="users"> <map-store enabled="true"> <class-name>com.hazelcast.example.MapStorage</class-name> <write-delay-seconds>0</write-delay-seconds> </map-store> </map>
  • 14. JCache API // Retrieve the CachingProvider which is automatically baced by // the chosen Hazelcast server or client provider CachingProvider cachingProvider = Caching.getCachingProvider(); // Create a CacheManager CacheManager cacheManager = cachingProvider.getCacheManager(); // Cache<String, String> cache = cacheManager // .getCache( name, String.class, String.class ); // Create a simple but typesafe configuration for the cache CompleteConfiguration<String, String> config = new MutableConfiguration<String, String>() .setTypes( String.class, String.class );
  • 15. JCache API // Create and get the cache Cache<String, String> cache = cacheManager .createCache( "example", config ); // Alternatively to request an already existing cache Cache<String, String> cache = cacheManager .getCache( name, String.class, String.class ); // Put a value into the cache cache.put( "world", "Hello World" ); // Retrieve the value again from the cache String value = cache.get( "world" ); System.out.println( value );
  • 16. Eviction • Unless deleted, entries remain in the map. • Use eviction policies to prevent OOM situations. • Eviction Triggers run LFU or LRU. time-to-live max-idle-seconds max-size (PER_NODE(number of entries), PER_PARTITION(number of entries), USED_HEAP_SIZE(mb), USED_HEAP_PERCENTAGE(mb)) • Setting eviction-percentage removes that % of entries when eviction is triggered.
  • 17. Features Description MultiMap Store multiple values against one Key Replicated Map Cluster wide replication, all entries on all nodes. Good for read heavy use cases Near Cache Map Entries on Client/Application local memory RingBuffer Stores data in a ring-like structure, like a circular array with given capacity More Distributed Structures
  • 18. Java Collection API: Map, List, Set, Queue Jcache High Density Memory Store Hibernate 2nd Level Cache Web Session Replication: Tomcat, Jetty Predicate API: Indexes, SQL Query Persistence: Map/Queue Store & Loader. Write Behind/Through Spring Compliance Transactions: Local & XA WAN & DR Replication IM Data Store (Caching) Features
  • 19. Java: Will it make the cut? Garbage Collection limits heap usage. G1 and Balanced aim for <100ms at 10GB. Unused Memory 64GB 4GB 4s Heap Java Apps Memory Bound GC Pause Time Available Memory GC Off-Heap Storage No low-level CPU access Java is challenged as an infrastructure language despite its newly popular usage for this Heap
  • 20. GC Pause 64 Gb JVM 4 Gb JVM 1 4 Gb JVM 2 4 Gb JVM 3 4 Gb JVM 4 4 Gb JVM 5 4 Gb JVM 16 GC Pause 0 GB 64 GB 64 Gb in-memory data time Standard Impediments of Caching
  • 21. GC Pause 64 Gb JVM 4 Gb JVM 1 4 Gb JVM 2 4 Gb JVM 3 4 Gb JVM 4 4 Gb JVM 5 4 Gb JVM 16 GC Pause 0 Gb 64 Gb 64 Gb in-memory data time 64 Gb HD 2 Gb JVM GC Pause HD Memory Caching with HD Memory
  • 22. On-Heap Memory Store (Objects Stored as Objects) High-Density Memory Store (Objects Serialized and Stored as Bytes) s.m.Unsafe s.m.Unsafe 2-4GB (Limited by Garbage Collection) 0-1TB (Limited by Machine RAM) Memory Stores •Member •Client (Near Cache) RAM in JVM Process APIs JCache (ICache) Map (IMap) HD Memory
  • 23. On Heap Vs. High-Density Memory Management On Heap Memory HD Memory 0 MB HD 3.3 GB 3.9 GB Heap Storage 0.6 GB 9 (4900 ms) Major GC 0 (0 ms) 31 (4200 ms) Minor GC 356 (349 ms) Node Used Heap Total Heap Max. Heap Heap Usage Percentage Used Heap: 0.2 GB 192.168.1.10:5701 57 MB 229 MB 910 MB 6.28% Memory Utilization Home Offheap-test Node Used Heap: 3.9 GB 192.168.1.10:5701 3933 MB 4658MB 4653MB 84.45% Memory Utilization Home Used Heap Total Heap Max. Heap Heap Usage Percentage Example: On Heap Memory Example: HD Memory
  • 24. Hazelcast Servers Hazelcast Server JVM [Memory] A B C Business Logic Data Data Data CE = Compute Engine Result Business / Processing Logic Result TCP / IP Client Client Distributed Computing
  • 25. IM Distributed Computing Feature HD Cache Dist. Compute Dist. Message Java Concurrency API (Lock, Semaphore, AtomicLong, AtomicReference, Executor Service, Blocking Queue) Entry and Item Listeners Entry Processor Aggregators Map/Reduce Data Affinity Continues Query Map Interceptors Delta Update
  • 26. Executor Service API public interface com.hazelcast.core.IExecutorService extends java.util.concurrent.ExecutorService HazelcastInstance hz = getHazelcastInstance(); //java.util.concurrent.ExecutorService implementation IExecutorService es = hz.getExecutorService("name"); es.executeOnAllMembers(buildRunnable()); es.executeOnKeyOwner(buildRunnable(), "Peter"); es.execute(buildRunnable()); Map<..> futures = es.submitToAllMembers(buildCallable()); Future<..> future = es.submitToKeyOwner(buildCallable(), "Peter"); es.submitToAllMembers(buildCallable(), buildCallback()); es.submitToKeyOwner(buildCallable(), "Peter", buildCallback());
  • 28. Lock API HazelcastInstance hz = getHazelcastInstance(); // Distributed Reentrant L​ock l​ock = hz.getLock("myLock"); l​ock.lock(); try { // Do something } finally { l​ock.unlock(); } Distributed Lock
  • 31. Map/Reduce API HazelcastInstance hz = getHazelcastInstance(); Map users = hz.getMap("users"); JobTracker tracker = hz.getJobTracker("default"); KeyValueSource source = KeyValueSource.fromMap(users); Job job = tracker.newJob(source); ICompleteFuture future = job.mapper(new MyMapper()) .reducer(new MyReducer()) .submit(); Map result = future.get();
  • 32. Aggregations API HazelcastInstance hz = getHazelcastInstance(); Map users = hz.getMap("users"); int sum = users.aggregate( Supplier.all((user) -> user.getSalary()), Aggregations.longSum() );
  • 33. Hazelcast Distributed Topic Bus Hazelcast Topic Hazelcast Node 1 Hazelcast Node 2 Hazelcast Node 3 MSG Subscribes Delivers Subscribes Delivers Distributed Messaging
  • 34. Queue Topic (Pub/Sub) Event Listeners Ring Buffers IM Distributed Messaging Features HD Cache Dist. Compute Dist. Message
  • 35. Queue API interface com.hazelcast.core.IQueue<E> extends java.util.concurrent.BlockingQueue HazelcastInstance hz = getHazelcastInstance(); //java.util.concurrent.BlockingQueue implementation IQueue<Task> queue = hz.getQueue("tasks"); queue.offer(newTask()); queue.offer(newTask(), 500, TimeUnit.MILLISECONDS); Task task = queue.poll(); Task task = queue.poll(100, TimeUnit.MILLISECONDS); Task task = queue.take();
  • 36. Topic API public class Example implements MessageListener<String> { public void sendMessage { HazelcastInstance hz = getHazelcastInstance(); ITopic<String> topic = hz.getTopic("topic"); topic.addMessageListener(this); topic.publish("Hello World"); } @Override public void onMessage(Message<String> message) { System.out.println("Got message: " + message.getMessageObject()); } }
  • 37.
  • 38. Data Distribution and Resilience 38
  • 39. Distributed Maps Fixed number of partitions (default 271) Each key falls into a partition partitionId = hash(keyData)%PARTITION_COUNT Partition ownerships are reassigned upon membership change A B C
  • 48. Data Safety on Node Failure 48
  • 59. © 2017 Hazelcast Inc. Confidential & Proprietary 59 Topologies Deployment Architectures
  • 60. © 2017 Hazelcast Inc. Confidential & Proprietary 60 Embedded Mode Great for microservices, OEM and ops simplification IMDG Application Java API IMDG Application Java API IMDG Application Java API
  • 61. © 2017 Hazelcast Inc. Confidential & Proprietary 61 Embedded Mode Embedded model, for example in a J2EE container Service and storage in the one JVM Easy for getting started With three nodes, 1/3rd of the data is local, good read performance -> This benefit diminishes as you scale up -> With ten nodes, only 1/10th of the data is local If the application needs to go offline, so does the data
  • 62. © 2017 Hazelcast Inc. Confidential & Proprietary 62 Client-Server Mode Great for scale-up or scale-out deployments with cluster lifecycle decoupled from app servers Use for non-Java clients Node.js API Application Node.js API Application Node.js API Application Node.js API ApplicationJava API Application .Net API Application C++ API Application C++ API Application C++ API Application C++ API Application .Net API Application .Net API Application .Net API ApplicationJava API Application Java API Application IMDG IMDG IMDG
  • 63. © 2017 Hazelcast Inc. Confidential & Proprietary 63 Client-Server Mode Follows the traditional Client-Server approach Clients do not change cluster topology Allows you to segregate service from storage Smart clients connect to all cluster members. Operations are routed directly to a member holding the necessary data. -> One network hop Other clients connect to a single cluster member and use it as a proxy to issue requests. -> Two network hops if the proxy doesn’t have the data needed
  • 64. © 2017 Hazelcast Inc. Confidential & Proprietary 64 Pushing Into New Language Ecosystems Hazelcast.org/Plugins • Fully featured • Hazelcast supported • Open Source • Common Hazelcast Feature Set • Hazelcast supported • Open Source • High Performance • Hazelcast supported • Open Source • Client and Member APIs • Scala 2.11 support • Hazelcast Supported • Open Source • Community supported • Open Source • Fully Featured • Hazelcast supported • Open Source • Industry Leading Performance • High-Density Memory Store Near Cache • Hazelcast Supported • Open Source Feature Matrix: https://hazelcast.org/clients-languages
  • 65. © 2017 Hazelcast Inc. Confidential & Proprietary 65 Hazelcast IMDG Microservices Deployment Options Microservices deployed embedded with isolated Hazelcast IMDG clusters Maximum service isolation Microservices deployed client server with a shared Hazelcast IMDG cluster Transition to microservices MS 1 IMDG IMDG IMDG MS 1 MS 1 .NET Client MS 1 IMDG IMDG IMDG IMDG .NET Client MS 1 Node.js Client MS 2 Node.js Client MS 2 C++ Client MS 3 Java Client MS 4 Java Client MS 4 MS 2 IMDG IMDG IMDG MS 2 MS 2 MS 3 IMDG IMDG IMDG MS 3 MS 3
  • 66. © 2017 Hazelcast Inc. Confidential & Proprietary 66 Hazelcast IMDG Microservices Deployment Options Microservices deployed client server with isolated Hazelcast cluster per service Maximum service isolation and non-Java clients .NET Client MS 1 IMDG IMDG IMDG .NET Client MS 1 .NET Client MS 1 .NET Client MS 1 .NET Client MS 1 Python Client MS 2 IMDG IMDG IMDG Python Client MS 2 Python Client MS 2 Python Client MS 2 Python Client MS 2
  • 67. Same Features – Embedded vs Client-Server // Creating a new Hazelcast node HazelcastInstance hz = Hazelcast.newHazelcastInstance(); // Getting a Map, Queue, Topic, ... Map map = hz.getMap("my-map"); Queue queue = hz.getQueue("my-queue"); ITopic topic = hz.getTopic("my-topic"); //Creating a Hazelcast Client HazelcastInstance client = HazelcastClient.newHazelcastClient(); // Shutting down the node hz.shutdown();
  • 68. © 2017 Hazelcast Inc. Confidential & Proprietary 68 Hazelcast IMDG – Any Cloud, Anytime Apache jClouds PaaS Environment Integrations Cloud & Cloud Discovery Plugins Deployment Options YARN
  • 69. © 2017 Hazelcast Inc. Confidential & Proprietary 69 Hazelcast IMDG WAN Replication Deployment Options Independent clusters, sharing selected data to maintain alignment One-way or two-way Choose which data to share, select by type, can de-select records by attributes One-way  keep DR site populated and ready-to-go Two-way  worldwide operations, access your nearest data copy API Applications API Applications API Applications IMDG IMDG IMDG IMDG IMDG IMDG IMDG
  • 71. © 2017 Hazelcast Inc. Confidential & Proprietary 71 Hazelcast IMDG High Level Roadmap Hi-Density Caching In-Memory Data Grid 2015 2016 JCache | Scalability | HD Memory | Hot Restart | Pre-Load Clients Cloud Discovery SPI | Azure Cloud Foundry | AWS j.u.c. | Low-Latency | Elasticity | Resiliency | Performance | Rolling Upgrades Open Cloud Platform 2017 j.u.s | Jet | Cascading | Streaming | Mesos | Reactive Expanded In-memory Computing 2018
  • 72. © 2017 Hazelcast Inc. Confidential & Proprietary 72 Recent Releases & Roadmap Release Date Hazelcast IMDG 3.7 August 2016 (Released) Hazelcast IMDG 3.8 February 2017 (Released) Hazelcast IMDG 3.9 August 2017 Hazelcast IMDG 4.0 September 2017
  • 73. © 2017 Hazelcast Inc. Confidential & Proprietary 73 New Hazelcast IMDG 3.8 Features Features Description Fast Aggregations Based on Queries Make aggregations significantly faster by moving in-built aggregations to be on top of the query infrastructure of Hazelcast IMDG. Another goal is to expose a simplified Aggregations API. User Code Deployment Ability to access and use user classes without having them on the classpath. Projection for Queries Provide projection in queries to allow returning specific fields of an entry, not the whole entry. This will cut the network and serialisation overhead and increase the throughput. RingBuffer Store Implementation of the storage mechanism for the RingBuffer, analogous to the existing QueueStore mechanism. It will also allow users of the ReliableTopic to replay messages older than the messages in the underlying RingBuffer. Continuous Query The previously Enterprise feature Continuous Query Cache becomes open source with 3.8.
  • 74. © 2017 Hazelcast Inc. Confidential & Proprietary 74 New Hazelcast IMDG 3.8 Features Features Description Unified Near Cache Implementation for IMap and JCache A technical debt to have just 2 Near-Cache implementations: OnHeapNearCache and HDNearCache. These 2 implementations should be used by both IMap and JCache as well as member and client. High Performance Near Cache Several Improvements on Near Cache including pre-loading near cache. Eventually Consistent Near Cache A guarantee of delivery of cache invalidations to the near cache. HyperLogLog Implementation of the HyperLogLog data structure on top of Hazelcast IMDG. It is a probabilistic structure used to estimate cardinality. Scheduled Executor Service Distributed version of java.util.concurrent.ScheduledExecutorService. Cluster Quorum for Queue and Lock Quorum support for Hazelcast IMDG Queue and Lock.
  • 75. © 2017 Hazelcast Inc. Confidential & Proprietary 75 New Hazelcast IMDG 3.8 Enterprise Features Features Description Rolling Member Upgrades for Minor Releases Extending current rolling upgrade capability of Hazelcast IMDG (client rolling upgradeability, patch version rolling member upgrades) to allow heterogeneous versions of Hazelcast IMDG members to be rolling upgraded without service interruption for minor version changes. Copying HRS Data from a Source to a Target Cluster Ability to copy the Hot Restart store files from a source environment to a target environment (for example, product to test environment). This feature also applies to running sources. One-Off WAN Sync with Dynamically Added Endpoint Ability to copy all Maps’ content to a target cluster dynamically in runtime. SSL Performance Improvements (Java) Large performance increase in SSL when using the Java client. LDAP Integration for Management Center Authentication and authorizarion with LDAP in Management Center. Many Management Center Enhancements More metrics, stats, and improved usability and monitoring.
  • 76. © 2017 Hazelcast Inc. Confidential & Proprietary 76 New Hazelcast IMDG 3.8 Clients Features Features Description Mutual Auth with TLS Certificates Allow using client certificates to identify the client to the server. Simulator Integration for Hazelcast .NET Client Integration of the .NET client into Hazelcast Simulator. Node.js Client Enhancements RingBuffer, Topic, and Near Cache Support for Node.js client. SSL Support for Node.js Client SSL support for Hazelcast Node.js Client to secure the connection between the client and the members. Near Cache Support for C++ Client Configurable near cache support for IMap in the Hazelcast C++ client. AWS Discovery for C++ Client AWS cluster discovery implementation for Hazelcast C++ Client.
  • 77. © 2017 Hazelcast Inc. Confidential & Proprietary 77 New Hazelcast IMDG 3.8 Cloud & Plugins Features Features Description Pivotal® CloudFoundry Integration Hazelcast IMDG on Pivotal CloudFoundry as a Tile. IBM DynaCache API Support A WebSphere DynamicCache provider, which could be used either by traditional WebSphere or WebSphere Liberty Profile applications. Discovery SPI based AWS Module Discovery SPI compatible hazelcast-aws module. AWS users would take advantage of ZONE_AWARE (availability zone aware partition groups) without using Apache jclouds. Striim Hot Cache Adapter HotCache functionality with Striim based on Change Data Capture ("CDC") of underlying database updates. Hibernate 5.2 Support Improvements in distributed L2 cache for Hibernate to support Hibernate 5.2.
  • 78. © 2017 Hazelcast Inc. Confidential & Proprietary 78 Hazelcast 3.9 Roadmap August 2017
  • 79. © 2017 Hazelcast Inc. Confidential & Proprietary 79 3.9 Features Features Description Near Cache: Key Store-by- Reference Option Key store-by-reference option for IMap and Hazelcast JCache Near Cache. Dynamic Creation of Specificly Configured Distributed Objects Ability to define new data structures with custom configuration on the fly. Gigantic Cache Migration Enhancements Improvements in partition migration process to allow better gigantic cache migrations. Lazy Initiation for Java Client An async client factory to connect lazily and in a non-blocking manner. Client Support for User Code Deployment Dynamic Classloading from clients for User Code Deployment. Query Async API Async query methods on the IMap
  • 80. © 2017 Hazelcast Inc. Confidential & Proprietary 80 3.9 – Security Features Features Description Mutual TLS Auth Java Client Allow a member to authenticate a client using a certificate installed in the client Mutual TLS Auth C# Client Allow a member to authenticate a client using a certificate installed in the client Management Center TLS TLS from members to Man Center. TLS from browser. TLS for REST endpoint TLS for REST endpoint on Members General Security Hardening Implement things like password blacklisting, password complexity requirements, toggling off of non-secure mechanisms when secure version is enabled and so on Fast SSL/TLS Make SSL/TLS almost as fast as unencrypted.
  • 81. © 2017 Hazelcast Inc. Confidential & Proprietary 81 Fast SSL/TLS
  • 82. © 2017 Hazelcast Inc. Confidential & Proprietary 82 New Hazelcast IMDG 3.9 Enterprise Features Features Description Dynamic Addition of WAN Endpoints Ability to add WAN Targets to a running cluster. NearCache Stats in Management Center Provision of showing data inside Near-Cache of client or member, similar to IMap/Cache in Management Center. Search Indexes in HD Memory Support for sorted and unsorted indexes in Hazelcast HD Memory Store.
  • 83. © 2017 Hazelcast Inc. Confidential & Proprietary 83 New Hazelcast IMDG 3.9 Clients Features Features Description Simulator Integrations of C++, Python, and Node.js Clients Integration of C++, Python, and Node.js clients into Hazelcast Simulator. SSL Performance Improvements for .NET Client Large performance increase in SSL when using the .NET Client. Projections for .NET Client Add projections (returning attributes rather than full values) to .NET. Paging Predicate for .NET Client Support for paging predicates in Hazelcast .NET Client. ExecutorService for C++ Client Ability to use the distributed ExecutorService from Hazelcast C++ Client. Near Cache for C++ Client Add near cache to C++ client TLS for C++ Client Add TLS encryption for C++ Clients communicating with members
  • 84. © 2017 Hazelcast Inc. Confidential & Proprietary 84 New Hazelcast IMDG 3.9 Cloud & Plugins Features Features Description Google Cloud Platform Integration SPI Plugin for GCP for Cloud Discovery with a support for partition groups. AWS support for C++ Client We have comprehensive cloud support for Java. Adding AWS to C++. Stats for Hibernate LocalRegionCache Ability to provide statistics for Hibernate LocalRegionCache. Web Sessions for WebSphere Liberty Web sessions replication plugin for WebSphere Liberty.
  • 85. © 2017 Hazelcast Inc. Confidential & Proprietary 85 New Hazelcast IMDG 4.0 Features Features Description Java 8 Language Level Lift the language level for Hazelcast members and their API to Java 8. Add a new Java 8 client. Continue to maintain the Java 6 compatible client. This will allow us to take advantage of Java 8 language features. Java 9 Modularisation Java 9 modularisation for Hazelcast IMDG, which introduces Jigsaw - a new modular systems with a strict enforcement of module boundaries. Proper Separation of Public and Private APIs Re-design of the whole private/public API to keep as much private as possible. Public and Stable SPI Make SPI a second public and stable API. This work includes many refactorings, client protocol enhancements, documentation efforts, and reference implementations. EntryProcessor Contract Formalisation Optimization of the EntryProcessor which would bring the IMap behaviour inline with the JCache behaviour. Unification of Client / Member Behaviour in Exceptional States Equalization of exceptions in client-side and member-side so that clients can throw the same exceptions as server whenever possible. This work also includes a documentation effort on exceptional cases for each method.
  • 86.
  • 87. What’s Hazelcast Jet? • General purpose distributed data processing framework • Based on Direct Acyclic Graph to model data flow • Built on top of Hazelcast • Comparable to Apache Spark or Apache Flink 8 7
  • 91. Service Offerings Hazelcast (Apache Licensed) • Professional Subscription – 24x7 support* Hazelcast Enterprise Support • Available with Hazelcast Enterprise software subscription - 24x7 support* Additional Services • Development Support Subscription – 8x5 support* • Simulator TCK • Training • Expert Consulting • Development Partner Program * All subscriptions include Management Center
  • 92. © 2016 Hazelcast Inc. Confidential & Proprietary 92 Support Subscriptions What’s Included 100% SUCCESS RATE ON CUSTOMER ISSUES: “As usual, the response was timely beyond expectations, and very good technical content returned. Exemplary support, hard to find in any company…” - Fortune 100 Financial Services Customer ENTERPRISE HD ENTERPRISE PROFESSIONAL OPEN SOURCE SUPPORT WINDOW 24/7 24/7 24/7 RESPONSE TIME FOR CRITIAL ISSUES 1 Hour 1 Hour 2 Hours SUPPORTED SOFTWARE Hazelcast & Hazelcast Enterprise Hazelcast & Hazelcast Enterprise Hazelcast SUPPORT CONTACTS 4 4 2 SUPPORT CHANNELS Email, IM & Phone Email, IM & Phone Email, IM & Phone PATCH LEVEL FIXES REMOTE MEETINGS (via GoToMeeting) CODE REVIEW (with a Senior Solutions Architect) 2 Hours 2 Hours 2 Hours QUARTERLY REVIEW OF FEATURE REQUES* QUARTERLY REVIEW OF HAZELCAST ROADMAP*
  • 93. © 2016 Hazelcast Inc. Confidential & Proprietary 9 3 Best In Class Support Support from the Engineers who wrote the code SLA Driven – 100% attainment of support response time Follow the Sun Portal, Email and Phone access Go Red, Go Green. Reproduction of issues on Simulator. Proof of fix on Simulator. Periodic Technical Reviews Meet your production schedule and corporate compliance requirements Ensure the success of your development team with training and best practices
  • 95. © 2016 Hazelcast Inc. Confidential & Proprietary 95 Support Testimonials
  • 96. © 2016 Hazelcast Inc. Confidential & Proprietary 96 Release Lifecycle • Regular Feature release each 4-5 months, e.g. 3.3, 3.4, 3.5 • Maintenance release approximately each month with bug fixes based on the current feature release, e.g. 3.4.1 • For older versions, patch releases made available to fix issues • Release End of Life per support contract