This talk will discuss an upcoming feature in MemSQL 6.5 showing how advanced stream processing use cases can be tackled with a combination of stored procedures (new in 6.0) and MemSQL's pipelines feature.

1. Stream Processing with Pipelines
and Stored Procedures
Tewei Luo
Software Engineer

2. CREATE PIPELINE AS LOAD DATA
[KAFKA | S3 | FS]
INTO TABLE tbl

3. Stream into more tables?

4. Stream into more tables?
Example:
- 2 fields in source
- 2 tables, each takes one column

5. CREATE PIPELINE AS LOAD DATA
[KAFKA | S3 | FS]
INTO PROCEDURE split

6. CREATE PROCEDURE
split(batch Query(c1 int, c2 text)) AS
BEGIN
INSERT INTO tb1 SELECT c1 from batch;
INSERT INTO tb2 SELECT c2 from batch;
…
END

7. 7
DEMO

8. - Split stream to multiple tables
- Update statistics
- Complex logic using existing tables
- Delete obsolete data automatically

9. 9
Questions?

10. Thank you
memsql.com