Tokyo APAC Groundbreakers tour - The Complete Java Developer

すみません、私は日本語を話せません

Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
The Complete
Developer
Meaner, faster code via SQL!
3

Connor McDonald

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Stuff
youtube bit.ly/youtube-connor
blog bit.ly/blog-connor
twitter bit.ly/twitter-connor
400+ posts mainly on database & SQL
250 technical videos, new uploads every week
my thoughts on tech and the world :-)

etc...
facebook bit.ly/facebook-connor
linkedin bit.ly/linkedin-connor
instagram bit.ly/instagram-connor
slideshare bit.ly/slideshare-connor

9
https://asktom.oracle.com

16
Java
html/css
node/angular
git/svn
logstash/MQ
Junit/Selenium
Cloud/REST/etc
etc
etc
etc

17
Java
html/css
node/angular
git/svn
logstash/MQ
Junit/Selenium
Cloud/REST/etc
???ORM Containers
KubernetesDocker

we developers love cool stuff
23

26
"fine-grained to perform a single function"
"Each service is ... minimal, and complete"
https://en.wikipedia.org/wiki/Microservices
select COUNT(*)
from PEOPLE
where GENDER = 'MALE'

30
"By abstracting the underlying implementation"
"describes the expected behaviour ... but can have multiple implementations"
https://en.wikipedia.org/wiki/Application_programming_interface
select NAME, STREET_NO, ZIP_CODE
from PEOPLE p,
ADDRESS a
where p.AGE > 50
and p.ADDRESS_ID = a.ADDRESS_ID;

... faster, simpler, better applications
35

this session is not about ...
37

being a smart-ass
38
知ったかぶり

we can do anything in SQL ...
39

SQL> with x( s, ind ) as
2 ( select sud, instr( sud, '.' )
3 from ( select replace(replace(
4 replace(replace(:board,'-'),'|'),' '),chr(10)) sud
5 from dual )
6 union all
7 select substr(s,1,ind-1)||z||substr(s,ind+1)
8 , instr(s,'.',ind+1)
9 from x
10 , ( select to_char( rownum ) z
11 from dual connect by rownum <= 9 ) z
12 where ind > 0
13 and not exists (
14 select null
15 from ( select rownum lp from dual
16 connect by rownum <= 9 )
17 where z = substr(s,trunc((ind-1)/9)*9+lp,1)
40

18 or z = substr(s,mod(ind-1,9)-8+lp*9,1)
19 or z = substr(s,mod(trunc((ind-1)/3),3)*3
20 +trunc((ind-1)/27)*27+lp
21 +trunc((lp-1)/3)*6,1)
22 )
23 ),
24 result as (
25 select s
26 from x
27 where ind = 0 )
28 select
29 regexp_replace(substr(s,(idx-1)*9+1,9),
30 '(...)(...)(...)',
31 '1|2|3')||
32 case when mod(idx,3)=0 then chr(10)||rpad('-',11,'-') end soln
33 from result,
34 ( select level idx
35 from dual
36 connect by level <= 9 )
41
Ack: Anton Scheffer,
https://technology.amis.nl

SQL> variable board varchar2(1000)
SQL> begin :board :=
2 '53.|.7.|...
3 6..|195|...
4 .98|...|.6.
5 -----------
6 8..|.61|..3
7 4..|8.3|..1
8 7..|.2.|..6
9 -----------
10 .6.|...|28.
11 ...|419|..5
12 ...|.8.|.79
13 ';
14 end;
42
5 3 7
6 1 9 5
9 8 6
8 6 1 3
4 8 3 1
7 2 6
6 2 8
4 1 9 5
8 7 9

SOLUTION
-----------
534|678|912
672|195|348
198|342|567
-----------
859|761|423
426|853|791
713|924|856
-----------
961|537|284
287|419|635
345|286|179
-----------
43
sud.sql

44
100%
% of developers that
will need to solve Sudoku
as part of their job

45
100%
% of developers that need
to get real work done

47
Java
html/css
node/angular
git/svn
logstash/MQ
Junit/Selenium
Cloud/REST/etc
SQL
Database

... do two things with data
49

a quick primer on (database) memory

memory access = controlled access

SGA
protected by
2) access memory

SGA
protected by
3) release latch

SGA
protected by
someone must wait ...

spinning
can I have the latch ?
71

you ... get ... nothing ... done

“But how is that related
to reading data?”

to read data, you read blocks...

SQL> select *
2 from EMP
3 where ...
/d01/ts/my_data1.dat
Cache

but (database) memory access is complex
82
multi-process

SQL> select *
2 from EMP
3 where …
SQL> update EMP
2 set …
3 where …
SQL> select *
2 from EMP
3 where …
SQL> insert into
2 EMP
3 values (…)
SQL> select *
2 from CUSTOMER
3 /
SQL> select max(EMP)
2 from DEPT
SQL> delete
2 from DEPT
3 /

• Get latch
• Search along list of blocks in memory (or "chain")
• Read block
• Extract row of interest
• Release latch
• Give row back to client
"fetch a row"
86

rs = stmt.executeQuery("...");
while(rs.next()) {
v1 = rs.getInt(1);
v2 = rs.getString(2);
...
}
• Get latch
• Walk along chain
• Get block
• Extract single row
• Release latch
• Give row back to client
• Get latch (again)
• Walk along chain (again)
• Get block (again)
• Release latch (again)
• Give row back to client (again)
Row #1
Row #2
• Get latch (again)
• Walk along chain (again)
• Get block (again)
• Release latch (again)
• Give row back to client (again)
Row #3
90

a better strategy.... “pinning”

"fetch a row"
• Get latch
• Get block
• Pin the block
• Release latch
• Give row 1 back to client
• Give row 2 back to client
…
• Give row n to client
• Get latch
• Remove my pin on the block
• Release latch
“and .. I’ll will need
several rows from
this block”

how do you say
“I may want many rows” ?

DBA relationship is important
100
oc00.sql

Statistics
-------------------------------
13 recursive calls
1 db block gets
527 consistent gets
103
Statistics
------------------------------
0 recursive calls
0 db block gets
377 consistent gets
versus

the most expensive SQL ...
104

... is one we've never seen
105

Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)
SQL> select *
2 frmo emp;
frmo emp
*
ERROR at line 2:
ORA-00923: FROM keyword not found where expected
107

Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)
SQL> select empnoo
2 from emp;
select empnoo
*
ERROR at line 1:
ORA-00904: invalid column name

Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)
PLAN
-------------------------------------
SELECT STATEMENT
TABLE ACCESS BY INDEX ROWID EMP
INDEX RANGE SCAN EMP_PK

EMP_PK EMP
Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)

Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)
111

Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)

could be most of execution time

SQL> select name, gets
2 from v$latch
3 where name like 'library%' or name like 'shared%';
NAME GETS
------------------------------ ----------
shared pool 333907
shared pool simulator 7191
shared pool sim alloc 133
shared server configuration 6
shared server info 1
SQL> select 0 from dual;
0
----------
0
GETS
----------
333991 = 84
7196 = 5
135 = 2
6
1

previously executed statements
119

select surname, firstname from emp where empno = 123
probability of reuse low ?

select surname, firstname from emp where empno = ?
binding
parse this...
now run it with ? = 7362
now run it with ? = 123

let's make it real
ParseDemo2 nn
ParseDemo2Bind nn

it takes 5 minutes to hack you

think blocks not rows
134
when appropriate

commit when it makes sense to
135

demo
136
InsertDemo3
InsertDemo4 nn

rows to columns, colums to rows
146

"I need sales by product
for each calendar quarter....now!"
147

SQL> select product, quarter, sum(quantity) total
3 from SALES
4 group by product, quarter
5 order by 1,2;
PRODUCT MTH TOTAL
-------------------- --- ----------
CHAINSAW JAN 251473
CHAINSAW APR 254830
CHAINSAW JUL 251994
CHAINSAW OCT 243748
HAMMER JAN 249889
HAMMER APR 256566
HAMMER JUL 252992
HAMMER OCT 249104
SCREW DRIVER JAN 245988
SCREW DRIVER APR 249219
SCREW DRIVER JUL 252128
SCREW DRIVER OCT 244721
SPADE JAN 242434
SPADE APR 254090
SPADE JUL 259613
...
148

150
PRODUCT JAN APR JUL OCT
CHAINSAW 128371 128798 282371 827919
HAMMER 138329 934580 840132 488201
SCREW DRIVER 934810 127319 934892 747613
SPADE 378641 874368 471209 348472
WHEEL BARROW 471380 748893 836471 461721
Annual Report

IT
151
CAN
ALL
DONE
WITH
SQL !
BE

SQL> select *
2 from (select product,
3 quarter,
4 quantity
5 from sales )
6 pivot( sum(quantity) for quarter in
7 ( 'JAN',
8 'APR',
9 'JUL',
10 'OCT') )
11 order by 1
12 /
PRODUCT JAN APR JUL OCT
-------------------- ---------- ---------- ---------- ----------
CHAINSAW 251473 254830 251994 243748
HAMMER 249889 256566 252992 249104
SCREW DRIVER 245988 249219 252128 244721
SPADE 242434 254090 259613 248428
WHEEL BARROW 243899 249327 252296 254137
153

"Employee salary list,
plus department total,
plus grand total"
157

rs = stmt.executeQuery(
"select empno, ename, sal, deptno from emp
order by deptno");
EMPNO ENAME SAL DEPTNO
---------- ---------- ---------- ----------
7782 CLARK 2450 10
7839 KING 5000 10
7934 MILLER 1300 10
...
7900 JAMES 950 30
7698 BLAKE 2850 30
7654 MARTIN 1250 30
159
rs = stmt.executeQuery("
select deptno,
sum(sal)
from emp
group by deptno
order by deptno");
DEPTNO SUM(SAL)
---------- ----------
10 8750
20 10875
30 9400
rs = stmt.executeQuery(
"select sum(sal) from emp");
SUM(SAL)
----------
29025

EMPNO SAL DEPTNO
---------- ---------- ----------
7782 2450 10
7839 5000 10
7934 1300 10
7566 2975 20
7902 3000 20
7876 1100 20
7369 800 20
160
DEPTNO SUM(SAL)
---------- ----------
10 8750
20 10875
30 9400
7782 2450 10
7839 5000 10
7934 1300 10
7566 2975 20
10 8750

IT
161
CAN
ALL
DONE
WITH
SQL !
BE

select deptno,
nvl2(rownum,max(empno),null) empno,
nvl2(rownum,max(ename),null) ename,
sum(sal)
from emp
group by rollup(deptno,rownum)
order by deptno,empno;
DEPTNO EMPNO ENAME SUM(SAL)
---------- ---------- ---------- ----------
10 7782 CLARK 2450
10 7839 KING 5000
10 7934 MILLER 1300
10 8750
20 7369 SMITH 800
20 7566 JONES 2975
...
30 7900 JAMES 950
30 9400
29025 162

select deptno,job,sum(sal) from scott.emp
group by CUBE(deptno,job)
order by deptno,job;
DEPTNO JOB SUM(SAL)
---------- --------- ----------
10 CLERK 1300
10 MANAGER 2450
10 PRESIDENT 5000
10 8750
20 ANALYST 6000
20 CLERK 1900
20 MANAGER 2975
20 10875
30 CLERK 950
30 MANAGER 2850
30 SALESMAN 5600
30 9400
ANALYST 6000
CLERK 4150
MANAGER 8275
PRESIDENT 5000
SALESMAN 5600
29025
164

select deptno, job, mgr, sum(sal) from emp
group by grouping sets (
(deptno),
(job,mgr), () ) ;
DEPTNO JOB MGR SUM(SAL)
---------- --------- ---------- ----------
CLERK 7902 800
PRESIDENT 5000
CLERK 7698 950
CLERK 7788 1100
CLERK 7782 1300
SALESMAN 7698 5600
MANAGER 7839 8275
ANALYST 7566 6000
10 8750
20 10875
30 9400
29025
166

"Show the employees in order
of their salary...
...and show the sequence they were employed"
168

EMPNO ENAME JOB HIREDATE SAL HIRE_SEQ
---------- ---------- --------- --------- ---------- ----------
7369 SMITH CLERK 17-DEC-80 800 1
7900 JAMES CLERK 03-DEC-81 950 10
7876 ADAMS CLERK 12-JAN-83 1100 14
7521 WARD SALESMAN 22-FEB-81 1250 3
7654 MARTIN SALESMAN 28-SEP-81 1250 8
7934 MILLER CLERK 23-JAN-82 1300 12
7844 TURNER SALESMAN 08-SEP-81 1500 7
7499 ALLEN SALESMAN 20-FEB-81 1600 2
7782 CLARK MANAGER 09-JUN-81 2450 6
7698 BLAKE MANAGER 01-MAY-81 2850 5
7566 JONES MANAGER 02-APR-81 2975 4
7902 FORD ANALYST 03-DEC-81 3000 10
7788 SCOTT ANALYST 09-DEC-82 3000 13
7839 KING PRESIDENT 17-NOV-81 5000 9

IT
170
CAN
ALL
DONE
WITH
SQL !
BE

select empno, ename, job, hiredate, sal,
rank() OVER (order by hiredate) as hire_seq
from emp
order by sal;
EMPNO ENAME JOB HIREDATE SAL HIRE_SEQ
---------- ---------- --------- --------- ---------- ----------
7369 SMITH CLERK 17-DEC-80 800 1
7900 JAMES CLERK 03-DEC-81 950 10
7876 ADAMS CLERK 12-JAN-83 1100 14
7521 WARD SALESMAN 22-FEB-81 1250 3
7654 MARTIN SALESMAN 28-SEP-81 1250 8
7934 MILLER CLERK 23-JAN-82 1300 12
7844 TURNER SALESMAN 08-SEP-81 1500 7
7499 ALLEN SALESMAN 20-FEB-81 1600 2
7782 CLARK MANAGER 09-JUN-81 2450 6
7698 BLAKE MANAGER 01-MAY-81 2850 5
7566 JONES MANAGER 02-APR-81 2975 4
7902 FORD ANALYST 03-DEC-81 3000 10
7788 SCOTT ANALYST 09-DEC-82 3000 13
7839 KING PRESIDENT 17-NOV-81 5000 9

"6 month moving average of employee
salary based on hiredate,
for each department"
173

IT
174
CAN
ALL
DONE
WITH
SQL !
BE

SQL> select deptno, ename, hiredate, sal,
2 avg(sal) over (
3 partition by deptno
4 order by hiredate
5 range between interval '6' month preceding and current row ) mov
6 from emp
7 order by deptno, hiredate;
DEPTNO ENAME HIREDATE SAL MOV
---------- ---------- --------- ---------- --------
10 CLARK 09-JUN-81 2450 2450
10 KING 17-NOV-81 5000 3725
10 MILLER 23-JAN-82 1300 3150
20 SMITH 17-DEC-80 800 800
20 JONES 02-APR-81 2975 1888
20 FORD 03-DEC-81 3000 3000
20 SCOTT 09-DEC-82 3000 3000
20 ADAMS 12-JAN-83 1100 2050
30 ALLEN 20-FEB-81 1600 1600
30 WARD 22-FEB-81 1250 1425
30 BLAKE 01-MAY-81 2850 1900
30 TURNER 08-SEP-81 1500 2175
30 MARTIN 28-SEP-81 1250 1867
30 JAMES 03-DEC-81 950 1233

"If my salary is increased to $3000,
where would I rank in each department?"
181

IT
182
CAN
ALL
DONE
WITH
SQL !
BE

SQL> select
2 deptno,
3 rank(3000) within group ( order by sal ) as ranking
5 from emp
6 group by deptno;
DEPTNO RANKING
---------- ----------
10 2
20 4
30 7
40 2

suspicious activity
187
anti money laundering

"Find 10 or more consecutive bets in a
24 hour period,
then a large win within three days of
the last bet,
at a different location"
189

ACCT TSTAMP TYP AMT LOC
---------- ------------------ --- ---------- ----
54261 25/01/13 17:20:55 Bet 100 Tokyo
54261 25/01/13 17:56:58 Bet 165 Tokyo
54261 26/01/13 11:24:14 Bet 30 Tokyo
54261 26/01/13 11:47:53 Bet 45 Kyoto
54261 26/01/13 12:59:38 Bet 100 Tokyo
54261 26/01/13 13:26:04 Bet 80 Osaka
54261 26/01/13 14:41:09 Bet 50 Tokyo
54261 26/01/13 14:53:12 Bet 50 Osaka
54261 26/01/13 15:15:05 Bet 50 Yokahama
54261 26/01/13 15:51:17 Bet 50 Tokyo
54261 26/01/13 16:15:02 Bet 120 Tokyo
54261 26/01/13 16:36:51 Bet 100 Kobe
54261 26/01/13 16:55:09 Bet 100 Tokyo
54261 26/01/13 18:07:17 Win -5000 Osaka
190

IT
191
CAN
ALL
DONE
WITH
SQL !
BE

SQL> select acct, tstamp, win_tstamp, txn_type, amt
2 from account_txns
3 MATCH_RECOGNIZE
4 ( partition by acct
5 order by tstamp
6 measures
7 Bet.tstamp Bet_tstamp,
8 win.tstamp win_tstamp
9 all rows per match
10 pattern ( Bet{10,} win )
11 define
12 Bet as
13 txn_type = 'Bet',
14 win as
15 txn_type = 'Win'
16 and last(Bet.tstamp)-first(Bet.tstamp) < interval '1' day
17 and win.tstamp - last(Bet.tstamp) < interval '3' day
18 and win.location != last(Bet.location)
19 ) 192

193
https://tinyurl.com/pattern-match

the default data access language
198

199
Hive
Spark
Tez
Presto
Drill
Hawq
Cloudera
Vertica
BigSQL
Jethro

ksql> CREATE STREAM ATM_TXNS (
account_id VARCHAR,
atm VARCHAR,
location STRUCT,
amount INT,
transaction_id VARCHAR)
WITH (
KAFKA_TOPIC='atm_txns',
VALUE_FORMAT='JSON');
ksql> SELECT ACCOUNT_ID, TRANSACTION_ID, AMOUNT, ATM FROM ATM_TXNS;
201

Thank you
youtube bit.ly/youtube-connor
blog connor-mcdonald.com
twitter @connor_mc_d

Tokyo APAC Groundbreakers tour - The Complete Java Developer

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Tokyo APAC Groundbreakers tour - The Complete Java Developer

Semelhante a Tokyo APAC Groundbreakers tour - The Complete Java Developer (20)

Mais de Connor McDonald

Mais de Connor McDonald (20)

Último

Último (20)

Tokyo APAC Groundbreakers tour - The Complete Java Developer

Notas do Editor