DevoxxFR 2024 Reproducible Builds with Apache Maven
JSON SQL Injection and the Lessons Learned
1. JSON SQL Injection
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
and
the Lessons Learned
DeNA Co., Ltd.
Kazuho Oku
1
2. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Who am I?
Field:
⁃ network and web-related architecture
Works:
⁃ Palmscape / Xiino (web browser for Palm OS)
⁃ Author of various Perl modules
• Class::Accessor::Lite, HTTP::Parser::XS,
Parallel::Prefork, Server::Starter, Starlet,
Test::Mysqld, ...
⁃ Author of various MySQL extensions
• Q4M, mycached, ...
⁃ Also the author of:
• JSX, picojson, ...
2
3. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Agenda
What is an SQL Query Builder?
JSON SQL Injection
SQL::QueryMaker and strict mode of SQL::Maker
The Lessons Learned
3
4. What is an SQL Query Builder?
4
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
5. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
What is an SQL Query Builder?
is a library for building SQL queries
⁃ exists below or as a part of an object-relational
mapping library (ORM)
• ORM understands the semantics of the database
schema / query builder does not
5
Application
ORM Library
SQL Query Builder
Database API (DBI)
Database Driver (DBD)
6. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Popular Implementations
SQL::Abstract
SQL::Interp
SQL::Maker
⁃ used by Teng
⁃ is today's main focus
6
Application
ORM Library
SQL Query Builder
Database API (DBI)
Database Driver (DBD)
7. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Query Builders are Great!
Easy to build query conditions
⁃ by using hashrefs and arrayrefs
⁃ arrayref is considered as IN queries
• e.g. foo => [1,2,3] becomes `foo` IN (1,2,3)
⁃ hashref is considered as (operator, value) pair
• e.g. foo => { '<', 30 } becomes `foo`<30
⁃ the API is common to the aforementioned query
builders
7
8. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Examples
use SQL::Maker;
...
$maker->select('user', '*', { name => 'tokuhirom' });
# => SELECT * FROM `user` WHERE `name`='tokuhirom';
$maker->select('user', '*', { name => [ 'tokuhirom', 'yappo' ] });
# => SELECT * FROM `user` WHERE `name` IN ('tokuhirom','yappo');
$maker->select('user', '*', { age => { '>' => 30 } });
# => SELECT * FROM `user` WHERE `age`>30;
$maker->delete('user', { name => 'sugyan' });
# => DELETE FROM `user` WHERE `name`='sugyan';
8
9. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Examples (cont’d)
use SQL::Maker;
...
$maker->select('user', '*', { name => [ 'tokuhirom', 'yappo' ] });
# => SELECT * FROM `user` WHERE `name` IN ('tokuhirom','yappo');
$maker->select('user', '*', { name => [] });
# => SELECT * FROM `user` WHERE 1=0;
$maker->select('user', '*', {
age => 30,
sex => 0, # male
});
# => SELECT * FROM `user` WHERE `age`=30 AND `sex`=0;
9
10. So What is JSON SQL Injection?
10
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
11. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
The Problem
User-supplied input might not be the expected type
# this API returns the entries of a blog (specified by $json->{blog_id})
my $json = decode_json($input);
my ($sql, @binds) = $maker->select(
'blog_entries', '*', { id => $json->{blog_id} });
my $rows = $dbi->selectall_arrayref($sql, { Slice => {} }, @binds);
send_output_as_json([ map { +{
entry_id => $_->{id},
entry_title => $_->{title},
} } @$rows ]);
11
What if $json->{name} was not a scalar?
12. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
The Problem (cont’d)
Will return a list of all blog entries if the supplied
JSON was: { "blog_id": { "!=": -1 } }
# this API returns the entries of a blog (specified by $json->{blog_id})
my $json = decode_json($input);
# generated query: SELECT * FROM `blog_entries` WHERE `id`!=-1
my ($sql, @binds) = $maker->select(
'blog_entries', '*', { id => $json->{blog_id} });
my $rows = $dbi->selectall_arrayref($sql, { Slice => {} }, @binds);
send_output_as_json([ map { +{
entry_id => $_->{id},
entry_title => $_->{title},
} } @$rows ]);
12
13. Can it be used as an attack vector?
13
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
15. Information Leakage in a Twitter-like App.
# this API returns the tweets of a user specified by $json->{user_id}, who is following the authenticating user
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
if (! is_following($session->{user_id}, $json->{user_id})) {
return "cannot return the tweets of an user who is not following you";
}
my ($sql, @binds) = $maker->select('tweet', '*', { user => $json->{user_id} });
My $rows = $dbi->selectall_arrayref($sql, { Slice => {} }, @binds);
send_output_as_json([ map { ... } @$rows ]);
sub is_following {
my ($user, $following) = @_;
# builds query: SELECT * FROM following WHERE user=$user AND following=$following
my ($sql, @binds) = $maker->select(
'following', '*', { user => $user, following => $following });
my $rows = $dbi->selectall_arrayref($sql, @binds);
return @$rows != 0;
}
15
16. Information Leakage in a Twitter-like App.
(cont'd)
# in case the JSON is: { user_id: { "!=": -1 } }
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
if (! is_following($session->{user_id}, $json->{user_id})) {
return "cannot return the tweets of an user who is not following you";
}
# generated query is SELECT * FROM `tweet` WHERE `user`!=-1, returns tweets of all users
my ($sql, @binds) = $maker->select('tweet', '*', { user => $json->{user_id} });
My $rows = $dbi->selectall_arrayref($sql, { Slice => {} }, @binds);
send_output_as_json([ map { ... } @$rows ]);
sub is_following {
my ($user, $following) = @_;
# the generated query becomes like bellow and the function likely returns TRUE:
# SELECT * FROM `following` WHERE `user`=$user AND `following`!=-1
my ($sql, @binds) = $maker->select(
'following', '*', { user => $user, following => $following });
my $rows = $dbi->selectall_arrayref($sql, @binds);
return @$rows != 0;
}
16
17. Is the problem in the SQL query builders
using hash for injecting arbitrary operators?
17
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
19. Handling of Array is problematic as well
# in case the JSON is: { user_id: [ 12, 34 ] }, returns tweets of both users if either is following the authenticating
user
if (! is_following($session->{user_id}, $json->{user_id})) {
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
return "cannot return the tweets of an user who is not following you";
}
# generates: SELECT * FROM `tweet` WHERE `user` IN (12,34)
my ($sql, @binds) = $maker->select('tweet', '*', { user => $json->{user_id} });
My $rows = $dbi->selectall_arrayref($sql, { Slice => {} }, @binds);
send_output_as_json([ map { ... } @$rows ]);
sub is_following {
my ($user, $following) = @_;
# generates: SELECT * FROM `following` WHERE `user`=$user AND `following` IN (12,34)
my ($sql, @binds) = $maker->select(
'following', '*', { user => $user, following => $following });
my $rows = $dbi->selectall_arrayref($sql, @binds);
return @$rows != 0;
}
19
20. Does the same problem exist in web
application frameworks written in
programming languages other than Perl?
20
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
21. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Yes.
Many web application frameworks (e.g. Ruby
on Rails) automatically convert condition
specified by an array into an IN query.
21
22. Is the problem only related to the handling
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
of JSON?
22
24. Query decoders may return nested data
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
in case of PHP and Ruby on Rails
query?id=1 => { id: 1 }
query?id[]=1&id[]=2 => { id: [1, 2] }
query?user[name]=yappo => { user: { name: "yappo" } }
⁃ also when using Data::NestedParams in Perl
Catalyst switches from scalars to using arrayrefs
when the property is defined more than once
query?id=1 => { id => 1 }
query?id=1&id=2 => { id => [1, 2] }
not the case for CGI.pm and Plack
24
25. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
SQL::QueryMaker
and
the strict mode of SQL::Maker
25
26. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Overview of SQL::QueryMaker
Provides a fail-safe API
⁃ provides functions to specify the SQL operators;
that return blessed refs to represent them
(instead of using arrayrefs / hashrefs)
sql_eq(name => 'yappo') # `name`='yappo'
sql_in(id => [ 123, 456 ]) # `id` IN (123,456)
sql_and([ # `sex`=1 AND `age`<=30
sex => 1, # female
age => sql_ge(30),
])
Added strict mode to SQL::Maker
⁃ raises error when arrayref / hashref is given as a
condition
26
27. The snippet becomes safe in strict mode
# this API returns the tweets of a user specified by $json->{user_id}, who is following the authenticating user
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
my $maker = SQL::Maker->new(..., strict => 1);
if (! is_following($session->{user_id}, $json->{user_id})) {
return "cannot return the tweets of an user who is not following you";
}
# in strict mode, SQL::Maker raises an error if $json->{user_id} is not a scalar
my ($sql, @binds) = $maker->select('tweet', '*', { user => $json->{user_id} });
My $rows = $dbi->selectall_arrayref($sql, { Slice => {} }, @binds);
send_output_as_json([ map { ... } @$rows ]);
sub is_following {
my ($user, $following) = @_;
# ditto as above
my ($sql, @binds) = $maker->select(
'following', '*', { user => $user, following => $following });
my $rows = $dbi->selectall_arrayref($sql, @binds);
return @$rows != 0;
}
27
28. Existing code may not work under strict mode
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
my $maker = SQL::Maker->new(..., strict => 1);
# equality comparison works as is
$maker->select(..., { foo => 123 });
# non-equality comparison needs to be rewritten
$maker->select(..., { foo => [ 123, 456 ]);
=> $maker->select(..., { foo => sql_in([ 123, 456 ]) });
$maker->select(..., { foo => { '<' => 30 } });
=> $maker->select(..., { foo => sql_lt(30) });
28
29. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Supported in Teng >= 0.24
my $teng = My::DB->new({
...,
sql_builder_args => {
strict => 1,
}
,});
29
30. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
The Lessons Learned
30
31. Always validate the type of the input
do not forget to validate the type of the input
⁃ even if you do not need to check the value of
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
the input
checks can be done either within the Controller or
within the Model (in case of SQL::Maker)
⁃ validation in the Controller is preferable
31
32. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Write fail-safe code
do not change the behavior based on the type of
the input
⁃ instead, provide different method for each type
of the input
⁃ e.g. sql_eq vs. sql_in
32
33. Use blessed refs for dynamic behavior
use blessed refs in case you need to change the
behavior based on the type of the input
⁃ this is a common idiom; many template engines
use blessed refs to determine whether if a string
is already HTML-escaped
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
# in case of Text::MicroTemplate
sub escape_html {
my $str = shift;
return ''
unless defined $str;
return $str->as_string
if ref $str eq 'Text::MicroTemplate::EncodedString';
$str =~ s/([&><"'])/$_escape_table{$1}/ge;
return $str;
}
33
34. Use blessed refs for dynamic behavior (cont'd)
do not use serialization libraries with support for
blessed objects (e.g. YAML or Storable) for user
input
⁃ such use may lead to XSS or other code
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
injection vulnerability
⁃ always only accept scalars / arrays / hashes and
validate their type and value
34
35. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Thanks to
Toshiharu Sugiyama
⁃ original reporter of the issue
⁃ http://developers.mobage.jp/blog/2014/7/3/jsonsql-injection
@tokuhirom, @cho45
⁃ for coordinating / working on the fix
@miyagawa
⁃ for the behavior of Catalyst, Ruby, etc.
@ockeghem
⁃ for looking into other impl. sharing the problem
⁃ http://blog.tokumaru.org/2014/07/json-sql-injectionphpjson.
html
35