SlideShare uma empresa Scribd logo
1 de 24
23.08.14 - Page 1
Département
Office
SQL::Abstract::FromQuery
Building DB requests from Web queries
YAPC::EU::2014, Sofia
laurent.dami@justice.ge.ch
Etat de Genève, Pouvoir Judiciaire Département
Office
Power search : the "widget" way
Source: https://rt.cpan.org/Search/Build.html
Power search : the "parsed" way
Name : Smi*
Gender : M
Salary : > 4000
Job : ! programmer, analyst
Birth : BETWEEN 01.01.1970 AND 31.12.1990
Address: ! NULL
Agenda
• Background : SQL::Abstract & SQL::Abstract::More
• SQL::Abstract::FromQuery
– purpose
– API
• internals
– Regexp::Grammars
– multiple inheritance
Background : SQL::Abstract
SQL::Abstract::More
Translating Perl datastructures
into SQL
SQL::Abstract ('SQLA') : example
my $sqla = SQL::Abstract->new;
my ($sql, @bind) = $sqla->select(
$table_name,
[qw/col1 col2 col3/],
{ col1 => 'val1',
col2 => {-in => [qw/foo bar/]},
col3 => {-between => [qw/2012-01-01 2012-07-01/]},
col4 => {">" => 999},
col5 => {-like => 'foo%'},
},
[qw/col1 col2/],
);
# SELECT col1, col2, col3 FROM $table_name
# WHERE col1 = ? AND col2 IN (?, ?) AND col3 … etc
# ORDER BY col1, col2
Purpose
• generate SQL from Perl data structures
– direct from config or from user input
– avoid painful string manipulations with regex / join / etc.
• placeholders and bind values
– avoid datatype distinctions (strings / numbers /dates / etc.)
– avoid SQL injection
  used by DBIx::Class, DBIx::DataModel, …
"Special operators" in SQLA
• support for user-defined SQL extensions. For ex :
– fulltext index
– multivalued columns
– case-insensitive
– LIKE / GLOB / etc.
• useful for DBMS-independent queries in apps
– the "special operator" translates to DBMS-specific syntax
{ col1 => {-contains => [qw/YAPC Sofia/]},
col2 => {-anyval => [12, 34, 56]},
col3 => {-glob => 'foo*'},
}
SQL::Abstract::More ('SQLAM')
• fully compatible with SQLA
• named parameters
• more SQL constructs (group by, limit/offset, union/intersect, etc.)
• more syntax (aliases, joins, ordering direction, etc.)
my $sqla = SQL::Abstract::More->new;
my ($sql, @bind) = $sqla->select(
-columns => [qw/table1.col|ct1 table2.col|ct2 … /],
-from => $table_name,
-where => { col1 => 'val1',
col2 => {-in => [qw/foo bar/]},
… },
-union => [-columns => …, -from => …, -where => …],
-order_by => [qw/+col1 -col2/],
);
23.08.14 - Page 1
Département
Office
SQLA, SQLAM are for programmers
 what about users ?
SQL
Perl
datastructure
SQLA, SQLAMHTTP
query
SQLA::FromQuery
HTTP query
Name : Smi*
Gender : M
Salary : > 4000
Job : ! programmer, analyst
Birth : BETWEEN 01.01.1970 AND 31.12.1990
Address: ! NULL
Expected SQLA structure
{
Name => {-like => 'Smi%'},
Gender => 'M',
Salary => {'>' => 4000},
Job => {-not_in => [qw/programmer analyst/]},
Birth => {-between => [qw/1970-01-01 1990-12-31/]},
Address => {'<>' => undef},
}
SELECT * FROM people
WHERE Name LIKE 'Smi%'
AND Gender = 'M'
AND Salary > 4000
AND Job NOT IN ('programmer', 'analyst')
AND Birth BETWEEN 1970-01-01 AND 1990-12-31
AND Address IS NOT NULL
SYNOPSIS
my $parser = SQL::Abstract::FromQuery->new();
my $criteria = $parser->parse($http_query);
# $http_query is an object with a ->param() method
# .. or it can be just a hashref
my ($sql, @bind)
= $sqla->select($table, @cols, $criteria);
# or
my ($sql, @bind) = $sqlam->select(
-columns => @cols,
-from => $table,
-where => $criteria,
);
Options
my $parser = SQL::Abstract::FromQuery->new(
# additional components (optional)
-components => [qw/FR Oracle/],
# grammar rules for some specific fields (optional)
-fields => {
standard => [qw/field1 field2 .../],
bool => [qw/bool_field1/],
...
}
);
Components
• FR
– dates au format dd.mm.yyyy
– mots-clés en français (VRAI, FAUX, ENTRE, NUL, etc.)
– messages d'erreur en français
• Oracle
– queries on date and time
•  ["to_date(?, '$date_format')", $date]
•  ["to_date(?, '$time_format')", $date]
 WHERE D_BIRTH > TO_DATE('DD.MM.YYYY', ?)
• Contains
– generate fulltext queries
• … (user-defined)
Field types : which grammar to apply
• Standard
– a plain value (number, string, date or time).
– a list of values, separated by ','.
– a negated value or list of values; negation is expressed by ! or != or - or
<>
– a comparison operator <=, <, >=, > followed by a plain value
– the special word NULL
– BETWEEN val1 AND val2
– boolean values YES, NO, TRUE or FALSE
• Restricted
– string / bool / date / … (user-defined)
Internals
Regexp::Grammars
use Regexp::Grammars; qr{
<grammar: SQL::Abstract::FromQuery>
<rule: standard>
<MATCH=between> | <MATCH=op_and_value> | <MATCH=values>
<rule: values>
<[value]>+ % ,
<token: compare>
<= | < | >= | >
<rule: value>
<MATCH=null> | <MATCH=date> | <MATCH=time> | <MATCH=string>
<rule: date>
<day=(dd?)>.<month=(dd?)>.<year=(ddd?d?)>
| <year=(ddd?d?)>-<month=(dd?)>-<day=(dd?)>
Autoactions hooked to the grammar
sub between {
my ($self, $h) = @_;
return {-between => [$h->{min}, $h->{max}]};
}
sub values {
my ($self, $h) = @_;
my $n_values = @{$h->{value}};
return $n_values > 1 ? {-in => $h->{value}}
: $h->{value}[0];
}
Grammar Inheritance
<grammar: SQL::Abstract::FromQuery::FR>
<extends: SQL::Abstract::FromQuery>
<rule: null>
NULL?
<rule: between>
<SQL::Abstract::FromQuery::between>
| ENTRE (*COMMIT) (?: <min=value> ET <max=value> | <error:> )
<rule: bool>
O(?:UI)? (?{ $MATCH = 1 })
| V(?:RAI)? (?{ $MATCH = 1 })
| NO?N? (?{ $MATCH = 0 })
| F(?:AUX|ALSE)? (?{ $MATCH = 0 })
| Y(?:ES)? (?{ $MATCH = 1 })
Rules inheritance
package SQL::Abstract::FromQuery::Oracle;
use parent 'SQL::Abstract::FromQuery';
use mro 'c3';
¨sub date {
my ($self, $h) = @_;
my $date_format = $self->{date_format}
|| 'YYYY-MM-DD';
my $date = $self->next::method($h);
return  ["to_date(?, '$date_format')", $date];
}
23.08.14 - Page 1
Département
Office
Conclusion
Features
• flexible syntax for user input
– plain value
– list of values
– comparison operators
– patterns
– special constructs (BETWEEN, MATCH, etc.)
– Including SQLA "special ops"
• internationalization
• automate specific behaviours for some form fields
– data conversion
– inject special ops
Usage
• See App::AutoCRUD
  come to tomorrow's talk 

Mais conteúdo relacionado

Mais procurados

Database2011 MySQL Sharding
Database2011 MySQL ShardingDatabase2011 MySQL Sharding
Database2011 MySQL ShardingMoshe Kaplan
 
Airbnb Search Architecture: Presented by Maxim Charkov, Airbnb
Airbnb Search Architecture: Presented by Maxim Charkov, AirbnbAirbnb Search Architecture: Presented by Maxim Charkov, Airbnb
Airbnb Search Architecture: Presented by Maxim Charkov, AirbnbLucidworks
 
Rupy2012 ArangoDB Workshop Part2
Rupy2012 ArangoDB Workshop Part2Rupy2012 ArangoDB Workshop Part2
Rupy2012 ArangoDB Workshop Part2ArangoDB Database
 
mongodb-aggregation-may-2012
mongodb-aggregation-may-2012mongodb-aggregation-may-2012
mongodb-aggregation-may-2012Chris Westin
 
Importing Data into Neo4j quickly and easily - StackOverflow
Importing Data into Neo4j quickly and easily - StackOverflowImporting Data into Neo4j quickly and easily - StackOverflow
Importing Data into Neo4j quickly and easily - StackOverflowNeo4j
 
Bubbles – Virtual Data Objects
Bubbles – Virtual Data ObjectsBubbles – Virtual Data Objects
Bubbles – Virtual Data ObjectsStefan Urbanek
 
Languages and tools for web programming
Languages and tools for web  programmingLanguages and tools for web  programming
Languages and tools for web programmingAlamelu
 
MongoDB Aggregation MongoSF May 2011
MongoDB Aggregation MongoSF May 2011MongoDB Aggregation MongoSF May 2011
MongoDB Aggregation MongoSF May 2011Chris Westin
 
Grokking TechTalk #20: PostgreSQL Internals 101
Grokking TechTalk #20: PostgreSQL Internals 101Grokking TechTalk #20: PostgreSQL Internals 101
Grokking TechTalk #20: PostgreSQL Internals 101Grokking VN
 
Schema tools-and-trics-and-quick-intro-to-clojure-spec-22.6.2016
Schema tools-and-trics-and-quick-intro-to-clojure-spec-22.6.2016Schema tools-and-trics-and-quick-intro-to-clojure-spec-22.6.2016
Schema tools-and-trics-and-quick-intro-to-clojure-spec-22.6.2016Metosin Oy
 
Paul Dix (Founder InfluxDB) - Organising Metrics at #DOXLON
Paul Dix (Founder InfluxDB) - Organising Metrics at #DOXLONPaul Dix (Founder InfluxDB) - Organising Metrics at #DOXLON
Paul Dix (Founder InfluxDB) - Organising Metrics at #DOXLONOutlyer
 
N hidden gems in forge (as of may '17)
N hidden gems in forge (as of may '17)N hidden gems in forge (as of may '17)
N hidden gems in forge (as of may '17)Woonsan Ko
 
Neo4j: Import and Data Modelling
Neo4j: Import and Data ModellingNeo4j: Import and Data Modelling
Neo4j: Import and Data ModellingNeo4j
 
Scaling ArangoDB on Mesosphere DCOS
Scaling ArangoDB on Mesosphere DCOSScaling ArangoDB on Mesosphere DCOS
Scaling ArangoDB on Mesosphere DCOSMax Neunhöffer
 
Sql php-vibrant course-mumbai(1)
Sql php-vibrant course-mumbai(1)Sql php-vibrant course-mumbai(1)
Sql php-vibrant course-mumbai(1)vibrantuser
 

Mais procurados (18)

Database2011 MySQL Sharding
Database2011 MySQL ShardingDatabase2011 MySQL Sharding
Database2011 MySQL Sharding
 
Airbnb Search Architecture: Presented by Maxim Charkov, Airbnb
Airbnb Search Architecture: Presented by Maxim Charkov, AirbnbAirbnb Search Architecture: Presented by Maxim Charkov, Airbnb
Airbnb Search Architecture: Presented by Maxim Charkov, Airbnb
 
Rupy2012 ArangoDB Workshop Part2
Rupy2012 ArangoDB Workshop Part2Rupy2012 ArangoDB Workshop Part2
Rupy2012 ArangoDB Workshop Part2
 
mongodb-aggregation-may-2012
mongodb-aggregation-may-2012mongodb-aggregation-may-2012
mongodb-aggregation-may-2012
 
Importing Data into Neo4j quickly and easily - StackOverflow
Importing Data into Neo4j quickly and easily - StackOverflowImporting Data into Neo4j quickly and easily - StackOverflow
Importing Data into Neo4j quickly and easily - StackOverflow
 
Bubbles – Virtual Data Objects
Bubbles – Virtual Data ObjectsBubbles – Virtual Data Objects
Bubbles – Virtual Data Objects
 
Languages and tools for web programming
Languages and tools for web  programmingLanguages and tools for web  programming
Languages and tools for web programming
 
MongoDB Aggregation MongoSF May 2011
MongoDB Aggregation MongoSF May 2011MongoDB Aggregation MongoSF May 2011
MongoDB Aggregation MongoSF May 2011
 
Grokking TechTalk #20: PostgreSQL Internals 101
Grokking TechTalk #20: PostgreSQL Internals 101Grokking TechTalk #20: PostgreSQL Internals 101
Grokking TechTalk #20: PostgreSQL Internals 101
 
Introduction to AJAX
Introduction to AJAXIntroduction to AJAX
Introduction to AJAX
 
SFScon 2020 - Peter Hopfgartner - Open Data de luxe
SFScon 2020 - Peter Hopfgartner - Open Data de luxeSFScon 2020 - Peter Hopfgartner - Open Data de luxe
SFScon 2020 - Peter Hopfgartner - Open Data de luxe
 
Schema tools-and-trics-and-quick-intro-to-clojure-spec-22.6.2016
Schema tools-and-trics-and-quick-intro-to-clojure-spec-22.6.2016Schema tools-and-trics-and-quick-intro-to-clojure-spec-22.6.2016
Schema tools-and-trics-and-quick-intro-to-clojure-spec-22.6.2016
 
Paul Dix (Founder InfluxDB) - Organising Metrics at #DOXLON
Paul Dix (Founder InfluxDB) - Organising Metrics at #DOXLONPaul Dix (Founder InfluxDB) - Organising Metrics at #DOXLON
Paul Dix (Founder InfluxDB) - Organising Metrics at #DOXLON
 
N hidden gems in forge (as of may '17)
N hidden gems in forge (as of may '17)N hidden gems in forge (as of may '17)
N hidden gems in forge (as of may '17)
 
Neo4j: Import and Data Modelling
Neo4j: Import and Data ModellingNeo4j: Import and Data Modelling
Neo4j: Import and Data Modelling
 
Scaling ArangoDB on Mesosphere DCOS
Scaling ArangoDB on Mesosphere DCOSScaling ArangoDB on Mesosphere DCOS
Scaling ArangoDB on Mesosphere DCOS
 
Sql php-vibrant course-mumbai(1)
Sql php-vibrant course-mumbai(1)Sql php-vibrant course-mumbai(1)
Sql php-vibrant course-mumbai(1)
 
Search@airbnb
Search@airbnbSearch@airbnb
Search@airbnb
 

Semelhante a Sql abstract from_query

DBIx-DataModel v2.0 in detail
DBIx-DataModel v2.0 in detail DBIx-DataModel v2.0 in detail
DBIx-DataModel v2.0 in detail Laurent Dami
 
Ruby on Rails Oracle adaptera izstrāde
Ruby on Rails Oracle adaptera izstrādeRuby on Rails Oracle adaptera izstrāde
Ruby on Rails Oracle adaptera izstrādeRaimonds Simanovskis
 
Cassandra Summit 2015: Intro to DSE Search
Cassandra Summit 2015: Intro to DSE SearchCassandra Summit 2015: Intro to DSE Search
Cassandra Summit 2015: Intro to DSE SearchCaleb Rackliffe
 
DataStax: An Introduction to DataStax Enterprise Search
DataStax: An Introduction to DataStax Enterprise SearchDataStax: An Introduction to DataStax Enterprise Search
DataStax: An Introduction to DataStax Enterprise SearchDataStax Academy
 
Bag Of Tricks From Iusethis
Bag Of Tricks From IusethisBag Of Tricks From Iusethis
Bag Of Tricks From IusethisMarcus Ramberg
 
PerlApp2Postgresql (2)
PerlApp2Postgresql (2)PerlApp2Postgresql (2)
PerlApp2Postgresql (2)Jerome Eteve
 
Beyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the codeBeyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the codeWim Godden
 
It's Time to Get Ready for the Power of PL/SQL and JavaScript Combined
It's Time to Get Ready for the Power  of PL/SQL and JavaScript CombinedIt's Time to Get Ready for the Power  of PL/SQL and JavaScript Combined
It's Time to Get Ready for the Power of PL/SQL and JavaScript CombinedRodrigo Mesquita
 
PHP and Rich Internet Applications
PHP and Rich Internet ApplicationsPHP and Rich Internet Applications
PHP and Rich Internet Applicationselliando dias
 
Presto anatomy
Presto anatomyPresto anatomy
Presto anatomyDongmin Yu
 
[245] presto 내부구조 파헤치기
[245] presto 내부구조 파헤치기[245] presto 내부구조 파헤치기
[245] presto 내부구조 파헤치기NAVER D2
 
Write Faster SQL with Trino.pdf
Write Faster SQL with Trino.pdfWrite Faster SQL with Trino.pdf
Write Faster SQL with Trino.pdfEric Xiao
 
Lazy vs. Eager Loading Strategies in JPA 2.1
Lazy vs. Eager Loading Strategies in JPA 2.1Lazy vs. Eager Loading Strategies in JPA 2.1
Lazy vs. Eager Loading Strategies in JPA 2.1Patrycja Wegrzynowicz
 
Singpore Oracle Sessions III - What is truly useful in Oracle Database 12c fo...
Singpore Oracle Sessions III - What is truly useful in Oracle Database 12c fo...Singpore Oracle Sessions III - What is truly useful in Oracle Database 12c fo...
Singpore Oracle Sessions III - What is truly useful in Oracle Database 12c fo...Lucas Jellema
 
Oracle APEX Performance
Oracle APEX PerformanceOracle APEX Performance
Oracle APEX PerformanceScott Wesley
 
All Things Open 2016 -- Database Programming for Newbies
All Things Open 2016 -- Database Programming for NewbiesAll Things Open 2016 -- Database Programming for Newbies
All Things Open 2016 -- Database Programming for NewbiesDave Stokes
 
Angular js - 4developers 12 kwietnia 2013
Angular js - 4developers 12 kwietnia 2013Angular js - 4developers 12 kwietnia 2013
Angular js - 4developers 12 kwietnia 2013Marcin Wosinek
 

Semelhante a Sql abstract from_query (20)

DBIx-DataModel v2.0 in detail
DBIx-DataModel v2.0 in detail DBIx-DataModel v2.0 in detail
DBIx-DataModel v2.0 in detail
 
Ruby on Rails Oracle adaptera izstrāde
Ruby on Rails Oracle adaptera izstrādeRuby on Rails Oracle adaptera izstrāde
Ruby on Rails Oracle adaptera izstrāde
 
Cassandra Summit 2015: Intro to DSE Search
Cassandra Summit 2015: Intro to DSE SearchCassandra Summit 2015: Intro to DSE Search
Cassandra Summit 2015: Intro to DSE Search
 
DataStax: An Introduction to DataStax Enterprise Search
DataStax: An Introduction to DataStax Enterprise SearchDataStax: An Introduction to DataStax Enterprise Search
DataStax: An Introduction to DataStax Enterprise Search
 
Bag Of Tricks From Iusethis
Bag Of Tricks From IusethisBag Of Tricks From Iusethis
Bag Of Tricks From Iusethis
 
Php summary
Php summaryPhp summary
Php summary
 
PerlApp2Postgresql (2)
PerlApp2Postgresql (2)PerlApp2Postgresql (2)
PerlApp2Postgresql (2)
 
Beyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the codeBeyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the code
 
It's Time to Get Ready for the Power of PL/SQL and JavaScript Combined
It's Time to Get Ready for the Power  of PL/SQL and JavaScript CombinedIt's Time to Get Ready for the Power  of PL/SQL and JavaScript Combined
It's Time to Get Ready for the Power of PL/SQL and JavaScript Combined
 
PHP and Rich Internet Applications
PHP and Rich Internet ApplicationsPHP and Rich Internet Applications
PHP and Rich Internet Applications
 
Presto anatomy
Presto anatomyPresto anatomy
Presto anatomy
 
Tt subtemplates-caching
Tt subtemplates-cachingTt subtemplates-caching
Tt subtemplates-caching
 
[245] presto 내부구조 파헤치기
[245] presto 내부구조 파헤치기[245] presto 내부구조 파헤치기
[245] presto 내부구조 파헤치기
 
Write Faster SQL with Trino.pdf
Write Faster SQL with Trino.pdfWrite Faster SQL with Trino.pdf
Write Faster SQL with Trino.pdf
 
Lazy vs. Eager Loading Strategies in JPA 2.1
Lazy vs. Eager Loading Strategies in JPA 2.1Lazy vs. Eager Loading Strategies in JPA 2.1
Lazy vs. Eager Loading Strategies in JPA 2.1
 
Oracle Material.pdf
Oracle Material.pdfOracle Material.pdf
Oracle Material.pdf
 
Singpore Oracle Sessions III - What is truly useful in Oracle Database 12c fo...
Singpore Oracle Sessions III - What is truly useful in Oracle Database 12c fo...Singpore Oracle Sessions III - What is truly useful in Oracle Database 12c fo...
Singpore Oracle Sessions III - What is truly useful in Oracle Database 12c fo...
 
Oracle APEX Performance
Oracle APEX PerformanceOracle APEX Performance
Oracle APEX Performance
 
All Things Open 2016 -- Database Programming for Newbies
All Things Open 2016 -- Database Programming for NewbiesAll Things Open 2016 -- Database Programming for Newbies
All Things Open 2016 -- Database Programming for Newbies
 
Angular js - 4developers 12 kwietnia 2013
Angular js - 4developers 12 kwietnia 2013Angular js - 4developers 12 kwietnia 2013
Angular js - 4developers 12 kwietnia 2013
 

Mais de Laurent Dami

PowerPivot_model_Geneva_Justice_20230531.pptx
PowerPivot_model_Geneva_Justice_20230531.pptxPowerPivot_model_Geneva_Justice_20230531.pptx
PowerPivot_model_Geneva_Justice_20230531.pptxLaurent Dami
 
Studying geneva real estate prices using perl grammars
Studying geneva real estate prices using perl grammarsStudying geneva real estate prices using perl grammars
Studying geneva real estate prices using perl grammarsLaurent Dami
 
Yapceu2015 geneva courts
Yapceu2015 geneva courtsYapceu2015 geneva courts
Yapceu2015 geneva courtsLaurent Dami
 
DBIx::Class vs. DBix::DataModel
DBIx::Class vs. DBix::DataModelDBIx::Class vs. DBix::DataModel
DBIx::Class vs. DBix::DataModelLaurent Dami
 
Gestion documentaire pour les tribunaux genevois
Gestion documentaire pour les tribunaux genevoisGestion documentaire pour les tribunaux genevois
Gestion documentaire pour les tribunaux genevoisLaurent Dami
 
Working with databases in Perl
Working with databases in PerlWorking with databases in Perl
Working with databases in PerlLaurent Dami
 
Emacs, a performant IDE for Perl
Emacs, a performant IDE for PerlEmacs, a performant IDE for Perl
Emacs, a performant IDE for PerlLaurent Dami
 
Managing Geneva's law courts, from Cobol to Perl
Managing Geneva's law courts, from Cobol to PerlManaging Geneva's law courts, from Cobol to Perl
Managing Geneva's law courts, from Cobol to PerlLaurent Dami
 

Mais de Laurent Dami (8)

PowerPivot_model_Geneva_Justice_20230531.pptx
PowerPivot_model_Geneva_Justice_20230531.pptxPowerPivot_model_Geneva_Justice_20230531.pptx
PowerPivot_model_Geneva_Justice_20230531.pptx
 
Studying geneva real estate prices using perl grammars
Studying geneva real estate prices using perl grammarsStudying geneva real estate prices using perl grammars
Studying geneva real estate prices using perl grammars
 
Yapceu2015 geneva courts
Yapceu2015 geneva courtsYapceu2015 geneva courts
Yapceu2015 geneva courts
 
DBIx::Class vs. DBix::DataModel
DBIx::Class vs. DBix::DataModelDBIx::Class vs. DBix::DataModel
DBIx::Class vs. DBix::DataModel
 
Gestion documentaire pour les tribunaux genevois
Gestion documentaire pour les tribunaux genevoisGestion documentaire pour les tribunaux genevois
Gestion documentaire pour les tribunaux genevois
 
Working with databases in Perl
Working with databases in PerlWorking with databases in Perl
Working with databases in Perl
 
Emacs, a performant IDE for Perl
Emacs, a performant IDE for PerlEmacs, a performant IDE for Perl
Emacs, a performant IDE for Perl
 
Managing Geneva's law courts, from Cobol to Perl
Managing Geneva's law courts, from Cobol to PerlManaging Geneva's law courts, from Cobol to Perl
Managing Geneva's law courts, from Cobol to Perl
 

Último

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingEdi Saputra
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfOverkill Security
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDropbox
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024The Digital Insurer
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWERMadyBayot
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxRustici Software
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Orbitshub
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobeapidays
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Victor Rentea
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamUiPathCommunity
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays
 
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUKSpring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUKJago de Vreede
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Victor Rentea
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyKhushali Kathiriya
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Zilliz
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistandanishmna97
 

Último (20)

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUKSpring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 

Sql abstract from_query

  • 1. 23.08.14 - Page 1 Département Office SQL::Abstract::FromQuery Building DB requests from Web queries YAPC::EU::2014, Sofia laurent.dami@justice.ge.ch Etat de Genève, Pouvoir Judiciaire Département Office
  • 2. Power search : the "widget" way Source: https://rt.cpan.org/Search/Build.html
  • 3. Power search : the "parsed" way Name : Smi* Gender : M Salary : > 4000 Job : ! programmer, analyst Birth : BETWEEN 01.01.1970 AND 31.12.1990 Address: ! NULL
  • 4. Agenda • Background : SQL::Abstract & SQL::Abstract::More • SQL::Abstract::FromQuery – purpose – API • internals – Regexp::Grammars – multiple inheritance
  • 6. SQL::Abstract ('SQLA') : example my $sqla = SQL::Abstract->new; my ($sql, @bind) = $sqla->select( $table_name, [qw/col1 col2 col3/], { col1 => 'val1', col2 => {-in => [qw/foo bar/]}, col3 => {-between => [qw/2012-01-01 2012-07-01/]}, col4 => {">" => 999}, col5 => {-like => 'foo%'}, }, [qw/col1 col2/], ); # SELECT col1, col2, col3 FROM $table_name # WHERE col1 = ? AND col2 IN (?, ?) AND col3 … etc # ORDER BY col1, col2
  • 7. Purpose • generate SQL from Perl data structures – direct from config or from user input – avoid painful string manipulations with regex / join / etc. • placeholders and bind values – avoid datatype distinctions (strings / numbers /dates / etc.) – avoid SQL injection   used by DBIx::Class, DBIx::DataModel, …
  • 8. "Special operators" in SQLA • support for user-defined SQL extensions. For ex : – fulltext index – multivalued columns – case-insensitive – LIKE / GLOB / etc. • useful for DBMS-independent queries in apps – the "special operator" translates to DBMS-specific syntax { col1 => {-contains => [qw/YAPC Sofia/]}, col2 => {-anyval => [12, 34, 56]}, col3 => {-glob => 'foo*'}, }
  • 9. SQL::Abstract::More ('SQLAM') • fully compatible with SQLA • named parameters • more SQL constructs (group by, limit/offset, union/intersect, etc.) • more syntax (aliases, joins, ordering direction, etc.) my $sqla = SQL::Abstract::More->new; my ($sql, @bind) = $sqla->select( -columns => [qw/table1.col|ct1 table2.col|ct2 … /], -from => $table_name, -where => { col1 => 'val1', col2 => {-in => [qw/foo bar/]}, … }, -union => [-columns => …, -from => …, -where => …], -order_by => [qw/+col1 -col2/], );
  • 10. 23.08.14 - Page 1 Département Office SQLA, SQLAM are for programmers  what about users ? SQL Perl datastructure SQLA, SQLAMHTTP query SQLA::FromQuery
  • 11. HTTP query Name : Smi* Gender : M Salary : > 4000 Job : ! programmer, analyst Birth : BETWEEN 01.01.1970 AND 31.12.1990 Address: ! NULL
  • 12. Expected SQLA structure { Name => {-like => 'Smi%'}, Gender => 'M', Salary => {'>' => 4000}, Job => {-not_in => [qw/programmer analyst/]}, Birth => {-between => [qw/1970-01-01 1990-12-31/]}, Address => {'<>' => undef}, } SELECT * FROM people WHERE Name LIKE 'Smi%' AND Gender = 'M' AND Salary > 4000 AND Job NOT IN ('programmer', 'analyst') AND Birth BETWEEN 1970-01-01 AND 1990-12-31 AND Address IS NOT NULL
  • 13. SYNOPSIS my $parser = SQL::Abstract::FromQuery->new(); my $criteria = $parser->parse($http_query); # $http_query is an object with a ->param() method # .. or it can be just a hashref my ($sql, @bind) = $sqla->select($table, @cols, $criteria); # or my ($sql, @bind) = $sqlam->select( -columns => @cols, -from => $table, -where => $criteria, );
  • 14. Options my $parser = SQL::Abstract::FromQuery->new( # additional components (optional) -components => [qw/FR Oracle/], # grammar rules for some specific fields (optional) -fields => { standard => [qw/field1 field2 .../], bool => [qw/bool_field1/], ... } );
  • 15. Components • FR – dates au format dd.mm.yyyy – mots-clés en français (VRAI, FAUX, ENTRE, NUL, etc.) – messages d'erreur en français • Oracle – queries on date and time • ["to_date(?, '$date_format')", $date] • ["to_date(?, '$time_format')", $date]  WHERE D_BIRTH > TO_DATE('DD.MM.YYYY', ?) • Contains – generate fulltext queries • … (user-defined)
  • 16. Field types : which grammar to apply • Standard – a plain value (number, string, date or time). – a list of values, separated by ','. – a negated value or list of values; negation is expressed by ! or != or - or <> – a comparison operator <=, <, >=, > followed by a plain value – the special word NULL – BETWEEN val1 AND val2 – boolean values YES, NO, TRUE or FALSE • Restricted – string / bool / date / … (user-defined)
  • 18. Regexp::Grammars use Regexp::Grammars; qr{ <grammar: SQL::Abstract::FromQuery> <rule: standard> <MATCH=between> | <MATCH=op_and_value> | <MATCH=values> <rule: values> <[value]>+ % , <token: compare> <= | < | >= | > <rule: value> <MATCH=null> | <MATCH=date> | <MATCH=time> | <MATCH=string> <rule: date> <day=(dd?)>.<month=(dd?)>.<year=(ddd?d?)> | <year=(ddd?d?)>-<month=(dd?)>-<day=(dd?)>
  • 19. Autoactions hooked to the grammar sub between { my ($self, $h) = @_; return {-between => [$h->{min}, $h->{max}]}; } sub values { my ($self, $h) = @_; my $n_values = @{$h->{value}}; return $n_values > 1 ? {-in => $h->{value}} : $h->{value}[0]; }
  • 20. Grammar Inheritance <grammar: SQL::Abstract::FromQuery::FR> <extends: SQL::Abstract::FromQuery> <rule: null> NULL? <rule: between> <SQL::Abstract::FromQuery::between> | ENTRE (*COMMIT) (?: <min=value> ET <max=value> | <error:> ) <rule: bool> O(?:UI)? (?{ $MATCH = 1 }) | V(?:RAI)? (?{ $MATCH = 1 }) | NO?N? (?{ $MATCH = 0 }) | F(?:AUX|ALSE)? (?{ $MATCH = 0 }) | Y(?:ES)? (?{ $MATCH = 1 })
  • 21. Rules inheritance package SQL::Abstract::FromQuery::Oracle; use parent 'SQL::Abstract::FromQuery'; use mro 'c3'; ¨sub date { my ($self, $h) = @_; my $date_format = $self->{date_format} || 'YYYY-MM-DD'; my $date = $self->next::method($h); return ["to_date(?, '$date_format')", $date]; }
  • 22. 23.08.14 - Page 1 Département Office Conclusion
  • 23. Features • flexible syntax for user input – plain value – list of values – comparison operators – patterns – special constructs (BETWEEN, MATCH, etc.) – Including SQLA "special ops" • internationalization • automate specific behaviours for some form fields – data conversion – inject special ops
  • 24. Usage • See App::AutoCRUD   come to tomorrow's talk 