Advanced Web Programming outside of APEX

Advanced Web Programming outside of APEX Filipe Silva Silva . Filipe @ gmail . com Faculdade de Engenharia - Universidade do Porto Portugal

Advanced Web Programming outside of beyond APEX Filipe Silva Silva . Filipe @ gmail . com Faculdade de Engenharia - Universidade do Porto Portugal

Brief review of PL/SQL Web Toolkit Web Request processing Security Error management Improve DB use File upload/ download and generation Best Practices PL/SQL Gateways alternatives Topics

Main Aplication: Almost 2 million PL/SQL code lines 4 000 Java lines 120 000 calls to HTP Framework: 240 000 PL/SQL code lines 1 300 Java lines 11 000 calls to HTP Statistics for SIGARRA© Documents generated (at FEUP) September 17-9-2010 : 1 230 493 (high) 05-09-2010: 196 595 (low) month average: 555 868 /day

The HTTP Server receives a request from a client browser (1) …routes the request to the plsql gateway (adapter) …that ”maps” the request to an Oracle DB call (2) Web Request Processing http://<http_server_name>:<port>/<dad>/<procedure>?<var1>=<value_var1>

Web Request Processing (cont.) Use of PL/SQL Web Toolkit will create an output in a buffer array (3) If no unhandled exception was raise a COMMIT is made at the end of the PL/SQL call The plsql Gateway gets the OWA buffer content and sends it to the HTTP Server (some sanitization is made) (4) …thatdeliverstheresult to theclient. (5)

Allows to generate web content directly from Oracle (since 8.1.6) Has direct access to the DB (no-round trip) It’s easy to learn Keeps all the code in the same place (backups,…) You can keep all the files in DB PL/SQL Web Toolkit

PL/SQL Web Toolkit HTP – write to OWA (Web) output HTF – construct HTML (mainly) OWA_CACHE - writescachingheadertags OWA_UTIL - retrievesenvironment variables, write header,... OWA_COOKIE – send and receive browser cookies OWA_OPT_LOCK –handles optimistic lock OWA_IMAGE – get coordinates from user click in image map OWA_SEC – security programs OWA_CUSTOM - authorize function OWA_PATTERN – pattern matching OWA_MATCH – pattern matching OWA_TEXT - stringmanipulation WPG_DOCLOAD – BLOBs and BFILEs downloads

HTP.p OWA_UTIL.mime_header OWA_UTIL.redirect_url OWA_UTIL.status_line OWA_UTIL.http_header_close OWA_UTIL.get_procedure OWA_UTIL.get_cgi_env WPG_DOCLOAD.download_file PL/SQL Web Toolkit ,[object Object]

DECLARE (...) lv_cookies VARCHAR2 (2000) := 'TESTE=Teste;'; BEGIN li_version := OWA.initialize; OWA_COOKIE.init; -- so it will reload the cookies buffer la_name_arr (1) := 'HTTP_COOKIE'; la_value_arr (1) := lv_cookies; li_num := li_num +1; OWA.init_cgi_env (li_num, la_name_arr, la_value_arr); my_program(); irows := 99999999999; OWA.get_page (thepage => la_thepage, irows => li_irows); (...) END; Test Output with cookies OWA_COOKIE.cookie RECORD ( name varchar2(4096), vals vc_arr, num_vals integer ) PROCEDURE my_program IS (...) lrec_cookieOWA_COOKIE.cookie BEGIN (...) lrec_cookie := OWA_COOKIE.get ('TESTE'); IF lrec_cookie.num_vals > 0 THEN HTP.p ('Cookie value:' || lrec_cookie.vals (1)); ENDIF; (...) END;

Needs ACL (access control list) permissions in 11g Allows HTTP callouts from PL/SQL and SQL Takes a URL as parameter and returns corresponding Webpage Request :data is returned in the form of pieces, each piece is varchar2(2000) Request Pieces: all pieces obtained at once as a table type of Varchar2(2000) UTL_HTTP

DECLARE l_request UTL_HTTP.req; l_responseUTL_HTTP.resp; (…) BEGIN UTL_HTTP.set_proxy (proxy => ‘myproxy‘, no_proxy_domains => ‘dom1,dom2'); l_request := UTL_HTTP.begin_request ('http://someurl’ ,method => 'GET'); UTL_HTTP.set_header (l_request, 'User-Agent', 'Mozilla/4.0'); l_response := UTL_HTTP.get_response (l_request); DBMS_OUTPUT.put_line (' HTTP response: status code: ' || l_response.status_code || ' reason phrase: ' || l_response.reason_phrase); read_header (); read_lines (); UTL_HTTP.end_response (l_response); END; UTL_HTTP PROCEDURE read_header IS lv_header_name VARCHAR2 (256); lv_header_value VARCHAR2 (1024); BEGIN FOR i IN 1 .. UTL_HTTP.get_header_count (l_response) LOOP UTL_HTTP.get_header (l_response, i, lv_header_name, lv_header_value); DBMS_OUTPUT.put_line (lv_header_name || ': ' || lv_header_value); END LOOP; END read_header; PROCEDURE read_lines IS lv_line VARCHAR2 (32767); BEGIN LOOP UTL_HTTP.read_line (l_response, lv_line); DBMS_OUTPUT.put_line (lv_line); END LOOP; EXCEPTION WHEN UTL_HTTP.end_of_body THEN NULL; END read_lines;

Read RSS ReadWebservices (REST, …) ReadWebpages Get Documents via URL Example: crawling sites to index by Oracle Text … UTL_HTTP: Uses

HTTP is a “stateless” protocol Means credentials have to go with every request (i.e. cookies) Should use SSL for everything requiring authentication (https) Important answer status 200 – ok 204 – no content 304 – not modified 403 – forbidden 404 – not found About HTTP

Improve DB use (1) HTTP Requests Types HTTP request types supported: GET – parameters passed by query string (URL) POST – parameters passed by HTTP Request body (large amount of data, file uploads, password forms, etc..) HEAD – like GET but web server only returns header info So...only return header info and exit the procedure! (…) –- send header IF OWA_UTIL.get_cgi_env ('REQUEST_METHOD') = ‘HEAD' THEN RETURN; END IF; (…) –- send content

DAD configuration PlsqlErrorStyle – use ApacheStyle in production systems PlsqlExclusionList - Blacklist default: sys.*, dbms_*, utl_*, owa_*, owa.*, htp.*, htf.* PlsqlRequestValidationFunction – blacklist or whitelist booleanfunction_name (procedure_name IN varchar2) Security MOD_PLSQL

FUNCTIONmy_validation_check (procedure_nameINVARCHAR2) RETURNBOOLEAN IS li_check PLS_INTEGER; BEGIN SELECT 1 INTO li_check FROM my_pages WHERE name = UPPER (procedure_name); RETURNTRUE; EXCEPTION WHEN NO_DATA_FOUND THEN RETURNFALSE; END my_validation_check; PlsqlRequestValidationFunction

Use a whitelist Web procedure table Best practices (0)

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"> <html> <head> <title>Hello world</title> <link rel="stylesheet" href="mystyle.css"> </head> <body> Hello World! </body> </html> HTML

Why? HTML code in lowercase letters, and NEVER skip closing tags (like </p>) Parsing errors => stop parse! Not supported by totally of HTF (and HTP) XHTML

Make your own HTP and HTF packages The new HTP must call sys.HTP to send the buffer to the OWA Output and use the same buffer types Create procedures for new tags, opening and closing tags Delete/comment obsolete tags XHTML complain (lower case, close tags, …) Add ‘class’ attribute to tags (…) Allow buffer control Read without changing it (or clean it) Manipulation For instance, if you need to get some storage buffer (3 columns divs problem) Best practices (1)

Text files CSS JSON http://reseau.erasme.org/pl-sql-library-for-JSON http://sourceforge.net/projects/pljson/ etc.. XML RSS http://tylermuth.wordpress.com/2008/01/22/producing-rss-from-plsql/ etc.. What else?

PDF PL_FPDF - http://www.erasme.org/PL-FPDF,1337 PL/PDF - http://plpdf.com/ € AS_PDF - http://technology.amis.nl/blog/8650/as_pdf-generating-a-pdf-document-with-some-plsql Youcanalsomanipulateimage files using Oracle interMedia To createthumbnails Putwatermark (11.2) … What else?

QrCode http://sourceforge.net/projects/oracle-jutils/ ZIP (BLOBs) What else?

owa_util.mime_header('application/vnd.ms-excel'); Use SYLK or even HTML!! SYLK is a text-based interchange format for spreadsheets; it supports formulas, borders, fonts, point sizes, etc. Or CSV (Comma Separated File) (mime-type: text/csv) PL/SQL ExcelDocumentType http://www.jasonsdevelopercorner.com/?page_id=8 PL/XLS-XML € http://plpdf.com/ Excel

SVG (XML) FusionCharts http://www.jasonsdevelopercorner.com/?page_id=22 Needgraphics?

DAD configuration: PlsqlDocumentTablename - the table name for storing documents when file uploads are performed through the DAD Table definition: NAME VARCHAR(128) UNIQUE NOT NULL, MIME_TYPE VARCHAR(128), DOC_SIZE NUMBER, DAD_CHARSET VARCHAR(128), LAST_UPDATED DATE, CONTENT_TYPE VARCHAR(128), [[CONTENT LONG RAW], -- files with extensions in PlsqlUploadAsLongRaw [BLOB_CONTENT BLOB]] – other files (...) – can have more columns File upload

HTML: <form enctype="multipart/form-data" action=“my_pack.my_proc" method="POST"> <input type="file" name=“p_my_file“ /> (...) In DB: procedure my_proc (p_my_file IN varchar2, …) is lrt_docdocuments_table%ROWTYPE; begin select * into lrt_doc from documents_table where name=p_my_file; (…) File upload

Do not keep documents in the uploaded documents table Avoid collisions Provide distinct access privileges Backup polices Distinct Business logic … Best practice (2)

1) Send Header info OWA_UTIL.mime_header(prt_doc.mime_type, FALSE); -- specify mime_type and do not close header HTP.p ('Last-Modified: ' ||pv_last_updated); -- allow cache HTP.p ('Content-Length: ' ||prt_doc.doc_size); -- download info HTP.p ('Content-disposition: attachment; filename=“'|| prt_doc.filename||'” '); -- “save As” OWA_UTIL.http_header_close; --close header File download

2) Send Data 2a) If BLOB call: DECLARE lb_lob BLOB := prt_doc.content; -- due to need to be an IN OUT BEGIN WPG_DOCLOAD.download_file (p_blob => lb_lob); END; 2b) If CLOB: DECLARE lb_lob BLOB := clob_util.clob2blob (p_clob); BEGIN WPG_DOCLOAD.download_file (p_blob => lb_lob); END; File download

DECLARE offset PLS_INTEGER; len PLS_INTEGER; amount PLS_INTEGER := 32000; BEGIN len := NVL (DBMS_LOB.getlength (c_lob), 0); FOR i IN0 .. len / amount LOOP offset := (amount * i) + 1; HTP.prn(DBMS_LOB.SUBSTR(c_lob, amount , offset)); ENDLOOP; END; File download Or2b) If CLOB

FUNCTION clob2blob (p_clob IN CLOB) RETURN BLOB DETERMINISTICIS lbl_blob BLOB; li_dest_offset INTEGER := 1; li_src_offset INTEGER := 1; li_amount INTEGER := DBMS_LOB.lobmaxsize; ln_blob_csid NUMBER := DBMS_LOB.default_csid; li_lang_ctx INTEGER := DBMS_LOB.default_lang_ctx; li_warning INTEGER; BEGIN DBMS_LOB.createtemporary (lob_loc => lbl_blob, cache => TRUE, dur => DBMS_LOB.call); DBMS_LOB.converttoblob (dest_lob => lbl_blob ,src_clob => p_clob ,amount => li_amount ,dest_offset => li_dest_offset ,src_offset => li_src_offset ,blob_csid => ln_blob_csid ,lang_context => li_lang_ctx ,warning => li_warning); RETURN lbl_blob; END clob2blob; CLOB 2 BLOB

An Example: OWA_UTIL.mime_header ('text/css', FALSE); -- sends mime-type -- and doesn’t closes the header HTP.P ('Last-Modified: ' || lv_updated_date); HTP.P ('Cache-Control: public, s-maxage=300, max-age=86400'); -- or -- HTP.p ('Cache-Control: private, max-age=86400'); --or -- HTP.p (‘Cache-Control: no-cache’); --or even no-store OWA_UTIL.http_header_close; --close the header (…) Cache – Last Modified

-- Thu, 30 Sep 2010 16:03:22 GMT lv_updated_date :=TO_CHAR ( SYS_EXTRACT_UTC ( CAST (NVL (pd_last_modified, SYSDATE) AS TIMESTAMP)), 'Dy, DD Mon YYYY HH24:MI:SS', 'NLS_DATE_LANGUAGE=''AMERICAN''') || ' GMT'; Last-Modified

Then I can check if the browser has the document in cache IF OWA_UTIL.get_cgi_env ('HTTP_IF_MODIFIED_SINCE') = lv_updated_date THEN OWA_UTIL.status_line (nstatus => 304, creason => 'Not Modified', bclose_header => TRUE); RETURN; END IF; Improve DB use (2)

Can also use ETag: HTP.p('ETag: ' || lv_file_etag); -- or owa_cache.set_cache(p_etag, p_level); -- length(lv_file_etag)<= 55 And to check: OWA_UTIL.get_cgi_env('HTTP_IF_NONE_MATCH') --needs PlsqlCGIEnvironmentList HTTP_IF_NONE_MATCH Cache - ETag

-> if so return status 503 -> exit

Output HTTP (Response) header info

Output ContentBest Practices (3)Improve DB use (1+2)

If a page can only be access by a POST form -> check it! owa_util.get_cgi_env( 'REQUEST_METHOD' ) =‘POST’ and owa_util.get_cgi_env( 'HTTP_REFERER‘) is not null -- maybe even check that the referer is from the same site or an exact page … If a page is only to be access by a AJAX -> check it! OWA_UTIL.get_cgi_env ('X-Requested-With') is not null -- needs DAD PlsqlCGIEnvironmentList X-Requested-With But an hacker can avoid that if they know of it  Disallow Access

begin (…) exceptions (…) whenothersthen qem_web.register_error; end; Error Manager

Avoid reporting system messages that the normal user cannot understand but an hacker could use (Error message SQL Injection). (i.e SQLERRM, …) The errors should be reported to a system maintainer even if the user doesn’t report them. Should have enough information to replicate the error. We based our solution on Quest Error Manager (http://qem.inside.quest.com/index.jspa) Error Manager

Quest Error Manager Log 4 PLSQL Logger PL/SQL Log & DBugLibrary PLJ_LG … Error and Log Libraries

Have an error handler for when others in every procedure that will generate your web pages The ones that are to called by URL They are in a whitelist table, right? With PL/Scope (11g) you can have a script that check the use of your error report program inside the procedures Best Pratices (4)

The OWASP Top 10 Web Application Security Risks for 2010 are: A1: Injection A2: Cross-Site Scripting (XSS) A3: Broken Authentication and Session Management A4: Insecure Direct Object References A5: Cross-Site Request Forgery (CSRF) A6: Security Misconfiguration A7: Insecure Cryptographic Storage A8: Failure to Restrict URL Access A9: Insufficient Transport Layer Protection A10: Unvalidated Redirects and Forwards Security

One of the more devastating attacks on a web application One of the most common attacks Probably the most popular technique used for database intrusion caused by inefficient input validation Some real World examples (2005-2010): http://en.wikipedia.org/wiki/SQL_injection#Real-world_examples SQL injection: what?

PROCEDUREmy_bad_proc (v inVARCHAR2) IS TYPE cursortyp ISREFCURSOR; cursordual cursortyp; result_v VARCHAR2 (300); sql_stmt VARCHAR2 (300); BEGIN sql_stmt := 'select * from dual where dummy=''' || v || '''; DBMS_OUTPUT.put_line (sql_stmt); OPEN cursordual FOR sql_stmt; LOOP FETCH cursordual INTO result_v; EXITWHEN cursordual%NOTFOUND; DBMS_OUTPUT.put_line (result_v); ENDLOOP; END my_bad_proc; Sql injection: example BEGIN my_bad_proc(v=> 'aaa'' union select login from logins -- END; select * from dual where dummy='aaa' union select login from logins -- …… Mark Steven (…)

In Oracle when concatenating non-constants to be used as a statement in EXECUTE IMMEDIATE v_string; OPEN v_cursor FOR v_string; DBMS_SQL.parse (..., v_string, …) Safe code: SELECT … INTO … And above if not concatenating with non- constants OPEN … USING var1, var2,… EXECUTE IMMEDIATE …. USING var1, var2,… DBMS_SQL.bind_variable SQL Injection: where?

We need to convert the Text value to a SQL text Literal. If needed by design replace all the quotation marks with two quotations replace(v,'''','''''') -- for instance to allow O’Neil Use DBMS_ASSERT.ENQUOTE_LITERAL: Enclose the literal with quotation Check with DBMS_ASSERT.enquote_literal the validy catching the exception VALUE_ERROR it raises if invalid OR Use DBMS_ASSERT.enquote_literal to enclose the literal with in quotation marks and it raises VALUE_ERROR if invalid. Best Pratices (4)

PROCEDUREmy_good_proc1 (v inVARCHAR2) IS (…) lv_literal VARCHAR2(100):='''' || REPLACE (v, '''', '''''') || ''''; BEGIN BEGIN lv_literal :=sys.DBMS_ASSERT.enquote_literal(lv_literal ); EXCEPTION WHENVALUE_ERROR THENDBMS_OUTPUT.put_line ('Value entered invalid'); RETURN; END; sql_stmt := 'select * from dual where dummy=' || lv_literal; OPEN cursordual FOR sql_stmt; (…) END my_good_proc1; DBMS_ASSERT.enquote_literal (1)

PROCEDUREmy_good_proc2(v VARCHAR2) IS (…) BEGIN BEGIN sql_stmt := 'select * from dual where dummy=' ||sys.DBMS_ASSERT.enquote_literal(v); EXCEPTION WHENVALUE_ERROR THEN DBMS_OUTPUT.put_line ('Value entered invalid'); RETURN; END; OPEN cursordual FORsql_stmt; (...) END my_good_proc2; DBMS_ASSERT.enquote_literal (2)

PROCEDUREmy_good_proc2(v VARCHAR2) IS (…) BEGIN BEGIN sql_stmt := 'select * from dual where dummy=‘ ||sys.DBMS_ASSERT.enquote_literal(REPLACE (v, '''', '''''') ); EXCEPTION WHENVALUE_ERROR THEN DBMS_OUTPUT.put_line ('Value entered invalid'); RETURN; END; OPEN cursordual FORsql_stmt; (...) END my_good_proc2; DBMS_ASSERT.enquote_literal (2)

DBMS_ASSERT SIMPLE_SQL_NAME QUALIFIED_SQL_NAME SCHEMA_NAME SQL_OBJECT_NAME ENQUOTE_LITERAL – single quotes check ENQUOTE_NAME – double quotes check

Protect the database and web server (patches) Use minimum privileges AUTHID CURRENT_USER when possible Use of prepared statement Data API Sanitize Input: from users URL parameters Cookies avoid cursor injection (11g): DBMS_SQL.open_cursor(security_level => 2); SQL Injection: Prevention

XSS is a form of injection where the interpreter is the browser and attacks are “buried” in the HTML document Cross-Site Scripting (XSS)

Sanitize input HTML Escapeuntrusted data (htf.escape_sc) if HTML input allowed…check it Use the new Content Security Policy header Examples: If no external script and no inline-script X-Content-Security-Policy: allow 'self'; If inline script exists but no external script X-Content-Security-Policy: allow 'self'; options inline-script; Cross-Site Scripting (XSS): Prevention

Avoid iframe/frame X-Frame-Options: [deny | sameOrigin] X-Content-Security-Policy: frame-src: ... HTTPOnly cookie flag (OAS version >= 10.1.3.5.0) ... Cross-Site Scripting (XSS): Prevention (cont.)

Advanced Web Programming outside of APEX

Recomendados

Recomendados

Mais conteúdo relacionado

Último

Último (20)

Destaque

Destaque (20)

Advanced Web Programming outside of APEX