Overview of Intel(r) Quick Assist Technology and NGINX* and OpenSSL*. Authors: Joel Auernheimer & Joel Schuetze

1. Unleashing End-To-End TLS Security
Leveraging NGINX with Intel®
QuickAssist Technology and
OpenSSL-1.1.0

2. 1. Market Trends
2. Overview of Intel® QuickAssist Technology
3. NGINX* -1.10 HTTPS Serving Results
4. OpenSSL* -1.1.0 Asynchronous Feature?
5. Summary
Agenda!
Copyright © 2016 Intel Corporation. All rights reserved. Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and/or other countries.
*Other names and brands may be claimed as the property of others.

3. 3
Encryption: Market Trends
• Largest sources of streaming applications
(Netflix* & YouTube*) committed to using Https
• Video traffic continues to grow2
• Mobile Traffic grew 74% in 20152
More Encrypted Traffic
70% of Internet Traffic Latin &
North America will be Encrypted
in 20161
1 Source: 2016 Global Internet Phenomena, Latin & North America, Sandvine*
2 Source: 2016 Cisco* Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2015–2020 White Paper
• Privacy
• Video
• Mobile
• Cloud Computing
Copyright © 2016 Intel Corporation. All rights reserved. Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and/or other countries.
*Other names and brands may be claimed as the property of others.

4. 4
Enabling SDI for NFV & Hyperconvergence with
Intel® QuickAssist Technology
Compute
Network
Storage
App 1
App 2
App 3
QuickAssist
Static
Purpose-built, Proprietary
Resource Pool
Orchestration SW
App 1 App 2App 3
Dynamic Software Defined
Infrastructure
Compute
Network Storage
Copyright © 2016 Intel Corporation. All rights reserved. Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and/or other countries.
*Other names and brands may be claimed as the property of others.

5. 1. Market Trends
2. Overview of Intel® QuickAssist Technology
3. NGINX* -1.10 HTTPS Serving Results
4. OpenSSL* -1.1.0 Asynchronous Feature?
5. Summary
Agenda!
Copyright © 2016 Intel Corporation. All rights reserved. Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and/or other countries.
*Other names and brands may be claimed as the property of others.

6. Technology
What it is, How it’s deployed
• Security (symmetric
encryption and
authentication) for data in
flight and at rest
Bulk
Cryptography
• Secure Key Establishment
(asymmetric encryption,
digital signatures, key
exchange)
Public Key
Cryptography
• Lossless data compression
for data in flight and at rest
Compression
Intel® QuickAssist Technology integrates hardware acceleration of compute intensive workloads
(specifically, cryptography & compression) on Intel® Architecture Platform Technology
Chipset
Connects to CPU
via on-board PCI
Express* lanes
PCI
Express*
Plugin
Card
Connects to CPU
via off-board PCI
Express* lanes
(slot)
SoC
Connects to CPU
via on-chip
interconnect
Copyright © 2016 Intel Corporation. All rights reserved. Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and/or other countries.
*Other names and brands may be claimed as the property of others.

7. Intel® QuickAssist Technology: Crypto
Usage Model
• Network security (IPsec, SSL/TLS), hashing for data-
deduplication, encrypted storage
Symmetric (Bulk) Cryptography
• Ciphers (AES, 3DES/DES, RC4, KASUMI*, Snow 3G)
• Wireless ciphers: KASUMI, Snow 3G
• Message digest/hash (MD5, SHA1, SHA2)
• Authentication (HMAC, AES in modes XCBC-MAC, CBC-MAC,
CMAC, GMAC)
• Algorithm chaining (one cipher and one hash/authentication in
a single operation)
• Authenticated encryption (AES-GCM, AES-CCM)
Asymmetric (Public Key) Cryptography
• RSA key generation, encryption/decryption and digital
signature generation/verification
• Elliptic Curve Cryptography: ECDSA, ECDHE
• Modular exponentiation for Diffie-Hellman (DH)
• DSA parameter generation and digital signature generation/
verification
Intel® DH8955 PCH
(Gen 1)
Network Security Protocols
TLS @ 4k records 50 Gbps
Inline IPSec @ 512B
Public Key Encryption
RSA Decrypt 2K 40k Ops/sec
TLS Handshakes with PFS
ECDHE + RSA2K:
17.6K Ops/sec
TLS Handshakes with PFS
ECDHE + ECDSA:
20K Ops/sec
Wireless Ciphers
ZUC/Snow 3G/KASUMI* F82 20 Gbps
Snow 3G 20 Gbps
Cipher or Hash Only
AES XTS 50Gbs
SHA1, SHA256, SHA3, MD5 50Gbs
1. Bulk Crypto uses AES-CBC-HMAC SHA1/256/SHA3 or AES GCM
2. KASUMI-F8 (encryption) at 320B packets, 15 Gbs for 40B packets
3. PFS = Perfect Forward Secrecy
Results have been estimated based on internal Intel analysis and are provided for
informational purposes only. Any difference in system hardware or software design or
configuration may affect actual performance.
Copyright © 2016 Intel Corporation. All rights reserved. Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and/or other countries.
*Other names and brands may be claimed as the property of others.

8. 8
Intel® QuickAssist Technology: Data Compression
Usage Model
• Big data Acceleration
• WAN acceleration
• Http compression
• File System
• Databases
Compression and Decompression Using the Following Algorithms
• DEFLATE: LZ77 compression followed by Huffman coding, with
a gzip or zlib header
Other Features
• Support for multiple history sizes with deflate
• Support for stateful (de)compression
• Supports Batch and Pack (seekable compression)
• Supports Chaining of Compression & Crypto for data integrity
(SHAx, MD5) and data encryption with AES XTS, CBC
• Supports “Compress & Verify”
Performance Intel® DH8955
PCH (Gen 1)
Compression 24 Gbs1
Decompression 24 Gbs1
Compression + Decompression 24 Gbs1
1. Dynamic Deflate Level 1 using 64KB buffer size
2. Measured using Calgary and Silesia corpi
Results have been estimated based on internal Intel analysis and are provided for
informational purposes only. Any difference in system hardware or software design or
configuration may affect actual performance.
#NetworkZeroToSecureCopyright © 2016 Intel Corporation. All rights reserved. Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and/or other countries.
*Other names and brands may be claimed as the property of others.

9. 1. Market Trends
2. Overview of Intel® QuickAssist Technology
3. NGINX* -1.10 HTTPS Serving Results
4. OpenSSL* -1.1.0 Asynchronous Feature?
5. Summary
Agenda!
Copyright © 2016 Intel Corporation. All rights reserved. Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and/or other countries.
*Other names and brands may be claimed as the property of others.

10. Web Server Benchmark – Logical view
10
Client(s)
HTTPS
Server
H1ps request, Session
Establishment
Encrypted Web Page
Close ConnecAon
NGINX* + OpenSSL-1.1.0* + Intel®
QuickAssist Technology Engine on
Standard Linux*
HTTPS Requests
Encrypted ConnecKons
Bulk Throughput Test
Application Data 0 Byte File
Certificates RSA-2K
ECDHE-RSA-2K (P256)
ECDHE-ECDSA (P256)
Protocol TLS v1.2
Cipher Suite AES_128_CBC_SHA
Copyright © 2016 Intel Corporation. All rights reserved. Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and/or other countries.
*Other names and brands may be claimed as the property of others.

11. Web Server Key Exchange Details
Source as of August 2016: Intel internal measurements with one E5-2699 v3 (18C, 2.3GHz, 145W), HSW-C1, HT & Turbo Enabled, NUMA & COD mode, Linux kernel: 3.11.10, gcc-4.8.3-7, BIOS GRNDCRB1.86B.
0032.R02.1405090848, OpenSSL-1.1.0. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark* and MobileMark*, are
measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you
in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. Any difference in system hardware or software design or configuration may affect actual performance.
Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. For more information
go to http://www.intel.com/performance
1
1
Intel® QuickAssist Technology + OpenSSL*-1.1.0
Async gain over software!
RSA-2K ~5.6x!
ECDHE-RSA-2K ~3.7x!
ECDHE-ECDSA ~1.25x*!
* further improvements in development.

12. 1. Market Trends
2. Overview of Intel® QuickAssist Technology
3. NGINX* -1.10 HTTPS Serving Results
4. OpenSSL* -1.1.0 Asynchronous Feature?
5. Summary
Agenda!
Copyright © 2016 Intel Corporation. All rights reserved. Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and/or other countries.
*Other names and brands may be claimed as the property of others.

13. Asynchronous OpenSSL*
OpenSSL, an open-source implementation of the TLS protocols, is estimated to encrypt 66% of all
active websites1
Asynchronous OpenSSL enables applications to get increased parallelism and throughput at lower
costs
1 Source: Netcraft* April 2014 Web Server Survey
Post
Descriptor(s)
Thread
Completion
Notification(s)
Synchronous Mode, Multiple Threads/Processes
Context Swap
Post
Descriptor(s)
Thread
Completion
Notification(s)
Asynchronous Mode, Single Thread/Process
Copyright © 2016 Intel Corporation. All rights reserved. Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and/or other countries.
*Other names and brands may be claimed as the property of others.

14. New Features in 1.1.0 (associated with async)
JOB Infrastructure Operations https://www.openssl.org/docs/manmaster/crypto/ASYNC_start_job.html
• Adds the ability for the engine to pause an existing operation flow (such as an SSL struct/
connection) and return to the calling application an error that indicates the JOB has been
PAUSED. The application can then activate the JOB again at any time through the JOB
infrastructure. (these are very light weight contexts).
Event Notifications: https://www.openssl.org/docs/manmaster/ssl/SSL_get_all_async_fds.html
• These API’s access the event file descriptors associated with a SSL connection. These allow
the application to query the active FD’s and subsequently use them in a call to select/epoll/
poll.
SYNC/ASYNC Intel® QuickAssist Technology support:
https://www.openssl.org/docs/manmaster/ssl/SSL_CTX_set_mode.html
• The API simply allows enabling a connection for async support.
Pipelining: https://www.openssl.org/docs/manmaster/ssl/SSL_CTX_set_split_send_fragment.html
• Allows multiple SSL records (or encryption JOBs) to be submitted to the engine as a single
request/API invocation.
Copyright © 2016 Intel Corporation. All rights reserved. Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and/or other countries.
*Other names and brands may be claimed as the property of others.

15. Asynchronous Advantages
Source as of August 2016: Intel internal measurements with one E5-2699 v3 (18C, 2.3GHz, 145W), HSW-C1, HT & Turbo Enabled, NUMA & COD mode, Linux kernel: 3.11.10, gcc-4.8.3-7, BIOS GRNDCRB1.86B.
0032.R02.1405090848, OpenSSL-1.1.0. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark* and MobileMark*, are
measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you
in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. Any difference in system hardware or software design or configuration may affect actual performance.
Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. For more information
go to http://www.intel.com/performance
Asynchronous support in
OpenSSL*-1.1.0, combined
with Intel® QuickAssist
Technology and
NGINX*-1.10 results in a
5.6 gain over software
and 3.9 gain over synch
invocation for the same
configuration.
Note: Measured using a single worker
process per logical core. Configurations
were kept consistent for each mode of
operation; software, sync and async

16. OpenSSL*-1.1.0 added JOB/Fibre support
Encapsulates a running piece of work such as an
SSL connection and allows it to be:
Paused, woken up and signaling status
JOB’s will be used to manage SSL connections
Allowing a connection to pause at any point during
processing.
Jump directly back to that execution point
JOB’s will have associated FD’s that an application
can epoll/poll/select on
JOB’s are based on the POSIX primitives:
makecontext, getcontext, setjmp, longjmp
Note: Start and Destroy context switches will be amortized
across multiple calls.
1
6
EVP API
ASYNC_start_job
QAT EngineASYNC JOB
RSA_sign
QAT Driver
cpaCyRsaDecrypt (non-blocking)
ASYNC_pause_job
ASYNC_FIBRE_swapcontext
ASYNC_FIBRE_swapcontext
RSA_sign
ASYNC_start_job
ASYNC_FIBRE_swapcontext
rsa_decrypt
delivery result
return RSA sign result
Copyright © 2016 Intel Corporation. All rights reserved. Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and/or other countries.
*Other names and brands may be claimed as the property of others.

17. QuickAssist Technology
Features
Intel QuickAssist Technology Engine released: https://github.com/01org/QAT_Engine
Asymmetric PKE Offload
• RSA Support with PKCS1 Padding for Key Sizes 1024/2048/4096.
• DH Support for Key Sizes 768/1024/1536/2048/3072/4096.
• DSA Support for Key Sizes 160/1024, 224/2048, 256/2048, 256/3072.
• ECDH Support for the following curves:
• NIST Prime Curves: P-192/P-224/P-256/P-384/P-521.
• NIST Binary Curves: B-163/B-233/B-283/B-409/B-571.
• NIST Koblitz Curves: K-163/K-233/K-283/K409/K-571.
• ECDSA Support for the following curves:
• NIST Prime Curves: P-192/P-224/P-256/P-384/P-521.
• NIST Binary Curves: B-163/B-233/B-283/B-409/B-571.
• NIST Koblitz Curves: K-163/K-233/K-283/K409/K-571.
Symmetric Chained Cipher Offload
• AES128-CBC-HMAC-SHA1/AES256-CBC-HMAC-SHA1.
• AES128-CBC-HMAC-SHA256/AES256-CBC-HMAC-SHA256.
TLS/SSL specific algorithms
• Pseudo Random Function (PRF) offload.
• For TLSv1.2, v1.1 and SSLv3.0
Future Work
• Pipeline support
Copyright © 2016 Intel Corporation. All rights reserved. Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and/or other countries.
*Other names and brands may be claimed as the property of others.

18. Modifications to NGINX* to
support Event architecture in
OpenSSL*-1.1.0
NGINX configuration:
• ‘ssl_asynch on;’ added to allow enabling per server block. This will enable all SSL connections for that server to run
asynchronously if supported.
ngx_connection_t:
• Each connection is aware of FD’s associated with an SSL_CTX (typically one).
NGINX event notifications:
• JOB FD’s will notify NGINX of data availability on an asynchronous JOB from OpenSSL. As such they get added to
the top level epoll FD instance.
SSL connection events:
• Are treated as another event source, in a similar manner to socket read/write. To re-enable the JOB (source of the
event) NGINX simply calls the associated SSL_* API (SSL_read, SSL_write, SSL_do_handshake etc.)
Copyright © 2016 Intel Corporation. All rights reserved. Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and/or other countries.
*Other names and brands may be claimed as the property of others.

19. 1. Market Trends
2. Overview of Intel® QuickAssist Technology
3. NGINX* -1.10 HTTPS Serving Results
4. OpenSSL* -1.1.0 Asynchronous Feature?
5. Summary
Agenda!
Copyright © 2016 Intel Corporation. All rights reserved. Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and/or other countries.
*Other names and brands may be claimed as the property of others.

20. Summary
• Intel® QuickAssist Technology + NGINX*-1.10 + OpenSSL*-1.1.0 provide vast performance improvements
for HTTPs transactions
• Engage with Intel Today to Optimize Crypto & Compression in your Workloads
For further resources/information:
• For more information on Intel® QuickAssist Technology & Intel® QuickAssist Software Solutions can be found here:
• Software Package available at 01.org: Intel QuickAssist Technology | 01.org
• Intel® QuickAssist Technology engine for OpenSSL-1.1.0: https://github.com/01org/QAT_Engine
• For more details on Intel® QuickAssist Technology visit: http://www.intel.com/quickassist
• Intel Network Builders: https://networkbuilders.intel.com/ecosystem
• Intel QuickAssist Chipset: http://ark.intel.com/products/80372/Intel-DH8955-PCH
• Intel QuickAssist Adapter: http://ark.intel.com/products/79483/Intel-QuickAssist-Adapter-8950
• OpenSSL 1.1.0 to Support Intel® QuickAssist and other optimizations
• http://www.servethehome.com/openssl-1-1-0-quickassist-optimizations/
• OpenSSL Webinar: https://www.brighttalk.com/webcast/12229/172667
2
0
Copyright © 2016 Intel Corporation. All rights reserved. Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and/or other countries.
*Other names and brands may be claimed as the property of others.

21. Copyright © 2016 Intel Corporation. All rights reserved. Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and/or other countries.
*Other names and brands may be claimed as the property of others.
Questions?!