AWS WAF introduction and live demo - Pop-up Loft Tel Aviv
•Transferir como PPTX, PDF•
5 gostaram•2,435 visualizações
Distributed application security at scale with AWS WAF. Learn about how the AWS WAF can help protect your web and application services and run through a demo showing how the WAF blocks IP addresses, prevents SQL Injection, and how you can use string matching to defend against Bots, Crawlers, and XSS .
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Destaque
Destaque (6)
Semelhante a AWS WAF introduction and live demo - Pop-up Loft Tel Aviv
Semelhante a AWS WAF introduction and live demo - Pop-up Loft Tel Aviv (20)
Mais de Amazon Web Services
Mais de Amazon Web Services (20)
Último
Último (20)
AWS WAF introduction and live demo - Pop-up Loft Tel Aviv
- 1. AWS WAF Tel Aviv Loft 9 March 2016 Tom Witman, Business Development AWS Seattle
- 2. Today’s Agenda AWS WAF Overview AWS WAF with CloudFront AWS WAF Automation with AWS Lambda Customer Example / Use Case: Magazine Luiza
- 3. What is a WAF? •A Web Application Firewall (WAF) is an appliance, server plugin, or software filter that applies a set of rules to HTTP traffic providing Layer 7 (application) protection •WAFs Come in Four Flavors • Pure Play: stand alone appliance or software • CDN: bundled with Content Delivery Network • Load Balancer: bundled with a load balancer • Universal Threat Manager (UTM): catch-all for misc. security
- 4. What is AWS WAF? Web application firewall (WAF) that gives you control over who (or what) can access your web applications. • Full-feature API • Customizable security • Integrated with Amazon CloudFront - protection at the edge • Use cases: protection against exploits, abuse, and application DDoS
- 5. Web site without AWS WAF Good users Attackers Web site Exploit
- 6. Web site with Amazon CloudFront + AWS WAF Good users Web site Exploit Attackers CloudFrontAWS WAF
- 7. Amazon CloudFront, Amazon Route 53, and AWS WAF Locations 54 CloudFront Edge Locations (PoPs), 38 Cities, 5 Continents CloudFront Amazon Route 53 AWS WAF
- 8. North America Cities: 15 PoPs: 21 Ashburn, VA (3) Atlanta, GA Chicago, IL Dallas/Fort Worth, TX (2) Hayward, CA Jacksonville, FL Los Angeles, CA (2) Miami, FL New York, NY (3) Newark, NJ Palo Alto, CA San Jose, CA Seattle, WA South Bend, IN St. Louis, MO Amazon CloudFront, Amazon Route 53, and AWS WAF Locations 54 CloudFront Edge Locations (PoPs), 38 Cities, 5 Continents South America Cities: 2 PoPs: 2 Rio de Janeiro, Brazil São Paulo, Brazil Europe / Middle East / Africa Cities: 10 PoPs: 16 Amsterdam, The Netherlands (2) Dublin, Ireland Frankfurt, Germany (3) London, England (3) Madrid, Spain Marseille, France Milan, Italy Paris, France (2) Stockholm, Sweden Warsaw, Poland Asia Pacific Cities: 11 PoPs: 15 Chennai, India Hong Kong, China (2) Manila, the Philippines Melbourne, Australia Mumbai, India Osaka, Japan Seoul, Korea (2) Singapore (2) Sydney, Australia Taipei, Taiwan Tokyo, Japan (2) CloudFront Amazon Route 53 AWS WAF Edge location AWS Region
- 9. AWS WAF Set Up Questions 1. What do I want to take action on? (Conditions – IP / String Match Set / SQL injection match sets) 2. Should I block, allow, count? (Rules - Precedence / Rule / Action) 3. What sites/distributions need these rules? (CloudFront Distribution) 4. What should I call the container of these rules? (Web Access Control Lists – WebACLs) 5. How do I see if the rules are working? (Real Time Metrics, Sampled Web Requests)
- 10. AWS WAF: WebACLs •WebACLs contain a set of conditions, rules, and actions. •WebACLs are applied to one or many CloudFront distributions. •WebACLs show you Real-Time Metrics & Sampled Web Requests for each rule.
- 11. AWS WAF: Conditions Conditions are lists of criteria that identify components of web requests. Conditions include matching on the following: • IP address i.e., /8, /16, /24, /32 • Strings, i.e., URI, query string, header, etc. • SQL injection, i.e., looks for valid SQL statements Conditions are logically disjoined, i.e. “OR”.
- 12. /login?x=test%20Id=10%20AND=1 /login?x=test%27%20UNION%20ALL%20select%20NULL%20-- /login?x=test’ UNION ALL select NULL -- Transform: URL Decode True Match: SQL Injection False Match Conditions: SQLi
- 13. AWS WAF: Rules Rules are sets of conditions with a predetermined action. Available actions are: • Block • Allow • Count Rules can logically join conditions, i.e., “AND”. Rules can be applied to many WebACLs.
- 14. AWS WAF: Resources WebACLs: applied to CloudFront distributions today •Rule reusability: use one WebACL for all distributions •Flexibility: use individual WebACL for each distribution AWS Partners: developing integrations with AWS WAF • Trend Mirco: Deep Security • GitHub: https://github.com/deep-security/aws-waf • Blog: http://aws.trendmicro.com/aws-waf-and-deep-security/ • Imperva: Threat Radar • Alert Logic: Web Security Manager
- 15. AWS WAF: Reporting & Logs Real-Time Metrics (CloudWatch): • Blocked web requests • Allowed web requests • Counted web requests Adjustments to rules in response to real time analysis. Time period can be adjusted by sliding graph end points or via filters.
- 16. AWS WAF: End to End Flow 1. Create WebACL 2. Create Conditions (IP, string match, SQL) 3. Create Rules and Actions (order, rule, action) 4. Associate WebACL to CloudFront distribution 5. Review and Deploy
- 17. AWS WAF: API & Data Types API Actions Create Delete Get List Update Data Types • ChangeToken • ChangeTokenStatus • WebACL • IPSet • StringeMatchSet • SQLinjectionMatchSet • Rule
- 18. AWS WAF: APIs 1. Get Change Token – a change token can only be used once to make a change to WAF resources. 2. Use Token to Make a Change – provide the change token to the change request 3. Check Status Using Token – use token to determine the status of your changes. INSYNC means changes were propagated
- 19. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS WAF + AWS Lambda = Automatic Protection
- 20. What is AWS Lambda? Lambda automatically runs your code without requiring you to provision servers. • “Server-less” scripting; event driven actions • Integrated with other AWS services • Use cases: scheduled events, provisioning services, and customer analysis
- 21. • Bad guys are adaptive and persistent • Better protection • Integrate application specific or open-source data sources • Sophisticated out of band analysis Why build automated security?
- 22. Automated security Good users Logs Threat analysis Rule updater Web siteRules Exploit Attackers
- 23. Automated security – traditional data center Good users Logs Threat analysis Rule updater Web site Exploit Attackers Rules
- 24. Automated security – AWS makes it easier Good users Logs Threat analysis Rule updater Web site Exploit Attackers Rules
- 25. Other AWS Services we’ll use Amazon CloudFront Amazon CloudWatch AWS CloudFormation Amazon S3 Amazon API Gateway
- 26. Types of attacks that can be automated HTTP floods Scans & probesIP reputation lists Bots & scrapers Attackers
- 27. IP reputation lists Collection of IP addresses with a bad reputation based on sending history • Open proxies or known hosts that send spam/trojans/viruses • Constantly changing/updating • Solution: import open source lists (i.e., Emerging Threats, SSL blacklist, Tor Node list) and update lists using CloudWatch events
- 29. HTTP Floods Legitimate requests at a level that excessively consume web server resources • Requests targeted at expensive components, i.e., login, product search, etc. • Different than other types of flood attacks because requests follow protocol. • Creates the problem of identifying attack from flash crowd. • Solution: count number of requests in CloudFront access logs and block offenders Attackers
- 30. HTTP Floods
- 31. Scans & probes Program that communicates with web application front end to identify potential vulnerabilities • Initiated by you – good; initiated by someone else – bad • Someone (something) with bad intentions • Consume resources by requesting URLs that don’t exist • Solution: count 40x error in access logs and block offenders
- 32. Bots & scrappers Software application that run automated tasks over the internet. • Good bots (search engines, weather, price comparison) vs bad bots (scrape content, steal data, malware) • Aggressive vs conservative days • Constantly changing/updating • Solution: use robots.txt and “honeypot” file to identify & block offenders
- 33. Bots & scrappers
- 34. Customer Story Magazine Luiza • One of the largest retail chains in Brazil • More than 700 stores, 24K staff, & 8 distribution centers • e-commerce platform customers use for purchases • Moving “all in” to AWS over the past 2-3 years • Breaking up monolithic app
- 35. Customer Story Challenges • Balance security with performance & cost • Traditional WAFs didn’t work: 1. Inflated models – lots of rules & based on vm or hardware 2. Couldn’t scale - constrained by bandwidth & CPU 3. Automation meant more hardware • Need to block bad bots (based on IP) without affecting search & shopping experience • Have solution in place by Black Friday
- 38. Customer Story Milestones Before Black Friday • September – October: confirmed new architecture and started building. • October – new architecture ready to go • November – started countdown and moved over all production traffic
- 39. Customer Story Black Friday • November 26: jumped from 4 to 28.9 million views/day • November 26 all hands on deck for the last infrastructure scale up! • All hands on deck until midnight, everyone went home, 5 people decided to sleep in our leisure room, I kept following monitoring. • Traffic started to ramp up again November 27 6:00AM and kept high page views and sales during the weekend.
- 40. Customer Story Advice to Others • Do analysis in house & start small • Use the right library for the job • Identify what needs protection • Think about the time it takes to process logs • Defense in Depth: simple security rules at perimeter, complex security rules closer to app
- 41. Conclusion Types of Attacks that can be Automated • IP reputation lists • HTTP floods • Web probes & scans • Bots & scrappers Thanks to our Solution Architects: • Heitor Vital, Ben Potter, Vlad Vlasceanu, and Lee Atkinson • Tutorials available on https://aws.amazon.com/waf/preconfiguredrules/
- 42. Thank you!
Notas do Editor
- And you’ll see how the customizability, APIs and integration with AWS can help you improve your website security.
- If you’ve operated a website in the internet, then you know there are a lot of bad requests flying around. Website operators need tools to help them find and block bad requests. Bad requests can be anything from specially craft requests design to exploit a vulnerability in your web application to steal data or takeover your server or volumetric attacks designed to take down your website or just bots scraping all of your content to be posted on some other website
- AWS WAF lets you create rules to detect and block bad stuff before it reaches your website. We provide you with new visibility tools to see what kinds of requests are hitting your website and a flexible rule language that allows you to block requests.
- Amazon CloudFront and Amazon Route 53 services are offered at AWS Edge Locations
- Our built-in SQL injection match condition checks for valid SQL statements, not just simple keywords. SQL injection usually occurs within query string parameters and request body. To check for query string, use a URL decode transform to prevent URL encode evasions, and configure a match set to check the query string.
- The WAF API is a Restful API that has five simple commands and five parameters. In addition the API requires a change token to be used when calling commands. The combination of a command and parameter is an API action that can be carried out by the AWS WAF There are two types of criteria that can be used to block or allow requests from being passed on to CloudFront or an ELB. The criteria are ByteMatch Set and an IP Set. A ByteMatch set includes syntax that matches a header value, http method, http version, query string, or URI. A SQL injection parameter is also considered a variant of a ByteMatch set. Actions are also known as default action types: ALLOW, BLOCK or COUNT
- Our demos prominently feature the use of Lambda, a scripting environment that allows you to develop and operator you own automation, without having to manage server infrastructure. By combining AWS WAF and AWS Lambda you can create automation to improve the security of your website.
- Provides better protection Out-of-band analysis doesn’t affect performance Experts are moving towards an automated model A traditional security model tends to make use of set and forget rules…
- Automated web security looks like this Amazon uses techniques just like this Find fraud, and bad bots
- … in a non-AWS world, this kind of analysis is a challenge Logs means lots of storage – Lots of compute and scaled reliable blocking is challenging
- In AWS, all of this is possible without managing servers – no server instances
- AWS WAF connects seamlessly with the AWS ecosystem
- Thanks Nate for the introduction. As Nate mention, my name is Gleicon Moraes, and I’m the Infrastructure and Architecture manager for Magazine Luiza. I handle a lot of the day-to-day operations with our architecture in AWS as well as some our of initiatives with big data. A little introduction to Magazine Luiza, we’re one of the top three retail chains in Brazil. We have over 700 stores, we employ about 24 thousand people, and have 8 distribution centers across the country. You can think of us as Amazon.com but with a dedicated focus to the Brazilian market. As part of the service we offer to customers, we also have an e-commerce platform where customers can buy products directly from us. During a normal month, our site sees anywhere from 120 to 200 million requests, and during high traffic times about 2 times more. About 2-3 years ago, we made the decision to go “all in” with AWS by leaving our data centers and moving our backend servers, web application frontend servers, and content delivery over to AWS. Instead of just porting over the application to cloud, we also started breaking up some of our monolithic applications into smaller applications based on Java to take advantage of the scale and architecture of AWS.
- As part of this transition to AWS, we’re also levering a lot of the security features AWS services offer. With our site being an ecommerce site, we of course see a lot of bots and scrapers that can affect our availability but also put a strain on our resources. With every new security feature we add to protect our site, we have to balance those against performance. During the process of improving our security, we looked at traditional WAFs but found that they didn’t work for us for three main reasons. First, traditional WAFs come with inflated models. By inflated models I mean these WAFs come pre-loaded with lots of rules that are application specific. In the end we found that these rules didn’t work for us because they were for applications that we didn’t use or had too many false positives for us to manage effectively. The second reason was scale. The more things we did with a traditional WAF, the more it would cost in CPU, and that meant the WAF could handle less traffic. For example, it we wanted to use some of the log analytic features these WAFs offer, it would spike the CPU which meant the WAF could handle less and less traffic. If we were really aggressive with the features, we found that this would have an impact on performance, which ultimately affected the shopping experience for our customers: and this was the one thing we wanted to protect. The third reason was automation. If we wanted to automated our security response like log storage, analysis, and rule updates, all of these steps required new hardware. Just like Nate showed in a previous slide, we had to purchase a server that would store the logs, purchase another server to run the analysis, and another to update the rules. Because of these things, a traditional WAF really didn’t meat our needs. A practical example of this was throttling bad bots without affecting our search. Our site relies heavily on search engine results as Brazilian customers are searching for certain products that we sell. At the same time, we see lots of bots that scrape our content, which can put an enormous resource load on our application. What we see is that these bots don’t just scrape our content, but they do catalog searches and product indexes which can impact other customers’ shopping experience as well as run the risk of stealing our content. When we looked at traditional WAFs, the solutions forced us to estimate the bandwidth or CPU, which put us in a hard place because if we exceeded these limits, we were actually causing the problem we were trying to avoid. And to make things even more complicated, we were coming up on Black Friday, when we anticipated our traffic was going to increase dramatically.
- So in September of last year, before we came up with a solution to controlling bots this is how we looked in AWS. We were using CloudFront to distribute static content, and a fleet of EC2 servers running Varnish to handle our navigation and security before requests hit our web application. Essentially a user would come in, navigate around using Varnish which we managed. Requests for static content were served directly by CloudFront, while requests for dynamic content was served through ELB and EC2. The problem we had with this architecture is that Varnish was becoming a big problem to manage. For example, we have a fleet of Varnish servers in a couple of regions, but as our traffic grew we had to duplicate each configuration in all regions. This required a lot of development on our side and operational resources. Additionally, even though AWS had 11 regions where we could host our Varnish servers, it didn’t compare to the reach of CloudFront which had 53 edge locations. This was a very compelling reason for us to increase our reach using a CDN where we could scale a lot easier. Lastly, although Varnish allowed us to do security for our site, simple things like counting web requests and tying those base to IP addresses would have required that we create some sort of database and manage yet another set of servers. When WAF was released in October and we saw this type of functionality already built it, we decided to make some changes that would allow us to scale and build better security.
- The first thing we did was move all our traffic over to CloudFront, both static and dynamic. With the new architecture requests for all our web content and navigation came through CloudFront. This allowed us to scale much better, increased performance for both types of content, simplified our architecture and took a huge operational burden off our hands. The second thing we did was to move over our security rules from our Varnish fleet to AWS WAF. With AWS WAF we were able to quickly count requests and start identifying which requesters were utilizing our resources the most. This took us some trial and error and we had to play with the right thresholds, but by using AWS WAF and Lambda we were able to free up a lot of resources from our web server because the analyses was happening out-of-band and didn’t affect the shopping experience for our customers.
- So to tell a good story, I wanted to share some of the milestones we had when we moved all our traffic over to CloudFront and started using WAF and Lambda. By the end of September, we had made the decision on what our new architecture would look like and by the first of October started making the changes. During this time we were running some component tests, adjusting some of our assets, and by the first of November we had everything in place to move all production traffic. From there, we started counting down the days until Black Friday. On the first week of November, we did a full capacity planning meeting with AWS and notification to warm up on capacity. On November 19, we moved all our production traffic over to the new architecture, and from that day to November 26, we were doing twice-a-day checkpoints as a team on our traffic with capacity planning and making sure everything was running according to plan.
- Then on November 26, the traffic really started to hit. We went from 4 million views per day to about 29 million views. On that day, it was all hands on deck to ensure that our infrastructure was scaling properly and that customers weren’t having any problems with the site or purchases. Even though things were going well, we wanted to be extra cautious. So most of my team stayed on deck until about midnight that night. From there, most of the team went home except for 5 people, who decided to sleep in our leisure room and respond in case anything happened. I continued to watch our monitoring and reporting for the rest of the night, but things started to die down after most of the team left. The next morning, traffic again started to spike about 6am and really keep high for the whole weekend. However, we were really proud of the solution we had built. We didn’t have any problems with scaling, and our automated solution with AWS WAF and Lambda kept pace with our traffic spikes and was able to successfully control bots to ensure our users were able to complete their purchases. And I finally got to go to bed that night, which was a big bonus.
- We’re continuing to use the solution for our day-to-day operations, and we’re still fine tuning things. But I thought that now that we’re done with the big traffic push, I’d share some insights with the audience that may help them implement their own version of the solution. First do your security analysis in house. If you’re new to the area, keep it small to start with. Think of WAF a rule engine that is going to enforce your security rules, but the best logic on what is bad traffic should come from you because you know your application better than any vendor. Again, if you’re new here, start small by looking at the logs and digging into what normal request volume should be and identify those requests that don’t fit the norm. Second use the right library for the job. We like to use python because the run time is smaller compared to like node.js and the data types are much cleaner. Both libraries are useful, but python really saved us from lots of coding and was able to execute the rule updates much faster. Third identify what points need protection. With our application, the dynamic requests are much more costly in CPU than the static requests. Had we only looked at top requests, those for static content would have surpassed dynamic content requests. But putting rules around those requests really don’t save us any CPU. The rules we created were for the points that needed the most protection, and made sure we had resources to serve our customers. Fourth, remember that there is time lag in between when requests come in and when CloudFront writes access logs. Think about this time because it will have an impact on your rules. If you set your throttle too low with the delta you run the risk of blocking valid users along with the bots and scrapers. Start with a high threshold with the time delta in mind, and then you can always come down when you start to better understand your traffic volumes. Last, don’t put all your security rules in one place. Some people call this a defense in depth strategy, but in our case blocking high requesters at the edge didn’t completely solve the problem. There are still some bots that are under this threshold that can affect our application. However, by blocking a lot of these requests at the edge, the traffic reduced to a volume where I could put in some additional rules using the apache module. The good thing about simple throttling at the edge is it reduces my traffic to a point that my application servers can handle more complex throttling. Also if any one of these points fails, I still have another tool to use to make sure my sites is available. So this concludes my story. I hope our audience can find value in the solution we created and the advice to others. With that, let me turn it back over to Nate.