Serverless OCR for NASA EVA: AWS Meetup DC 2017-12-12
•
4 gostaram•740 visualizações
NASA's EVA group needs to extract text (OCR) from 100,000 pages per month to improve Astronaut safety. This presentation shows how we used AWS Lambda serverless tech to solve the problem quickly and cost-effectively.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Serverless OCR for NASA EVA: AWS Meetup DC 2017-12-12
Semelhante a Serverless OCR for NASA EVA: AWS Meetup DC 2017-12-12 (20)
Mais de Chris Shenton
Mais de Chris Shenton (8)
Último
Último (20)
Serverless OCR for NASA EVA: AWS Meetup DC 2017-12-12
- 1. Serverless Optical Character Recognition in Support of NASA Astronaut Safety Chris Shenton CTO V! Studios AWS DC Meetup 2017-12-12
- 2. Talk Overview ● The Problem ● The Challenge ● Architectures: server, cloud, serverless ● Lambda: FaaS, Events, Benefits, Limitations ● NASA EVA OCR Architecture ● Security, FedRAMP, ATO ● Serverless Framework ● Gotchas! ● Happy Customer ● Future Challenges and Opportunities
- 3. Problem: Life-Threatening Spacesuit Failure On July 16, 2013, water filled the helmet of Italian astronaut Luca Parmitano, creating a life threatening scenario which forced NASA to abort his spacewalk.
- 4. The Challenge ● Designs on paper or scanned without OCR ability ● Current reporting processes and procedures cannot be changed ● About 60 Discrepancy Reports (20 pages) and 190 Task Performance Sheet reports (500 pages) per month ● Started OCR in 2015, stopped due to server load ● Overwhelmed the EVA Data Integration pipeline 100,000 pages/month
- 5. Architecture evolution: server to cloud to serverless ● Datacenter: no scaling ● Cloud servers: scaling ● Cloud Containers: scaling ● Serverless: fast, painless scaling
- 6. Architecture [1a]: Datacenter, no scaling PDF doc Server OCR process TXT doc
- 7. Architecture [1b]: Datacenter, no scaling PDF doc Server OCR process TXT doc PDF doc PDF doc PDF doc PDF doc Load overwhelms OCR server
- 8. Architecture [2]: Cloud with scaling PDF doc PDF doc PDF doc PDF doc PDF doc Pro: ● Scaling handles load spikes Con: ● Complicated to set up ● Scale out takes a few minutes per server ● Still have to manage OS, security Autoscaling group SQS Queue Server OCR Server OCR Server OCR Server OCR Server OCR TXT doc TXT doc TXT doc TXT doc TXT doc
- 9. Architecture [3]: Cloud Containers with scaling PDF doc PDF doc PDF doc PDF doc PDF doc Pro: ● Scaling handles load spikes ● Can deploy immutable instances Con: ● Have to manage scaling ● Have to manage placement, orchestration SQS Queue TXT doc TXT doc TXT doc TXT doc TXT doc Container Serve Container OCR Container OCR Container OCR Container OCR Container OCR Container Server Container OCR Container OCR Container OCR Container OCR Container OCR
- 10. Automatic scaling Architecture [4a]: Serverless Cloud with built-in scaling PDF doc PDF doc PDF doc PDF doc PDF doc Pro: ● Scaling is automatic, nearly instant ● No patching, open ports Con: ● Some limits on size, lifetime TXT doc Lambda OCR Lambda OCR Lambda OCR Lambda OCR Lambda OCR TXT doc TXT doc TXT doc TXT doc
- 11. PDF page Automatic scaling Architecture [4b]: Serverless Cloud with built-in scaling PDF doc PDF doc PDF doc PDF doc PDF doc PDF page PDF pageLambda split doc Lambda split doc Lambda split doc Lambda split doc Lambda split doc PDF page PDF page PDF page PDF page PDF page PDF page PDF page Automatic scaling Lambda OCR Lambda OCR Lambda OCR Lambda OCR Lambda OCR Lambda OCR Lambda OCR Lambda OCR Lambda OCR Lambda OCR TXT doc TXT doc TXT doc TXT doc TXT doc TXT doc TXT doc TXT doc TXT doc TXT doc With instant, automatic scaling, we can split PDF docs into PDF pages and OCR each page to text in parallel, with minimal extra effort. Exploiting parallelism gives us our results much faster at no extra cost.
- 12. Lambda, FaaS, Event-Driven Computing Say What?
- 13. AWS Lambda is a “Function as a Service” (FaaS) Function as a service (FaaS) is a category of cloud computing services that provides a platform allowing customers to develop, run, and manage application functionalities without the complexity of building and maintaining the infrastructure typically associated with developing and launching an app. Building an application following this model is one way of achieving a “serverless” architecture, and is typically used when building microservices applications. Wikipedia FaaS Products ● AWS Lambda ● Google Cloud Functions ● Microsoft Azure Functions ● IBM OpenWhisk
- 14. Event-Driven Computing: trigger Functions based on events S3 ObjectCreated DynamoDB Row Changed API Gateway GET, PUT, POST, DELETE Lambda Function processes event
- 15. Event-Driven Computing: Lambdas can trigger Lambdas S3 ObjectCreated DynamoDB Row Changed API Gateway GET, PUT, POST, DELETE Lambda Function processes event Lambda invoke Sync or Async
- 16. Event-Driven Computing: allows interesting architectures Lambda: return new S3 location DynamoDB Row Changed store info to DynamoDB GET /newUpload {uploadUrl: s3://bucket/newKey} PUT /newKey ...data… S3 ObjectCreated {bucket: b, key: newKey} Application Search Engine Send metadata to application via HTTP {method: GET, url: /newUpload, data: none} {uploadUrl: s3://bucket/newKey} 1 2
- 17. Event-Driven Computing: AWS events ● Amazon S3 ● Amazon DynamoDB ● Amazon Kinesis Streams ● Amazon Simple Notification Service ● Amazon Simple Email Service ● Amazon Cognito ● AWS CloudFormation ● Amazon CloudWatch Logs ● Amazon CloudWatch Events ● AWS CodeCommit ● Scheduled Events ● AWS Config ● Amazon Alexa ● Amazon Lex ● Amazon API Gateway ● AWS IoT Button ● Amazon CloudFront ● Amazon Kinesis Firehose ● Invoking a Lambda Function On Demand
- 18. Lambda Benefits Example application: process 1000 2-second requests/day ● Server: $16.84/month (AWS t2.small, 24x7) ● Lambda: $1.50/month No Servers to Manage: no patching, open ports or logins Subsecond Metering: no idle capacity Continuous Scaling: high availability
- 19. Lambda Limitations ● Languages: NodeJS, Python, Java, C# (Golang soon) ● Maximum execution duration: 300 seconds ● Memory: 128 MB - 3 GB ● Ephemeral disk: 512 MB ● Invoke request payload: 6 MB (sync), 128 KB (event/async) ● File descriptors: 1024 ● Processes and threads: 1024
- 20. Lambda Limitations: some work-arounds Maximum execution duration: 300 seconds Memory: 128 MB - 3 GB ● Decompose function into smaller microservices ● Chain Lambdas with events like SNS or Lambda async invocation Ephemeral disk capacity: 512 MB ● Read/write from/to external storage: S3, DynamoDB, etc
- 21. EVA OCR Architecture EVA Data Integration systems Text Documents Text Pages PDF Pages PDF Docs EDI App EDI Search API Split Output PDF doc JSON doc PDF doc AWS Autoscaling LambdasAWS S3 Storage OCR for EVA TXT pages OCROCROCR CombineCombineCombine PDF page PDF page PDF page TXT page TXT page TXT page
- 22. EVA OCR Architecture: Big Wins ● Architecture designed for lowest operational cost possible: ○ S3 files removed after 24 hours: minimal data charges, better security ○ no database cost ● Architectural patterns we used instead of database: ○ track progress with directory prefixes ○ propagate information using S3 object metadata ● Lambda autoscaling, fast scaling, pay only for active use ● Serverless Framework simplified deployment ● but see the Gotchas in a few slides...
- 23. EVA OCR Architecture: securely connect with cloud policies EDI App IAM Role: eva-app-role EVA OCR S3 bucket eva-ocr-dev ● /doc_pdf/ ● /page_pdf/ ● /page_txt/ ● /doc_txt/ EVA Search API on 3x EC2 HTTPS API on port 5333 Security Group: sg-002: eva-search ● allow from sg-001 ● to port 5333 EVA OCR Lambda Functions IAM Role: eva-ocr-dev-us-east-1-lambda Security Group: sg-001: ocreva-lambda-output EVA code uploads PDF to /doc_pdf/ HTTP POST {docid: ‘mydocid’, page: 42, text: ‘ocr text…’} Lambdas read/write pdf and txt in various folders IAM Role: eva-app-role Policies: ● ocreva-s3-write-doc_pdf ● other EDI policies... IAM Policy: ocreva-s3-write-doc_pdf allow write arn:aws:s3:::eva-ocr-dev/doc_pdf/ No auth servers were harmed in the making of this service
- 24. EVA OCR Security Controls EVA OCR S3 Storage Even though Lambda is currently undergoing FedRAMP certification, cloud security group provided ATO based on the following controls: ● GovCloud for sensitive data ● IAM policies, roles and Security Groups restrict access ● Separate VPC for Lambdas ● No VPC network egress for Lambdas ● Security Group allows output of final Lambda to EDI Search API ● Encrypted data in transit and at rest ● Static Code Analysis EVA OCR Autoscaling Lambdas Lambda VPC private IP space /16 = 65535 IPs EVA Data Integration systems EDI VPC NASA IP space limited IPs SG allows to port 5333 S3 VPC Endpoint
- 25. Serverless Framework: from the horse’s mouth Serverless is your toolkit for deploying and operating serverless architectures. Focus on your application, not your infrastructure. serverless.com npm install serverless -g serverless create --template hello-world serverless deploy curl http://xyz.amazonaws.com/hello-world
- 26. Serverless Framework: overview ● Controlled by file serverless.yml ● Defines Resources: S3, Lambda, … ● Defines Events: ObjectCreated, … ● Wires Events to trigger Lambdas ● Defines Security ● Uses CloudFormation underneath ● Simple CLI to deploy to cloud 1. service: myservicename 2. provider: 3. name: aws 4. runtime: python3.6 5. functions: 6. s3created: 7. handler: myservice.s3created 8. events: 9. - s3: 10. bucket: myservice-dev 11. event: s3:ObjectCreated:* 12. rules: 13. - prefix: doc_pdf/ 14. formget: 15. handler: myservice.formget 16. events: 17. - http: 18. path: form 19. method: get 20. formpost: 21. handler: myservice.formpost 22. events: 23. - http: 24. path: form 25. method: post
- 27. Gotchas! ● Will get duplicate events if Lambda exits unsuccessfully ○ this is a good thing ● May get duplicate events ○ detect and possibly ignore them (idempotent) ● Timeouts if job takes longer than 300 seconds ○ may have to chain Lambdas ● Overloading destinations is likely due to scale ○ detect, back-off ○ may require handling like Timeouts ● Fast scaling can exhaust limited IP addresses in a VPC ○ use separate VPC for Lambda with large private IP space, e.g., /16 with 65,535 IPs ● S3 eventual consistency ○ use UUIDs in S3 keys to force read consistency
- 28. Happy Customer “The work you’ve accomplished is a big step proving out this new technology for NASA” Cuong Q Nguyen, JSC/NASA EVA Office
- 29. Future Challenges and Opportunities NASA’s Cuong Nguyen has told us he needs to track assembly, subassembly and part hierarchies. Can we extract structured text? He also needs to identify inspector and approval “stamps”. This is not OCR but hard image processing.
- 30. Future Work: structured content extraction
- 31. Future Work: “stamp” detection and identification
- 32. Questions? Reach out to us! chris@v-studios.com @shentonfreude