securing-your-software-delivery-pipelines-with-a-slight-shift-to-the-left.pdf
•
0 likes•97 views
Buildkite is a CI/CD platform with security and flexibility at the core of it's product. In my presentation for Programmable conference in Sydney, I talk about the big security risks in CI/CD and introduce some measures to mitigate them.
Recommended
Recommended
More Related Content
Similar to securing-your-software-delivery-pipelines-with-a-slight-shift-to-the-left.pdf
Similar to securing-your-software-delivery-pipelines-with-a-slight-shift-to-the-left.pdf (20)
Recently uploaded
Recently uploaded (20)
securing-your-software-delivery-pipelines-with-a-slight-shift-to-the-left.pdf
- 1. Securing your delivery pipelines with a slight shift to the left
- 2. I’m OK at Computers.
- 8. We should do better. We can do better.
- 10. Supply Chain Levels for Software Artefacts (SLSA) A framework designed to help organisations improve the integrity of their software supply chains.
- 14. The Secure Software Development Framework (SSDF) is a set of fundamental, sound, and secure software development practices based on established secure software development practice documents from organizations such as BSA, OWASP, and SAFECode. Few software development life cycle (SDLC) models explicitly address software security in detail, so practices like those in the SSDF need to be added to and integrated with each SDLC implementation.
- 15. The SSDF outlines solid practices for embedding secure software development practices in the delivery lifecycle, that don’t just identify threats but actually address them. Source: https://csrc.nist.gov/Projects/ssdf
- 18. 33% of respondents described their security strategy as having a mix of prevention and detection. Source: Left and Right of Boom in Cybersecurity ,Elastic, 2022 82% said they plan to implement, are implementing or have implemented.
- 19. 33% of respondents described their security strategy as having a mix of prevention and detection. Source: Left and Right of Boom in Cybersecurity ,Elastic, 2022 82% said they plan to implement, are implementing or have implemented.
- 20. The road to hell is paved with good intentions.
- 21. “would pursue laws to establish liability for software companies that sell technology that lacks cybersecurity protections” The Biden-Harris National Cybersecurity Strategy
- 22. Security is our Responsibility
- 23. • Linting • Unit tests • Integration tests Continuous Integration • Staging deploy • Prod deploy • Smoke tests Continuous Deployment Commit & Push Git Hello World
- 24. CI CD Git
- 25. Top 10 CI/CD SECURITY RISKS SECURITY RISKS The Open Worldwide Application Security Project (OWASP)
- 26. SECURITY RISKS SECURITY RISKS 1 — Insufficient Flow Control Mechanisms 2— Inadequate Identity and Access Management 3— Dependency Chain Abuse 4— Poisoned Pipeline Execution (PPE) 5 — Insufficient PBAC (Pipeline-Based Access Controls) 6 — Insufficient Credential Hygiene 7 — Insecure System Configuration 8— Ungoverned Usage of 3rd Party Services 9 — Improper Artifact Integrity Validation 10 — Insufficient Logging and Visibility
- 27. SECURITY RISKS SECURITY RISKS 1 — Insufficienct Flow Control Mechanisms 2— Inadequate Identity and Access Management 3— Dependency Chain Abuse 4— Poisoned Pipeline Execution (PPE) 5 — Insufficient PBAC (Pipeline-Based Access Controls) 6 — Insufficient Credential Hygiene 7 — Insecure System Configuration 8— Ungoverned Usage of 3rd Party Services 9 — Improper Artifact Integrity Validation 10 — Insufficient Logging and Visibility 4 — Poisoned Pipeline Execution (PPE) 5 — Insufficient PBAC (Pipeline-Based Access Controls) 2 — Inadequate Identity and Access Management 6 — Insufficient Credential Hygiene 1 — Insufficient Flow Control Mechanisms 3 — Dependency Chain Abuse 2— Inadequate Identity and Access Management
- 28. Our goal is to limit the blast radius.
- 29. Is executing build scripts within all build contexts okay?
- 30. Executing scripts within all build contexts is not ok.
- 31. How about running `terraform plan` in all build contexts?
- 32. Executing arbitrary code in all build contexts is not ok.
- 33. SECURITY RISKS SECURITY RISKS 1 — Insufficient Flow Control Mechanisms 2— Inadequate Identity and Access Management 3— Dependency Chain Abuse 5 — Insufficient PBAC (Pipeline-Based Access Controls) 6 — Insufficient Credential Hygiene Poisoned Pipeline Execution (PPE) 7 — Insecure System Configuration 8— Ungoverned Usage of 3rd Party Services 9 — Improper Artifact Integrity Validation
- 34. Poisoned Pipeline Execution (PPE) • Have isolated pipeline environments and contexts • Sensitive and Non-Sensitive contexts • Use branch protection rules in GitHub/GitLab/BitBucket etc.
- 35. Upload Pipeline Build Docker Image Linting Security Scans RSpec Jest Code Coverage Bundle Analysis Branch Build Non-sensitive context - no access to secrets - no pipeline to prod Sensitive context - access to secrets - additional permissions Upload Pipeline Build Docker Image Linting Security Scans RSpec Jest Code Coverage Bundle Analysis Main Build Prepare for Deploy Deploy to Prod
- 36. SECURITY RISKS SECURITY RISKS 1 — Insufficient Flow Control Mechanisms 2— Inadequate Identity and Access Management 3— Dependency Chain Abuse 4— Poisoned Pipeline Execution (PPE) 6 — Insufficient Credential Hygiene 7 — Insecure System Configuration 8— Ungoverned Usage of 3rd Party Services 9 — Improper Artifact Integrity Validation Insufficient PBAC (Pipeline-Based Access Controls)
- 37. • Restrict the scope of a pipeline's access & permissions • Use granular access controls Insufficient PBAC (Pipeline-Based Access Controls)
- 38. ECS Service Agent Job ECS deploy role Agent API (Pipelines)
- 39. ECS Service Agent Job Agent API (Pipelines) OIDC provider OIDC token
- 44. • Restrict the scope of a pipeline's access & permissions • Apply granular access controls: • job-tokens • OIDC • Use these things with a dedicated Secrets Manager: • Hashicorp Vault (Buildkite plugin) • AWS Secure Secrets Manager (Buildkite plugin) • Have ingress/egress filters to the internet: • Tailscale • Cloudflare etc. • Always terminate agents and wipe VMs/Machines! Insufficient PBAC (Pipeline-Based Access Controls)
- 45. SECURITY RISKS SECURITY RISKS 1 — Insufficient Flow Control Mechanisms 2— Inadequate Identity and Access Management 3— Dependency Chain Abuse 4— Poisoned Pipeline Execution (PPE) 6 — Insufficient Credential Hygiene 7 — Insecure System Configuration 8— Ungoverned Usage of 3rd Party Services 9 — Improper Artifact Integrity Validation Insufficient PBAC (Pipeline-Based Access Controls)
- 46. SECURITY RISKS SECURITY RISKS 1 — Insufficient Flow Control Mechanisms 2— Inadequate Identity and Access Management 3— Dependency Chain Abuse 4— Poisoned Pipeline Execution (PPE) 5 — Insufficient PBAC (Pipeline-Based Access Controls) 7 — Insecure System Configuration 8— Ungoverned Usage of 3rd Party Services 9 — Improper Artifact Integrity Validation Insufficient Credential Hygiene
- 47. • Limit the blast radius of potential breaches. • Reduce risk of Poisoned Pipeline Execution (PPE): • Limit what code is executed in certain contexts • Have sensitive/non-sensitive build contexts • Have strong Pipeline-Based Access Controls (PBAC): • Limit scope of what builds/pipelines have access to • Use ephemeral/tightly scoped access tokens Insufficient Credential Hygiene
- 48. Let machines do the work!
- 49. • Use a dedicated secret manager: • HashiCorp Vault, AWS Secure Secrets Manager etc. • Automatically scan for leaked keys and credentials: • GitGuardian, GitHub’s configurable Secret Scanning etc. Insufficient Credential Hygiene
- 50. Alerts are only useful if they’re seen and acted on.
- 51. SECURITY RISKS SECURITY RISKS 2— Inadequate Identity and Access Management 3— Dependency Chain Abuse 4— Poisoned Pipeline Execution (PPE) 5 — Insufficient PBAC (Pipeline-Based Access Controls) 6 — Insufficient Credential Hygiene 7 — Insecure System Configuration 8— Ungoverned Usage of 3rd Party Services 9 — Improper Artifact Integrity Validation Insufficient Flow Control Mechanisms
- 52. we accept mistakes are part of software delivery. CI/CD exists because
- 53. Insufficient Flow Control Mechanisms LGTM • Unreviewed code can’t trigger deployment pipelines • Code reviews & approvals should be part of the merge process. • Configure this process in your Source Control Manager: • 2 human approvals prior to a PR being merged • For teams with additional compliance regulations consider using a `block step` in your pipeline.
- 54. SECURITY RISKS SECURITY RISKS 1 — Insufficient Flow Control Mechanisms 2— Inadequate Identity and Access Management 4— Poisoned Pipeline Execution (PPE) 5 — Insufficient PBAC (Pipeline-Based Access Controls) 6 — Insufficient Credential Hygiene 7 — Insecure System Configuration 8— Ungoverned Usage of 3rd Party Services 9 — Improper Artifact Integrity Validation Dependency Chain Abuse
- 56. Open Source NPM, Yarn, PyPi, RubyGems, all the things…
- 57. Dependency Chain Abuse • Get visibility into CVEs and act on them, use tools like: • GitHub Dependabot (Enterprise orgs) • Identifies & notifies users about vulnerable dependencies • Opens PRs to keep dependencies updated • Use services like Snyk or Cloudsmith who: • Integrate with most CI/CD providers • Do application/container scanning • Asset Discovery and tagging (so you can pin versions) • Avoid latest versions • Verify the checksum
- 58. Software Bill of Materials An immutable list of what’s in an application: • Open source libraries (languages, imports/dependencies) • Plugins, extensions, add-ons used • Application code (versioned) • Information about versions, licensing status and patch status of these components An SBOM for a SaaS application can include info like: • APIs • 3rd party services required to run the SaaS application.
- 60. SBOM > F-BOMB
- 61. CC/CD CI/
- 64. Dependency Chain Abuse • Get visibility into packages + CVEs with tools and act on them • GitHub Dependabot • Snyk • Avoid latest versions • Verify the checksum • Practice Continous Compliance (Put a CC in CI/CD) • Generate SBOMs for your applications, using: • GitHub, Cloudsmith, SonaType, JFrog, ReversingLabs • Create action oriented workflows around your SBOMs!
- 67. Aim to limit the blast radius
- 69. Lean on tooling & automation
- 70. Work together to create and adapt the human processes.
- 73. @MelissaKaulfuss
- 74. OWASP Top 10 CI/CD Security risks 2022 State of DevOps Report Supply Chain Levels for Software Artifacts (SLSA) Secure Software Development Framework (SSDF) US National Cybersecurity Strategy (March 2023) Auth0's Open ID Connect Handbook Software Bill of Materials (SBOM) Automating Governance Risk and Compliance Creating Actionable SBOMs with Cloudsmith & Buildkite GitHub self-service SBOM Resources