Configuration of Spring Boot applications using Spring Cloud Config and Spring Cloud Vault.
Presentation given at the meeting of the Java User Group Freiburg on October 24, 2017
Diamond Application Development Crafting Solutions with Precision
Externalized Spring Boot App Configuration
1. S P R I N G C L O U D
C O N F I G &
V A U LT
C H R I S TO P H L U D W I G
H A U F E G R O U P, F R E I B U R G
J AVA U S E R G R O U P F R E I B U R G , 2 4 . 1 0 . 2 0 1 7
2. AGENDA
• What’s the noise about app configuration?
• Spring Cloud Config
• HashiCorp Vault
– Generic Secrets
– PKI: Vault as Certification Authority
– Client Authentication
• Usage Scenarios
• Extensions in Haufe Projects
– Vault-based Config Server Discovery (or: Where do I get the config server credentials
from???)
– Keystores for client & server configurations in Vault
3. E X T E R N A L I Z E D
S P R I N G A P P L I C A T I O N
C O N F I G U R A T I O N
5. 1 APPLICATION – MANY DEPLOYMENTS
Git
Developer
PC
CI
Environmen
t
Performance
, Load, and
Stress Test
Enviroment
Production
Enviroment
Branch A
Integration
Environmen
t
Branch B
Integration
Environmen
t
Demo
Environmen
t
6. TWELVE FACTOR APPS
I. Codebase
One codebase tracked in revision control,
many deploys
II. Dependencies
Explicitly declare and isolate dependencies
III. Config
Store config in the environment
IV. Backing services
Treat backing services as attached resources
V. Build, release, run
Strictly separate build and run stages
VI. Processes
Execute the app as one or more
stateless processes
VII. Port binding
Export services via port binding
VIII. Concurrency
Scale out via the process model
IX. Disposability
Maximize robustness with fast startup and
graceful shutdown
X. Dev/prod parity
Keep development, staging, and production
as similar as possible
XI. Logs
Treat logs as event streams
XII. Admin processes
Run admin/management tasks as
one-off processes
Source: https://12factor.net/
8. CONFIG SERVER CONCEPT
• Manageable # Environment Variables:
– Profile Names (Env ID, Feature Selectors)
– Config Server URL
– Config Server Credentials
• Configuration under revision control
• App fetches config details while
bootstrapping
• Config server scalable since state in repo
Git
App
Imag
e
Config
Server
App
Instanc
e 1
App
Instanc
e 2
App
Instanc
e …
Build
Deploy
Pull
Fetch
9. SPRING CLOUD CONFIG
Server:
• “Normal” Spring Boot Service
• URL path parameters:
– App name
– List of profile names
– Git label (revision / branch)
• Git-based:
– Uses (file:///…) or clones (git://…, ssh://…)
lokal Git repo
– Git access via JGit library
– On every client access, branch is checked
out and pulled from remote
• Alternative "native" profile:
– Config in simple file folder
– Ideal for local development
Client:
• Configuration in
– bootstrap.yml
– bootstrap-profile.yml usw.
– By discovery (e.g., Consul, Eureka)
• Adds PropertySources to the Environment
==> combined with other property
sources
• Periodic health check re-fetches config
11. SECRETS
• Don’t put plain passwords (for, say DB access) into config files!
• Don’t put passwords into your Git repo!
• Many developers perceive TLS key handling as complex
• Secrets in environment variables / command lines easy to read from a shell account
• When did you last change your DB passwords??
• All network transport of secrets must be protected by, say, TLS
• …
12. HASHICORP VAULT
• Tool for secure access to secrets:
– Encryption of data at rest
– Elaborate access control concept
– Audit log of every access
• Support for dynamic secrets:
– E.g., on-the-fly setup of DB users with their respective roles and credentials
– Issuing of freshly created X.509 certificates
• One-time tokens, cubbyhole backend
• Many storage backends (filesystem, cloud storage, databases, Etcd, …)
• Revocation of individual leases as well as complete immediate system lock-down.
• Most parts open source (Mozilla Public License 2.0)
• One shared app image serves both as daemon and as command line interface
• Server exposes REST API
13. VAULT AUTHENTICATION
How to authenticate a vault client?
• AppRole:
– Static Role Id / ephemeral Secret Id
– Secret typically created by deployment
pipeline
– Optional: Secret expires after use
– Alternative: No secret, but CIDR
• AWS:
– EC2 or IAM credentials
– Trusts AWS signatures
• LDAP:
– User credentials stored in LDAP / AD
– Optional MFA
• GitHub:
– GitHub personal access token
• Radius
• Client certificates
• Username / Password
Underlying Token Authentication:
• Explicit or under the hood by other mechanisms
• Tokens bound to policies (access control)
• Tokens expire if not renewed
• Tokens can be revoked by admin
• Most REST requests require token in HTTP header
14. DEPLOYMENT CONSIDERATIONS
One-time tokens are great, but…
• App operation in container clusters becomes more and more popular
• Containers often replicated (scale out, disruption free deployments, …)
• Cluster may migrate / restart containers at any time
• So far no cluster hooks for automated re-creation of tokens (abuse potential!)
==> Multi-use secrets for container vault authentication
In the Deployment Pipeline:
vault write -f –format=json auth/approle/role/myapprole/secret-id |
jq -r '.secret_id' |
docker secret create myapprole_secretid -
docker service create --name=“myapp"
--secret="myapprole_secretid" myapp:alpine
16. SPRING CLOUD VAULT
Secret Bootstrapping:
• Adds PropertySource to Spring Environment (similar to Cloud Config client)
• Runs in bootstrapping phase
• App name + profiles translate in Vault paths:
{backendName}/{appName}/{appName-profile}
RestTemplates for Vault Access:
• Ease Vault use from custom code
• Examples:
– Storage of secrets at runtime
– Interactions with PKI backend
– Transit backend: Encryption as a service
17. CONFIG SERVER + VAULT
Git
App
Imag
e
Config
Server
App
Instanc
e 1
App
Instanc
e 2
App
Instanc
e …
Build
Deploy
Pull
Fetch
Vault
18. S P R I N G
C L O U D V A U LT
E X T E N S I O N S
@ H A U F E
19. VAULT-BASED
CONFIG SERVER DISCOVERY
Issue:
• Even with secrets in Vault, configuration details give clues to potential attackers
==> Config Server access should require authorization
• Config Server passwords are secrets, belong into Vault
• Out of the box, config client won’t see properties fetched by vault client
(config client configuration too early in bootstrap process)
Solution:
• Leverage Cloud Config Discovery mechanism
• VaultPropertySourceLocator injected as @Resource into VaultBasedDiscoveryClient
20. PKI KEY- & TRUSTSTORE INTEGRATION
Issue:
• Keystore management (JCEKS, PKCS#12) cumbersome, frequent source of operation
errors
• Keystores in the file system should be password-protected
Solution:
• For (app) internal connections (= no “official” cert required):
– Leverage Vault’s PKI backend, issue certificates on-the-fly.
– Build on sample by Mark Paluch, extended with retrieval of trusted certificates
• For customer / partner facing clients & services:
– Represent key- and trust stores as JSON objects in Vault’s generic secret backend
– Auto-configuration for Web container & HTTP clients
• Fallback to keystores in the file system (for, e.g., local development, tests)
• Fallback to the trusted certificates of the runtime’s default X509TrustManager
Notas do Editor
When Spring resolves a property, it goes through the list of all registered property sources and takes the value from the first source that contains this property. Property values can refer to other properties (using the “${otherProperty}” syntax); the referenced property is then looked up in the list of property sources as well.
Once a deployable artifact (Jar, Docker image, …) is created from the application sources, we want to run it without modification in many different environments. Of course, sizing, URLs of backend systems, certificates of client-facing endpoints etc. will be different in the various environments.
However, if any parameter that might change becomes an environment variable, then the handling of these environment variables becomes unwieldy. Extra care is required for the handling of secrets.
JGit supports ssh-rsa keys only and does not understand hashed known hosts entry. This can lead to errors that are hard to diagnose if there is already a hashed entry for the remote Git host.
Encrypted secrets in the repo are still not ideal: At runtime, the code needs to decrpyt the secret – where you keep the decryption key? The code in the repo makes it obvious where the key comes from. Typically, a large group has access to the repo – often including _former_ project members or all developers in the company.
The Open Source version has no Web UI, unfortunately. There are 3rd party projects, but they have other restrictions (i.e., AuthN options).
As a consequence, I’d recommend to plan the structure of your secrets and access roles for different environments right from the beginning of a project and automate the vault setup.
The AuthN methods on the right are primarily for the login of developers; the methods on the left are well suited for the login of applications.
The example uses Docker Swarm mode. After changes to the secret store in recent releases of Kubernetes, a similar approach is possible if you are using a Kubernetes cluster.