Project Setup, Deplyoment and Maintenance of FLOW3 projects. By Christopher Hlubek - networkteam GmbH.

1. Project Lifecycle

2. The
Project Lifecycle
Project Setup, Deplyoment and Maintenance of FLOW3 projects.

3. About me
Christopher Hlubek
CTO networkteam GmbH
FLOW3 & TYPO3 Phoenix Core Team

4. Requirements
Support Architecture
Deploy Design
Test Implementation
Software
Development
Lifecycle

5. Code is not everything
Architecture Test Deploy Support
Software
Development
Lifecycle

6. Non-Functional
Project Risks
Framework updates and changes
Bugs in application code
Bugs in framework code
Bugs in 3rd party code
Erroneous deployment ...

7. Architecture

8. Composition
of a FLOW3 application
Build Application Packages
Configuration
Community Packages
Data
Framework Packages
Web
Distribution

9. Project Setup

10. Create a distribution
git clone --recursive -o typo3
git://git.typo3.org/FLOW3/Distributions/Base.git MyDistribution
git remote add origin ssh://myrepo.local/path/to/distribution.git

11. Versioning with Git
simple
Distribution
Application Packages
Your Git Server
TYPO3 / Community Git Community Packages
Server
TYPO3 Git Server
Framework Packages

12. Emergency Hotfix
- or -
Core Hacks

13. Versioning with Git
more flexible
Distribution
Application Packages
Your Git Server
TYPO3 / Community Git Community Packages
Server
TYPO3 Git Server
Framework Packages

14. Git Vendor Branching
Submodule
Pointer
origin/master [TASK] Latest ...
vendor/master
[FEATURE] Some feature ...
[BUGFIX] Some bugfix ...
[TASK] Lorem ipsum ...
Package Repository

15. Create a hotfix
git checkout master
git commit ...
git push origin master

16. Git Vendor Branching
Submodule
Pointer origin/master [HOTFIX] Custom patch
vendor/master[TASK] Latest ...
[FEATURE] Some feature ...
[BUGFIX] Some bugfix ...
[TASK] Lorem ipsum ...
Package Repository

17. Update the Package
git fetch vendor
git merge vendor/master master
git push origin master

18. Git Vendor Branching
Submodule
Pointer
origin/master Merge remote-tracking ...
vendor/master [TASK] New change ...
[HOTFIX] Custom patch
[TASK] Latest ...
[FEATURE] Some feature ...
[BUGFIX] Some bugfix ...
[TASK] Lorem ipsum ...
Package Repository

19. Test

20. Unit Tests vs.
Functional Tests
Application Class / Unit
} Unit Test
Application Package
Functional Test
FLOW3 Framework
Database

21. Unit Tests
Test complex logic in isolation
Find the source of a problem
Should test a restricted set of classes

22. Functional Tests
Test code with the framework and database
Check that a feature works
Isolation is difficult
Use mocks or fixtures for data or external services

23. Where to test?
High risk of failure
High complexity
Hard to test manually
Less Tests More Tests

24. Test strategies
101

25. Don‘t test everything
Test the important stuff

26. Test every feature with
one functional test
(at least)

27. Reproduce bugs
with tests
(they won‘t come back)

28. oyment Deployment Deploy
Deployment Deployment
Deployment

29. Environments
Integration Load-Testing
Live Staging
Development
Development

30. Manual deployment
# ssh testuser@my-staging-server.example.com
user123 $ cd /var/www/project
user123 $ git pull --rebase
user123 $ git submodule init
user123 $ git submodule update
user123 $ rm -rf Data/Temporary/Production
user123 $ ./flow3 doctrine:migrate
user123 $ ./flow3 cache:warmup
# ssh user123@my-live-server1.example.com
user123 $ cd /home/myproject/public_html
user123 $ git pull --rebase
user123 $ git submodule init
user123 $ git submodule update
user123 $ rm -rf Data/Temporary/Production
user123 $ ./flow3 doctrine:migrate
user123 $ ./flow3 cache:warmup
# ssh user123@my-live-server2.example.com
user123 $ cd /home/myproject/public_html
user123 $ git pull --rebase
Lot of work
user123 $ git submodule init
Erroneous
user123 $ git submodule update
user123 $ rm -rf Data/Temporary/Production
user123 $ ./flow3 doctrine:migrate
user123 $ ./flow3 cache:warmup

31. Automated
deployment
A custom script?
Rollback on errors?
Distributed deployment?

32. Meet Surf

33. Deployment with Surf
SSH Node
Surf
CI Server / Node
Dev Host
Git Repositories Git

34. The Surf Model
Deployment Environments
Workflow
Deployment
Application Code / Artifacts
Node Node Server

35. Deployment process
initialize Create release structure
update Git update
migrate Migration (e.g. Doctrine)
Stages finalize Finalize release
test Test release (smoke test)
switch Publish release to live state
cleanup Cleanup old releases

36. INFO Deploying vagrant
INFO Using workflow "Simple workflow"
DEBUG ====== Stage initialize ======
DEBUG **** Node vagrant ****
DEBUG * Application FLOW3 *
DEBUG Executing stage "initialize" on "vagrant" for application FLOW3
DEBUG Execute task "typo3.surf:createdirectories" on "vagrant" for application "FLOW3"
DEBUG $vagrant: "test -d /home/planetflow3/deploy"
DEBUG $vagrant: "mkdir -p /home/planetflow3/deploy/releases;mkdir -p /home/planetflow3/deploy/shared;mkdir -p /home/planetflow3/deploy/releases/20120331105449;cd /home/planetflow3/deploy/
releases;ln -snf ./20120331105449 next"
DEBUG Execute task "typo3.surf:flow3:createdirectories" on "vagrant" for application "FLOW3"
DEBUG $vagrant: "mkdir -p /home/planetflow3/deploy/shared/Data/Logs;mkdir -p /home/planetflow3/deploy/shared/Data/Persistent;mkdir -p /home/planetflow3/deploy/shared/Configuration"
DEBUG ====== Stage update ======
DEBUG **** Node vagrant ****
DEBUG * Application FLOW3 *
DEBUG Executing stage "update" on "vagrant" for application FLOW3
DEBUG Execute task "typo3.surf:gitcheckout" on "vagrant" for application "FLOW3"
DEBUG $vagrant: "git ls-remote git://github.com/chlu/Planetflow3-Distribution.git refs/heads/master | awk '{print $1 }'"
INFO > 7662553d5e666261a3a5bc2d6122096e7e21e40a
DEBUG $vagrant: "if [ -d /home/planetflow3/deploy/cache/localgitclone ]; then cd /home/planetflow3/deploy/cache/localgitclone && git fetch -q origin && git reset -q --hard
7662553d5e666261a3a5bc2d6122096e7e21e40a && git submodule -q init && for mod in `git submodule status | awk '{ print $2 }'`; do git config -f .git/config submodule.${mod}.url `git config -f .gitmodules
--get submodule.${mod}.url` && echo synced $mod; done && git submodule -q sync && git submodule -q update && git clean -q -d -x -ff; else git clone -q git://github.com/chlu/Planetflow3-
Distribution.git /home/planetflow3/deploy/cache/localgitclone && cd /home/planetflow3/deploy/cache/localgitclone && git checkout -q -b deploy 7662553d5e666261a3a5bc2d6122096e7e21e40a && git
submodule -q init && git submodule -q sync && git submodule -q update; fi"
INFO > synced Build/Common
INFO > synced Packages/Application/Libtextcat
INFO > synced Packages/Application/Planetflow3
INFO > synced Packages/Framework/Doctrine.Common
INFO > synced Packages/Framework/Doctrine.DBAL
INFO > synced Packages/Framework/Doctrine.ORM
INFO > synced Packages/Framework/Symfony.Component.Yaml
INFO > synced Packages/Framework/TYPO3.FLOW3
INFO > synced Packages/Framework/TYPO3.Fluid
INFO > synced Packages/Framework/TYPO3.Party
DEBUG $vagrant: "cp -RPp /home/planetflow3/deploy/cache/localgitclone/. /home/planetflow3/deploy/releases/20120331105449 && (echo 7662553d5e666261a3a5bc2d6122096e7e21e40a > /home/
planetflow3/deploy/releases/20120331105449REVISION)"
DEBUG Task "typo3.surf:flow3:symlinkdata" after "typo3.surf:gitcheckout" for application FLOW3
DEBUG Execute task "typo3.surf:flow3:symlinkdata" on "vagrant" for application "FLOW3"
DEBUG $vagrant: "mkdir -p /home/planetflow3/deploy/releases/20120331105449/Data;cd /home/planetflow3/deploy/releases/20120331105449;ln -sf ../../../shared/Data/Logs ./Data/Logs;ln -sf ../../../shared/
Data/Persistent ./Data/Persistent"
DEBUG Task "typo3.surf:flow3:symlinkconfiguration" after "typo3.surf:gitcheckout" for application FLOW3
DEBUG Execute task "typo3.surf:flow3:symlinkconfiguration" on "vagrant" for application "FLOW3"
DEBUG $vagrant: "cd /home/planetflow3/deploy/releases/20120331105449/Configuration;rm -f Production/*;rmdir Production;mkdir -p ../../../shared/Configuration/Production;ln -snf ../../../shared/
Configuration/Production Production"
INFO > rmdir: failed to remove `Production': No such file or directory
DEBUG ====== Stage migrate ======
DEBUG **** Node vagrant ****
DEBUG * Application FLOW3 *
DEBUG Executing stage "migrate" on "vagrant" for application FLOW3
DEBUG Execute task "typo3.surf:flow3:migrate" on "vagrant" for application "FLOW3"
DEBUG $vagrant: "cd /home/planetflow3/deploy/releases/20120331105449 && FLOW3_CONTEXT=Production ./flow3 typo3.flow3:doctrine:migrate"

37. Deployment structure
< Document Root

38. Deployment Features
Clustered deployment
Configuration overrides
Smoke tests
Custom tasks
Flexible workflows

39. ?
Questions?