This document summarizes a presentation on test automation for agile data warehousing and business intelligence teams. It discusses why test automation is important for agile teams, challenges to test automation adoption, and provides an overview of a path to test automation including starting with unit tests and regression tests. It also demonstrates a simple approach to data warehouse test automation using test queries, expected and actual results, and test execution.
Bring:
Big stickies with markers for the parking lot
Business Cards
Laptop and power cord
Agile requires that we include QA activities in every iteration to verify that we are building shippable quality code.
Agile requires that we include regression testing in every iteration to ensure that we aren’t breaking previously working components.
Agile requires that we automate our tests so that regression testing doesn’t become a development bottleneck.
Effective Agile testing involves 3 key factors:
Tools for automated testing – This is relatively easy to adopt and learn. Typically takes a technical team less than a week to stand up this environment.
Principles and practices for effective testing – This requires discipline and commitment, and is more challenging than tools and automation.
A culture and attitude that values high quality in everything – This is the hardest factor to “teach” and to change.
Unlike conventional testing, which relies heavily on system and integration testing, Agile testing is principally supported by unit and component testing.
Unit & Component Testing – The technology-facing tests that support the development team
Made up of robust and fully automated unit and component tests
Highest percentage of automated tests
Written in the same language as the system under test (e.g., SQL)
Customers do not need to be able to read these tests
These tests are by far the quickest and easiest to write
They provide the quickest feedback, making them highly valuable
These tests have the highest ROI by far!
Acceptance Tests - The business-facing tests that support the development team by testing underneath the GUI
Functional tests that validate that we are building the right thing
Tests that cover broader chunks of business meaning or semantics
These tests are easier to write since they are underneath the BI application layer
These tests generally run more slowly since they cover more ground than unit tests
Try to write these tests in a domain specific language (not SQL) so that customers can understand them. This takes more time.
The feedback is quick, but not as quick as unit tests. Therefore ROI is moderate.
BI Application Tests - The business-facing tests that validate that we’ve built the right product by testing the user experience
Testing through the UI to validate the proper behavior and data under all usage scenarios
More expensive to write and adversely affected by changes to the UI
These tests tend to be written after development is complete
These tests are slower to run since they rely on the UI layer
These test provide a lot of value, but a relative low ROI
Manual Tests - Actual users performing exploratory testing on the BI applications
Subjective but invaluable mechanism for customer feedback
ROI is high as long as the other layers are robust, otherwise ROI is low
Users gain confidence in data validity through lower layers, and provide usability feedback at this layer
The fundamental steps in database test automation process are:
Load a fixed set of test data
Run the process that is under test
Verify the result set
Return everything to the way you found it
Database test automation involves these elements:
The component under test is generally an ETL module, SQL script, stored procedure, or other functional component.
The static test data is carefully constructed, as small as possible, and stored in version control.
The test queries and expected results are constructed with respect to the static test data.
The test runner handles everything else.
