1. Copyright 2010 ISA. All Rights Reserved.
Security Certification –
A Critical Review
Dr. Ragnar Schierholz
Kevin McGrath
Standards
Certification
ABB Corporate Research
Education & Training
Publishing
Conferences & Exhibits
Distributed with permission of author(s) by ISA 2010 Presented at ISA Automation Week 2010; http://www.isa.org

2. Presenter Copyright 2010 ISA. All Rights Reserved.
Dr. Ragnar Schierholz Kevin McGrath
• Research Area Coordinator for • Technical lead for security in
Secure Remote Service ABB’s Industrial Communication
Infrastructure in ABB’s Industrial research program
Software Systems research • R&D project manager for
program technology development
• Voting member of ISA 99 projects
committee representing ABB
Distributed with permission of author(s) by ISA 2010 Presented at ISA Automation Week 2010; http://www.isa.org
2

3. Outline Copyright 2010 ISA. All Rights Reserved.
• Background
• Security certification explained
– Economic fundamentals
– History of certification
– (Current approaches in industrial automation)
• Analysis
– Learn from the past
• Conclusions
Distributed with permission of author(s) by ISA 2010 Presented at ISA Automation Week 2010; http://www.isa.org
3

4. Background Copyright 2010 ISA. All Rights Reserved.
• Security standardization
– Setting a minimum level of acceptable security
– Enabling technical interoperability
• Information asymmetry & market failure
– «Market actors having imperfect, asymmetric information»
is one condition which can lead to market failure
– Hidden characteristics
– Hidden action/information
– Hidden intention
– Security properties of a product are difficult to
assess for a customer (hidden characteristics)
Distributed with permission of author(s) by ISA 2010 Presented at ISA Automation Week 2010; http://www.isa.org
4

5. Security certification explained Copyright 2010 ISA. All Rights Reserved.
Economics
Transaction cost economics Principal-Agent theory
• Allocate different costs to • Explains effects of con-
different stages of a market flicting interests under
transaction asymmetric information
and suggests governance
Stage Examples for associated activities and costs models
Initiation identification of transaction partners, e.g. marketing (on the
vendor’s side) and product/supplier search and comparison
– Conflicts:
(on consumers’ side)
– Moral hazard
Negotiation consulting and administrative costs for contract closure,
coordination costs in specification, delivery planning, etc. – Adverse selection
Settlement costs for product delivery, management of the exchange of – Hold-up
products and payments, validation of delivery and payment
– Governance models
Monitoring monitoring of quality and timeliness of transaction execution
– Signalling/Screening
Adjustment modification of contracts according to changes in
requirements – Self selection
– Institutional hierarchy
Distributed with permission of author(s) by ISA 2010 Presented at ISA Automation Week 2010; http://www.isa.org
5

6. Security certification explained Copyright 2010 ISA. All Rights Reserved.
History of certification
Certification of cyber security properties of software
products has been attempted in other industries
– Trusted Computer System Evaluation Criteria (TCSEC or
Orange Book)
– US Government initiative for systems used by government agencies
– Characteristics
– Direct interaction between government (NSA) and product vendor
– Test of systems in their context of use (incl. security organization)
– NSA tested against different sets of defined requirements
(higher level of certification means more comprehensive or stronger
requirements)
– Expensive, long testing procedures
Distributed with permission of author(s) by ISA 2010 Presented at ISA Automation Week 2010; http://www.isa.org
6

7. Security certification explained Copyright 2010 ISA. All Rights Reserved.
History of certification
Certification of cyber security properties of software
products has been attempted in other industries
– Information Technology Security Evaluation Criteria (ITSEC) /
IEC 15408 (Common Criteria)
– EU driven initiative, now internationally standardized, generic
certification of software product security
– Characteristics
– Tests against profiles selected/defined by product vendor
(Protection Profile, Security Target, Security Function
Requirements, Security Assurance Requirements)
– Tested by independent certification labs, accredited for certification
(Commercial Licensed Evaluation Facility - CLEF)
– Certification levels (EALs) depend on rigor of test procedure – not
on different product requirements
– Cost of certification depends on certification lab’s procedures
Distributed with permission of author(s) by ISA 2010 Presented at ISA Automation Week 2010; http://www.isa.org
7

8. Security certification explained Copyright 2010 ISA. All Rights Reserved.
History of certification
Certification of cyber security properties of software
products has been attempted in other industries
– ISO/IEC 27000 series
– International standard for certification of generic system security
– Characteristics
– Test of systems in their context of use (incl. security organization)
– Guidelines of testing / auditing defined in standard
– Cost of certification depends on auditor’s procedures
– No certification levels, pass/fail certification
Distributed with permission of author(s) by ISA 2010 Presented at ISA Automation Week 2010; http://www.isa.org
8

9. Security certification explained Copyright 2010 ISA. All Rights Reserved.
Current approaches in industrial automation
• Several certification approaches exist or are being
developed in the automation industry
– Wurldtech Achilles Communication Certification (ACC)
– Wurldtech Achilles Practices Certification (APC)
– MuDynamics MUSIC certification
– Exiday Integrity Certification
– ISCI ISASecure Certification (EDSA)
• More on this from the other speakers in this session
Distributed with permission of author(s) by ISA 2010 Presented at ISA Automation Week 2010; http://www.isa.org
9

10. Analysis Copyright 2010 ISA. All Rights Reserved.
• Issues found with certification programs
(to learn from the history, not to repeat it)
– Certification criteria
– Must be meaningful measurements of actual security property1
– Must be transparent so the principal can check for fit
– Must take the context of use into account
– Race to the bottom
– Certification labs only compete on price, but have no liability
– Incentive is to reduce cost by lax testing / auditing
– Adverse selection
– Only vendors who can’t demonstrate security with more meaningful
(possibly more expensive) signals will pursue certification
– Lifecycle coverage
– Recertification dilemma with new vulnerabilities or attack paths
Distributed with permission of author(s) by ISA 2010 Presented at ISA Automation Week 2010; http://www.isa.org
1See also S. Pfleeger and R. Cunningham, "Why Measuring Security Is Hard," IEEE Security & Privacy Magazine, vol. 8, 2010, pp. 46-54. 10
and further references in the paper

11. Conclusions Copyright 2010 ISA. All Rights Reserved.
• Security is not only a technical matter
• Economic theories explaining the environment and
suggesting solutions are out there
– Transaction cost economics
– Principal-agent theory
• Certification of security properties is one approach
– Has been tried several times and has failed (almost) as often
– Learn from mistakes, don’t repeat them
• Don’t forget alternative approaches
– Leverage the characteristics of the automation domain
– Large, few market actors where individual interaction is common
– Framework contracts reduce the frequency of transactions
Distributed with permission of author(s) by ISA 2010 Presented at ISA Automation Week 2010; http://www.isa.org
11

12. Questions? Copyright 2010 ISA. All Rights Reserved.
Ask now or contact us later!
Dr. Ragnar Schierholz
Principal Scientist
Industrial Software Systems
ABB Switzerland
Corporate Research
Segelhofstr. 1K
CH-5405 Baden 5 Dättwil
Phone +41 58 586 82 97
E-Mail ragnar.schierholz@ch.abb.com
Kevin McGrath
Scientist
Industrial Communication
ABB Norway
Corporate Research
Bergerveien 12
NO-1375 Billingstad
Phone +47 22 874 624
E-Mailby ISAkevin.mcgrath@no.abb.com Week 2010; http://www.isa.org
Distributed with permission of author(s) 2010 Presented at ISA Automation
12