SlideShare a Scribd company logo
1 of 14
Download to read offline
Formal Verification Theory
and Practice
Tim Blackmore
Theory
Considerations for formal verification

      When writing properties need to consider
     1. Complexity
     2. Level of abstraction
     3. Completeness
     4. Consideration of reachable states




October 2008         Copyright © Infineon Technologies 2008. All rights reserved.   Page 3
1. Complexity

     For a good property checker complexity is not normally a major
     issue
        May be compilation issues e.g. for memories
         ¬ Black box and model
         May be issues due to property complexity
         ¬ Very long properties may need to be split into shorter properties
         ¬ Large multipliers will need special consideration




October 2008             Copyright © Infineon Technologies 2008. All rights reserved.   Page 4
2. Level of abstraction

     Properties should be closer to the specification than RTL
       Use of temporal languages (PSL, SVA, …)

     Whole transaction described in single property
      Bus access (request, address and data phase)
      Instruction execution (not individual pipe stages)

     Utilise built-in operators (e.g. arithmetic and logic operators to
     verify an ALU)




October 2008           Copyright © Infineon Technologies 2008. All rights reserved.   Page 5
3. Completeness

     A complete set of properties describes
       All interactions of DUV with system (outputs, registers)
       For all valid input sequences

     E.g. to verify a bus slave may need 3 properties to describe all
     possible input sequences
        Write
        Read
        No-op
     All the slave outputs are described in each property




October 2008          Copyright © Infineon Technologies 2008. All rights reserved.   Page 6
Assertions

     Level of abstraction and completeness of properties applies
     equally to assertions to be checked during simulation

     E.g. a good way to develop bus protocol checkers is to write a
     complete set of transaction-level assertions
       Ensure that checking bus agents are obeying protocol in every
       cycle of simulation
       1-2 weeks work

     Also 1-2 weeks work to write a complete set of transaction-level
     properties to check protocol adherence exhaustively but …




October 2008          Copyright © Infineon Technologies 2008. All rights reserved.   Page 7
4. Reachable states

     … they would all fail

     E.g. read, write and no-op properties need to start in general
     reachable state so can be placed end-on-end

     Property checkers cannot determine the reachable states of a
     design
       False failures due a property starting in an illegal state
       Not a problem for simulation - starts at reset and drives only
       legal inputs

     Property checker requires user input to exclude such false
     failures
        Often the major effort for formal verification
        BUT not always e.g. combinatorial designs



October 2008           Copyright © Infineon Technologies 2008. All rights reserved.   Page 8
Semi-formal verification

     Property checker can be used from reset (semi-formal
     verification, hybrid, ‘bug-hunting’)
       Can be effective at finding bugs but no practical, meaningful
       coverage metric
       Unlikely to be exhaustive – not formal verification




October 2008          Copyright © Infineon Technologies 2008. All rights reserved.   Page 9
Practice
Where we apply formal verification

     In theory could formally verify a complete system
       Large undertaking requiring many experts
     In practice tends to be targeted - based on RFE
       10-20% of verification effort spent on formal

     Lower effort blocks
       Combinatorial blocks – see next slide
       Can still provide high return

     Higher effort blocks with high return
       Critical sub-blocks e.g. complex blocks where bugs lead to
       unavoidable data corruption (fetch unit, cache controller)
       Highly re-usable blocks (bus MIFs and SIFs)
       Effort comparable to developing stand-alone test bench and
       applying coverage-driven verification



October 2008         Copyright © Infineon Technologies 2008. All rights reserved.   Page 11
Combinatorial Blocks

     Combinatorial ALUs can be (almost) exhaustively verified quickly
       E.g. arithmetic in processor (most integer and floating point
       arithmetic, load and store address calculation, branch address
       calculation)
       Ensures instruction set compliance between different core
       versions

     Error-correction codes
       Encoder/decoder pair can be showed to function correctly for
       all data values and error conditions in hours (including
       property development)
       Can be safety-critical




October 2008         Copyright © Infineon Technologies 2008. All rights reserved.   Page 12
Results we achieve

     Formal verification always finds bugs
       Number found depends on
         ¬ Block complexity
         ¬ Extensiveness of simulation-based verification
         ¬ How long block has been in the field

     Formal verification rarely misses a bug
       Bugs can be missed due to human error
         ¬ Property set incorrect and matches RTL (very rare)
         ¬ Incomplete property set (now tool assisted)




October 2008             Copyright © Infineon Technologies 2008. All rights reserved.   Page 13
Use of formal verification grows …

     Once you start using a formal tool you see new uses
      Register access verification – push button solution from
      register database to formal verification
      Clock domain crossing
      Simplification of coverage closure
      …

     Often reduce effort and increase confidence




October 2008         Copyright © Infineon Technologies 2008. All rights reserved.   Page 14

More Related Content

Viewers also liked

Educational research qualitative sampling
Educational research  qualitative samplingEducational research  qualitative sampling
Educational research qualitative samplingRidwanul Mosrur
 
Language Mechanics
Language MechanicsLanguage Mechanics
Language MechanicsMGC1987
 
Need help with academic writing
Need help with academic writingNeed help with academic writing
Need help with academic writinghaleyhansens
 
Writing For Audience Curation
Writing For Audience CurationWriting For Audience Curation
Writing For Audience CurationMaroutcha Mouawad
 
Sampling in qualitative research
Sampling in qualitative researchSampling in qualitative research
Sampling in qualitative researchDr. V Vorvoreanu
 
Sampling in qualitative researc
Sampling in qualitative researcSampling in qualitative researc
Sampling in qualitative researckavita yadav
 
Mechanics Of Writing Chapter 3 (MLA)
Mechanics Of Writing Chapter 3 (MLA)Mechanics Of Writing Chapter 3 (MLA)
Mechanics Of Writing Chapter 3 (MLA)AISHA PERVEEN
 
Micro and Macro Skills of Writing Found in The Writing Exercises of The Bridg...
Micro and Macro Skills of Writing Found in The Writing Exercises of The Bridg...Micro and Macro Skills of Writing Found in The Writing Exercises of The Bridg...
Micro and Macro Skills of Writing Found in The Writing Exercises of The Bridg...Dwi Firli Ashari
 
Identifying the Inquiry and Stating the Problem
Identifying the Inquiry and Stating the ProblemIdentifying the Inquiry and Stating the Problem
Identifying the Inquiry and Stating the ProblemDAPHNIE MONTEVERDE
 
The Mechanics of Writing a Research Report
The Mechanics of Writing a Research ReportThe Mechanics of Writing a Research Report
The Mechanics of Writing a Research Reportvimal nair
 
Identifying the inquiry and stating the problem(Practical Research)
Identifying the inquiry and stating the problem(Practical Research)Identifying the inquiry and stating the problem(Practical Research)
Identifying the inquiry and stating the problem(Practical Research)majoydrew
 
Research title & knowing the problem
Research title & knowing the problemResearch title & knowing the problem
Research title & knowing the problemBean Malicse
 
Academic Writing Skills 1
Academic Writing Skills 1Academic Writing Skills 1
Academic Writing Skills 1Dilip Barad
 
Sample size
Sample sizeSample size
Sample sizezubis
 
Types of academic writing
Types of academic writingTypes of academic writing
Types of academic writingvmargutti
 

Viewers also liked (17)

Educational research qualitative sampling
Educational research  qualitative samplingEducational research  qualitative sampling
Educational research qualitative sampling
 
Mechanics of writing
Mechanics of writingMechanics of writing
Mechanics of writing
 
Language Mechanics
Language MechanicsLanguage Mechanics
Language Mechanics
 
Need help with academic writing
Need help with academic writingNeed help with academic writing
Need help with academic writing
 
Writing For Audience Curation
Writing For Audience CurationWriting For Audience Curation
Writing For Audience Curation
 
Sampling in qualitative research
Sampling in qualitative researchSampling in qualitative research
Sampling in qualitative research
 
Sampling in qualitative researc
Sampling in qualitative researcSampling in qualitative researc
Sampling in qualitative researc
 
Mechanics Of Writing Chapter 3 (MLA)
Mechanics Of Writing Chapter 3 (MLA)Mechanics Of Writing Chapter 3 (MLA)
Mechanics Of Writing Chapter 3 (MLA)
 
Micro and Macro Skills of Writing Found in The Writing Exercises of The Bridg...
Micro and Macro Skills of Writing Found in The Writing Exercises of The Bridg...Micro and Macro Skills of Writing Found in The Writing Exercises of The Bridg...
Micro and Macro Skills of Writing Found in The Writing Exercises of The Bridg...
 
Identifying the Inquiry and Stating the Problem
Identifying the Inquiry and Stating the ProblemIdentifying the Inquiry and Stating the Problem
Identifying the Inquiry and Stating the Problem
 
The Mechanics of Writing a Research Report
The Mechanics of Writing a Research ReportThe Mechanics of Writing a Research Report
The Mechanics of Writing a Research Report
 
Identifying the inquiry and stating the problem(Practical Research)
Identifying the inquiry and stating the problem(Practical Research)Identifying the inquiry and stating the problem(Practical Research)
Identifying the inquiry and stating the problem(Practical Research)
 
Research title & knowing the problem
Research title & knowing the problemResearch title & knowing the problem
Research title & knowing the problem
 
Academic Writing Skills 1
Academic Writing Skills 1Academic Writing Skills 1
Academic Writing Skills 1
 
Academic Writing
Academic WritingAcademic Writing
Academic Writing
 
Sample size
Sample sizeSample size
Sample size
 
Types of academic writing
Types of academic writingTypes of academic writing
Types of academic writing
 

Similar to Bristol 2009 q1_blackmore_tim

YuryMakedonov_GUI_TestAutomation_QAI_Canada_2007_14h
YuryMakedonov_GUI_TestAutomation_QAI_Canada_2007_14hYuryMakedonov_GUI_TestAutomation_QAI_Canada_2007_14h
YuryMakedonov_GUI_TestAutomation_QAI_Canada_2007_14hYury M
 
Stateful mock servers to the rescue on REST ecosystems
Stateful mock servers to the rescue on REST ecosystemsStateful mock servers to the rescue on REST ecosystems
Stateful mock servers to the rescue on REST ecosystemsNuno Caneco
 
Speed Up Synchronization Locks: How and Why?
Speed Up Synchronization Locks: How and Why?Speed Up Synchronization Locks: How and Why?
Speed Up Synchronization Locks: How and Why?psteinb
 
Paremus service fabric
Paremus service fabricParemus service fabric
Paremus service fabricpjhInovex
 
Hybrid Automation Framework Developement
Hybrid Automation Framework DevelopementHybrid Automation Framework Developement
Hybrid Automation Framework DevelopementGlasdon Falcao
 
DO-254 for dummies 7
DO-254 for dummies 7DO-254 for dummies 7
DO-254 for dummies 7DMAP
 
EasyTest Test Automation Tool Introduction
EasyTest Test Automation Tool IntroductionEasyTest Test Automation Tool Introduction
EasyTest Test Automation Tool IntroductionZhu Zhong
 
Agile Development in Aerospace and Defense
Agile Development in Aerospace and DefenseAgile Development in Aerospace and Defense
Agile Development in Aerospace and DefenseJim Nickel
 
Kirin User Story: Migrating Mission Critical Applications to OpenStack Privat...
Kirin User Story: Migrating Mission Critical Applications to OpenStack Privat...Kirin User Story: Migrating Mission Critical Applications to OpenStack Privat...
Kirin User Story: Migrating Mission Critical Applications to OpenStack Privat...Motoki Kakinuma
 
How to Make Istio Work with Your App
How to Make Istio Work with Your AppHow to Make Istio Work with Your App
How to Make Istio Work with Your AppKarenBruner
 
How to Make Istio Work with Your App
How to Make Istio Work with Your AppHow to Make Istio Work with Your App
How to Make Istio Work with Your AppStackRox
 
Plantwide benefits of EtherNet IP Seminar
Plantwide benefits of EtherNet IP Seminar Plantwide benefits of EtherNet IP Seminar
Plantwide benefits of EtherNet IP Seminar RoutecoMarketing
 
Top 5 Automation Challenges Webinar
Top 5 Automation Challenges WebinarTop 5 Automation Challenges Webinar
Top 5 Automation Challenges WebinarPerfecto by Perforce
 
What's New with Perfecto? - May 2017
What's New with Perfecto? - May 2017What's New with Perfecto? - May 2017
What's New with Perfecto? - May 2017Lizzy Guido (she/her)
 
Leverage Virtual Design to Build a Better System
Leverage Virtual Design to Build a Better SystemLeverage Virtual Design to Build a Better System
Leverage Virtual Design to Build a Better SystemRockwell Automation
 
The Verification Methodology Landscape
The Verification Methodology LandscapeThe Verification Methodology Landscape
The Verification Methodology LandscapeDVClub
 
Advancing From Fault Management To Fault Resolution Framework
Advancing From Fault Management To Fault Resolution FrameworkAdvancing From Fault Management To Fault Resolution Framework
Advancing From Fault Management To Fault Resolution FrameworkTTI Telecom
 

Similar to Bristol 2009 q1_blackmore_tim (20)

YuryMakedonov_GUI_TestAutomation_QAI_Canada_2007_14h
YuryMakedonov_GUI_TestAutomation_QAI_Canada_2007_14hYuryMakedonov_GUI_TestAutomation_QAI_Canada_2007_14h
YuryMakedonov_GUI_TestAutomation_QAI_Canada_2007_14h
 
Stateful mock servers to the rescue on REST ecosystems
Stateful mock servers to the rescue on REST ecosystemsStateful mock servers to the rescue on REST ecosystems
Stateful mock servers to the rescue on REST ecosystems
 
Speed Up Synchronization Locks: How and Why?
Speed Up Synchronization Locks: How and Why?Speed Up Synchronization Locks: How and Why?
Speed Up Synchronization Locks: How and Why?
 
Paremus service fabric
Paremus service fabricParemus service fabric
Paremus service fabric
 
Hybrid Automation Framework Developement
Hybrid Automation Framework DevelopementHybrid Automation Framework Developement
Hybrid Automation Framework Developement
 
DO-254 for dummies 7
DO-254 for dummies 7DO-254 for dummies 7
DO-254 for dummies 7
 
EasyTest Test Automation Tool Introduction
EasyTest Test Automation Tool IntroductionEasyTest Test Automation Tool Introduction
EasyTest Test Automation Tool Introduction
 
Agile Development in Aerospace and Defense
Agile Development in Aerospace and DefenseAgile Development in Aerospace and Defense
Agile Development in Aerospace and Defense
 
Kirin User Story: Migrating Mission Critical Applications to OpenStack Privat...
Kirin User Story: Migrating Mission Critical Applications to OpenStack Privat...Kirin User Story: Migrating Mission Critical Applications to OpenStack Privat...
Kirin User Story: Migrating Mission Critical Applications to OpenStack Privat...
 
How to Make Istio Work with Your App
How to Make Istio Work with Your AppHow to Make Istio Work with Your App
How to Make Istio Work with Your App
 
How to Make Istio Work with Your App
How to Make Istio Work with Your AppHow to Make Istio Work with Your App
How to Make Istio Work with Your App
 
Plantwide benefits of EtherNet IP Seminar
Plantwide benefits of EtherNet IP Seminar Plantwide benefits of EtherNet IP Seminar
Plantwide benefits of EtherNet IP Seminar
 
Top 5 Automation Challenges Webinar
Top 5 Automation Challenges WebinarTop 5 Automation Challenges Webinar
Top 5 Automation Challenges Webinar
 
What's New with Perfecto? - May 2017
What's New with Perfecto? - May 2017What's New with Perfecto? - May 2017
What's New with Perfecto? - May 2017
 
Leverage Virtual Design to Build a Better System
Leverage Virtual Design to Build a Better SystemLeverage Virtual Design to Build a Better System
Leverage Virtual Design to Build a Better System
 
The Verification Methodology Landscape
The Verification Methodology LandscapeThe Verification Methodology Landscape
The Verification Methodology Landscape
 
Vishal_Resume
Vishal_ResumeVishal_Resume
Vishal_Resume
 
Birendra_resume
Birendra_resumeBirendra_resume
Birendra_resume
 
Advancing From Fault Management To Fault Resolution Framework
Advancing From Fault Management To Fault Resolution FrameworkAdvancing From Fault Management To Fault Resolution Framework
Advancing From Fault Management To Fault Resolution Framework
 
Next-gen Automation Framework
Next-gen Automation FrameworkNext-gen Automation Framework
Next-gen Automation Framework
 

More from Obsidian Software (20)

Zhang rtp q307
Zhang rtp q307Zhang rtp q307
Zhang rtp q307
 
Zehr dv club_12052006
Zehr dv club_12052006Zehr dv club_12052006
Zehr dv club_12052006
 
Yang greenstein part_2
Yang greenstein part_2Yang greenstein part_2
Yang greenstein part_2
 
Yang greenstein part_1
Yang greenstein part_1Yang greenstein part_1
Yang greenstein part_1
 
Williamson arm validation metrics
Williamson arm validation metricsWilliamson arm validation metrics
Williamson arm validation metrics
 
Whipp q3 2008_sv
Whipp q3 2008_svWhipp q3 2008_sv
Whipp q3 2008_sv
 
Vishakantaiah validating
Vishakantaiah validatingVishakantaiah validating
Vishakantaiah validating
 
Validation and-design-in-a-small-team-environment
Validation and-design-in-a-small-team-environmentValidation and-design-in-a-small-team-environment
Validation and-design-in-a-small-team-environment
 
Tobin verification isglobal
Tobin verification isglobalTobin verification isglobal
Tobin verification isglobal
 
Tierney bq207
Tierney bq207Tierney bq207
Tierney bq207
 
The validation attitude
The validation attitudeThe validation attitude
The validation attitude
 
Thaker q3 2008
Thaker q3 2008Thaker q3 2008
Thaker q3 2008
 
Thaker q3 2008
Thaker q3 2008Thaker q3 2008
Thaker q3 2008
 
Strickland dvclub
Strickland dvclubStrickland dvclub
Strickland dvclub
 
Stinson post si and verification
Stinson post si and verificationStinson post si and verification
Stinson post si and verification
 
Shultz dallas q108
Shultz dallas q108Shultz dallas q108
Shultz dallas q108
 
Shreeve dv club_ams
Shreeve dv club_amsShreeve dv club_ams
Shreeve dv club_ams
 
Sharam salamian
Sharam salamianSharam salamian
Sharam salamian
 
Schulz sv q2_2009
Schulz sv q2_2009Schulz sv q2_2009
Schulz sv q2_2009
 
Schulz dallas q1_2008
Schulz dallas q1_2008Schulz dallas q1_2008
Schulz dallas q1_2008
 

Bristol 2009 q1_blackmore_tim

  • 1. Formal Verification Theory and Practice Tim Blackmore
  • 3. Considerations for formal verification When writing properties need to consider 1. Complexity 2. Level of abstraction 3. Completeness 4. Consideration of reachable states October 2008 Copyright © Infineon Technologies 2008. All rights reserved. Page 3
  • 4. 1. Complexity For a good property checker complexity is not normally a major issue May be compilation issues e.g. for memories ¬ Black box and model May be issues due to property complexity ¬ Very long properties may need to be split into shorter properties ¬ Large multipliers will need special consideration October 2008 Copyright © Infineon Technologies 2008. All rights reserved. Page 4
  • 5. 2. Level of abstraction Properties should be closer to the specification than RTL Use of temporal languages (PSL, SVA, …) Whole transaction described in single property Bus access (request, address and data phase) Instruction execution (not individual pipe stages) Utilise built-in operators (e.g. arithmetic and logic operators to verify an ALU) October 2008 Copyright © Infineon Technologies 2008. All rights reserved. Page 5
  • 6. 3. Completeness A complete set of properties describes All interactions of DUV with system (outputs, registers) For all valid input sequences E.g. to verify a bus slave may need 3 properties to describe all possible input sequences Write Read No-op All the slave outputs are described in each property October 2008 Copyright © Infineon Technologies 2008. All rights reserved. Page 6
  • 7. Assertions Level of abstraction and completeness of properties applies equally to assertions to be checked during simulation E.g. a good way to develop bus protocol checkers is to write a complete set of transaction-level assertions Ensure that checking bus agents are obeying protocol in every cycle of simulation 1-2 weeks work Also 1-2 weeks work to write a complete set of transaction-level properties to check protocol adherence exhaustively but … October 2008 Copyright © Infineon Technologies 2008. All rights reserved. Page 7
  • 8. 4. Reachable states … they would all fail E.g. read, write and no-op properties need to start in general reachable state so can be placed end-on-end Property checkers cannot determine the reachable states of a design False failures due a property starting in an illegal state Not a problem for simulation - starts at reset and drives only legal inputs Property checker requires user input to exclude such false failures Often the major effort for formal verification BUT not always e.g. combinatorial designs October 2008 Copyright © Infineon Technologies 2008. All rights reserved. Page 8
  • 9. Semi-formal verification Property checker can be used from reset (semi-formal verification, hybrid, ‘bug-hunting’) Can be effective at finding bugs but no practical, meaningful coverage metric Unlikely to be exhaustive – not formal verification October 2008 Copyright © Infineon Technologies 2008. All rights reserved. Page 9
  • 11. Where we apply formal verification In theory could formally verify a complete system Large undertaking requiring many experts In practice tends to be targeted - based on RFE 10-20% of verification effort spent on formal Lower effort blocks Combinatorial blocks – see next slide Can still provide high return Higher effort blocks with high return Critical sub-blocks e.g. complex blocks where bugs lead to unavoidable data corruption (fetch unit, cache controller) Highly re-usable blocks (bus MIFs and SIFs) Effort comparable to developing stand-alone test bench and applying coverage-driven verification October 2008 Copyright © Infineon Technologies 2008. All rights reserved. Page 11
  • 12. Combinatorial Blocks Combinatorial ALUs can be (almost) exhaustively verified quickly E.g. arithmetic in processor (most integer and floating point arithmetic, load and store address calculation, branch address calculation) Ensures instruction set compliance between different core versions Error-correction codes Encoder/decoder pair can be showed to function correctly for all data values and error conditions in hours (including property development) Can be safety-critical October 2008 Copyright © Infineon Technologies 2008. All rights reserved. Page 12
  • 13. Results we achieve Formal verification always finds bugs Number found depends on ¬ Block complexity ¬ Extensiveness of simulation-based verification ¬ How long block has been in the field Formal verification rarely misses a bug Bugs can be missed due to human error ¬ Property set incorrect and matches RTL (very rare) ¬ Incomplete property set (now tool assisted) October 2008 Copyright © Infineon Technologies 2008. All rights reserved. Page 13
  • 14. Use of formal verification grows … Once you start using a formal tool you see new uses Register access verification – push button solution from register database to formal verification Clock domain crossing Simplification of coverage closure … Often reduce effort and increase confidence October 2008 Copyright © Infineon Technologies 2008. All rights reserved. Page 14