1. Regulations: To Evaluate or Not
to Evaluate?
Catherine Dymond
Stacey Prieur
June 10, 2013

2. • International Context
• Domestic Context
• CIC Regulatory Context
• CIC Challenges
• Recent Policy Change
• So Now What?
2
Introduction and Overview

3. • New question in the international regulatory community:
how do we evaluate regulations?
• Organization for Economic Co-operation and Development
(OECD): intergovernmental organization that develops best
practices for member states
• Canada chaired OECD’s Regulatory Policy Committee in 2012
– Canada was key thought leader, and seen by many as paving
the way in practice, on the subject of how to monitor the
performance of regulations and how they should be evaluated
3
International Context

4. • In 2012, the OECD’s Council on Regulatory Policy and
Governance adopted 12 key recommendations, which
included recommendation #5:
“Conduct systematic programme reviews of the stock of
significant regulation against clearly defined policy goals,
including consideration of costs and benefits, to ensure that
regulations remain up to date, cost justified, cost effective
and consistent, and deliver the intended policy objectives.”
(emphasis added)
4
International Context

5. • Between 1975 and 1999, the Government of Canada
enacted approximately 25,000 regulations
• Deregulation and regulatory quality became a policy goal in
Canada in the 1980s
• Government of Canada Regulatory Policy 1992, 1995, 1999
– Focus on smaller, simpler, more efficient/effective regulation
– Some mention of monitoring impacts by TBS and departments
• Cabinet Directive on Streamlining Regulation (2007)
– Focus on maximizing net benefits to society and exploring
alternatives to regulation, through cost-benefit analysis
– Performance Measurement and Evaluation Plan (PMEP)
required for high impact regulations
5
Domestic Context

6. 6
PMS versus PMEP
Performance Measurement Strategy Performance Measurement and
Evaluation Plan
Policy /
Directive
TBS Policy on Evaluation;
Directive on the Evaluation Function
Cabinet Directive on Streamlining
Regulation
Policy Authority Centre of Expertise for Evaluation (CEE),
TBS
Centre of Regulatory Expertise (CORE),
TBS
Purpose Ensure credible and reliable performance
information is available to monitor and
assess results of programs
Ensure that regulatory activities meet
initial policy objectives and are
accordingly renewed on an ongoing basis
Requirement All programs, including transfer payment
programs, have a PMS
When the proposal is triaged as “high”
impact
Evaluation
responsibility
Heads of Evaluation responsible for
"reviewing and providing advice on PMSs”
Heads of Evaluation “must review the
PMEP”
Components • Program Profile
• Logic Model
• Performance Measurement Strategy
Framework
• Evaluation Strategy
• Overview of Regulatory Proposal
• Logic Model
• Performance Measurement Strategy
Framework
• Evaluation Strategy

7. • CIC is governed by two key Acts and 14 Regulations
– Immigration and Refugee Protection Act
– Citizenship Act
• Ways in which CIC uses regulation
– Change conditions (e.g., for applying, appealing)
– Legal enabler (e.g., to provide authority)
– Change behaviour (e.g., to deter)
7
CIC Regulatory Context

8. • 6 PMEPs completed to-date
– Visa Requirements for Mexican Nationals: changed the
conditions under which individuals could enter Canada
– Biometrics: provided legal authority to collect fingerprints on
temporary residents
– Refugee Reform “Designated Countries of Origin”: Changed
processing rules for individuals from typically non-refugee
producing countries to deter them from claiming asylum in
Canada
• 1 PMEP was underway
8
CIC involvement in PMEPs

9. • Nature of CIC regulations
– How do you monitor performance of a legal authority?
– How do you separate the impact of the regulations from a
larger program or initiative?
• Duplication of effort (PMS versus PMEP)
– Why the need for 2 documents? Who does the reporting?
– Drains resources in program areas and with stakeholders
• Policy Gap between CDSR and Evaluation Policy (2009)
– CDSR points to Evaluation Policy
– Evaluation Policy does not provide details for how to evaluate
regulations
9
CIC Challenges in Meeting the CDSR

10. • According to CORE, it has received just over 30 PMEPs from
seven different departments
• Evaluation divisions have not always been involved in the
development of the PMEP
– Why has Evaluation not been involved?
10
Other Departmental Involvement in PMEPs

11. • Cabinet Directive on Regulatory Management (2012)
– Focus on the life-cycle approach to regulatory management
– No mention of a PMEP requirement
• Identify the intended results, ensure that monitoring and reporting
activities are effective, integrate performance measures, collect
performance information (s. 44)
– TBS still required a PMEP for a high-impact regulatory
proposal
• Recent decision by TBS no longer requires a PMEP for high
impact proposals
– Unclear specific rationale for change in policy direction
11
Most Recent Policy Context

12. 1. Do we need to have a specific focus on performance
measurement and evaluation for regulations?
2. If yes, how do we do this and how do we add the most value?
– Measurement (PMS, PMEP, RIAS, DPR)
– Evaluation (What does the evaluation focus on? How do you
incorporate it into existing evaluation?)
3. If no, what is the risk of not doing it?
4. What kind of support /direction do we need from TBS (guidance,
policy instruction, etc.)?
12
So Now What – As Evaluators do We Care?