Overall aim: develop a methodology to improve understanding of the systemic and other factors influencing impacts arising from the implementation of regulatory change Objectives: Understand the form and magnitude of errors in current impact assessments Identify potential improvements in the general methodology in order to reduce errors Develop a framework methodology for the conduct of ex-post assessment of regulations

1. Developing approaches to ex-post
assessment of regulatory change impacts
at the farm level: prototype study
John Powell
Jane mills
CCRI
University of Gloucestershire

2. Aims and objectives
Overall aim:
• develop a methodology to improve understanding of
the systemic and other factors influencing impacts
arising from the implementation of regulatory change
Objectives:
• Understand the form and magnitude of errors in
current impact assessments
• Identify potential improvements in the general
methodology in order to reduce errors
• Develop a framework methodology for the conduct of
ex-post assessment of regulations

3. Role of IA in the policy
process

4. Research Design
1. What does the Literature say?
2. Selection of regulations
3. Collection of empirical data
4. Comparison of ex-ante and ex-post impact
assessments
5. Identification of causal factors accounting for
differences
6. A framework methodology for undertaking ex-post
impact assessment of regulatory change.

5. Evaluating regulatory
impact assessments
Reasons for ex- Recent studies
ante /ex-post • Significant variation in
differences quality
– Use of worst case • Limited policy options
scenarios
considered
– Strategic behaviour
• Limited impacts
– Compliance issues
might alter costs explored
– Incorrect assumption • Limited utility

6. Selection of Regulations
•Criteria for selection
•Nitrate Pollution Control regulations 2008
•Agricultural waste Regulations 2006

8. Comparison of ex-ante and ex-
post impacts: Nitrate Regulations 2008
Costs ex-ante: Costs ex-post:
– £655 - 1,009 million – £300 - 900 million
Total present value concentrated in the short-
discounted over 20yrs at term period after 2008
3.5%
• Benefits • Benefits
– £28 - 274 million TVP – No monetary estimate,
over 20 years potentially long-term and
small
• Comparisons between • Main differences related
measures to behavioural changes

9. Ex-ante/ex-post differences –
Nitrate Regulations
Category Magnitude of difference
Total costs Low
Total benefits Unclear - Low
Compliance costs:
Storage capacity Medium
Covered yards/run-off reduction measures Low
Reduction in stocking rate Unclear
Additional spreading costs Unclear
Spreading techniques Low
Compliance costs:
Moving and making more efficient use of slurry High
Reduction in fertiliser use Unclear
Compliance costs High
Increased farm labour/time costs High
Dairy Derogation Unclear – possibly high.
Planning costs Low
Implementation and enforcement Costs Unclear – possibly low
Small firms impact test Low
Competitiveness Unclear – possibly high

10. Comparison of ex-ante and
ex-post impacts: Agricultural Waste
Regulations 2006
Ex-ante: Ex-post:
• Costs: • Costs:
– Haz. Waste £28.7 – 69.8 – Haz. Waste £35.4 m/yr
m/yr – On + off farm £124-750 or
– On-farm £177-430/yr ave. £219/farm/yr.
– Off-farm 1 – 12% income
• Benefits – high, linked to • Benefits – not estimated –
improvements in soil and water Environmental possibly lower;
quality indirect benefits higher
• Comparisons with ex-post • Main differences due to high
measures implementation costs; farmer
behaviour; level of re-use and
recycling

11. Ex-ante/ex-post differences
– Waste Regulations
Category Magnitude Reason for difference
Environmental Benefits Environmental impacts overstated
Ammonia; Loss of fishery value; Bathing High Many practices altered to comply with
waters pollution; Direct impact of faecal farm assurance schemes.
pathogens to soil/water No mention of emissions from burning
plastic and other wastes.
Compliance Most farmers already in compliance
•On farm hazardous waste storage Low Largest waste stream (plastic) is
•On-farm disposal/recovery recycled.
•Off-farm disposal/recovery Farmers not disposing of all wastes.
Implementation costs Communication costs not considered –
The Environment Agency (EA) will face High in particular those of associated
increased costs in administration, stakeholders outside of government.
monitoring and enforcement.
Costs to the court system Lack of waste focused inspections.
Estimated 11 court prosecutions involving High EA prefers light touch approach
the agricultural sector.
Small firms Unclear No consideration of significance of
plastics recycling.

12. Exploring reasons for the
differences in estimates
Causal Category Description
Assumptions Differences caused by assumptions
Methodological Difference caused by ineffective or inaccurate
methods
Technological Unforeseen changes in technology; anticipated
changes that did not develop, or not had an impact
Sector Knowledge Erroneous or imperfect understanding of the sector
External drivers Failure to account for the impact of external forces
Implementation support Erroneous accounting
Systemic Failure to carry out actions or undertake studies
required; or, specific ‘ways of doing’ that cause
errors.

13. Exploring the differences in
estimates: Nitrate regulations
Assumptions
•Derogations
Technological
•Anaerobic digestion
•Reduction in stocking rates
•Spreading of slurry External drivers
•Fertiliser prices
Methodological Implementation
•Costs annualised over 20 yrs
•Increased time to manage nitrates
support
•Provision of advice/guidance
•Mapping and appeals
•monitoring
Systemic
•Timing
•Options explored
Sector knowledge
•Renting land
•Methodological
•Variable impacts

14. Exploring the differences in
estimates: Waste regulations
Assumptions Implementation
•Recycling behaviour support
•Provision of advice/guidance
•Communicating complex definitions
Methodological •IT system - registration
•Estimating on farm costs
External drivers
•Value of waste materials
Systemic
•Limited options explored
Sector knowledge
•Effects on other stakeholders

15. Suggestions for improving
ex-ante approaches
• Workshops/case study methods
• Scenario modelling
• Guidelines for implementation impact
measures
• Market forces
• Cost-benefit estimation

16. A framework for the conduct of
ex-post assessment of regulations

17. Proposed approach: key
issues
•IA documentation
•Identification of
impacts
•Collecting data
•small sample size
•Qualitative
•Validating data
•Accounting for
differences

18. Evaluation of the
proposed approach:
Strengths
•Impacts considered from the perspective of the
business unit affected
•Business and external influences are brought together
•Focus on identifying causal
•Pattern matching approach
•Qualitative assessment of costs and benefits
•Feeds recommendations into the next round of policy
review

19. Evaluation of the proposed
approach: Weaknesses
•Level of compliance
•Capturing variability
•Systemic differences
•Benefit measures
•Timing
•Resource requirements

20. Resource implications of
the modified approaches
Ex-ante IA
•70-90 person days
PIR
•35-75 person days
•Size/complexity of
legislation
•Potential benefits