How to assess your FSMS (Food Safety Management System)

Food Quality and Safety
Diagnosis of your food safety
management system
performance : output
Pieternel Luning & Willem Marcelis (WUR),
Liesbeth Jacxsens (UGent)

Topics
• Introduction
• Food Safety Management System (FSMS)
diagnostic instrument
• Food Safety Performance Indicators (SPI)
diagnose
• Example of case company X

Introduction
Safe products
Food safety assurance requirements
PRP, CODEX, GMP,HACCP, BRC, ISO22000, SQF, etc………
Company specific Food Safety Management System
Stakeholders
Government, retailers, branch organisations, sector boards, etc
Dynamic environment
Socio-demographic changes, technological developments,
emerging pathogens, global sourcing, etc

This slide illustrates that companies have to operate in a dynamic environment, and
that stakeholders (that are government, retailers, branch organisations, etc) put
assurance demands on the company’s FSMS.
They require that the set-up of the company’s FSMS is according to e.g. PRP, HACCP
guidelines and or BRC standard, and or ISO22000 standard, etc.
So the stakeholders affect the company’s FSMS set up by demanding the
implementation of certain QA guidelines/standards.
Keep in mind however, that each company has a UNIQUE FSMS which is a
translation of all the QA requirements into their own specific system

Food Safety Management System
Food Safety Management System
Stakeholder
requirements
Food safety assurance activities
to control safety management system and to provide
evidence and confidence to stakeholders about
meeting safety requirements
System requirements feedback
Assurance
on
Product
Safety
Food safety control activities
aim at keeping product properties, production
processes, and human processes between certain
acceptable tolerances
Product
safety

This slide is to show the companies that there are actually to outputs of their
FSMS.
One output is the realisation of safe products, which is the result of all actual
control activities as executed by equipment and people during daily operation
under the given contextual situation.
The other output is assurance of product safety, which means being able to give
the assurance that product as made under the company’s conditions are safe.
Control activities contribute to the first output whereas assurance (indirectly affects
the first output) but results in assurance.
So it does not mean that if a company is not/scarcely elaborating assurance
activities that products are not safe….but it induces a risk!!!

FSMS diagnostic instrument :
general picture
Product
characteristics
Process
characteristics
Organisational
characteristics
Environmental
characteristics
Product
safety
feedbackSystem requirements
Food safety assurance
Food safety control
Preventive measures design
Intervention processes design
Monitoring system design
Operation control strategies
Setting system requirements
Validation
Verification
Documentation and record keeping
Assurance
on product
safety

FSMS Diagnostic Instrument :
control activities
CORE CONTROL ACTIVITIES
•Sophistication hygienic design equipment & facilities
•Specificity of sanitation program
•Extent personal hygiene requirements
•Raw material control, etc….
Intervention processes design
•Adequacy intervention equipment
•Specificity maintenance program
•….etc
Monitoring system design
•Appropriateness CCP analysis
•Adequacy analytical equipment
•…..etc
Operation control strategies
•Appropriateness and compliance to procedures
•Actual performance cooling, intervention, measuring equipment ,..etc

FSMS Diagnostic Instrument :
assurance activities
Food safety control system
CORE ASSURANCE ACTIVITIES
Defining system set-up
•Sophistication translating external requirements
•Extent of systematic use of feedback information
Validation
•Sophistication validation
preventive measures
•etc
Verification
•Extent of verification of people
related performance
•Extent of verification equipment
related performance
Documentation and record-keeping

This slide shows the core assurance activities. The primary objectives of assurance
are to control the system, check its effectiveness, check whether it works in practice
as designed, and adapt the system when changes are necessary. All in order to
provide evidence and confidence to stakeholders that your system is able to comply
with the safety and assurance requirements.
An important assurance activity is defining the specific system set up for the company
(based on stakeholder requirements and internal feedback information about the
FSMS performance).
Validation is aimed ate checking effectiveness in advance, whereas verification is a
check afterwards (when to check if the system is functioning properly in practice).
Documentation and record keeping support above activities and play a major role in
providing evidence and confidence to stakeholders.

Example of grid
Indicator: Specificity cleaning and disinfection program
Mechanism
Complete, tailored
sanitation program
with appropriate
cleaning agents,
supported with
appropriate user
instructions better
prevents
contamination 
positive contribution
to food safety
Level 1
•Incomplete program
equal for all
•Equipment & facilities.
•Common cleaning
agents not specific
for production system.
•Cleaning instructions
derived from
information on label
or company
experience
Level 2
•Complete programme
equal for all equipment
and facilities.
•Cleaning agents
selected based on
advices of suppliers;
•specific agents for
typical product
applications.
•Idem for instructions
Level 3
•Complete programs,
tailored different
equipment/facilities
•Cleaning agents
•specifically modified
•and tested on
effectiveness for
specific food
production system.
•Instructions based
on test results.

Criteria to differentiate FSMS
(=FSC + FSA) levels
Control activities
• Level 3 high i.e. scientifically
underpinned accurate,
complete, stable, predictable,
tailored
• Level 2  medium, i.e. Best
practice knowledge, sometimes
variable, not fully predictable,
generic information
• Level 1  low i.e. lack of
scientific evidence, use
company experience, history,
variable, unknown,
unpredictable, common
materials, equipment
Assurance activities
• Level 3 high i.e. systematic,
pro-active, scientific evidence,
independent, experimental
trials, well-documented
• Level 2  medium, i.e. regular,
reactive, expert
knowledge/opinion, internal
independent, no actual testing,
restricted documentation
• Level 1  low i.e. ad hoc,
reactive, history/ company
knowledge, dependent, no
testing, no documentation

Product characteristics
Organisational characteristics
• Lack of technical workforce
• Variability in workforce composition
• Insufficiency operators competence
• Lack of commitment
• Deficiency of employee involvement
• Absence of formalisation
• Insufficiency information support
systems
Process characteristics
Environmental characteristics
Food safety assurance
system
Food safety control
system
Product
safety
Assuranc
e on
product
safety
CONTEXTUAL FACTORS FOOD SAFETY MANAGEMENT SYSTEM OUTCOME

Criteria to differentiate context situations
• Situation 1 : not
critical, not
vulnerable
• Situation 2 :
potentially critical,
potentially
vulnerable
• Situation 3 : highly
critical, highly
vulnerable
• Situation 1: high ability,
advanced, specific
• Situation 2:
constrained ability,
restricted advanced,
specific
• Situation 3: low ability,
not advanced not
specific
•Situation 1: low
dependency, not
vulnerable
•Situation 2: restricted
dependency,
potentially vulnerable
•Situation 3: highly
dependent, highly
vulnerable
Product/process Organisational Environmental

interpretation of results
• Addresses:
– Core control activities
– Core assurance activities
– The contextual factors of company
• Assesses
– at which level control & assurance activities are
executed (0,1,2,3, simple sophisticated)
– in which risky contextual situation the FSMS has to
operate (1,2,3 not demanding very demanding and
riskfull)
• Aggregates
– Insights in relationships between context, activities
and food safety performance

• Output of the assessment gives indication of:
– Insights in relationships between context, activities
and food safety performance
– At which level the specific control and assurance
activities of food safety control and assurance are
now
– How to go – what is needed for the higher level in
control and assurance activities
– Middle or long term improvements can be identified
• NOT:
– Judgement tool  no audit is performed
interpretation of results

Interpretation results of contextual factors
• The contextual situations are represented in spiderweb diagrams
 A more coloured spiderweb diagram is associated with a more
demanding and risky contextual situation
 A more risky contextual situation is expected to result more easily in
food safety problems, which will put higher demands on the FSMS.
• To obtain an overall picture of the contextual situation, overall scores
have been assigned  For this purpose the mean scores for a certain
set of characteristics are calculated by dividing the sum of situation
numbers by the number of situation assessments for this specific set of
characteristics (see the legend of a spiderweb diagram).
• The overall score has been assigned by interpreting the mean score of
all contextual characteristics as:
– Overall score 1: if mean score of characteristics is 1 - 1.2
– Overall score 1-2: if mean score of characteristics is 1.3-1.7
– Overall score 2: if mean score of characteristics is 1.8 - 2.2
– Overall score 2-3: if mean score of characteristics is 2.3 - 2.7
– Overall score 3: if mean score of characteristics is 2.8 - 3.0

Interpretation results of Food Safety Control and
Assurance Activities
• The levels of the core safety control and assurance activities are
represented in spiderweb diagrams
The spiderweb diagrams represent the detailed levels of the core safety
control activities and the core safety assurance activities
 A more coloured spiderweb diagram is associated with a higher/more
sophisticated level of control activities and assurance activities.
• To obtain an overall picture of the FSMS activities, overall scores have
been assigned For this purpose the mean scores of levels for a
certain set of activities were calculated by dividing the sum of activity
levels by the number of activities of this specific set.
• The overall score has been assigned by interpreting the mean score:
– Overall score 1: if mean score of activities is 0 - 1.2
– Overall score 1-2: if mean of score of activities is 1.3-1.7
– Overall score 2: if mean score of activities is 1.8 - 2.2
– Overall score 2-3: if mean score of activities is 2.3 - 2.7

Assumption behind diagnosis
• High risks products and/or processes (= higher overall
score in contextual factors) put higher demands on
FSMS to achieve safety requirements
• Less supporting organisational conditions and/or higher
chain dependency (= higher overall score in contextual
factors) result in higher impact on FSMS to achieve
safety requirements
Which FSMS levels is necessary?
It depends !!

Assumption of output
Context FSMS FS
3
1
2
1
2
3
1
2
3
3=most dangerous 3=highest level 3=best performance

Food safety performance indicators
• The microbiological food safety output of a companies
Food Safety Management System can be measured via
microbiological analysis of raw materials, intermediated
products and final food products
• But the food safety output can as well be measured via
Food Safety Performance Indicators (SPI)
• Indicators are measurable, objective, quantitative
measures of key system elements performance
• They indicate the extent upto which a certain Food
Safety Management System meets the needs and
expectations towards microbiological food safety
• They can be applied for self-evaluation or for
benchmarking
• They are applied in this work to measure the food safety
output

For each indicator, a food safety level is attributed :
• Level 0 : not conducted, not present, not appropriate
• Level 1 : low performance of food safety  problems
regarding food safety can be expected and are not
under control by the current food safety management
system
• Level 2 : medium performance of food safety  problems
regarding food safety can be expected and are under
control by the current food safety management system
• Level 3 : high performance of food safety  no problems
regarding food safety can be expected

• To obtain an overall picture of the food safety
performance, overall scores have been assigned.
• For this purpose the mean scores of levels for a certain
set of food safety performance indicators need to be
calculated by dividing the sum of performance levels by
the number of indicators of this specific set.
• The overall score has been assigned by interpreting the
mean score:
– Overall score 1: if mean score of activities is 0 - 1.2
– Overall score 1-2: if mean of score of activities is 1.3-1.7
– Overall score 2-3: if mean score of activities is 2.3 - 2.7

Assumption behind Food Safety
Performance Indicators diagnosis
• More sophisticated FSMS would be better
able to realise products with lower
contamination levels and less deviation in
contamination loads
Different FSMS level/context
situations
may result in good FS !!

Profiles contextual situation
company X
Product and process characteristics
0
1
2
3
Risk level raw materials
Risk level most critical
product groups
Safety contribution of
packaging concept
Intervention steps
Level of production
process
changes
Rate of product / process
design changes
1-2 Environmental characteristics
0
1
2
3
Safety contribution in chain
position
Power in supplier
relationship
Authority in customer
relationship
Strictness
of legislative
requirements
2
Organizational characteristics
0
1
2
3
Technical workforce
Variability workforce
composition
Operators' competence
Management commitmentEmployee involvement
Formalisation
Supporting information
systems
2-3
• Overall context score:
2

Profiles of core control activities
company X
0
1
2
3
Product specific measures
Hygienic design
Critical cooling facilitiesSanitation program
Personal hygiene
requirements
1-2 Monitoring system design
0
1
2
3
CCP analysis
Standards and tolerances
assessment
Analytical methods
Measuring equipmentCalibration program
Sampling design
Corrective actions
2
Operation of FSCS
0
1
2
3
Appropriateness of
procedures
Compliance to
procedures
Actual hygienic
performance of equipment
and facilities
Actual cooling capacity
Actual process
capability
of intervention process
Actual performance of
measuring equipment
Actual analytical
equipment performance
2
Intervention process design
0
1
2
3
Intervention equipment
Maintenance programIntervention methods
0
Overall score control
activities 1-2

Profile of core assurance activities
company X
Food safety assurance system
0
1
2
3
Translating external
requirements
Use of feedback
information
Validating
preventive measures
Validating
intervention systems
Validating monitoring system
Verifying people related
performance
Verifying equipment and
methods related
performance
Documentation and record
keeping system
Overall score
assurance activities
1-2
System req.: 1-2
Validation: 1
Verification: 2-3
Documentation: 2

Assigning overall scores to mean scores
of control and assurance activity levels
• Overall score 1 if mean score of activities 0 - 1.2
• Overall score 1-2 if mean of score of activities 1.3-
1.7
• Overall score 2 if mean score of activities 1.8 - 2.2
• Overall score 2-3 if mean score of activities 2.3 - 2.7
• To calculate the overall score for FSMS one should
take all original scores as basis!!!

of contextual situations
• Overall score 1-2 if mean of score of activities 1.3-
1.7
• To calculate the overall score for context one should
take all original scores as basis!!!

company x
Overall score  2 (because average of the score is 2)

of food safety performance indicators
• Overall score 1-2 if mean of score of activities 1.3-1.7
• To calculate the overall score for FSMS one should take
all original scores as basis!!!

Overall results company X
Context FSMS FS
3
1
2
1
2
3
1
2
3
3=most dangerous 3=highest level 3=best performance
1-2

Conclusion : overall results company X
• Assumption for this company : In context 2 we
expect FSMS level 2 and FS level 3
• so, FS level lower than expected probably due to
FSMS level being lower than 2
• With FS level 2 there is a potential risk on
microbiological food safety
• When FS level perceived as unacceptable by
the company, measures could be discussed in
FSMS activities (towards level 2/3) and or in the
context (towards level 1)
• Possible measures/interventions can be
discussed from a detailed analysis of the spider-
webs

Detailed discussion FSMS interventions
for company X
• Score 0: Product specific measures; intervention strategies
• Score 1: Extent of personal hygiene requirements; corrective actions;
appropriateness of procedures; use of feed back information and
validation
• Score 2: Sophistication hygienic design, specificity sanitation program,
compliance to procedures, sampling design and tolerances design
(important for control raw materials)
• Resulting in discussions about:
-preventing raw material contamination by product specific measures
and or raw material control (including sampling design, tolerances
design of monitoring system)
-possible techological intervention strategies (because now score 0)
-personal hygiene, hygienic design, hygiene control, and cleaning and
disinfection program
-use of information for feedback and corrective actions
-adequacy of procedures and instructions and compliance to procedures
-validation of preventive measures and monitoring system

Detailed discussion context interventions
for company X
• Score 3: Risk level of raw materials and products; lack of
technological staff; lack of information systems; lack of
formalisation/procedures; lack of management commitment;
strictness of legislative requirements
• This results in discussions about:
-reducing the risk level of raw materials (i.e.. Reducing initial load
and variation in initial load) in collaboration with suppliers
-redesigning their information system (to support decision making)
-the quality management system with its procedures (introduce
some extent of formalisation)
-people in quality staff positions (technological staff, people
responsible for FSMS activities), enhance competences/skills (e.g.
by training)
-management commitment

Thanks for your attention
and
for contributing to our research
2 2 3

How to assess your FSMS (Food Safety Management System)

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (6)

Semelhante a How to assess your FSMS (Food Safety Management System)

Semelhante a How to assess your FSMS (Food Safety Management System) (20)

Último

Último (20)

How to assess your FSMS (Food Safety Management System)