Think it's impossible to make a significant change in government? Well, it's not easy, but GoLeanSixSigmna.com Black Belt Pamela Kuehling made a real difference from the inside. She overturned countless "honest wrong beliefs" with facts and data by modeling the Lean Six Sigma approach in her work and won over the doubters. Initially the overall procurement process was the focus, but when that proved to be too big, she scoped it down to a more manageable project that can now be the basis for an expanded improvement. Her improvement was meaningful, but a bigger victory was in bringing critical and logical thinking into an environment often impacted by less scientific influences!

1. Black Belt Project Storyboard
Reduction in Cycle Time
for Bid Tab Creation
Pamela Kuehling, Director of Procurement
City of St. Louis, MO

2. BACKGROUND
• The City of St. Louis procures about $115 million in non-construction products and services through the
Supply Division. Products purchased range from pencils to firetrucks, from car wash services to computer
software. This is done using a “green screen” Cobol-based system from 1983, and Excel.
• The Supply Division services 69 City Departments/Divisions/Agencies with 10 full time staff (3 are Buyers).
76% of our dollars and 37% of the quantity of items are purchased via individual bids, rather than on contract.
Individual bids are usually used for more expensive, less-frequently used items. In FY 2018, Supply processed
more than 4,800 individual bids.
• The Supply Division operates in a very rigid legal structure including the City Charter, City Ordinances, Civil
Service Rules, and State and Federal laws and regulations. Process ownership is often unclear, and progress
may not be possible based on politics, in addition to lack of technology or normal fear of change.
• Our internal customers are the City Departments who need items, and the external customers are Vendors
who sell us the items. Vendors change frequently.
• Procurement is so complex that the Supply Division does not control most of the process. This is due to legal
requirements, no real technology/overnight batch processing, and redundant approval processes which are
all non-value added.
• Procurement delays cause:
• Department frustration, which can lead to “going around the system,” which may cost the City more money
• Wasted time and taxpayer money
• Possible legal exposure from disgruntled vendors
• Most importantly, delays of services to the Citizens of St. Louis
2

3. Executive Summary
Key Take Away: The reduction in cycle time is clear, which means customers are
experiencing less controllable delays as a result of bid tab creation cycle time.
Executive Summary
Business Case Project Results
Bid Tabs are created after product bids are received from vendors, and are entered
in Excel spreadsheets to show an apples to apples comparison. This helps
Departments make better selection decisions.
The project was a great success. Most of the remaining delay time in the bid tab
creation process is legislated and out of scope/control. Goal was 49% reduction.
Vs. Same Time Last Year, we created a 22% reduction in average days, a 7%
reduction in standard deviation and an 11% reduction in the range of days for bid
tab creation cycle time.
Vs. 2013-2018 Historical Data, average days declined 58%, standard
deviation declined 48% and the range of days declined 67%.
Hopefully we get the added staff and Excel training we requested, which should
help even further with cycle time reduction.
The Team is engaged and motivated for further projects!
The scope was narrowed to advertised bids over $5,000, which had the highest level
of customer complaints. Why should we address this situation? (1) to save time to
reduce Department frustration (VOC), (2) to encourage proper purchasing behavior
by the Departments (compliance), (3), reduce possible legal exposure from
disgruntled vendors, and (4) potentially save money and delays for the taxpayers. If
we do not do this, we won’t be able to meet our Mission, which is “to ethically,
efficiently and cost-effectively source products and services that Departments
need in order to service the Citizens of the City of St. Louis.
Root Cause Analysis
Note: Most of the Root Causes were out of control of the Supply Division.
-- No formal backup plan for sick and vacation time
-- Buyers have limited capacity/handle too much volume. Need > staff
-- Buyers don’t report roadblocks upwards for help in removal
-- Excel training
-- Buyers possibly using wrong priorities to plan work
Graphical Display of Improvement
Number of Days for Bid Tab Creation – Before vs. After in Days
Solutions Implemented
Solutions could only focus on what the Supply Division controlled
-- Created formal backup plan for sick and vacation time
-- Asked for more staff and Excel training in Budget 2020 (will find out 7/1/19)
-- Created new procedure for reporting roadblocks to Management for help in
removing those roadblocks
3

4. Define Phase

5. Project Charter
Key Take Away: This project will immediately impact internal and external customer
satisfaction which is key to Supply Division alignment with our Mission.
Project Charter
Cycle Time for the Creation of Bid Tabs in Procurement (Supply Division)
Problem Statement Business Case & Benefits
Many purchases for the City of St. Louis are made via individual bids.
Long delays in the process lead to department frustration and attempts to
bypass proper channels. The Supply Division does not control most of
the parts of the process. The longest part we can address is the creation
of Bid Tabulations (Bid Tabs). Bid tabs are created by the Buyer after
bids are received from Vendors, and used by the Departments to compare
bids and determine which vendor will receive the award. About $115
million dollars are involved in this process.
Why should we address this situation: (1) to save time to reduce
Department frustration (VOC), (2) to encourage proper purchasing
behavior by the Departments (compliance), (3), reduce possible legal
exposure from disgruntled vendors, and (4) potentially save money and
delays for the taxpayers. If we do not do this, we won’t be able to meet
our mission, which is “to ethically, efficiently and cost-effectively source
products and services that Departments need in order to service the
Citizens of the City of St. Louis.
Goal Statement Timeline
Decrease the time it takes to create a Bid Tab from approximately 3 days
to 2 days by 5/30/19.
Phase Planned Completion Date Actual
Define: 11/30/18 11/12/18
Measure: 12/31/18 1/29/19
Analyze: 2/28/19 3/19/19
Improve: 4/19/19 4/24/19
Control: 5/15/19 5/9/19
Scope In/Out Team Members
1st Process Step Buyer gets copies of bids on closing date Position Person Title % of Time
Last Process
Step
Buyer emails Bid Tab to Department for award
confirmation and selection
Team Lead Pam K. Black Belt
Candidate
20%
Sponsor Todd W. Dir of Ops 5%
In: Purchasing processes with existing technology & staff Team Member Lynn C. Deputy Comm. 10%
Out: New ERP system, approval functions, IT functions,
legal requirements (Charter and Ordinance), and the
Budget Process
Team Member Dot Buyer 15%
Team Member Rico H. Contract
Specialist
15%
5

6. Voice of the Customer Translation Matrix
Customer Comment ID the Issue/Priority Customer Requirement
Department Approval Process
(Question A and E)
Timeliness Takes longer than they thought.
Supply does not control this part
of the process but used it for
illustration purposes
Approvals and Legal
Requirements take too long
(Question B & C)
Timeliness Legislated and can’t be changed.
All parties accept this fact.
Supply Division takes too long to
process our bids (Question D & F)
Timeliness 2 days instead of 3
Voice of the Customer
Key Take Away: Timeliness is important to internal customer satisfaction, and the
comments highlight frustration. Customers think it takes longer for us to process than it
actually does (5.74 days per the Departments vs. 2.26 days actually on average)
Note: Survey Questions and Answers are in the Appendix

7. SIPOC
S
Supplier
I
Inputs
P
Process
O
Outputs
C
Customer
Vendors who
send bid in for
potential City
orders
Bids for a
particular
Requisition
Number for a
Department
See Gemba
Results on next
slide for Bid Tab
Creation
Bid Tabulation
Excel
Spreadsheet
(ex. in Appendix)
Requesting
Department
(to be used for
Citizen)
Requirements: Customer wants 2
day turnaround on
bid tabulations
Key Take Away: The scope of the project is from when the Supply Division open the vendor
bids on the due date though when the Buyer sends the completed Bid Tab to the
requesting Department. The process is manual at this time.
7

8. “As Is” Current Detailed Map Segment
Bid Tabulation Creation and Distribution Process
Phase
Public Opening at
noon
Deputy Comm.
Gives Buyer the
bids received
for each Req.
Buyer
prioritizes
order of
completion
1) Hot item
2) Simple to
do
3) Technical
OFI-Variation
Buyer creates
spreadsheet
(copy template or
old bid)
Buyer enters
data into the
spreadsheet
Buyer reviews and
double checks
that Bid Tab
matches bids
Buyer
determines
low bid from
Bid Tab
Buyer reviews for
City Business
Preference match
w/in 2%
Buyer
documents low
bid at bottom
of Bid Tab
Buyer sends
Bids and Bid
Tab to Dept.
(scan or mail)
Takes time
Best Practice –
scan in order of
low to high bid
for Dept. ease
Buyer sends
email and gives
Dept. 7 days to
respond
Buyer follows up
in one week if
award decision
not made
Note: Some bids are more complicated than others because of what they are(ie. firetruck bids are more complicated than bids for seeds or plants for the Parks
Division). Supply has no control over the complexity of each potential bid. In addition, although scanning or snail mailing the bids and Bid Tab to the Department are
time consuming, they are required so that the Departments can make an educated decision as to whether the low bid actuallymeets specifications and will be
acceptable. All decisions by Supply are dictated by Charter constraints and Ordinances.
Since I am onlyreviewing the Bid Tab Creation process, I am not focused on giving the Departments less than 7 days to respond. That will be another test – to see if
that helps lower the overall cycle time without creating imperfect decision making, which is very time consuming in the long term.
The workload is heavy – some days we have 22 bids opening, some days less.
Key Take Away: Lots of room for technology and other improvements in this process
8

9. Measure Phase

10. Data Collection Plan
Key Take Away: Since this is a cycle time project, the measure is a continuous,
non-normally distributed measures of time. One day.
Measure Data Type Operational
Definition
Stratification
Factors
Who and
How
Bid Tab
Creation Cycle
Time
Continuous Number of
days from
when the
Buyer gets a
closing
Requisition and
all of the
vendor bids
received for
the Requisition
By last year
same month
By historical
data from
2013 – 2018
(Note: Civil Service
rules do not allow
measurement of
performance by
Buyer)
Buyer to create
an Excel Log of
when bids are
received and
bid tabs are
sent to the
Department for
approval. Data
will include
input from all 3
Buyers.
10

11. Data Collection Process
Sample Size Calculator - Continuous Data
Sample Data (user inputs):
Estimate of Standard Deviation S 1
Desired margin of error
delta / half-
interval 0.25
Population Size (optional) N
Confidence level (enter .95 for
95%) 100*(1-α)% 95.0%
Results:
Minimum Sample Size n 62
n (adjusted for
small N)
Descriptive Statistics Bid Tab Creation
Count 116
Mean 2.983
StDev 4.170
Range 24
Minimum 0
25th Percentile (Q1) 1
50th Percentile (Median) 1
75th Percentile (Q3) 3.750
Maximum 24
95.0% CI Mean 2.2158 to 3.7497
95.0% CI Sigma 3.6939 to 4.7887
Anderson-Darling Normality Test 11.042
P-Value (A-D Test) 0.0000
Skewness 2.653
P-Value (Skewness) 0.0000
Kurtosis 8.726
P-Value (Kurtosis) 0.0000
Key Take Aways: The range of days reflects most of the
issues. The data is very skewed, and is non-normal.
Because the mean is closer to the lower specification
unit we use Cp1, which is 0.11, which means that our
current process has poor capability to meet customer
requirements.
Note: Both Pam and Rico got sample data from
documents, and then switched 30 of them and got the
exact same results. We did not do a full blown MSA. 11

12. Baseline Run Chart – Cycle Time in Days
Key Take Away: There is lots of variation in the
process and the average time to create a Bid
Tabulation
Nonparametric Runs Test: Duration
Number of Runs about Median: 53
Expected Number of Runs about Median: 58.931
Number of Points above Median: 56
Number of Points equal to or below Median: 60
P-Value for Clustering: 0.1340
P-Value for Mixtures: 0.8660
P-Value for Lack of Randomness (2-Sided): 0.2681
Number of Runs Up or Down: 78
Expected Number of Runs Up or Down: 77
P-Value for Trends: 0.5878
P-Value for Oscillation: 0.4122
12

13. Baseline – Bid Tab Cycle Time
Key Take Away: The range of number of days reflects the problem – reducing
the range, mean and standard deviation will make customers more satisfied. 13

14. Analyze Phase

15. Fishbone Diagram
Bid Tab Creation Takes too Long
Long term – create formal backup plan for
vacationing or sick buyers so their work gets done –
obvious fix, no testing needed. Legal Change needs
2/3 voter approval so may never happen.
Possible Test Options:
• Change/Update Laws -
improbable
• Use Clerical Help for Data
Entry (if Budget $)
• Excel Training (if Budget $)
• Tell Director re: Roadblocks
OLD System -
Need New
Excel
Training
Clerical Support Staff
Human
Capacity
Vacation Backup
Schedule
Policies and Procedures
Sick Leave Backup
Schedule
Charter & Ordinance Requirements
Minimum Dept. Purchasing Level
Advertising Requirements
Civil Service
Multiple, Unnecessary Approval Layers
Yellow = Out of Control/Scope
Green = Requested in 2020
Budget but can’t test now
Possible Causes of Delays in Bid Tab Creation
Complexity of
Bid
15
The 5 Why’s are
shown in the
Appendix

16. Value Added Flow Analysis
Key Take Away: Process analysis showed most processes were NOT value added nor in our
control at the Supply Division
Value Added Flow Analysis - Bid Tabulation Creation Cycle Time
Time is measured in days - there is large variation in cycle times and data is non-normal
Process Step
Step Label (VA, NVA,
NVA-r)
Value Added
Time
NVA & NVA-r Required Work
Time
NVA - Wait
Time Notes
Clerk logs req into book, Deputy assigns to Buyer & sets open
date NVA 1 3 steps done on the same day
Buyer review/research potential vendors VA 1 Saves money if we get more competition
Deputy creates journal ad & uploads to web - required by law NVA-r 2-5 days Only printed once/week so timing can create delays
Advertising Period - required by law. Receive sealed bids NVA-r 21 Required minimum by law
Bids opened & given to Buyer VA 1 Usually same day as above
Buyer reviews bids & creates Bid Tabulation VA 1-6 days Too much variation
Buyer sends Bid Tab to Department VA 1 via email or snail mail if complex bid
Dept. reviews for award VA 1-10 days Could be shorter
Buyer gets missing docs (if any) from winner NVA-r 0-5 days Not frequent but very time consuming
Deputy reviews for accuracy NVA 1-4 days Deputy is very busy
Director Approves NVA-r 1-2 days Required
Comptroller Approves NVA-r 1-6 days Required
Clerk picks up Req and gives to Buyer NVA 1 Not electronic - quicker than mailroom
Buyer reviews & creates Purchase Order VA 3 If there is a typo, must fix and reprint overnight
PO prints overnight NVA 1 OLD System
Buyer reviews PO for accuracy. Director approves NVA-r 1 2 steps on same day
PO scanned & emailed to vendor VA 1
Most of the wasted time is required by law -- the advertising and multiple required approval levels. Out of the 8.1 weeks, 3 weeks are required for advertising, so real "work" time is about 5 weeks. Of that, over a week
is spent on required approvals & duplicate signatures. This was an eye-opener. Of what processes the Supply Division controls, most are not likely seen as "value added." The Bid Tabulation Creation process is the
longest value added process we control.
16

17. Hypothesis Testing Plan – Focus on Buyers telling
Management about roadblocks on day 3 of bid tab creation
Possible X
Null
Hypothesis
Alternative
Hypothesis
Hypothesis
Test
P Value Results
Tell Management
about Roadblocks
so they can be
cleared faster
There is no
difference in the bid
tab creation cycle
time if a Buyer tells
Management about
roadblocks on day 3
or if they don’t
Management
intervention to clear
up road blocks will
reduce the cycle
time for bid tab
creation
Two Sample Test P = .000 Reject Null
Practical Problem: Buyers may be wary of telling Management of issues/delays. We have a new Deputy Supply
Commissioner who was recently promoted from within. Change management and participation will be needed.
The test was run for 21 work days. Buyers told Pam K. when a bid tab was incomplete after 3 days. At that time, Pam
immediately intervened to help expedite the process. The “Director” title can expedite responses in a control and
command structured environment like the City of St. Louis.
Key Take Aways: The histograms that follow show the tightening of data points. The run charts also show the
improvement in the new vs. last year same month vs. historical data pattern.
The two sample t-Test for the new process vs. historical data showed a P value of zero, which supports the alternative
hypothesis that the new process impacted cycle time significantly.
An email update was sent to all Stakeholders 3/24/19 and a Team meeting was held on 3/27/19 (photos in Appendix)
17

18. Management Intervention Matters
Mean, Std. Dev. and Trend Declined
Key Take Aways: Versus last year at the same time, variability & mean cycle time were reduced
18

19. Bid Tab Creation Cycle Time – New Process Improvement Is Even
Greater vs. Historical Data from 2013 - 2018
Key Take Aways: Even more visible improvement in mean and standard
deviation exists compared to historical data 19

20. 2013 – 2018 Historical Data Run Chart
“Before” Cycle Time Data from 2013-2018
Key Take Away: Baseline bid tab creation cycle time data was out of control 20

21. April 2018 Run Chart
“Before Data” – Same Month Last Year
Key Take Away: April of 2018 was not as out of control as the overall
historical data for 2013-2018, but not at an acceptable level. 21

22. Key Take Away: April of 2018 was not
as out of control as the overall
historical data for 2013-2018.
April 2019 Run Chart
22
“After” Data with New Process in 04/19

23. Key Take Away: Using a Two Sample t-Test we found that there was improvement in the process cycle time new vs.
last year same time. The mean decreases from 1.6 to 1.25 days, or 22%, and the standard deviation went from 2.354
to 2.179, a 7% reduction. However, since the P value is > 0.05, we can not reject the null hypothesis that the mean
difference = 0. This root cause is not statistically significant. However, versus historical data, the P value = 0, so we
can reject the null hypothesis that there is no difference in the means.
New vs. Last
April
New vs. 2013 –
2018 Historical
Data
Test Results – Verification of Root Causes
23

24. Improve Phase

25. Based on the matrix, the Team proposed the
following Action Plan:
1) Actively participate in the City-wide effort
to implement an ERP system (out of
scope). This will improve cycle time for all
Supply processes.
2) During the Budget process, I requested 2
new FTE’s to help manage and level load
workflows.
3) Low impact, easy process change was a
quick win. It caused backlash at first, but
we are nurturing and rewarding a team
focus vs. a “me” mentality.
4) We decided not to focus on Low Impact,
difficult processes. I have requested
funding in the Budget for Excel Training,
but are not sure of the direct impact on
cycle time. Errors using old bid tabs as
template don’t occur frequently, so we
disregarded that as an improvement tactic.
5) The Team decided to emphasize the easier,
high impact ideas as focus to achieve
goals. Those will be our solutions to
approach.
Selected Solutions
Key Take Away: The visual aid of the Impact/Effort
Matrix made decision-making easier for the Team 25

26. Improve Phase Implementation Plan
In looking at overall cycle time for the bid process, we
implemented cross training as a quick win. It was
approached at the same time we got a new Buyer, so
all Buyers got a “refresher course,” and were cross
trained in other buying categories. We will see over
time if this helps, but can’t really design an
experiment around it. It should help with workload
balancing and load leveling for the Buyers.
• Since Buyers sit at their computers all day, work
cell design was not a critical factor in the Team’s
perception. Each Buyer has made their space
comfortable for them, and we recently bought new
chairs and ergonomic keyboards for everyone.
How this will impact bid cycle time is unclear.
• The Kanban as a visual signal to trigger action was
approved by the Team and created as a reminder
in the Lunch Room. The notice reminds Buyers
what to do if a bid tab takes longer than 3 days to
create.
• The Team used brainstorming techniques to
determine the x factors that will help reduce bid tab
creation cycle time.
Key Take Away: The A3 test was another useful visual for the Team to assess options and
brainstorm possibilities

27. Implementation Plan –
“3 Days to Remove Roadblocks” Pilot Plan
Key Take Away: This Implementation Plan was key in terms of all team members
knowing their role in the roll-out, and participation increased buy-in. 27

28. SUCCESS
Level Settings: Compare number of days for
the cycle time, the average number of days,
standard deviations, and the range of
number of days.
Compare: Compare historical bid tab
creation time to time under the new
methodology of Management Assistance
after 3 days.
To eliminate any noise from timing issues,
we compared the same month from this
year vs. last year in addition to comparing
this month vs. historical data.
Going Forward Action Plan: The Supply
Division will permanently incorporate the “3
Day Rule” into our Process Manual
Key Take Away: The results were clear enough to make the path an easy choice

29. Risk Management - FMEA
Key Take Away: A new ERP system/technology will be greatly useful in minimizing risk by forcing compliance
These occur before the Bids are received, but contribute to the delays in Bid Tab creation
Process
Step/Input
Potential
Failure
Mode
Potential
Failure Effects
Severity
Potential
Causes
Occurrence
Current Controls
Detection
RPN
Action
Recommendation
Responsible
Person Action Taken
Severity
Occurrence
Detection
RPN
Bid Receipt Complex Bid
Delays in
Procurement 7
Complex
need or
product 5
Visual Inspection
by Buyer 6 210
Ask Depts for
Clarification on
Specs Each Buyer
Buyers call
Dept. if
needed 7 4 5 140
Bid Receipt
Incomplete
Bid
Delays in
Procurement 6 Vendor Error 4
Visual Inspection
by Buyer 6 144
Remind Vendors
in Instructions Lynn C.
Added bolded
directions 6 4 6 144
Bid Receipt
Unclear
Specs
Delays in
Procurement 9 Dept. Error 3
Visual Inspection
by Buyer 9 243
Train
Departments
Pam and
Lynn
Purchasing
School 9 2 8 144
These occur after the Bids are received
Process
Step/Input
Potential
Failure
Mode
Potential
Failure Effects
Severity
Potential
Causes
Occurrence
Current Controls
Detection
RPN
Action
Recommendation
Responsible
Person Action Taken
Severity
Occurrence
Detection
RPN
Bid Tab
Creation Complex Bid
Incorrect Bid
Tab 9
Vendor not
follow
directions,
technical
specs 5
Reject Bid or Ask
for Clarification
(Rework) 7 315
Use spreadsheets
for bid
responses, wait
for ERP Solution Pam and Dot Incomplete
TB
D TBD TBD TBD
Bid Tab
Creation
Incomplete
Bid
Delay in
approval of
award 9
Vendor not
follow
directions 4
Buyer reviews
each bid 6 216
Reject all
incomplete bids
now. Hardcode
into ERP system
in future Pam Incomplete
TB
D TBD TBD TBD
Bid Tab
Creation Any
Delay in
procurement 7 Any 5
Buyer reviews
each bid 6 210
Tell Mgt. about
Roadblocks
Pam and
Lynn Implemented 5 2 5 50
F M E A Form
29

30. “To Be” Going Forward Process Map
Key Take Away: Management intervention works well in a hierarchical
environment and has improved bid tab creation cycle time
Revised Bid Tabulation Creation Process
Phase
Public Opening at
noon
Deputy Comm.
Gives Buyer the
bids received
for each Req.
Buyer
prioritizes
order of
completion
1) Hot item
2) Simple to
do
3) Technical
Buyer creates
spreadsheet
(copy template or
old bid)
Buyer enters
data into the
spreadsheet
Buyer reviews and
double checks
that Bid Tab
matches bids
Buyer
determines
low bid from
Bid Tab
Buyer reviews for
City Business
Preference match
w/in 2%
Buyer
documents low
bid at bottom
of Bid Tab
Buyer sends
Bids and Bid
Tab to Dept.
(scan or mail)
Best Practice –
scan in order of
low to high bid
for Dept. ease
Has it been 3
days?
Request
Management
Roadblock
Assistance
Has it been 3
days?
This may look more complicated but it has sped up the
process and reduced Buyer frustration
No No
Yes Yes
30

31. Proof of Improvement
Mean: 1.25
0
1
2
3
4
5
6
7
8
RunChart:BidTabCreationTime
NUMBER OF DAYS FOR BID TAB CREATION-NEWVERSION
Nonparametric Runs Test: Bid Tab Creation Time P-Value for Lack of Randomness (2-Sided): 0.6547
Number of Runs about Median: 3 Number of Runs Up or Down: 7
Expected Number of Runs about Median: 2.833 Expected Number of Runs Up or Down: 7.667
Number of Points above Median: 1 P-Value for Trends: 0.3102
Number of Points equal to or below Median: 11 P-Value for Oscillation: 0.6898
P-Value for Clustering: 0.6726
P-Value for Mixtures: 0.3274
Mean: 1.60
0
1
2
3
4
5
6
7
8
9
RunChart:BidTab
NUMBER OF DAYS FOR BID TAB CREATION-APRIL 2018 VERSION
Nonparametric Runs Test: Bid Tab Number of Runs Up or Down: 8
Expected Number of Runs Up or Down: 9.667
Number of Runs about Median: 7 P-Value for Trends: 0.1382
Expected Number of Runs about Median: 6.867 P-Value for Oscillation: 0.8618
Number of Points above Median: 4
Number of Points equal to or below Median: 11
P-Value for Clustering: 0.5372 P-Value for Lack of Randomness (2-Sided): 0.9256
P-Value for Mixtures: 0.4628
Nonparametric Runs Test: Duration
Number of Runs about Median: 53 P-Value for Lack of Randomness (2-Sided): 0.2681
Expected Number of Runs about Median: 58.931
Number of Points above Median: 56 Number of Runs Up or Down: 78
Number of Points equal to or below Median: 60 Expected Number of Runs Up or Down: 77
P-Value for Clustering: 0.1340 P-Value for Trends: 0.5878
P-Value for Mixtures: 0.8660 P-Value for Oscillation: 0.4122
Number of Days for Bid Tab Creation 2013-2018 Data
Continuous historical data is not available due to
turnover in the Supply Division 31

32. Control Phase

33. Monitoring & Response Plan
Key Take Away: As long as Buyers keep to the new process of the “3 Day Plan,” we
should continue to see improved cycle times. 33

34. Control Chart of Project Y
Key Take Away: The team is continuing to work on the special causes of variation – and
the Upper Control Limit of 5 days is the Response Plan trigger. 34

35. Project Closure
Lessons Learned Customer Impact
Do's and Don'ts for Future Efforts Positive Impacts on Customers
Defining the Process is the hardest part, but it must be done correctly to
improve Understanding the reality of the data was helpful for internal customers.
Stakeholders need sound communication Stakeholder Interest grew as the process continued.
You don't control as much of the process as you think you do. Make sure
you understand that situation before wasting time/effort wanting to
change what you can't. Less frustration for internal and external customers.
Small, incremental change can have a big impact on performance. Quick,
small wins motivate participation Steady improvement in performance, reductions in delays.
Visual communication works well in addition to verbal
Clear communication reduced frustration and sped up the removal of
roadblocks.
Innovation is helped by collaboration, but not a democracy
Enthusiasm and education help the change management process Improved engagement in the Procurement process
Removing roadblocks improves trust in Management and processes.
Mentoring others to learn Six Sigma will help with future efforts
All eyes focused on the Voice of the Customer and the customer
experience
Don't try to have one person do everything Tasks completed faster, more accurately
Final Calculations of Savings or Gains: Soft Savings - Cost or Time - Short Term
Almost a day was saved in the Bid Tab creation process
Hard Dollar Savings: None at this time. Buyers have more time to focus on other tasks
Less departmental frustration may improve compliance
Process Owner (Pam K)
Understands process change Soft Savings - Cost or Time - Long Term
Willing to continue monitoring new process Lower department frustration may improve compliance
Created new process documentation Taxpayer savings due to error correction and detection
Shared new process documentation with Team
The Project Sponsor, Todd W. has signed off on this project.
35

36. Appendix
Key Words/Definitions
• Bid Tabulation (Bid Tab) = Excel spreadsheet summarizing the bids received for a particular
Requisition or Departmental need
• City Charter = Constitution of the City of St. Louis. Legal framework for Governance. Requires
approval of 2/3 of all City voters to change a provision
• City Ordinances = Laws created by Board of Aldermen that can be changed by a new law.
Changes need not go to the voters.
• Civil Service Rules = Civil Servants are hired by the City, and protected by rules and regulations
promoting safety, fairness, etc. Generally Political Appointees are not protected by Civil Service
rules.
• Comptroller = Elected Official responsible for financial reporting, payments of bills, cash
receipting, and other non-treasury functions. Comptroller must approve funds sufficiency before
items may be ordered and bills may be paid.
• Departments = City functions such as Parks, Health, Corrections, Police, Fire, Human Services…
• Fiscal Year = The City of St. Louis Fiscal Year runs from July 1 – June 30th.
• Requisition = Document starting the procurement process. Includes specifications for the product,
delivery instructions, billing information, etc.
• Supply Division = name of the Procurement Function for the City of St. Louis
• Vendors = Companies and non-profits that want to sell goods and services to the City
36

37. Key Takeaway Summary
37
Key Take Aways: Slide #
The reduction in cycle time is clear, which means customers are experiencing less controllable delays as a result of bid tab creation cycle time. 3
This project will immediately impact internal and external customer satisfaction which is key to Supply Division alignment with our Mission. 5
Timeliness is important to internal customer satisfaction, and the comments highlight frustration. Customers think it takes longer for us to process than it actually does 6
The scope of the project is from when the Supply Division open the vendor bids on the due date until the Buyer sends the completed Bid Tab to the requesting Department.
The process is manual at this time. 7
Lots of room for technology and other improvements in this process 8
Since this is a cycle time project, the measure is a continuous, non-normally distributed measures of time. One day. 10
The range of days reflects most of the issues. The data is very skewed, and is non-normal. Because the mean is closer to the lower specification unit we use Cp1, which is 0.11,
which means that our current process has poor capability to meet customer requirements. 11
There is lots of variation in the process and the average time to create a Bid Tabulation 12
The range of number of days reflects the problem – reducing the range, mean and standard deviation will make customers more satisfied. 13
Process analysis showed most processes were NOT value added not in our control 16
The histograms show the tightening of data points. The run charts also show the improvement in the new vs. last year same month vs. historical data pattern.
17
The two sample t-Test for the new process vs. historical data showed a P value of zero, which supports the alternative hypothesis that the new process impacted cycle time
significantly. 17
Versus last year at the same time, variability & mean cycle time were reduced 18
Even more visible improvement in mean and standard deviation exists compared to historical data 19
Baseline bid tab creation cycle time data was out of control 20
April of 2018 was not as out of control as the overall historical data for 2013-2018. 21
April of 2018 was not as out of control as the overall historical data for 2013-2018. 22
Using a Two Sample t-Test we find that there is improvement in the process cycle time new vs. last year same time. The mean decreases from 1.6 to 1.25 days, or 22%, and the
standard deviation went from 2.354 to 2.179, a 7% reduction. However, since the P value is > 0.05, we can not reject the null hypothesis that the mean difference = 0. This
root cause is not statistically significant. However, versus historical data, the P value = 0, so we can reject the null hypothesis that there is no difference in the means.
23
This visual aid of the Impact/Effort Matrix made decision making easier for the Team 25
The A3 test was another useful visual for the Team to assess options and possibilities 26
This Implementation Plan was key in terms of all team members knowing their role in the roll-out, and participation increased buy-in 27
The results were clear enough to make the path an easy choice 28
A new ERP system/technology will be greatly useful in minimizing risk by forcing compliance 29
Management intervention works well in a hierarchical environment and has improved bid tab creation cycle time 30
As long as Buyers keep to the new process of the “3 day plan,” we should continue to see improved cycle times. 33
The team is continuing to work on the special causes of variation – and the Upper Control Limit of 5 days is the Response Plan trigger. 34

38. Appendix: Team Photos
38

39. Voice of the Customer Survey and Results
39

40. SUP B2. Standard Advertised Bid Award
B2a.SUPBid
Receipt/Open
B2c.SUPAward
Process
B2d.Comptroller’s
OfficeRole
B2e.ITSARole
Bids are time
stamped upon
arrival, but left
sealed
Sealed bids with
Req. # outside arrive
in mail or delivered
per SUP B1.
Bids are filed
securely until
their bid
opening date
All bids are opened in public
forum at noon in the Supply
Division Conference Room on
the appropriate bid opening
date
Bid Opening Date is
determined by Deputy
Commissioner based on 21
day time lapse from bid
announcement/posting
Each Buyer is
given a copy of
bids received
by Req. #
Buyers review
bids and create
a Bid Tab
(tabulation)
spreadsheet
Buyers email or IO
mail Bid Tab to
requestor for
review & Approval
of low bid
Dept. emails to
accept low bid
or justify
choosing
higher bid
Buyer brings
justification to Deputy
Comm. or Comm. to
approve justification
Does Dept.
choose low bid?
Deputy or
Comm.
discusses with
Dept. until
decision made
Delays
No
Does exception
get approval?
Buyer attaches
documentation to
Bid Award
No
Yes Yes
Buyer routes Bid
Award to Deputy
& Comm. for
signed approval
Bid sent to
Comptroller’s
Office for
Approval
Account Clerk
II gives Req.
back to Buyer
Buyer enters
Purchase
Order into
MSA System
Bid returned to
Supply
Delays
P.O. Prints
Overnight
P.O. is
delivered to
Supply
Account Clerk
II passes out
paperwork to
Buyers
Buyer proofs
for accuracy
Is P.O. accurate?
Buyer fixes
error in MSA
System
No
Buyer returns
correct P.O. to
Account Clerk II
Yes Account Clerk II
gives correct P.O.
to Commissioner
to approve/sign
Commissioner
logs savings into
Excel spreadsheet
& returns papers
to Account Clerk II
Account Clerk
II emails bid
award to
vendor & files
paperwork
Delays
Delays
See SUP B1b
for Bid
Opening
Process
Complete Bid Award Process -- for Reference/Context
40

41. Example of a Simple Bid Tab that could be completed quickly
41

42. Example of a Complex Bid Tab – 4 Pages of Calculations.
Could not currently be done in 2-3 days
42

43. The 5 Whys for the Bid Tabulation Creation Process
Why is the range of days so high for bid tab creation? Why are the variation and median
so high?
• Question: Why does it take so long to create a bid tabulation (bid tab)?
• I am too busy to do everything at once
• Question: Why are you so busy?
• We handle a lot of volume of paperwork?
• Question: Why do you have so much paperwork?
• Lots of amendments, rebids, many forms, variety of specifications… (REWORK)
• Question: Why can’t you talk with the departments/agencies about their issues &
complexity?
• Chain of Command issues – especially with Police, Water and Airport. Also the Point of Contact (POC)
may be in the field and have limited access to phone or email
• Question: Why don’t you ask the Director or Deputy (Supply Management) to get involved?
• I did not realize it was that big of a delay. I don’t want to get the POC in trouble. Would Supply
Management going to his boss help or hurt?
• Question: Why would using Chain of Command NOT help?
• No good answer for that question.

44. Bid Tab Creation Process Only – Sample 116 Historical Data Points
Data is non-normal and skewed right
44

45. Bid Tab Creation
Process with sample
of 116 data points
Bid Tab Creation Sample Population Data
45

46. Complete Data Set Test Results – Descriptive
Statistics before limiting to a sample
Days to create bid tabs – in Supply
control with big variability!
Days for Dept/Div/Agency to approve and award bid.
Mean = 7, range = 0-149 days! They consistently
underestimate their cause of delay in the process.
46

47. Improve Phase Data Examples
April 2018April 2019

48. 48
• On the photos, you see the top picture is of the City of St. Louis, which is to always
remind us of why we do our jobs. The mission of the Supply Division is to help City
Departments/Divisions/Agencies serve the Citizens of St. Louis. We do that by quickly
and efficiently sourcing the products they need at the most competitive prices for the
taxpayers.
• The CQI Team also decided to post the 5S reminder. At first, there was severe
resistance to cleaning up, but now all agree that the workplace is less stressful and
more productive.
Control Phase – Visual Management