Parallel session 4 - Wednesday 21 September 2016

1. The views expressed are those of the author(s) and should not be attributed to the Economic Research Service or USDA.
Putting the Subject Back into Subject-
Based Innovation Research: Latent Class
Analysis in the 2014 ERS Rural
Establishment Innovation Survey
Timothy R. Wojan
Economic Research Service/USDA
Paper presented at OECD Blue Sky III
Ghent, Belgium
19-21 September, 2016

2. The views expressed are those of the author(s) and should not be attributed to the Economic Research Service or USDA.
Outline of Talk
• Strong priors that rural innovation is rare and
largely inconsequential
• Challenge to conventional wisdom requires
credible measure of substantive innovation
• Assume experiences of substantive
innovators unique and can be elicited with
simple questions
• Do identified substantive innovators satisfy
tests of internal and external validity?
• Feasibility and assessment of “rural
innovation policy” requires credible measure
of substantive rural innovators

3. The views expressed are those of the author(s) and should not be attributed to the Economic Research Service or USDA.
CIS findings contradict but do not
overturn conventional wisdom
• NBER, Brookings, World Bank either
wholly disregard or disqualify rural in
regional studies of innovation
• CIS findings on rural innovation based on
response to single ambiguous question
• North and Smallbone (2000): 49% of
rural UK mftrs regarded selves as
“innovative” based on CIS response but
industry experts rated only 24% as
“highly innovative”

4. The views expressed are those of the author(s) and should not be attributed to the Economic Research Service or USDA.
2014 ERS Rural Establishment
Innovation Survey
• First nationally representative self-
reported innovation survey for Rural
America
• Oversampled rural establishments but
allocated a quarter of the sample to urban
establishments for comparison
• Sample size 11,000 for all establishments
with 5 or more employees in nonfarm,
tradable sectors
• Sought more efficient way of IDing
substantive innovators

5. The views expressed are those of the author(s) and should not be attributed to the Economic Research Service or USDA.
Assume that struggling with innovation
alters responses to key questions
• EU CIS core questions in combination with
other observable characteristics
–New or significantly improved goods, services,
processes, logistics, marketing methods.
–Are innovation investments capital constrained?
–Acknowledge failed innovation initiatives?
–Possess intellectual property worth protecting?
–Does data drive decision-making?

6. The views expressed are those of the author(s) and should not be attributed to the Economic Research Service or USDA.
IDing Substantive and Nominal
Innovators Using Latent Class Analysis
(LCA)
• Assumes that sample drawn from distinct
but unobservable subpopulations inferred
from the data
• Latent class analysis resolves two main
problems of classification in large datasets:
– Classification is probabilistic
– Can be estimated incorporating complex
sample design with the MPlus statistical
package

7. The views expressed are those of the author(s) and should not be attributed to the Economic Research Service or USDA.
Latent Class Analysis with Covariates
Schematic
Outcomes
Outcome
Explanatory
Vars.
Latent
Classes
Covariates
Explaining
Class
Membership
y1 y2 y3
y4
zi … … …. zk
33.09% 36.79%30.12%
xis xks
Core Innovation
Covariates
Data Driven Decision
Making Covariates
Substantive
Innovators
Nominal
Innovators
Non-
Innovators
Source: 2014 Rural
Establishment Innovation Survey

8. Source: 2014 Rural Establishment
Innovation Survey
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
Innovation Projects
Abandoned and/or
Incomplete
Intellectual Property
Protection
Surplus Funds Used
for Innovation-
Definitely
Track Employee
Training
Customer
Satisfaction Analysis
--Regularly
Customer
Satisfaction Analysis
--Never
Change Process Due
to Complaint--
Regularly
Enterprise Resource
Planning Software
Affirmative Responses to Variables Used to
Determine Latent Class Membership
Substantive Innovators Data-Drvien Nominal Innovators Non-Innovators
Core Innovation Questions
Data Driven Decision-Making
%
A
n
s
w
e
r
i
n
g
Y
E
S

9. The views expressed are those of the author(s) and should not be attributed to the Economic Research Service or USDA.
Whether these subpopulations truly
exist is an empirical question
• Initial results will be in cross-section:
– Do auxiliary questions provide a sufficient threshold?
– Are establishments in more innovation intensive sectors
more likely to respond affirmatively to auxiliary questions?
• Linking REIS to the longitudinal business data
at BLS or Census will provide dynamic
performance data to compare substantive with
nominal innovators
• Broad but shallow survey research
supplemented with narrow but deep case
study research
Source: 2014 Rural Establishment
Innovation Survey
17.38%
16.63%
8.35%
30.97% 31.23%
6.02%
2.31%
1.10%
6.15%
8.31%
4.69%
3.02%
0.96%
5.66%
8.11%
PURCHASE OR LICENSE PATENTS PARTICIPATED IN A PATENT
APPLICATION
REGISTERED AN INDUSTRIAL DESIGN REGISTERED A TRADEMARK PRODUCE MATERIAL ELIGIBLE FOR
COPYRIGHT
Validity wrt Survey Responses: Innovation Related Activities
Substantive Innovators Data-Drvien Nominal Innovators Non-Innovators
% Answering YES

10. The views expressed are those of the author(s) and should not be attributed to the Economic Research Service or USDA.
Source: Shackelford 2013 and 2014 Rural
Establishment Innovation Survey
0
2
4
6
8
10
12
14
16
18
Rank Order Correlation Between NSF and REIS Innovation Intensive
Industries Removing Likely Outlier (NAICS 3342 Communications Equip.)
NSF Patent Apps REIS
(21)
(43)
(24)
(48)
(9)
(13)
N = 13 but no
Metro
Substantive
innovators
(9)
(135)
(45)
(75)
(112)
N in
(2374)
(751)
(195)
(431)
(1932)
(4206)
Rank Order Correlation =
0.433**
N
S
F
a
n
d
R
E
I
S
R
a
n
k
o
f
I
n
d
u
s
t
r
y

11. The views expressed are those of the author(s) and should not be attributed to the Economic Research Service or USDA.
The central question: Are rural substantive
innovators common or rare?
Substantive Innovators Data Driven Nominal Innovators Non-Innovators
Nonmetro 22.56 38.52 38.92
Metro 31.27 32.26 36.47
Small Establishments
Nonmetro 18.02 38.29 43.69
Metro 26.00 33.18 40.83
Medium Establishments
Nonmetro 28.53 41.12 30.35
Metro 41.10 31.96 26.94
Large Establishments
Nonmetro 52.14 29.99 17.87
Metro 48.36 22.97 28.67
Hi-tech Manufacturing
Nonmetro 44.04 29.53 26.43
Metro 35.56 30.26 34.19
Hi-tech Services
Nonmetro 32.71 26.75 40.54
Metro 40.41 24.21 35.38
Source: 2014 Rural Establishment Innovation Survey

12. The views expressed are those of the author(s) and should not be attributed to the Economic Research Service or USDA.
How Reliable Measures of Rural
Innovation Can Aid Rural Policy
• Does rural policy need to address the
problems that emerge from innovation-led
growth?
• Are market failures that plague sparsely
populated areas impeding grassroots
innovation?
• How are rural areas best able to ameliorate
the disadvantages of distance and kindle
the creative spark?

13. The views expressed are those of the author(s) and should not be attributed to the Economic Research Service or USDA.
Thank you
Comments? Questions?
twojan@ers.usda.gov

14. The views expressed are those of the author(s) and should not be attributed to the Economic Research Service or USDA.
Is the innovation measure picking up
things that citizens care about?
• Associating substantive innovation with
establishment performance such as
productivity, exports, employment growth,
survivability, etc. must wait for these data to
become available
• In the meantime, retrospective employment
experience possible based on 2014 county-
industry innovativeness estimate and county-
industry employment growth in recovery
2009-2014.

15. The views expressed are those of the author(s) and should not be attributed to the Economic Research Service or USDA.
Table 9: Regressions of County-
Industry Employment Growth, 2009-
2014Variable Parameter
Estimate
Standard Error t Value Pr > |t|
Probability
Substantive Innovator
82.69 43.02 1.92 0.0546
Share Introducing New
Products or Processes
(CIS Equivalent)
-60.61 37.88 -1.60 0.1097
Probability Nominal
Innovator
-116.0698 54.081 -2.15 0.0319
Probability Non-
Innovator
-14.59 54.01 -0.27 0.7870
Source: 2014 ERS Rural Establishment Innovation Survey and BLS Quarterly Census of Employment and Wages
Coefficient estimates for intercept, population, and industry controls not reported

16. The views expressed are those of the author(s) and should not be attributed to the Economic Research Service or USDA.
Table 10: Regressions of County-Industry
Employment Growth, 2009-2014, Selected Sectors
Source: 2014 ERS Rural Establishment Innovation Survey and BLS Quarterly Census of Employment and Wages
Coefficient estimates for intercept, population, and industry controls not reported
Industrial Sector Variable Parameter
Estimate
Standard Error t Value Pr > |t|
Fiber Probability Substantive
Innovator
38.64 132.15 0.29 0.7709
Fiber Share Introducing New
Products or Processes (CIS Eq.)
484.33 83.795 5.78 <.0001
Food Probability Substantive
Innovator
-146.081 52.49 -2.78 0.0057
Food Share Introducing New
Products or Processes (CIS Eq.)
-110.174 52.933 -2.08 0.0383
Information Probability Substantive
Innovator
412.369 76.328 5.40 <.0001
Information Share Introducing New
Products or Processes (CIS Eq.)
200.25 62.53 3.20 0.0015