https://dl.acm.org/doi/10.1145/3321705.3329849

1. Copyright©2019 NTT corp. All Rights Reserved.
A Pilot Study on Consumer IoT Device Vulnerability Disclosure
and Patch Release in Japan and the United States
Asuka Nakajima1, Takuya Watanabe1, Eitaro Shioji1, Mitsuaki Akiyama1,
Maverick Woo2
1 NTT R&D, Tokyo, Japan
2 Carnegie Mellon University, Pittsburgh, United States
AsiaCCS 2019

2. 1Copyright©2019 NTT corp. All Rights Reserved.
Background [1/2]
-1-
Our Society Continues to Increase
Its Reliance on Computers
Vulnerabilities and Their Lifecycle Management
are Gradually Becoming a Matter of Public Safety
Vulnerability Disclosure
& Patch Release

3. 2Copyright©2019 NTT corp. All Rights Reserved.
Background [2/2]
-2-
Finder
(BugHunter)
Coordinator
(e.g.CERT/CC)
Publicly Accessible
Resources (e.g.Blog)
Deployer
(User)
Attacker
Vendor
Vulnerability Disclosures & Patch Release
No prior work that focused on
Consumer IoT Devices
 Previous Research
 Patch Release Behaviors
 Characteristics of Patches
Provides
Patch
Vulnerability

4. 3Copyright©2019 NTT corp. All Rights Reserved.
Background [2/2]
✔
✔&
Trends and findings of the study might differ in different markets
Purchase Decisions of IoT Devices are Heavily Localized….!
Selected
Japan United States
-3-
 NTT(JP) and CMU(US)
 Market Size
 Have two oldest national CSIRTs✔
Finder
(BugHunter)
Coordinator
(e.g.CERT/CC)
Publicly Accessible
Resources (e.g.Blog)
Deployer
(User)
Attacker
Vendor
No prior work that focused on
Consumer IoT Devices
 Previous Research
 Patch Release Behaviors
 Characteristics of Patches
Provides
Patch
Vulnerability
Vulnerability Disclosures & Patch Release

5. Data Collection Method
-4-

6. 5Copyright©2019 NTT corp. All Rights Reserved.
Data Collection Method [1/3]
・CVE-IDs
Vendor List
Identify Target Vulnerabilities
NVD
Identify Relevant Vulnerabilities Related to Included Vendors
STEP1 STEP 2 STEP 3
STEP1
-5-
Vendor List
Country Vendor
Japan Buffalo
IO-DATA
NEC (Aterm*1)
United States Netgear
Linksys
D-Link
*1: “Aterm” is a brand name for the router products released from NEC.

7. 6Copyright©2019 NTT corp. All Rights Reserved.
Data Collection Method [2/3]
Collect Vulnerability Information
NVD
Internet
ArchiveEDB
JVN
External
Reference
STEP1 STEP 2 STEP 3
CVE-IDs1) Affected Products
2) Affected Versions
3) Patched Versions
4) Public Disclosure Date
5) Exploit Release Date
・(1)~(5)
STEP2
Collect Vulnerability Info
 Vulnerability Info (1)~(5)
Source Name Details
NVD NVD Data Feed(+External Reverences)
EDB
Exploit-DB
Metasploit
JVN Japan Vulnerability Notes iPedia
Vendor’s
website
Official website of each vendor
(Security Advisories)
Vendor's
Website
-6-

8. 7Copyright©2019 NTT corp. All Rights Reserved.
Data Collection Method [3/3]
Collect Patch Release Information
Vendor's
Website
Released
Software
STEP1 STEP 3STEP 2
STEP3
Collect Patch Info
・(6)
Vuln Info
Release
note
Internet
Archive
 (6)Patch Release Date
 Extract the date from release note
-7-

9. 8Copyright©2019 NTT corp. All Rights Reserved.
Country Vendor
# CVE-IDs #
Products
#
Patches
#
ExploitTotal CVSS:Low CVSS:Med CVSS:High
JP
Buffalo 20 1 15 4 71 105 0
IO-DATA 24 3 11 10 57 88 0
NEC(Aterm) 3 0 3 0 26 35 0
JP Total 47 4 29 14 154 228 0
US
Netgear 25 1 12 12 107 106 21
Linksys 17 2 3 12 31 40 12
D-Link 61 6 27 28 158 177 35
US Total 103 9 42 52 296 323 68
Total 150 13 71 66 450 551 68
 Dataset Summary
 Collected 150 CVE-IDs spanning 2006 – 2017, involving 450 products,
551 patches, and 68 published exploits*2
 Start from 298 CVEs and dropped non-IoT CVEs. (15 CVEs) (298->283)
 Dropped total 133 CVEs that we could not find the patch release date(->150)
 Classified collected patches based on CVSSv2 (Low/Medium/High)
Dataset
Dataset
-8-
Router 70%
Network Camera 16%
NAS 9%
*2 All the data in our dataset is collected before Jan/2019

10. 1) Patch Availability Delay
2) Minimum Exploit Windows
3) Incremental Patch Release
4) Patch Release Timeliness Over Time
5) Overall Patch Release Timing
6) Fix Prioritization
Characterization of Patch Releases
-9-

11. 10Copyright©2019 NTT corp. All Rights Reserved.
Characterization of Patch Releases [1/6]
1) Tends to Release Patches Before the Disclosure Date
2) Tends to Release Patches Around the Disclosure Date
3) Tends to Release Patches After the Disclosure Date
 Patch Availability Delay【 tp – td 】
 Based on the median patch availability delay, we can classify the
vendor into three categories
td Public Disclosure Date
tp Patch Release Date
te Exploit Release Date
-10-

12. 11Copyright©2019 NTT corp. All Rights Reserved.
Characterization of Patch Releases [1/6]
1) Tends to Release Patches Before the Disclosure Date
2) Tends to Release Patches Around the Disclosure Date
3) Tends to Release Patches After the Disclosure Date
 Patch Availability Delay【 tp – td 】
 Based on the median patch availability delay, we can classify the
vendor into three categories
Box-plots of Patch Availability Delay for Each Included Vendor
Netgear
Buffalo
NEC(Aterm)
IO-DATA
D-Link
Linksys
Before
Around
After
Summary
td Public Disclosure Date
tp Patch Release Date
te Exploit Release Date
(±5 days)
(+23 days)
(around-100 days)
-11-

13. 12Copyright©2019 NTT corp. All Rights Reserved.
 Minimum Exploit Windows【 tp – te 】
 Measures the time between patch availability and the release of
the first known exploit known to us
Characterization of Patch Releases [2/6]
Exploit-DB Metasploit
-12-

14. 13Copyright©2019 NTT corp. All Rights Reserved.
 Minimum Exploit Windows【 tp – te 】
 Measures the time between patch availability and the release of
the first known exploit known to us
Characterization of Patch Releases [2/6]
Vendor CVE-ID First Patch
Release Date
Exploit
Release Date tp – te
D-Link 2015/Mar/02 2015/Feb/26 4 days
Netgear
2017/Jan/16 2016/Dec/26 21 days
2017/Jan/16 2016/Dec/28 21 days
2017/Jan/16 2016/Dec/28 21 days
2016/Dec/22 2016/Dec/07 15 days
2016/Dec/26 2013/Aug/22 1222 days
2009/Mar/03 2008/Nov/13 110 days
CVE-2015-1187
CVE-2016-10176
CVE-2016-10175
CVE-2016-6277
CVE-2016-10174
CVE-2008-6122
CVE-2013-4775
Summary
 7 exploits were released before their corresponding patches are released
 All exploits are in the US dataset (Database Bias?)
Exploit-DB Metasploit
-13-

15. 14Copyright©2019 NTT corp. All Rights Reserved.
Characterization of Patch Releases [3/6]
-14-
 Incremental Patch Release
 Releasing a series of patches to the same vulnerability but for
different devices over time
Patch Release
Timeline
Example: CVE-2016-10175
WNR2000v5
2017/Jan/12
R6020/R6080
2018/May/18
WNR2000v3
2017/Jan/16
WNR2000v4
2017/Jan/17
491 Days
Vendor: Netgear, Product: Wireless Router

16. 15Copyright©2019 NTT corp. All Rights Reserved.
Characterization of Patch Releases [3/6]
 In our dataset, 62.4% of the patches were released incrementally
which are associated with 40 CVE IDs
 All 6 vendors practiced incremental patch release
Summary
-15-
 Incremental Patch Release
 Releasing a series of patches to the same vulnerability but for
different devices over time
Patch Release
Timeline
Example: CVE-2016-10175
WNR2000v5
2017/Jan/12
R6020/R6080
2018/May/18
WNR2000v3
2017/Jan/16
WNR2000v4
2017/Jan/17
491 Days
Vendor: Netgear, Product: Wireless Router

17. 16Copyright©2019 NTT corp. All Rights Reserved.
 Patch Release Timeliness Over Time
 Measured the timeliness of patch release with a break down on CVSS
severity(Low/Medium/High) over 2006~2017
 Categorized the patch releasing timing:
• 1.Before Disclosure 2.Concurrent with Disclosure 3.After Disclosure
Characterization of Patch Releases [4/6]
-16-

18. 17Copyright©2019 NTT corp. All Rights Reserved.
Characterization of Patch Releases [4/6]
Timeliness of patch release
(Partitioned by vulnerability severity)
Summary
 Patches which released after
the public disclosure (black
colored part) account for a
large portion across all CVSS
severities.
 Unfortunately, no sign of
reduction over time
-17-
 Patch Release Timeliness Over Time
 Measured the timeliness of patch release with a break down on CVSS
severity(Low/Medium/High) over 2006~2017
 Categorized the patch releasing timing:
• 1.Before Disclosure 2.Concurrent with Disclosure 3.After Disclosure

19. 18Copyright©2019 NTT corp. All Rights Reserved.
Characterization of Patch Releases [5/6]
 Overall Patch Release Timing
 Count the number of included patches based on their timeliness
• 1.Before Disclosure 2.Concurrent with Disclosure 3.After Disclosure
-18-

20. 19Copyright©2019 NTT corp. All Rights Reserved.
Characterization of Patch Releases [5/6]
 Overall Patch Release Timing
 Count the number of included patches based on their timeliness
• 1.Before Disclosure 2.Concurrent with Disclosure 3.After Disclosure
Number of Patches Released
Before / Concurrent with / After Disclosure
Summary
 Over 1/2 of the included
patches (total 551) were
released pre-disclosure
Details will be shown at
JP vs. US analysis section
320
41
190
+
 About 1/3 were released
post-disclosure
 We see a stark contrast,
when we break down
the dataset by market
-19-

21. 20Copyright©2019 NTT corp. All Rights Reserved.
Characterization of Patch Releases [6/6]
 Fix Prioritization
 Q. Do high severity vulnerabilities get patched more quickly？
 Plot a cumulative distribution function (CDF) graph of patch availability delay
(tp – td) for each CVSSv2 severity category (Low/Medium/High) to visualize the fix speed
-20-

22. 21Copyright©2019 NTT corp. All Rights Reserved.
Characterization of Patch Releases [6/6]
CDF of the patch availability delay [tp – td]Summary
A. No
 CDF of the high severity
vulnerabilities remains
around 0.9 well into 1 year
post-disclosure
 In contrast, low/medium
severity vulnerabilities are
all fixed
-21-
 Fix Prioritization
 Q. Do high severity vulnerabilities get patched more quickly？
 Plot a cumulative distribution function (CDF) graph of patch availability delay
(tp – td) for each CVSSv2 severity category (Low/Medium/High) to visualize the fix speed

23. Japan vs. the United States
-22-

24. 23Copyright©2019 NTT corp. All Rights Reserved.
Japan vs. the United States [1/2]
 Significant Difference Shown in Patch Release Timing Behavior
 Most of the patches in JP dataset were released either concurrently or
before public disclosures
Number of Patches Released
Before / Concurrent with / After Disclosure
184
6
+
190
Japanese Vendors/Finders
tend to perform
Coordinated Disclosure… ?
-23-

25. 24Copyright©2019 NTT corp. All Rights Reserved.
Japan vs. the United States [2/2]
 Disclosure Process Classification
 Classified the disclosure process of each vulnerability
• 1. Coordinated Disclosure 2. Full Disclosure 3. Unknown
 [Source] JVN, NVD(+External References) Security Advisories, Blog Posts/ML
-24-

26. 25Copyright©2019 NTT corp. All Rights Reserved.
Japan vs. the United States [2/2]
Summary
% of Each Disclosure Process [JP vs. US]
 Over 97% of the vulnerabilities in
the JP dataset where disclosed via
Coordinated Disclosure
 Over 37% of the included
vulnerabilities in the US dataset
where disclosed via Full Disclosure
97.9%
 Finders of the 30 of the 53 CVE
entries in the JP dataset was from
local security company named
“Mitsui Bussan Secure Directions,
Inc.”
-25-
37.5%
 Disclosure Process Classification
 Classified the disclosure process of each vulnerability
• 1. Coordinated Disclosure 2. Full Disclosure 3. Unknown
 [Source] JVN, NVD(+External References) Security Advisories, Blog Posts/ML

27. 1) Incremental Patch Release (Shown in page 17-18)
2) Unsynchronized Patch Release
3) Implicit End-of-Support (EoS)
Significant 1-Day Risk Uncovered
-26-

28. 27Copyright©2019 NTT corp. All Rights Reserved.
Significant 1-Day Risk Uncovered [1/2]
 Unsynchronized Patch Release
 Regional subsidiaries of some vendors would often release a patch
against the same vulnerability on different dates.
 We dub this risk “Geographical Arbitrage”
Example: CVE-2017-7852
Patch Release
Timeline
DCS-932L RevA
2015/Nov/18
DCS-932L RevA
2016/Jul/19
244 Days Vendor: D-Link, Product: Network Camera
-27-

29. 28Copyright©2019 NTT corp. All Rights Reserved.
Significant 1-Day Risk Uncovered [1/2]
 Unsynchronized Patch Release
 Regional subsidiaries of some vendors would often release a patch
against the same vulnerability on different dates.
 We dub this risk “Geographical Arbitrage”
Example: CVE-2017-7852
Patch Release
Timeline
DCS-932L RevA
2015/Nov/18
DCS-932L RevA
2016/Jul/19
244 Days Vendor: D-Link, Product: Network Camera
Summary Vendor Region
#
Patches
Average
(Days)
Median
(Days)
Max
(Days)
Buffalo 12 -58 0.5 1
D-Link
103 23.7 2 366
62 2.5 -1 218
Netgear 51 31 8 346
Extended our Dataset Japan(JP)
Germany(DE)
Australia(AU)
China(CN)
-28-
 D-Link US is behind DE in 58.3%
Patch releases by these
subsidiaries are indeed
often unsynchronized

30. 29Copyright©2019 NTT corp. All Rights Reserved.
Significant 1-Day Risk Uncovered [2/2]
 Implicit End-of-Support (EoS)
 Many regional subsidiaries appeared to have stopped releasing
patches to products that were still being supported in at least one
other region but posted no EoS announcement
Example: CVE-2016-1556, WN604
Patch Release
Timeline[US]
Ver. 3.0.2
2012/Apr
Patch Release
Timeline[CN]
Ver. 3.0.2
2012/Dec No firmware has been released after ver 3.0.2.
& No End-of-Support Announcement
Ver. 3.3.1
2015/May
Ver. 3.3.3
2016/Mar
Ver. 3.3.2
2015/Jul
Security Update
-29-
Vendor: Netgear, Product: Wireless Router

31. 30Copyright©2019 NTT corp. All Rights Reserved.
Significant 1-Day Risk Uncovered [2/2]
 Implicit End-of-Support was found in Buffalo US/D-Link AU/Netgear CN
 We found total 15 patches which shows Implicit EoS !
Summary
-30-
 Implicit End-of-Support (EoS)
 Many regional subsidiaries appeared to have stopped releasing
patches to products that were still being supported in at least one
other region but posted no EoS announcement
Example: CVE-2016-1556, WN604
Patch Release
Timeline[US]
Ver. 3.0.2
2012/Apr
Patch Release
Timeline[CN]
Ver. 3.0.2
2012/Dec No firmware has been released after ver 3.0.2.
& No End-of-Support Announcement
Ver. 3.3.1
2015/May
Ver. 3.3.3
2016/Mar
Ver. 3.3.2
2015/Jul
Security Update
Vendor: Netgear, Product: Wireless Router

32. 31Copyright©2019 NTT corp. All Rights Reserved.
Suggestions / Conclusion
 Suggestions
-31-
• Consider leveraging natural language processing
techniques when parsing the release notes
• Consider coordinating among subsidiaries to synchronize
their patch release & publish EoS
• Release notes should be in a machine readable format
(JSON/XML) and distributed via RSS with security advisories
• Consider requiring vendors to publicly disclose the dates
of all discovered vulnerabilities
To
Researcher
To
Vendor
To
Policy Maker

33. 32Copyright©2019 NTT corp. All Rights Reserved.
 Suggestions
Acknowledgement We thank to Allen Householder for insightful discussion and his suggestion of the term “geographical arbitrage”
 We conducted a pilot study on consumer IoT device vulnerability
disclosure and patch release in Japan and the United States
 Investigated 150 CVE entries and characterize the vendors behavior
 [JP vs. US] Significant difference shown in patch release timing behavior
 Our investigation has uncovered 3 significant risks of 1-day exploits
Conclusion
-32-
• Consider leveraging natural language processing
techniques when parsing the release notes
• Consider coordinating among subsidiaries to synchronize
their patch release & publish EoS
• Release notes should be in a machine readable format
(JSON/XML) and distributed via RSS with security advisories
• Consider requiring vendors to publicly disclose the dates
of all discovered vulnerabilities
To
Researcher
To
Vendor
To
Policy Maker
Suggestions / Conclusion

34. Q&A?
-33-