For enterprise software applications and related processes, highly accurate and synchronized time is a necessity. An inaccurate
computer clock can cause significant problems. A discrepancy of a minute or two could cause a significant and unacceptable margin of error, since many applications require that the time be kept accurate to the nearest second or less.
2. Introduction
Every computer has a variety of clock styles to choose from: Steven Teppler, Senior Counsel at KamberEdelson, LLC in New
analog, digital or a Big Ben lookalike. For most users, the York City, notes astutely that not only are there regulatory
accuracy of the basic clock is likely sufficient, even if it is off by a imperatives to maintaining accurate records, but emerging legal
minute or two. discovery and evidence management court decisions are
beginning to impose severe sanctions and penalties on parties in
For enterprise software applications and related processes, highly lawsuits who engage in time-based data manipulation. Teppler
accurate and synchronized time is a necessity. An inaccurate says the implication is that time must be accurate and
computer clock can cause significant problems. A discrepancy of synchronized to the extent possible within the network, and that
a minute or two could cause a significant and unacceptable this accurate and synchronized time must also be reflected in its
margin of error, since many applications require that the time be association with enterprise computer-generated records in a
kept accurate to the nearest second or less. manner also sufficient to withstand a legal challenge.
For example, computers in financial institutions are required to Perhaps the greatest benefit of effective time synchronization is
keep highly accurate records of when a transaction was that it won’t make IT look foolish. The picture of the Boeing 757
completed. Similarly, software used in the manufacturing process hitting the Pentagon on the morning of September 11, 2001, is
requires mixtures to be executed at a precise time. Internet, one of the most heartbreaking pictures of the last decade.
radio and TV stations also need computers that can switch feeds Regrettably, the time stamp on the video was “September 12,
or link up with remote links at the correct time. When the time 2001 5:37PM.” The picture, unfortunately, is used extensively
on enterprise network devices is incorrect, the effects can be amongst the 9/11 conspiracy community. Having correct time on
costly and significant. the video monitors would have obviated such misuse.
An accurate time source, as well as time synchronization between This white paper looks at the need for accurate and synchronized
two devices, is a necessity. However, clocks on computers cannot enterprise time, what products are available to provide this
be depended on for this source because of their propensity to drift. capability and how to implement time synchronization.
They use oscillator circuits or a battery-driven, quartz crystal clock
(mainly for cost savings), which can drift up to minutes per day.
With that, serious timing errors can quickly occur.
In addition to inaccurate clocks, an organization needs to defend
its timing infrastructure against malicious attacks from internal and
external attackers. One of the ways an attacker will try to hide their
tracks is by modifying the clock on systems they have breached.
1
3. Need and Risk Cost/ROI
Doing things on time is a frequent requirement as many activities Given the legal, practical and operational realities, adding time
need to be synchronized with others in order to operate at peak services functionality to your enterprise network is no longer an
levels. But the reality is that synchronized time is a relatively option. The beauty of implementing a time services infrastructure
new phenomenon, as it was just 125 years ago, on November to your organization is that it will not break the bank. The
18, 1883, when Standard Time was created. Prior to 1883, local approximate cost varies between $2,000 and $10,000 depending
mean time was used throughout the USA, which resulted in on the level of accuracy required, and if redundancy is needed.
plethora of local times. This alone caused chaos to train
schedules, with travelers often missing their trains. The time server infrastructure itself initially can be up and
running in a day, but will take longer (exactly how long depends
Effective time synchronization can illustrate improprieties. on the organization and requirements) to fully deploy. Some of
Perhaps the most significant case where time synchronization its many benefits are:
could have helped–or prevented–fraud is exemplified by Enron.
The CEO and CFO of Enron made a habit of engaging in time- • Reduced downtime
based data manipulation. CFO Andrew Fastow and his team
altered financial data to suit whatever it was they wanted the • Prevention of operational failure
investing public or government authorities to know, or not know. • Avoidance of data loss
Specifically, Fastow backdated documents to manipulate Enron’s • Improved security
financial statements and, as a result, drained millions of dollars
that rightfully belonged to Enron and a bank that invested with • Mitigation of legal exposure
Enron. He also backdated documents to overstate value of a • Time services ROI often measured in weeks or months
technology company in which Enron had invested.
Here is a practical example: An attacker illegally infiltrates your
Enron is not an isolated case. Many other companies, including system on Wednesday, October 29, 2008 between 16:38:39 and
NextCard, Autotote, RiteAid, Parmalot and Adelphia, acted in 17:25:37. Your system logs show that these events occurred
similar ways. And, in all of these cases, effective time starting at 19:49:12. The attacker has a dozen witnesses stating
synchronization would have provided data integrity assurance of that he was with them watching the final game of the World
financial reports, grant letters, loan reports, securities Series from 18:00 to 21:00. Most prosecutors wouldn’t take the
transactions, letters of credit and much more. case as the logs can’t be admitted as evidence.
The importance of accurate time and time synchronization
is two-fold:
1. They allow events to occur at the proper time via event
synchronization. In this way, an organization can schedule a
process and ensure that it starts or stops on time, or runs for a
specified period regardless of when it starts or stops. This also
ensures that cooperating processes can interoperate correctly,
so that if one process hands a task off to a second process, that
process will in fact be ready to accept the handoff
2. They provide proof when events occurred or did not occur, in
other words, using time as a key feature of digital forensics.
If IT does not have synchronized time, it is important to determine
the associated risks. Organizations need to know how accurate
their clocks ought to be–be it minutes, seconds or milliseconds.
Don’t underestimate the risks of inaccurate time; if you don’t
practice due care pertaining to the time on your network and
application, the organization can be legally liable for negligence.
2
4. Regulatory Imperatives for Network Time Protocol
Time Synchronization No discussion about time synchronization would be complete
From a regulatory perspective, more and more industry without mention of the Network Time Protocol. NTP has been in
standards are requiring time synchronization. Some of these use for nearly 30 years and remains the longest running,
standards and standards-making bodies are: continuously operating, Internet application protocol.
• 21 CFR Part 11 NTP is a User Datagram Protocol (UDP)-based protocol. With
• Payment Card Industry Data Security Standard (PCI DSS) UDP, without requiring prior communications, computer
applications send messages, known as datagrams, to other hosts
• GLBA to set up special transmission channels or data paths. UDP is
• Sarbanes-Oxley known as an unreliable protocol, and is used for service and
• HIPAA speed, but not for reliability or data integrity.
• European Telecommunications Standards Institute (ETSI)
NTP was designed to synchronize the clock on a client device
• National Emergency Number Association with the clock on a network time server. Note that NTP is simply
• Public Safety Answering Point Master Clock Standard the protocol and the use of NTP requires separate client and
• National Fire Protection Association server applications.
• Standard #1221 - Installation, Maintenance and Use of
Emergency Services Communication Systems NTP is roughly accurate to within 10-100 milliseconds, and even
though it uses UDP, which is an unreliable protocol, it has been
One of the most detailed specifications around time architected to sustain levels of accuracy and robustness, even
synchronization is the October 2008 update to version 1.2. when used over numerous gateways and their respective delays.
Section 10.4 of the PCI DSS, which requires an entity to What NTP specifically does is determine the offset of the client’s
“synchronize all critical system clocks and times.” clock relative to the time server’s clock. The client then sends a
UPD time request packet to the server, which is time stamped
The PCI testing procedures for requirement 10.4 are to obtain and and returned. The NTP client computes the local clock offset
review the process for acquiring and distributing the correct time from the time server and makes an adjustment.
within the organization, as well as the time-related system-
parameter settings for a sample of system components. You should The use of NTP can be broken up into the following five steps:
verify the following are included in the process and implemented:
1. NTP Design - Choose your NTP time source, either Internal
• A known, stable version of NTP (Network Time Protocol) or (more control, more management) or External (less control,
similar technology, kept current per PCI DSS Requirements 6.1 less management).
and 6.2, is used for time synchronization.
2. NTP Topology - Issues include the desired level of time
• Internal servers are not all receiving time signals from external accuracy, number of NTP clients, network infrastructure
sources. [Two or three central time servers within the redundancy and network physical topology and geography.
organization receive external time signals directly from a Investigate how the sites are connected as round trip delays
special radio, GPS satellites or other external sources based on can impact NTP and negatively affect time accuracy.
International Atomic Time and UTC (formerly GMT), peer with
each other to keep accurate time, and share the time with 3. Feature evaluation - Determine which NTP features to use,
other internal servers.] basic security (authentication, access control) and
redundancy (redundancy between peers, redundancy
• Specific external hosts are designated from which the time configuration on clients).
servers will accept NTP time updates (to prevent a malicious
individual from changing the clock). Optionally, those updates 4. Management - How much you need to manage your NTP
can be encrypted with a symmetric key, and access control infrastructure is dependent on how important synchronized
lists can be created that specify the IP addresses of client time is to your organization.
machines that will be provided with the NTP service (to 5. Audit - Your time infrastructure must be able to prove that
prevent unauthorized use of internal time servers). the time on any monitored system was correctly
The implications of PCI non-compliance are significant–from fines synchronized at a particular time and date with a specified
levied by Visa and MasterCard, to having your payment processor time source. This is often required by industry specific
charge higher fees, to negative publicity and more. Non-compliance regulations. Note that the audit logs must be used within the
is risky, costly and can quickly bankrupt a merchant. context of digital forensics. Your staff needs to know and
follow the rules of evidence.
3
5. NTP Alternative Time synchronization must be made part of the corporate IT
systems and security policies. As an example, the following policy
Some organizations are reluctant to use NTP given the is quite effective: “Time synchronization to an accurate time
requirement to punch yet another hole in their firewall to allow source is required on all enterprise network devices.” Without a
NTP port 123 through. The primary concern is that hackers will policy, there will be no impetus for staff to achieve the goal of
use port 123 as a point of entry to conduct extensive network accurate, synchronized time.
attacks. An additional concern about opening port 123 is that it
can provide information about the network, as well serve as an
avenue for attack. Some of the information that can be gathered
Time Synchronization Products
from port 123 includes: For those companies interested in using a time synchronization
appliance, there are a number of vendors offering state-of-the-
• System uptime art capabilities. Three leading vendors are:
• Time since reset
• Time server packets • Symmetricom www.symmetricom.com
• I/O, memory statistics • Spectracom www.spectracomcorp.com
• NTP peer list
• EndRun Technologies www.endruntechnologies.com
Also, the attacker can run a replay attack using captured packets,
All of these vendors’ products have roughly the same
or can stop security-related chronograph (cron) jobs from running
functionality, although each has its own strengths. It is
or cause them to run at incorrect times. For that reason, many
important, though, to focus on your specific requirements first,
organizations prefer to use a GPS-based synchronization system.
rather than focus on the feature set of each appliance.
GPS satellites have atomic clocks and GPS-based time servers
All of the major vendors have stratum 1 NTP/NTP time servers
synchronize with those clocks, which are accurate to
that use GPS via oven-stabilized crystal oscillator (OCXO) and
approximately one-millionth of a second to UTC. Since this
rubidium oscillators. These maintain time standard if time
occurs behind the corporate firewall, there is no need to open it
reference is lost, and also have a dial-out modem that provides
up to another protocol.
back up to GPS or functions as the primary reference, such as for
disaster recovery, and has accuracy to within a few microseconds
Time Synchronization Checklist over a heavy load.
The following time synchronization checklist is a good way to
Don’t forget that you must secure the time appliance itself.
start things rolling:
There are many ways in which this can be done. Some of the
most effective security features to protect a time server or
System administrators appliance are passwords, SSL and access control lists. Use all of
1. Manually ensure that all firewalls, routers, critical servers, these for maximum security and protection of the device.
etc. have the correct time.
2. At this point, synchronizing by calling the United States Naval Finally, realize that while time synchronization hardware is
Observatory Master Clock at 202/762-1402 is sufficient. relatively inexpensive for most organizations, is may be a cost
factor in some. Determine how much your organization can
Management afford to spend.
1. Identify all critical network devices in your organization that
require accurate time. Conclusion
2. Appoint a responsible technical staff member to be the time The need for synchronized time is a crucial business and
services liaison and to manage time services. technology requirement. As such, it is an integral part of an
3. Meet with vendors of time synchronization equipment to effective network and security architecture. Ensuring accurate
determine the solution that best fits your organization and time is relatively inexpensive and offers a significant return on
specific needs. investment. It is also a great way to be in compliance with your
various regulatory efforts and to stop your company from
4. Advise the CIO and CISO on the security risk of non- getting negative press.
synchronized time.
5. Get management approval for the purchase of time As organizations and IT processes become even more highly
synchronization equipment. synchronized, the importance of network time synchronization will
only increase, and so will the need for accurate, synchronized time.
6. Work with the CIO and CISO to ensure that time
synchronization is an enterprise policy.
4