Bob Braden, University of Southern California (USC) Information Sciences Institute (ISI) Fellow and Project Leader, attended the Working Group (WG) meeting of NASPI (North American SynchroPhasor Initiative) in Huntington Beach, California on February 20 - 21, 2013. This Working Group includes several hundred electrical engineers (EEs) and a few computer scientists, working towards standardization and deployment of instrumentation for monitoring and control of electric power transmission grids around the world. Their goal is a significant increase in reliability and efficiency of high voltage transmission. It is a part of a technical revolution that has sprung upon a technologically backwards industry, the "smart grid". This is a big deal. One fascinating session, at the meeting, analyzed the major regional blackout events in the past 10 years. The general conclusion was that the fine-grained instrumentation being planned by this Working Group should prevent many future blackouts, which are very costly to society as well as the utilities themselves. A synchrophasor is a measurement of the electrical state -- e.g., voltage, current, and frequency -- at a specific point in the electric power grid, repeating typically 60 times per second. "Synchro" refers to the use of GPS to time-synchronize these measurements, so collectively they provide an accurate and consistent picture of the grid state. The synchrophasor data is streamed over a computer network ("NASPInet") to a central control center for processing, to drive visual displays for the operators. The design of NASPInet and its relation to the Internet present interesting issues on internetwork architecture. The power grid is critical infrastructure, and the monitoring and control functions considered by the NASPI meetings form a cyber-physical system. Bob Braden has been attending NASPI meetings to promote the use of DeterLab for testing the proposed cyber-physical system architectures at moderate scale. Within the Data Networking and Management Task Team (DNMTT) of the NASPI Working Group, Bob presented DeterLab, and has described early experiments with DeterLab for testing middleware for synchrophasor data collection. At the February NASPI meeting, he presented a talk entitled "An InterNut Looks at NASPInet". This slide presentation summarized the key meta-principles behind Internet protocol design, and it presented Bob's perspective on NASPInet as a computer scientist with extensive Internet experience. For more information on DeterLab, visit: www.project-deter.org

1. DNMTT NASPI WG
DNMTT ‐‐ NASPI WG
Feb 20, 2013
An InterNut Looks at NASPInet
Bob Braden
Bob Braden
USC Information Sciences Institute
Marina del Rey, CA
Marina del Rey CA

2. • Last month was the 30th anniversary of the
Last month was the 30 anniversary of the
Internet.
• Is the Internet experience relevant to the
Is the Internet experience relevant to the
design of NASPInet?
• N dl
Needless re‐inventing the wheel would be
i i h h l ld b
costly.
2/20/2013 DNMTT Feb2913 2

3. Internet Protocol Suite (IPS):
Guiding Principles
d l
1.
1 Interoperability is essential
Interoperability is essential
2. Don’t over‐optimize
3.
3 Keep it as simple as possible (but no simpler)
i i l ibl (b i l )
4. Obey End‐to‐End principle whenever possible
5. Use distributed control
2/20/2013 DNMTT Feb2913 3

4. 1. Interoperability
p y
• Specify enough (but not more) so two
Specify enough (but not more) so two
conforming implementations will interoperate
correctly, out of the box.
correctly out of the box
– Allow multiple vendors
– Allow innovative implementations
Allow innovative implementations
• IETF Standardization rules: require at least 2
IETF Standardization rules: require at least 2
independent interoperable implementations.
2/20/2013 DNMTT Feb2913 4

5. … Interoperability
… Interoperability
• Before you accept any NASPInet proposal:
Before you accept any NASPInet proposal:
“show us the bits on the wire.”
• So vendor A and vendor B can interoperate
So vendor A and vendor B can interoperate
with different implementations.
• C
Comment: exposure of final NASPInet d i
f fi l NASPI design
in the IETF might save a lot of trouble in the
long run.
2/20/2013 DNMTT Feb2913 5

6. 2. Don t Over optimize
2. Don’t Over‐optimize
• Be willing to sacrifice some efficiency to gain
Be willing to sacrifice some efficiency to gain
flexibility & uniformity
– The problem will change
The problem will change
– The technology will rapidly evolve
• E “W
Eg “We can develop our own internetwork layer
d l i t t kl
protocol that will be more efficient that IP [for our
specific case].
specific case] ”
–BAD IDEA!!
2/20/2013 DNMTT Feb2913 6

7. NASPInet Example
NASPInet Example
• Two quite different NASPInet functions:
Two quite different NASPInet
1. Delivery of PMU data streams
2. Closed loop control
2 Closed loop control
• Could build two different NASPInets, one
optimized for each function.
ti i d f h f ti
• Probably a BAD IDEA (but not certain).
2/20/2013 DNMTT Feb2913 7

8. 3. Simplicity
3. Simplicity
• Simplicity ~ elegance in protocol design
Simplicity elegance in protocol design
• Complexity goes with:
– ugliness
li
– unmaintainability
– non‐interoperability
• Unfortunately, all protocols tend to accrete
features … !
2/20/2013 DNMTT Feb2913 8

9. 4. End to End Principle
4. End‐to‐End Principle
• Converse of the telephone system ‐‐
Converse of the telephone system
– Phones: smart network, dumb terminals
– Internet: smart terminals dumb network
Internet: smart terminals, dumb network
• Dumb network adaptable to new applications
– Minimal function in internetwork: => IP
– No per‐flow state in routers
2/20/2013 DNMTT Feb2913 9

10. … End to End Principle
… End‐to‐End Principle
• Those who would violate the E2E principle
Those who would violate the E2E principle
have to make a strong case in the IETF.
– IP multicast violates E2E principle
IP multicast violates E2E principle
– RSVP and Integrated Service violate E2E principle
– GridStat violates E2E principle
violates E2E principle
2/20/2013 DNMTT Feb2913 10

11. 5. Distributed Control
5. Distributed Control
• Avoid single points of failure
Avoid single points of failure
• Avoid implosion points
• Example:
– GridStat does pub/sub centrally
– IP MC, RSVP do pub/sub in distributed fashion
2/20/2013 DNMTT Feb2913 11

12. Untrue Rumors about the IPS
Untrue Rumors about the IPS
• IPS cannot stream real‐time data
IPS cannot stream real time data
• See VoIP, Skype
• IPS has no QoS mechanisms
IPS has no QoS mechanisms
• See Diff Service, Integrated Service
• IPS cannot bound E2E latency
IPS cannot bound E2E latency
• See Guaranteed Service under Int Serv
• IPS
IPS cannot do resource reservation
td ti
• See RSVP
2/20/2013 DNMTT Feb2913 12

13. Fact: Limited deployment
Fact: Limited deployment
• Backbone providers typically provide only
Backbone providers typically provide only
basic best‐effort service
• But some corporate intranets provide IP MC
But some corporate intranets provide IP MC,
diff‐serv, int‐serv, resource reservations, …
2/20/2013 DNMTT Feb2913 13

14. Some Common Fallacies
Some Common Fallacies
• The IPS* cannot stream real time data
The IPS cannot stream real time data
– Have you used Skype or VoIP lately?
• The IPS has no QoS provisions
The IPS has no QoS provisions
– See Differentiated Services, Integrated Services
• The IPS cannot provide bounded E2E delay
The IPS cannot provide bounded E2E delay
– See Guaranteed Service under Integrated Services
• Th I t
The Internet cannot do resource reservations
t td ti
– See RSVP *Internet Protocol Suite
2/20/2013 DNMTT Feb2913 14

15. Important Issues
Important Issues
In packet switching, important issues include:
In packet switching important issues include:
• Naming and Addressing
• C
Congestion
i
• Fragmentation
• Why aren’t you worried about fragmentation?
• Buffering
• Store‐and‐forward delays
2/20/2013 DNMTT Feb2913 15

16. Naming and Addressing
Naming and Addressing
• 61850 requires a 128 bit unique name for
61850 requires a 128 bit unique name for
every hardware box.
• A new name space => questions
A new name space => questions
– Who allocates names?
–HHow partition bits for distributed allocation?
i i bi f di ib d ll i ?
• Suggest: Use convenient /n notation from IP addr
• N d
Need new DNS‐like directory service to map
DNS lik di i
these hardware IDs to IP addresses.
2/20/2013 DNMTT Feb2913 16

17. • But if every box already has 128 bit unique IP
But if every box already has 128 bit unique IP
address, do we really need a new name space
at all?
at all?
2/20/2013 DNMTT Feb2913 17

18. NASPInet
• Will be a (more or less) crisply defined subset of
( ) py
the Internet.
– Completely isolated network? Never happen ‐‐
– “Backdoor” Internet connections inevitable
“Backdoor” Internet connections inevitable.
• Built on Internet protocol suite (IPS):
– IP at internetwork layer
IP at internetwork layer
– Some transport protocol (UDP, TCP, SCTP …)
• That positions NASPInet in:
– the Application layer (eg IP MC)
‐‐ or in middleware layer (eg GridStat)
2/20/2013 DNMTT Feb2913 18

19. The Isolation Myth
The Isolation Myth
• “If we build a new network isolated from the
If we build a new network isolated from the
Internet, we can get precisely the service we
need and avoid many security problems
need and avoid many security problems”
• It never works
–EE.g., need Internet back doors for management,
dI t tb kd f t
configuration, diagnosis, …
– Even with only one 300 baud modem connection
Even with only one 300 baud modem connection,
bad guys can get in and take over.
2/20/2013 DNMTT Feb2913 19

20. Proposed NASPInet P Architectures
Proposed NASPInet‐P Architectures
• IP MC – Myrda et al
IP MC Myrda et al
– Network layer MC, TA* at dest
• Chained PDCs
Chained PDCs
– App layer or network layer MC
– TA at intermediate PDCs and dest
• GridStat ‐‐ Bakken et al
– Middleware approach
*Time align
g
2/20/2013 DNMTT Feb2913 20

21. • Can we choose one of these?
Can we choose one of these?
• Must we choose one?
• What input do we need to choose wisely?
h i d d h i l ?
2/20/2013 DNMTT Feb2913 21

22. Some Opinions
Some Opinions
• We are underestimating the rapidity of
We are underestimating the rapidity of
evolution in communication software and
hardware
hardware
– Transformers can last 50 years, but specifying a 15
year lifetime for NASPInet seems foolish
year lifetime for NASPInet seems foolish
• Should think of a PG as a set of functions, not
as a device.
as a device
• PG currently has too many functions, is too
complex, in fact.
l i f t
2/20/2013 DNMTT Feb2913 22

23. More Opinions
More Opinions
• The “Data Bus” is an unfortunate and
The Data Bus is an unfortunate and
misleading metaphor. Should be cloud.
• Maybe we are underestimating the ability of
application builders to accommodate
li i b ild d
lost/delayed data.
– Buffering is cheap
– Physical system has lots of inertia
2/20/2013 DNMTT Feb2913 23

24. Danger Areas for NASPInet
Danger Areas for NASPInet
• NASPInet dynamics will be similar to Internet’s
NASPInet dynamics will be similar to Internet s
• Rapid technology change
• Ch i
Changing problem space
bl
– See: “An Internet‐Inspired Electricity Grid”, IEEE
Spectrum, 12‐14, Jan 2013.
• Success disaster
2/20/2013 DNMTT Feb2913 24

25. Robust, Interoperable
Implementation
Implementation
• Postel Principle: “Be conservative in what you
send and liberal in what you receive.
send and liberal in what you receive ” [RFC1122]
2/20/2013 DNMTT Feb2913 25

26. Thank You
2/20/2013 DNMTT Feb2913 26