The traditional security industry has somehow decided that they are the white knights who are going to save everyone from the horror of insecure powergrids, pipelines, chemical plants, and cookie factories. Suddenly, every consultant is an expert and every product fixes SCADA. And because they don't know what the hell they're talking about -- 'fake it till ya make it' doesn't work -- they're making all of us look stupid.
Attendees will gain a practical level of knowledge sufficient to keep them from appearing foolish should they choose to opine on any of the various real issues stemming from Industrial Control or SCADA systems. Attendees will also feel embarrassed for something they've said, empowered to call out charlatans, and much less worried about cyberhackers unleashing cyberattacks which cybercause cyberpipelines and cybermanufacturing plants to cybergonuts and cybertakeovertheplanet using cybercookiesofdeath.
Take control of your SAP testing with UiPath Test Suite
Notacon 7 - SCADA and ICS for Security Experts
1. SCADA and ICS for
Security Experts:
How to Avoid Cyberdouchery
James Arlen, CISA
Notacon 7 - Cleveland - 2010
1
2. Disclaimer
I am employed in the Infosec industry,
but not authorized to speak on behalf
of my employer or clients.
Everything I say can be blamed on
great food, mind-control and jet lag.
2
3. Credentials
15+ years information security specialist
staff operations, consultant, auditor, researcher
utilities vertical (grid operations, generation,
distribution)
financial vertical (banks, trust companies,
trading)
some hacker related stuff like game show host,
etc.
3
119. Credits, Links and
Notices
http://myrcurial.com and
Me: http://cyberdouchery.com
and sometimes http://
liquidmatrix.org/blog
All of you, My Family, Friends, Jeff
Moss (for demanding this talk)
Kaospunk, Froggy, Tyger and the
Thanks: Notacon Awesome Team.
Mentors/Luminaries: D. Anderson, M. Fabro, J.
Brodsky, R. Southworth, M. Sachs, C. Jager, B.
Radvanovsky and J. Weiss (all from whom I
twitter, fast music, caffeine, my lovely
borrowed material)
Inspirati
wife and hackerish children, blinky
on:
lights, shiny things, modafinil &
altruism.
http://creativecommons.org/licenses/by-nc-sa/2.5/ca/ 116
This ISN’T a talk about SCADA so much as it is a talk about TALKING ABOUT SCADA.
[twitter]http://myrcurial.com/N7/SCADA-N7.004.png[/twitter]
Around 2005, and almost all of a sudden, the infosec industry noticed SCADA. And immediately started identifying it as a market.
[twitter]http://myrcurial.com/N7/SCADA-N7.005.png[/twitter]
Of course, the simplest explanation is always the most likely. In this case, it was all about the money - there were regulators starting to breathe heavily (NERC 1200, ISA99)
[twitter]http://myrcurial.com/N7/SCADA-N7.006.png[/twitter]
And because a packet is a packet is a packet, there were suddenly a million security experts who were also scada experts. Let’s not even get started on the four letter security religion people and how they jumped on this one.
[twitter]http://myrcurial.com/N7/SCADA-N7.007.png[/twitter]
At this point, I was working in control systems security -- electricity in particular and as much as I could, I spent as much free time as possible pointing out these flawed responses to a very real problem.
[twitter]http://myrcurial.com/N7/SCADA-N7.008.png[/twitter]
And then the swarm of consultants and infosec dudes and even a few dudettes showed up and started telling me everything they “knew” about control systems security.
[twitter]http://myrcurial.com/N7/SCADA-N7.009.png[/twitter]
They tied a nice little bow on my problems, and told me they could fix it - just a few blinky lights and a few more shiny things and I was going to be fine.
[twitter]http://myrcurial.com/N7/SCADA-N7.010.png[/twitter]
I told you we were going to talk ABOUT SCADA systems. Here’s the short form. LANGUAGE is important - specificity is something that engineers really enjoy. They’re kind of like car people -- and our industry has been using words like “synchro-mesh transmission” to describe “derrailluer”
[twitter]http://myrcurial.com/N7/SCADA-N7.011.png[/twitter]
Between the experts pontificating and the media eating it up, well.
[twitter] http://myrcurial.com/N7/SCADA-N7.012.png[/twitter]
HIghly distributed systems used to control geographically dispersed assets (water supply systems, oil and gas pipelines, electrical powergrids, railways, etc.
[twitter]http://myrcurial.com/N7/SCADA-N7.013.png[/twitter]
HIghly distributed systems used to control geographically dispersed assets (water supply systems, oil and gas pipelines, electrical powergrids, railways, etc.)
[twitter] http://myrcurial.com/N7/SCADA-N7.014.png[/twitter]
Used where centralized data acquisition and control are critical or practical to overall system operation
[twitter] http://myrcurial.com/N7/SCADA-N7.015.png[/twitter]
Used where centralized data acquisition and control are critical or practical to overall system operation
[twitter] http://myrcurial.com/N7/SCADA-N7.016.png[/twitter]
When you’re talking about LARGE systems that are GEOGRAPHICALLY distributed and used for huge control undertakings like this... that’s scada.
[twitter] http://myrcurial.com/N7/SCADA-N7.017.png[/twitter]
Control Systems (CS) are used to control manufacturing processessuch as electric
power generation, oil and gas refineries, and chemical, food, and automotive
production.
[twitter] http://myrcurial.com/N7/SCADA-N7.018.png[/twitter]
Control Systems (CS) are used to control manufacturing processessuch as electric
power generation, oil and gas refineries, and chemical, food, and automotive
production.
[twitter] http://myrcurial.com/N7/SCADA-N7.019.png[/twitter]
Control Systems (CS) are used to control manufacturing processessuch as electric
power generation, oil and gas refineries, and chemical, food, and automotive
production.
[twitter] http://myrcurial.com/N7/SCADA-N7.020.png[/twitter]
Control Systems (CS) are used to control manufacturing processessuch as electric
power generation, oil and gas refineries, and chemical, food, and automotive
production.
[twitter]http://myrcurial.com/N7/SCADA-N7.021.png[/twitter]
Control Systems (CS) are used to control manufacturing processessuch as electric
power generation, oil and gas refineries, and chemical, food, and automotive
production.
[twitter] http://myrcurial.com/N7/SCADA-N7.022.png[/twitter]
Control Systems (CS) are used to control manufacturing processessuch as electric
power generation, oil and gas refineries, and chemical, food, and automotive
production.
[twitter] http://myrcurial.com/N7/SCADA-N7.023.png[/twitter]
CSare integrated as a control architecture containing a supervisory level of control overseeing multiple, integrated sub-systems that are responsible for controlling the details of a localized manufacturing process
[twitter] http://myrcurial.com/N7/SCADA-N7.024.png[/twitter]
Usually found in a designated critical infrastructure sector, a control system is a
collection of devices or components working together for a common process, controlled by
a master entity that can direct, regulate, and refine the behavior of those devices or
components through observations and commands.
[twitter] http://myrcurial.com/N7/SCADA-N7.025.png[/twitter]
Usually found in a designated critical infrastructure sector, a control system is a
collection of devices or components working together for a common process, controlled by
a master entity that can direct, regulate, and refine the behavior of those devices or
components through observations and commands.
[twitter] http://myrcurial.com/N7/SCADA-N7.027.png[/twitter]
These smaller and “contained” entities are the control systems -- they are generally PROCESS oriented. And we need to talk about them as separate entities. THERE ARE WAY MORE OF THESE THAN THERE ARE SCADA SYSTEMS.
[twitter] http://myrcurial.com/N7/SCADA-N7.028.png[/twitter]
This is the problem -- more than anything -- this incredible lack of understanding.
[twitter] http://myrcurial.com/N7/SCADA-N7.029.png[/twitter]
It doesn’t matter here whether we’re talking about SCADA or Control Systems... The computers are NOT that which is controlled - - And just like in so many other aspects of infosec - they are NOT the reason that YOU are involved.
[twitter] http://myrcurial.com/N7/SCADA-N7.030.png[/twitter]
“What happens when Edna falls into the reactant vessel” -- Just as you’d expect. The system STOPS. This is EXACTLY what happens when the computer breaks.
[twitter] http://myrcurial.com/N7/SCADA-N7.031.png[/twitter]
[twitter] http://myrcurial.com/N7/SCADA-N7.032.png[/twitter]
Protocols (partial list)
E/IP
DH+
ProfiBus
ANSI X3.28
BBC 7200
CDC Types 1 and 2
Conitel 2020/2000/3000
DCP 1
DNP 3.0
Gedac 7020
ICCP
Landis & Gyr
8979
OPC
ControlNet
Tejas 3 and 5
Modbus
TRW 9550
UCA
[twitter] http://myrcurial.com/N7/SCADA-N7.033.png[/twitter]
Protocols (partial list)
E/IP
DH+
ProfiBus
ANSI X3.28
BBC 7200
CDC Types 1 and 2
Conitel 2020/2000/3000
DCP 1
DNP 3.0
Gedac 7020
ICCP
Landis & Gyr
8979
OPC
ControlNet
Tejas 3 and 5
Modbus
TRW 9550
UCA
[twitter] http://myrcurial.com/N7/SCADA-N7.034.png[/twitter]
Mapping from the data to the process is HARD. There’s hours/days/weeks/months/YEARS of programming effort there. The protocol bitstream is just that -- a bitstream.
[twitter] http://myrcurial.com/N7/SCADA-N7.035.png[/twitter]
How do you know which device does what?
You need to find or see the mapping... not just the raw protocol data. One without the other isn’t terribly useful. Oh, I’m not kidding myself - there are some SERIOUS rockstar protocol reverse engineers out there. There are even some process reverse engineers. In all likelihood, you can BREAK the computer, but can you MAKE the computer do your bidding?
[twitter] http://myrcurial.com/N7/SCADA-N7.036.png[/twitter]
And guess what - you’re in a position to break part of it.... can you break all of the additional controls that have been emplaced? ALL OF THEM?
[twitter] http://myrcurial.com/N7/SCADA-N7.037.png[/twitter]
There’s a whole additional system under local control THAT IS NOT PART OF THE SCADA OR ICS/DCS system which keep equipment from going all Skynet/Terminator
[twitter] http://myrcurial.com/N7/SCADA-N7.039.png[/twitter]
So say that you manage to screw up the process -- the batch you were messing with... it hits the garbage pretty hard.
[twitter] http://myrcurial.com/N7/SCADA-N7.041.png[/twitter]
Because the organization cares enough to ensure that it only sends the right product out the door.
[twitter] http://myrcurial.com/N7/SCADA-N7.042.png[/twitter]
The most interesting part is that NONE of these systems are actually autonomous - they are all predicated upon having a human element - an operator, a controller, an organic mental component...
[twitter] http://myrcurial.com/N7/SCADA-N7.043.png[/twitter]
Partly because of liability issues and partly because Bags of Mostly Water are still much better at in-situ problem solving than any of the future silicon masters currently are.
[twitter] http://myrcurial.com/N7/SCADA-N7.044.png[/twitter]
Alright. So you’re a super-hacker. YOU busted the SCADA system. You pwnd them good.
[twitter] http://myrcurial.com/N7/SCADA-N7.045.png[/twitter]
Well... here’s the thing. They plan for that to happen. Most systems can handle two simultaneous failures without skipping a beat.
[twitter] http://myrcurial.com/N7/SCADA-N7.046.png[/twitter]
Because we’re sorta used to it.
[twitter] http://myrcurial.com/N7/SCADA-N7.047.png[/twitter]
Wires come down, and they get repaired.
[twitter] http://myrcurial.com/N7/SCADA-N7.048.png[/twitter]
pipelines break for all kinds of reasons - and they get repaired.
[twitter] http://myrcurial.com/N7/SCADA-N7.049.png[/twitter]
And nine hundred and ninety nine times out of a hundred... well, more like 99,999 out of 100,000.... you don’t feel it at all.
[twitter] http://myrcurial.com/N7/SCADA-N7.050.png[/twitter]
You’ve still got a cozy little house.
[twitter] http://myrcurial.com/N7/SCADA-N7.051.png[/twitter]
No one is wandering the streets looking for flesh to feed on.
[twitter] http://myrcurial.com/N7/SCADA-N7.052.png[/twitter]
Yup, under very controlled circumstances, with some modest efforts, and a known target surface (relatively turn key systems -- little to no customization) it is possible to make things go BANG. Suggesting that your garden variety NOTACON or DEFCON type hacker can achieve this in an afternoon is... well. Crap.
[twitter] http://myrcurial.com/N7/SCADA-N7.054.png[/twitter]
Make sure to go all kind of drifty -- notice SOMETHING in the audience and kinda “Snap” for the next slide.
[twitter] http://myrcurial.com/N7/SCADA-N7.055.png[/twitter]
All of you are perfectly smart. You’ve just got to pay attention and focus and HEY, SQUIRREL!!!!
[twitter] http://myrcurial.com/N7/SCADA-N7.056.png[/twitter]
Since you’ve solved all of your organizations security problems, you’ve got time.
[twitter] http://myrcurial.com/N7/SCADA-N7.057.png[/twitter]
Between the warring factions of business/asset owners, traditional IT departments and control systems IT departments...
[twitter] http://myrcurial.com/N7/SCADA-N7.058.png[/twitter]
But. Remember, you’re not the expert. Suck it the heck up. Buy some people some coffee.
[twitter] http://myrcurial.com/N7/SCADA-N7.059.png[/twitter]
EVEN though it feels disingenuous, become the student first, the teacher later.
[twitter] http://myrcurial.com/N7/SCADA-N7.060.png[/twitter]
Show a willingness to be the friend, the person who UNDERSTANDS that everyone is a unique and special person.
[twitter] http://myrcurial.com/N7/SCADA-N7.061.png[/twitter]
Ok. Here’s some things that I’ve discovered in my time as a control systems security dude.
[twitter] http://myrcurial.com/N7/SCADA-N7.062.png[/twitter]
Unions. Really. Woodshed talks down on the loading dock.
[twitter] http://myrcurial.com/N7/SCADA-N7.063.png[/twitter]
Hey, we’re in infosec, we all think we’re rock stars... right?
[twitter] http://myrcurial.com/N7/SCADA-N7.064.png[/twitter]
The VAST majority of the people that I’ve met in the control systems world would be perfectly happy with good ole 8-bit computers that knew their place in the world. You ARE the age of their kids, and therefore, you are a kid.
[twitter] http://myrcurial.com/N7/SCADA-N7.065.png[/twitter]
Yeah, you know you wanna.
[twitter] http://myrcurial.com/N7/SCADA-N7.066.png[/twitter]
UNDERSTAND the organization -- what the moving pieces are... look outside the IT department... shadow a few of the “workers” -- it’s a system like any other. Get all “Mitnick-y”
[twitter] http://myrcurial.com/N7/SCADA-N7.067.png[/twitter]
the doors begin to open... you’re starting to get things done.
[twitter] http://myrcurial.com/N7/SCADA-N7.068.png[/twitter]
Because hey... you can learn anything fast -- you’re an infosec rockstar. Make THEM change to suit the needs of the almighty altrusim -- KTLO, hold the Zombies at bay.
[twitter] http://myrcurial.com/N7/SCADA-N7.069.png[/twitter]
Just for review... because, believe it or not... you need to TEAR DOWN each of these preconceptions before you can build up what the glory of a real console feels like.
[twitter] http://myrcurial.com/N7/SCADA-N7.070.png[/twitter]
And actually, that’s the problem -- infosec people are fixated on the protocol and ignore the user interface.
[twitter] http://myrcurial.com/N7/SCADA-N7.071.png[/twitter]
And actually, that’s the problem -- infosec people are fixated on the protocol and ignore the user interface.
[twitter] http://myrcurial.com/N7/SCADA-N7.072.png[/twitter]
And actually, that’s the problem -- infosec people are fixated on the protocol and ignore the user interface.
[twitter] http://myrcurial.com/N7/SCADA-N7.073.png[/twitter]
And actually, that’s the problem -- infosec people are fixated on the protocol and ignore the user interface.
[twitter] http://myrcurial.com/N7/SCADA-N7.074.png[/twitter]
And actually, that’s the problem -- infosec people are fixated on the protocol and ignore the user interface.
[twitter] http://myrcurial.com/N7/SCADA-N7.075.png[/twitter]
And actually, that’s the problem -- infosec people are fixated on the protocol and ignore the user interface.
[twitter] http://myrcurial.com/N7/SCADA-N7.076.png[/twitter]
And actually, that’s the problem -- infosec people are fixated on the protocol and ignore the user interface.
[twitter] http://myrcurial.com/N7/SCADA-N7.077.png[/twitter]
Of course, you can have all different kinds of user interfaces...
[twitter] http://myrcurial.com/N7/SCADA-N7.078.png[/twitter]
And since you’ve got nothing but time... you’ve reviewed all of the log files...
[twitter] http://myrcurial.com/N7/SCADA-N7.079.png[/twitter]
And you’re just tired of doing the same ole same ole.... AND YOU”RE LOOKING IN THE WRONG PLACE FOR THE WEIRDNESS. Your effectiveness is in the toilet. Get your shit together.
[twitter] http://myrcurial.com/N7/SCADA-N7.080.png[/twitter]
And they just love trotting out these stories... kinda like the local news stations... “EXCESS DI-HYDROGEN OXIDE CAN KILL YOU... AND IT”S EVERYWHERE!!!!!!! MORE NEWS TONIGHT AT ELEVEN ON ACTION ONE NEWS!!!!!!!!”
[twitter] http://myrcurial.com/N7/SCADA-N7.087.png[/twitter]
Of course none of the 14 year olds I know (or was) are interested in world domination. They’re hormonally driven.
[twitter] http://myrcurial.com/N7/SCADA-N7.088.png[/twitter]
The conservatives want you to think of evil brown people.
[twitter] http://myrcurial.com/N7/SCADA-N7.089.png[/twitter]
But really, it’s middle aged white guys that are the hackers --- so easy a white guy can do it.
[twitter] http://myrcurial.com/N7/SCADA-N7.090.png[/twitter]
This story in the news Wednesday -- Booz Allen Hamilton is being paid has now landed the contract to build the Air Force’s cyberwar control center. For a measly $14.4 million in taxpayer money, the outfit will help build a new cyberwar bunker for the U.S. Cyber Command, a wing of the Air Force.
Additionally, Booz Allen Hamilton won another contract for $20 million to “foster collaboration among telecommunications researchers, University of Maryland faculty members and other academic institutions to improve secure networking and telecommunications and boost information assurance,” Washington Technology reports. While that might sound like a lot of money to set up a mailing list and a wiki, please don’t be cynical. Undoubtedly, McConnell’s crack team of consultants are providing the researchers with around-the-clock bodyguards and state-of-the-art bullet-proof monitors.
[twitter][/twitter]
Of course, this is what we’re all APT fraid of.
[twitter][/twitter]
And it’s right up near this as likely.
[twitter][/twitter]
And well... you know the internet is out to get you.
[twitter][/twitter]
[twitter][/twitter]
[twitter][/twitter]
Lack of security policy specific for control domain
• SCADA network separated only by VLANs and
rudimentary ACLs
• No change management policy
• Physical security policy richly enforced (but OPSEC
does not accommodate for access past defences)
• No Security Agreement (SA) with vendor, no SA
with contractors
Vendor default accounts and passwords have not been
changed
• Guest accounts still available
• No mechanism for schedule in place for
updates/upgrades
Primary HMIs do not require
username/password to get control
• HMIs may be secured physically but not
electronically
• VNC enabled EWS
LOTS of “shared” networks... internet access from HMI stations
Internet access TO HMI stations
“Run your process from your blackberry!”
Absence of testing of core OS
– Standard SCADA builds are rare (unused SW remains on systems)
– No testing in place for remaining applications
• Many insecure applications within key control servers
– To aid in operator boredom
– To aid in operator net access
– To aid in data manipulation
• Assessments discovered rogue applications trying to call
home
– Hostile ICMP payloads
– Covert channel over DNS
Vendor access (direct via VPN) into control network
• Access to main switch is by unsecured telnet, and main
switch gives all access to all comms
– Switches use default access credentials
– Traffic is not filtered by port (i.e. port filtering is not enabled
• No encryption or authentication on the control network
• Dynamic ARP is used with no ARP monitoring
• Firewalls have some interesting rules, sometimes very
simple:
# $fwadd-rule "allow udpfrom _any_ to _any_ 0-65535"
# $fwadd-rule "allow tcpfrom _any_ to _any_0-65535"
Vendor provides turnkey solution in each
customer location
• Commonality among deployments
–Same remote access mechanism
–Same username/password
–Same technology (brand, device, etc.)
–Same addressing schema
–Same vulnerabilities
PLCs unknowingly have embedded web
servers
• PLCs have embedded webserver enabled
• Data used as a significant step in
enumeration
• Compromised embedded servers allow
attacker to gain highest trust level
Basic flaws in programming can be
discovered and leveraged
• Vendors (proprietary) are very vulnerable
Least privilege
Least privilege
Buffer overflows
Buffer overflows (stack and
(stack and
heap)
heap)
Setuid
Setuiderrors
errors
Race conditions
Race conditions
Poor cryptography
Poor cryptography
Hard coded IP space
Hard coded IP space
RPC/DCOM
Telnet
Telnet
GUI
GUI
Password use/storage
Password use/storage
File Access
File Access
X
X-
-windows
windows
rsh
rsh(instead of
(instead of ssh
ssh)
)
sprintf
sprintf /
/ strcpy
strcpy
Accept all multicastRPC/DCOM Accept all multicast
Really. All of that stuff is real, seen it with my own eyes.
[twitter][/twitter]
Of course.
[twitter][/twitter]
[twitter][/twitter]
If we had any real “lateral thinkers” in the mix...
[twitter][/twitter]
But none of this is rocket science. In many repects, the control systems industry is living in the past - following the minimums of a modern hardening guideline would be good -- even though you’d likely seriously break the thing you were trying to fix.
[twitter][/twitter]
it’s just SUCK.
[twitter][/twitter]
And the machines only do as well as their masters.
[twitter][/twitter]
And the industry cannot seem to keep up with it’s own awesome. You can operate an HMI from your blackberry, and at the same time, they can’t fix the basics.
[twitter][/twitter]
[twitter][/twitter]
I cannot stress this point enough. become an infovore - consume knowledge - RTFM
[twitter][/twitter]
Generally speaking, someone who says they are an expert REALLY isn’t. Especially if they are really REALLY proud of being an expert.
[twitter][/twitter]
Project timelines are REALLY long, make little changes at the beginning.
[twitter][/twitter]
[twitter][/twitter]
[twitter][/twitter]
People who are putting themselves ‘out there’ as the mouthpieces... even the ones with actualy (albeit aged) cred... if your bullshit meter is going off, make sure other people know that. It’s on YOU to help catch and ?persecute? the charlatans out of out bidness. Call a Cyberdouche a Cyberdouche.
[twitter][/twitter]
You are not Zero Cool, Neo, The Plague, QQQQ John Travolta’s character, or any other uber 733t dude-ette. Impress with persuasion and humility rather than wearing your bravado and hackerdouchery. Also, shameless self-promotion -- please see my previous talk on the subject.
be the water drops. add requirements to the procurement process -- boil the frog. Also -- get to know your procurement people -- make friends EVERYWHERE.
[twitter][/twitter]
The overwhelming, vast, unbelievably dense history that we have as an industry is rich with comparable situations, problems found and solved, learn from them...
[twitter][/twitter]
Once upon a time, computers did what they were supposed to do. Help us to get there again.
[twitter][/twitter]
Thank you all so much for listening to me rant, I’m here for the rest of the day and tomorrow. Ask me anything and I’ll try to answer.
[twitter][/twitter]
Dave Anderson, Mark Fabro, Jake Brodsky, Ron Southworth, Marcus Sachs, Chris Jager, Bob Radvanovsky and Joe Weiss
[twitter][/twitter]