Securing the control system from IT

Friday, June 11, 2010 @ 11:06 AM gHale

By Dan Schaffer
In today’s cyber-sensitive environment, it is imperative IT and engineering work together to ensure a secure operating system. To compound that, it is also important for engineering to protect the control system from IT.
About 6 months ago, while visiting users in Michigan, we stopped in to see the principal controls engineer at one of the Big 3 auto makers.
When we sat down and started talking about various issues on the plant floor, we talked about a solution that allows a manufacturer to isolate a control network from IT and protect against unwanted traffic. His eyes lit up. It seems he had just had his entire test control network taken down (unintentionally) by IT while he was in the middle of a multi-day test.
When stuff first started acting odd and crashing, he called IT and asked if they were doing anything to cause this. The reply was a surly “no, don’t be ridiculous” type of statement.
But the problems persisted. His network was still sluggish and unstable so he used Wireshark, which is a free and open-source packet analyzer used for network troubleshooting, analysis, software and communications protocol development, to see if he could figure out what was wrong.
He found huge amounts of traffic hitting all of his devices. He called IT back and asked again if they were doing anything odd or unusual to cause his network grief; again the response was a resounding “nope.”
He then asked about the address that was the source of the bulk of the traffic in Wireshark. After checking around, IT begrudgingly told him it was “one of ours” and after a little more investigation on their part he found out one of the IT guys was running a utility to check who/what was out on the network.
It was essentially a flood or arp and ping-type traffic. Since there was no firewall or even a router (which doesn’t forward broadcast traffic) between the office network and the control network, the PLCs, HMIs, etc. got overwhelmed and started locking up.
He became a customer within a week.
Dan Schaffer is a networking security specialist at Phoenix Contact.

One Response to “Securing the control system from IT”

  1. alkoman says:

    Hey – I just saw this on Phoenix’s Facebook page. I do a lot of network consulting for different folks and I hear this stuff all the time.

    It sounds like their IT dept was doing a pretty poor job at understanding what was on their network and it’s importance to the business (not to mention knowing what was going on in their own dept, ouch!). Nevertheless the engineers were just as at fault (or more) for connecting their equipment to the office network! That’s pretty stupid…. No wonder their IT dept was trying to find out what was connected to their office switches! The engineers should have worked with IT long ago to get a separate network. How long would it have really taken to setup a VLAN?

    Also your whole argument about “protecting the controls system from IT” is silly – it is more likely that a virus on a user’s laptop would be spamming the office network and causing problems than an experienced IT professional. You should pitch protection from the “office” network instead.

    Anyway, I’m glad to see you offer solutions that will make both parties happy. With the possibility of completely separate network environments IT doesn’t have to worry about engineers plugging in rogue DHCP servers or STP devices into the office networks, and engineers can be in control of their own switches to configure them way they need without worrying about IT (and office viruses).

Leave a Reply

You must be logged in to post a comment.