Gilsdorf: Safety, Security Incident Costs Mount

Wednesday, August 31, 2011 @ 05:08 PM gHale

Editor’s Note: Norm Gilsdorf, president of Honeywell Process Solutions, sat down during the Honeywell User Group in Phoenix and talked with ISSSource Founder and Editor Gregory Hale and talked about how the importance of safety and security are continuing to grow throughout the manufacturing automation industry.

ISSSource: Last year you talked about the industry losing $20 billion each year in safety and security incidents. Is that numbers still valid?
Gilsdorf: It is not getting better I can say that. We actually have a site where we track incidents and it doesn’t appear to be getting any better. I am discounting the major incident we had in the last year, which would have added a lot more to those statistics. No, I don’t think it is necessarily getting any better – and you might ask why? I think we are under the same circumstances: The equipment is another year older; probably the experience level is another year older and about to turn many years younger because of the aging and turnover of the industry. The technology continues to get a little bit more sophisticated, but maybe all the tools and investments to leverage it are not fully in place yet because the industries have had another year of uncertainty and low profitability and some of the verticals had to consolidate in some markets. When you are in that environment, there is not a lot of investment and your assets continue to age. There is not a lot there that is going in the direction of making it better. We as a company keep driving to make solutions easier and better, but it takes time and investment. We are absolutely dedicated and committed to continue to bring better technology, software and training to the industry.

Q&A: Jump Starting Human Reliability
Emerson CTO: Keep Security Simple
Defend Against Stuxnet, ‘But Look at Big Picture’

ISSSource: The technology is there, but in many cases it all comes down to people, is there some type of training they have to undergo or is there a deep-seated lack of knowledge about security and safety?
Gilsdorf: No, I don’t think that is the case. Some of it may be training because there has been a fair amount of turnover and if training is not rigorous, you can run into some incidents. When I was starting up units in my earlier days at UOP, you would go into an operating unit and the operators would be very experienced. They knew their unit inside and out. I think many of them, if not all of them, have now retired. The size of the units has shrunk and the younger people that have come in to replace them are not interested in 30-year careers. That is a huge shift.

You reduce the amount of people that are 30-year experts and bring in guys who are spending 1, 2 or 3 years and then moving on. What I found before I left UOP, is there is a unit a that makes a lot of gasoline in the U.S. called a fluid catalytic cracker (FCC unit). FCC units years ago used to shut down every year or two for maintenance, but the licensors and equipment improved and they now shut down every four years. Once in awhile a massive thunderstorm would roar through the Midwest or the South and knock out a massive amount of power and shut down these FCC units. The companies would call out UOP to restart the plants because no one had ever been there during a start up. So, it’s a lack of training, turnover of talent, less people available to operate more sophisticated equipment and people not trained on the equipment.

In addition, there is the idea of alarm flooding. Now, when there is an incident, you overwhelm the young operator. We are trying to get more simulators and simulated training. I just don’t know if that has had a big enough impact.

Honeywell's Norm Gilsdorf.

Honeywell's Norm Gilsdorf.

ISSSource: You talk about data coming in and people have been talking about sensory overload for a long time. Have you gotten to the point where you are able to put that data in the proper context for the right people?
Gilsdorf: Yes, we have in three areas. One is around alarm management: How to better manage alarms. How to “bucketize” them. How to visualize them. How to position them and prioritize them so the operator can react in the right logical order and find the core root of the problem and not focus on the stuff that is not critical. That is one; the alarm management aspect and the visualization of it.

The second is by automating the procedures and having it right there on the control system so if something happens he is given a procedure and an action and recommendation to play.

The third one is about getting information in advance of a problem in the hands of the experts so they can fix the problem before becomes a problem. Matrikon (Honeywell acquired Matrikon last year) is very good at gathering the data managing it, organizing it and putting in a form on the desk of the right guy. That is, by the way, part of the hidden issue here is there is less engineers and less experts that can see the problem before there is a problem. So, with us getting the data cleaned up and scrubbed, we can create dashboards and other analytics that can help get the information to the right guy at the right time so he can predict something. The second part of that is the asset manager in the (Experion) R400, which can gather data around the aging assets and put analytics on the data so the operator can see what is happening, or the maintenance operator can see the bearing temperature has been going up a degree a day and put it in the maintenance schedule before it fails.

So we have asset management, data management, getting it to the expert, alarm management and the procedural steps to take. All five of those are focus areas and they all point toward better safety, reliability and operations.

ISSSource: You talked earlier about that big incident that would have added to the numbers. We live in a world where Stuxnet took down a control system that was keeping a nuclear plant up and running. Do you think users really know the seriousness behind that attack or do they feel it was such a focused attack they feel they are safe?
Gilsdorf: That is hard to answer. I can say a fair number of people recognized that and understood what it meant and they have taken steps they can take to make themselves safer. They have contacted us and we continue to work at putting safer systems out there. We always put our systems through rigorous testing. I think it made us redouble our efforts. There are things we have to do, but there are things customers need to do in terms of their patches and using different software to protect themselves. I think many of them understand this and they are asking questions of the IT and control departments to make sure they are safe.

ISSSource: In terms of the executive level, are they truly aware of the potential they are the next target?
Gilsdorf: I think it varies quite a bit. Maybe it is more regional. Certainly in the Middle East there are very aware of it. They bring it up quite frequently there. I haven’t heard it a lot in the U.S. I would say it is gaining momentum. In the beginning I think they would say it is not going to happen to me, but I think it is building and gaining attention.

ISSSource: It seems like more companies are coming out and saying they were victims of attacks. Why is that?
Gilsdorf: Before you didn’t want to draw attention to it. You didn’t want anyone to know you were under attack, or vulnerable to attack. You didn’t want to put a big red flag in front of the bull. Now people are willing to talk about.

There are still plenty of events people don’t talk about. I think there are people that don’t want their stockholders to think they weren’t talking about it because the information will probably get out. There is also the issue there was always these one off hackers that do it for fun. Now, there is a growing aura of state-sponsored hacking and that has brought more people forward.

When it is an individual hacker, people felt it could be managed because that hacker didn’t have enough resources. Today, as attacks are growing, there is a political aspect coming down. I think that type of hacking with the sophistication and quantity; people can begin to tell there is a lot money behind it.

ISSSource: Manufactures think safety, it is part of their DNA, is security part of their DNA now?
Gilsdorf: I remember you asked me the same question last year. I would have said last year security is (at a lower level than safety) I will tell you this year safety is up a little higher and security is not rising as much. It is not at the same level.

It still goes back to IT and the IT side of these organizations What we will probably see more is that it is not just the IT guy talking about it, but the talk will be much broader within the organization.

ISSSource: Poor security for SCADA systems has been exposed. Do you find these system are the low-hanging fruit for would be hackers?
Gilsdorf: I think some systems, like some SCADA systems, because they are dispersed over broad areas on broad networks may have a different exposure level than something that is concentrated within a room somewhere. So, I don’t think it is just SCADA systems necessarily. I don’t think all SCADA systems are susceptible.

ISSSource: Major safety incidents continue to be in the news, why?
Gilsdorf: This goes back to what I said earlier. The equipment is getting older, the experience is retiring, the new people have to come in, the turnover of people is higher and companies are trying to get the most out of their assets and you see what happens.

Here is an analogy to think about: Imagine if we made a rule that all the drivers in next year’s Indy 500 could only have driven in one Indy Car race before. That was the qualification for the race. What do you think the outcome of the race would be? How many cars would finish? How many people would be in trouble? The same could be said for the pit crews. They could be trained, but only worked in one previous race. Here they are in the race with rookie drivers. We are talking about high technology. We are talking about fast speeds. We are talking about running on the edge; people want to push their assets. I will take the analogy a little further: You can’t have any new parts in your car. You have to deal with parts that are one or two years old.

What do we have to do as an industry? It goes back to training. Better training, better asset management to tell which parts are bad and good, better training for the maintenance people, putting better dashboards and better software and controls to make it easier for the driver to make less mistakes.

There is a lot out there that is still in the systems today that requires intervention before you take action. We can help solve that by taking away some decisions by automating so we can help the operator.

ISSSource: How far do you go to take the decision making out the human’s hands? At what point does the process become totally automated?
Gilsdorf: That is where our expert groups of ASM and API and the different folks in the different industries we work with determine things like where it makes sense and it is safe to automate and where is it the human needs to make a decision. And that process will be constantly changing and moving. It is our job to put forward solutions that can help automate it in safe way and it is up to those industrial bodies to judge whether or not to take that next step.

ISSSource: Do you consider safety and security cost centers?
Gilsdorf: No.

ISSSource: What do you consider them?
Gilsdorf: I consider them fundamental parts of running a good business. To me safety, security, quality are fundamentals. If I don’t get my safety right, my security right and quality right, it is like a stool without any legs.

ISSSource: Do you see government getting more involved in cyber security?
Gilsdorf: All I can comment about there is when Secretary of State Hillary Clinton said some attacks could be considered an act of war. That is your answer very clearly. When the Secretary of State gets on the podium and talks about it. That tells me the government is getting more involved. I don’t know how or which way, but there will be more involvement.

One Response to “Gilsdorf: Safety, Security Incident Costs Mount”

  1. damian says:

    Nice article. I have worked as a developer in the industrial software / process historian area in the past and was often shocked by the huge reliance on the dated ‘perimeter’ security model, i.e. the assumption that anyone who is on the network is already authorized and so can do almost anything in terms of database connections/access and server access. Most oracle-driven apps I worked on for example included an authorized users table with passwords stored in clear text… some such apps were core to the companies daily operations and alarm management.

Leave a Reply

You must be logged in to post a comment.