30 STRONG: Seeking systems for all seasons

After 30 years of upgrades, Alyeska brings telecommunications, computer systems to peak

Rose Ragsdale

Advances in telecommunications and information technology have brought numerous improvements to the computer system that controls the trans-Alaska oil pipeline and the network that facilitates and reinforces its safe operation.

Moving at a slower but perhaps, surer pace than others in the industry, the 800-mile pipeline’s computer systems are finally coming into their own, demonstrating capabilities pipeline operator Alyeska Pipeline Service Co. could only dream about 30 years ago.

“Originally, when the pipeline was being constructed, there really wasn’t any communications infrastructure,” said Mike Joynor, vice president of oil movements at Alyeska.

Joynor leads a team of controllers and programmers who have marched the pipeline through three decades of changes. The pipeline’s owners hired AT&T to build a dual microwave system for telecommunications in the early 1970s. Erected along the pipeline’s right of way from Pump Station 1 to Valdez, it was an analog system based on serial communications.

“Back then, it was modern for the technology of the day,” Joynor said.

Nearly 25 years later in the late 1990s, Alyeska took another telecommunications company up on an offer to build a fiber optics system along the pipeline right of way and connect it with a sub-sea fiber cable then under construction from the Seattle area through Prince William Sound to Valdez and Anchorage.

“We moved to the fiber system in 2002, but we kept the analog microwave system as part of our voice communications radio system, or mobile communications system,” Joynor said.

Since then, Alyeska has worked with AT&T and GCI Inc. to fast forward its telecommunication infrastructure.

“AT&T went in and upgraded their entire analog microwave system to digital technology, making it fully redundant. With the two digital microwave systems and the fiber optic system, we now have a communication infrastructure that are independent of each other,” Joynor said.

The upgrades have given the pipeline ground-based and microwave-based telecommunications systems and enabled Alyeska to develop primary, secondary and tertiary communication links. Alyeska now has analog and digital microwave, fiber optic and digital satellite telecommunications available.

The three different levels of communications circuitry to the pump stations means that if any one system fails, Alyeska will retain its ability to see what is happening on the pipeline. In addition, the satellite link has proven to be very useful maintaining communications at high elevations such as Atigun Pass.

“Very seldom does anyone get in a position to have the best of all the telecommunications worlds,” Joynor said. “It gives us our 99.995 percent up time, which is the availability required by regulators for the pipeline. Outside of the military, that’s significantly higher than any other place you can find.”

Upgrades enhance control

The first two supervisory control and data acquisition systems were essentially mainframe computers that relied on serial, or point-to-point communications, but the one installed in 2005 is a network-based distributed system.

SCADA is an industry recognized acronym meaning “Supervisory Control And Data Acquisition system.”

Before the latest update, all of the SCADA system’s logic that controlled and protected the pipeline resided in the mainframe. This made the pipeline susceptible to power outages, said Paul Liddell, a computer systems engineer and supervisor of Alyeska’s SCADA system.

If the pipeline lost power, the controller working in our operations control center, or OCC, couldn’t take steps necessary to ensure a safe shutdown, Liddell said.

With the new distributed system, Alyeska has pushed the logic out as close to the field as possible, “such that if we lose communications, the pipeline can still safe itself out. (The system) can monitor for adverse conditions, and it can monitor for loss of communications with the OCC and can bring the pipeline down safely,” Liddell said.

The new system also allows Alyeska computer analysts to troubleshoot and monitor it.

“We can actually go into the programming logic that controls processes out in the pump stations and see what’s going on,” said Liddell.

The technology is similar to that used in the aircraft industry for diagnostic analysis of airplane engines while they are still flying in the sky.

Liddell said telecommunications is used to access the system and once inside, analysts use the computer system’s distributive nature “to peer around and see what’s going on.”

Though installation of the SCADA system is essentially complete, Alyeska programmers are up to their elbows in additional upgrades and reorganization designed to make the pipeline’s operation even safer and more efficient. This includes moving the OCC to Anchorage.

SIPPS started up this year

Part of this picture was startup earlier this year of a new PLC-based Safety Integrity Pressure Protection System, or SIPPS, that Alyeska launched to provide additional safeguards for running the pipeline.

Since February when the SIPPS station came on line, systems analysts have focused on fine-tuning it to essentially watch pipeline operations for mistakes and mishaps.

Liddell said SIPPS is a primary example of how the company has pushed the protective logic capabilities of the new computer system out into the field. Each pump station will have a SIPPS node when the work is completed, hopefully by 2009, and the nodes will be able to communicate with each other and exchange information continuously.

SIPPS will also monitor communications with other nodes on the same network and its own communications with the OCC, and take the appropriate action if a problem occurs in those areas, Liddell said.

Suppose a valve closes without receiving a command from the human controller at the OCC. SIPPS will respond by automatically shutting down pump stations upstream from the closed valve, Liddell said.

If SIPPS loses communications with the OCC or with other SIPPS nodes, it also will automatically shut down the pipeline.

“SIPPS is programmed to react if certain predetermined situations occur. It’s not artificial intelligence. AI implies a certain level of learning. We’d like to get to that point, but we’re not there yet,” Liddell said.

At the OCC, meanwhile, Alyeska is also examining each and every alarm that comes in to the controller and deciding whether the alarms need to be presented to the controller, Liddell said. “Obviously with one person sitting there trying to control the pipeline, being hit with more than 1,000 alarms in a work shift is a bit much to handle.”

Analysts are scrutinizing each alarm to determine its source, function, required response and whether it is even necessary. A new type of software developed by PAS of Seabrook, Texas, is helping with the automatic alarm analysis. The software intercepts all the alarms coming in and the operator’s actions going out. Then it suggests ways to make the process more efficient.

The plan to move the OCC to the Government Hill offices of AT&T in Anchorage is moving apace. Joynor said the new facility should be finished by year’s end and the OCC relocated in 2008.

“This is another area where technology advances have helped us,” Liddell said. “With the old point-to-point communications with a centralized mainframe, trying to coordinate this move would have been very difficult.

“With the new SCADA system, all we have to do is establish another point of control in Anchorage and we can run the pipeline in parallel or from one control point at a time,” he explained. “It really makes the transition over to Anchorage much, much easier.”

Off-the-shelf technology

“With the second generation SCADA system, we owned all the source code to it,” Liddell said. “When you buy UCOS, it’s essentially a toolbox. You can build yourself a house with that toolbox. “Before 2005, we had to build everything from scratch. It took a very high skill set to maintain, and quite honestly, a lot of us are getting a little long in the tooth and looking toward retirement.” (UCOS, an acronym for user configurable open system, is a complete control system solution. In layman terms UCOS is a set of software tools that allows a user to build a SCADA system that ‘out of the box’ can communicate using a large number of standard process control protocols – i.e. messages — over a wide variety of communication infrastructure, such as Ethernet, point-to-point, etc.)

Alyeska no longer needs to hire engineers who are specialists in programming to run the new system. “Our people can use the toolbox to configure the changes that they need,” Liddell said.

One reason Alyeska made the switch to off-the-shelf computer technology was that back in the ’70s the hottest things in computers were languages like FORTRAN and COBOL, etc.

“Most everybody who really understood those computer programs are in their 60s and 70s today,” Joynor said. “It’s about using computer software that is industry standard. You can bring in support personnel to keep your in-house people trained. But the farther you depart from commercial- or industrial-grade technology that’s available, the more you are building a system that requires specialists. You may have only one or two specialists, and if anything happens, you may have a gap in your ability to support your systems. That’s why we try to stay with commercial or industrially available technology.”

Alyeska’s leak detection program also got a boost from advancements in personal computing technology.

The complexities of hydraulic modeling of pipelines leads to some very difficult equations, and Alyeska engineers used to submit a problem to the mainframe computer and wait all day to get an answer.

With the newest PCs, the engineers can solve equations within minutes, Liddell said. “So our leak detection system is now based on PC technology.”

But these are not your standard PCs, Joynor cautioned. “They have a significant number of coprocessors in them, and they cost $10,000 to $15,000 each,” he said.

New capabilities, versatility

Flexibility and reliability, mostly.

“Once the OCC is in Anchorage, we’ll have routing through Glennallen to Fairbanks that connects us to the system that way and we will have routing that comes from Valdez to Anchorage through our alternate control center in the Palmer area,” Joynor said. “So if we had disruptions, say a fiber problem in Prince William Sound, we’re connected in Anchorage going north over the mountains and we can intercept and restore communications.

“We always make sure we have at least two and maybe three or four routes. Having three communications routes will be our standard when we’re done,” he said. “The underlying goal is for TAPS’ SCADA system to have the highest reliability that we can achieve with current technology. We never can be 100 percent, but we can be close.”