An Alaska natural gas pipeline project would be a mega project, with a price tag estimated at $20 billion to $25 billion.
Members of the Alaska Legislature got an overview of the risks associated with such projects — and mitigation measures that can address some of those risks.
Al Rogers of Virginia-based Independent Project Analysis told legislators May 19 at one of the administration’s gas pipeline fiscal contract overview sessions that IPA got its start as a result of the 1980s oil boom, the big projects of that time and the track record of those projects.
An abbreviated version of those projects, he said, goes something like this: “They ended up costing twice as much as predicted. They took twice as long. And they only worked half as well as predicted.”
Rogers, who has a Ph.D. in geology and geophysics and worked on operations and research assignments for Exxon and a small independent, has for the last 10 years been with Independent Project Analysis, a company that looks at projects and analyzes the outcomes compared to what was authorized. The company’s founder Ed Merrow worked at Rand Corp., and had the task under a U.S. government contract of identifying the drivers of those unsuccessful outcomes for 1980’s projects, Rogers said.
Merrow founded Independent Project Analysis, or IPA, and focused on statistical analysis of projects, both large and small.
IPA, Rogers said, “is the only company in the world” with extensive databases comparing what was promised when projects were authorized with the final outcomes.
Schedule acceleration guarantees difficulties
“The primary thing you can do to guarantee difficulty in a project of this magnitude is to try to accelerate the schedule,” Rogers said.
Mega projects, such as the proposed Alaska North Slope gas pipeline, aren’t like the broad range of some 10,000 projects IPA has in its database, Rogers said. Other projects have an outcome distribution from good to bad while mega projects have two outcomes: about as predicted and disasters.
Front-end loading, all the activities that occur before a project is authorized with a firm cost estimate, is something IPA has historically been associated with, he said. In IPA’s experience people want to get on with mega projects — selecting the route, cutting steel, welding — but successful mega projects involve what Rogers called “early-time preparatory activities.”
He said a mega project is one that meets one or more criteria: the cost is greater than $1 billion; “it’s being executed in a region where the project alone changes an environment” such as labor needs or infrastructure; or it represents a major step-out of complexity, size or location for the company.
Mega projects also attract a lot of attention, he said, both from formal stakeholders and from people who are not formal stakeholders but believe they have a right to be involved in the project.
Rogers said industry tends to have a narrow focus for projects and from looking at very big and mega projects it is necessary very early on to expand the scope of people who “legitimately or at least in their own mind might be stakeholders. Because the success for running fast after authorization is to make certain that all of those early-time potential stakeholders have either been satisfied or you’re reconciled somehow you’re going to handle their objections.”
He said that in studying mega project failures, “the common thread was pushing past somebody who thought they had a legitimate right to be a stakeholder” because if they raise objections later, the risk of delay “goes significantly higher.”
High failure rate
Rogers said IPA evaluates a project in four ways: compare it what IPA estimates industry would have spent on the project; how well did the project stay on schedule; how did it do against its own proposed budget; and on whether it had “severe and continuing operational problems” following completion. By these measures only 44 percent of mega projects were successful, with 42 percent failing on one measure, 32 percent on two measures, 21 percent on three and 5 percent failing on all four measures.
Compared to other projects, mega projects “rather than being robust because of their size are inherently very fragile,” Rogers said, can get off track very easily, and once they get off track “they’re almost impossible to get back. And so you lose control.”
Public information about Sakhalin provides an example, he said: they have had significant cost increases. But more important, he said, “the project team essentially lost control of what was happening. They eventually had project manager changes. Things had changed so much that they no longer had a guide or blueprint by which to go forward in the execution of the project.”
That is why IPA focuses so much on front-end work, he said.
All projects need front-end work to define, plan and schedule, he said, “but particularly mega projects.”
“If that work is left undone or it is allowed to slip, that turns out to be one of the primary attendant drivers of mega project failure.”
And how is success measured?
Good mega projects come in plus or minus 10 percent on cost and schedule, he said.
Front-end loading
Rogers said as much as 5 percent of the project’s cost may need to be spent on the front-end. One project offshore Eastern Canada had a 100 percent cost overrun and was two years late. He said those in charge of the project said the up-front costs were too high, that management would never approve them, and went ahead without any front-end definition, just began executing the project.
Integrated teams are also important, he said, and while industry does that pretty well, with mega projects they only do it half as well.
Joint venture projects are more successful, Rogers said: “Our data show that having more joint venturers leads to better results at the end even though the pathway is rockier and more contentious. The thing that more partners do is they bring different points of view.”
He said anecdotal evidence suggests that joint ventures are problematic but the data IPA has collected doesn’t support that. The data, he said, shows that mega projects with fewer JV partners have worse outcomes. That’s because the JV partners will have different experiences and different areas of expertise and will bring things to the team’s attention that need to be done early vs. late in the process.
“Projects that have a higher number of owners, JV partners, develop better front-end loading. They do their homework to a statistically significant better level,” he said.
Asked about how successful the gas pipeline sponsors, BP, ConocoPhillips and ExxonMobil, have been with mega projects, Rogers said they all have done successful mega projects. And all have done unsuccessful mega projects.
“And I’ll say this pretty loudly: it’s not the company, the company culture or policies and practices. It’s the team they put together and the amount of front-end work they do.”
Looking at the upstream industry in general, he said, “There’s a wide variation in outcomes based on the quality and effort of the team.”
Clear understanding of objectives
Rogers said there are a number of things to be considered when doing a mega project.
All members of the project team must have a clear understanding of the objectives when the project is authorized. IPA data shows that in 20 percent of mega projects that clear understanding is lacking. It’s easy to have multiple objectives when things are going smoothly: “The tough thing comes when you have a problem and you have to decide,” decisions such as what’s more important — cost or schedule; cost or operability? It is one area where having a joint venture may cause a problem, he said, because if the JV partners aren’t agreed on project objectives, then they have conflicting objectives and the project team doesn’t have clear guidance.
In addition to a definition of the project, the team needs to be robust — not a lot of holes in the organization chart and not a lot of boxes filled by the same person; and full staffing needs to occur early on in the front-end process.
If a project team can’t meet objectives the reason is likely that owners didn’t give them enough time, or enough money or enough people.
Rogers said that unsuccessful projects are not the result of stupid people, or people who didn’t work hard. “What happened is things occurred that distracted the project management,” drew them off to fight fires. There have to be enough people on staff that there is “the capability to fight fires as they occur to keep the project on track.”
The people who are going to operate also need to be involved early on so that their comments about what will work and what will make things reliable are heard early on. “One of the major sources of late changes in mega projects is bringing the operations people in late” and being told then that something won’t work for safety or reliability reasons.
As for the cost, Rogers said the amount of money saved by not doing it right upfront will be more than made up by cost overruns and late completion if the upfront work isn’t done. “That’s the entire thesis that the data says the late-time costs are significantly more than the early-time investment in all of these things we’re touting,” he said.
And a crucial part of doing that upfront work is having the team in place: “You can’t get to good FEL (front-end loading) unless you start with an adequately staffed good team.”
Team needs enough authority
Joint venture partners will bring different large-project processes to the table, Rogers said, and IPA’s data shows that it’s important to pick one of the processes, not to try to create a hybrid from all of the systems, because “nobody has experience with the hybrid system. Nobody knows how to operate in it.”
An issue that will probably be difficult for the state is giving the project team enough decision-making authority, he said. Otherwise so much time is wasted taking things to the governing committee for approval that the project gets off track.
What IPA recommends, he said, is that as long as a team is within an approved budget for each phase of the project the authority should rest with the team. “And if you’re not willing to give that team those authorities, then the question is whether you really trust the people that you’ve detailed to do this job.”
What drives lack of trust, Rogers said, “is nonalignment on business issues,” nonalignment on objectives.
With a project process the team asks for money to do things in “Gate 2” at the end of “Gate 1.” With a budget and a schedule the team has permission to do those things.
While integrated teams and adequate authority are important drivers of success, “the underlying driver is those characteristics are associated with better preparatory work — better front-end loading. That’s what the data says.”
What is front-end loading?
Rogers said front-end loading, where IPA puts its emphasis, is everything before project authorization.
This includes appraisal: what are the different ways things could be done? It includes selecting — making some final decisions about the scope of the project, such as the diameter of the pipeline, the route, major equipment, degree of instrumentation and degree of automation. Then the project is defined and engineering is done to specify how the project will implement the decisions.
That detail provides the information for a firm cost estimate, he said.
Rogers said he realized legislators were concerned because they didn’t know how much the project would cost and how long it would take, and are worried about what could happen in the bad case. That, he said, is a function of how much the project team does before the project gets final authorization.
How much is enough?
Advanced engineering status before final approval includes 18-30 percent of design complete; extensive input into design from owner/operator; and finalized process technology. About 55 percent of mega projects achieved advanced engineering status. The results: 3 percent cost deviation compared to 29 percent for other projects; 7 percent schedule slip compared to 21 percent; 11 percent of projects had operating problems compared to 50 percent; and 70 percent project success compared to 20 percent.
Scheduling
The long lead times required for some items is the biggest issue for project execution planning and scheduling, Rogers said.
Labor is also important, he said, noting that Alberta oil sands projects are running into a problem because labor is not as productive as expected.
The temptation is to let the contractor worry about labor, but Rogers said unless the project team “has their own resource level schedule (for labor), they really don’t know what is doable and what the impact is going to be on the environment of execution.”
The objection to doing resource loading is that it costs time, but he said the only way a team can “evaluate whether contractor bids are credible or not” is by having their own “independently devised schedule and manpower loading.”
And resource loading isn’t just manpower, he said: “It’s vehicles to lay pipe. It’s a whole host of equipment — things that may or may not be in short supply.”
Late changes — things not included in the plan — can have a big cost on the project, Rogers said. “If it’s late, it has cascading effects in terms of more costs ... a stretched schedule as well as poor operability,” he said. And late changes happen to even the best defined projects: almost half of mega projects “have major late changes,” he said. But, he said, 100 percent of projects with poor definition have late changes.
How much time does front-end loading take?
About half the time it takes to execute a project, Rogers, said, two and a half to three years of front-end loading for a project that takes five to six years to execute.
Aggressive schedule targets contribute to premature starts on detailed engineering and fabrication: 73 percent of projects driven to fulfill a schedule were failures, he said.
And when do you know how long a project will take, he asked. That is possible only after you look at all of the components, all of the activities that it takes to do the project. “And the tendency for everybody before they look at the inputs is to do what? Underestimate the time,” Rogers said.
While schedule-driven projects are expensive, “they’re not significantly faster,” he said, because once you get behind you spend more money trying to catch up and costs balloon.
Based on some mega projects that have “recaptured control” after bad things happened, “the best thing to do is to admit that you may not make the schedule, re-baseline and calibrate,” and don’t spend a lot of money trying to recapture the original schedule.
When should commitment be made?
Rogers was asked when during front-end loading the companies, and the State of Alaska, should commit to a project.
With three phases of FEL, he said, commitment would be at the end of FEL 2. At this point the team knows what the major scope decisions are, and the commitment can be made to approve the project if the team comes back at the end of FEL 3 with an estimate within the range of what they had at the end of FEL 2. A lot of money will be spent in FEL 3, Rogers said, “and you really don’t want to go forward and do that and at the end say, whoops, I don’t like the answers.”
As to whether the participation of a government increases or decreases the chances of project success, Rogers said “it’s how well that government company participated in team selection, business objectives and FEL.” He offered examples that went both ways. “The participation of a government in equity is not, in and of itself, a driver of success or failure. It’s how the government behaves and whether they endorse or demand good behavior.”
Asked about work commitment language in the gas pipeline contract Rogers said he thinks “there’s more work commitment language than there is in most of the other ones that I’ve looked at. And so my opinion is that it’s a pretty good balance.” Successful mega projects have early-time work, he said, and some of that “is characterized by the statements that are made in the work commitments package. In fact, it’s getting more than is in the work commitments package done that’s going to give you success, not just hewing to the letter,” he said.
