QRA for decision making – Getting the full benefit
Very often it is seen that Companies do not get the full benefit of a QRA and a QRA is simply a thick report on a shelf collecting dust, proving that the risk acceptance criteria of the plant have been met. The real value of a QRA is not identifying whether risk levels are within acceptable bounds, but rather gaining an understanding of the risks and the use of QRA techniques in the design and operational phase of the plant to aid decision-making.
This insight is for decision-makers and other end users of the QRA and will discuss:
Recommendations to a useful QRA for supporting decision makers.
Quantitative Risk Assessment (QRA) is a widely used and recognized methodology for analyzing the risk of process plants both onshore and offshore. Normally the fatality risk to people is quantified i.e. the risk that an individual or group of people suffers a fatality during a year of operation of the plant. Sometimes the risk to the environment and/or assets is also quantified.
In many cases, local regulations and/or Company rules require that a QRA is performed for plants with major accident hazard (MAH) potential to document that acceptable risk levels have been achieved.
For a QRA to be useful in supporting the decision-making process, it needs to fulfil the following number of things:
There needs to be a clear and transparent basis for QRA modelling detailed in the form of an assumption register;
The QRA model needs to contain the right level of detail or risk resolution to be able to support the intended decision-making process;
The QRA model needs to be performed in a timely manner;
The QRA model needs to be flexible and easy to modify design changes in the operational lifecycle of the plant;
The QRA model shall not only produce the end risk results/risk metrics to be compared to risk acceptance criteria but needs to present intermediate frequency and consequence results as well;
The QRA model needs to be quality-assured and validated.
A QRA without a clear, understandable, and detailed assumption register is a "black box" and in essence, is useless unless the decision maker has placed full trust in the QRA modeler. When developing a QRA model the importance of establishing the assumption register cannot be underestimated. "Garbage in will be garbage out" no matter how sophisticated the mathematical modelling performed is.
The QRA model must have a sufficient risk resolution to investigate the problem at hand. For example, the focus of onshore QRAs is normally on major accident hazards that can impact 3rd party personnel outside the fence of the plant. However, if it is intended that the QRA shall be applied for evaluating different layout options of the plant, the focus must not be only on major accidents but also on the many smaller hazards that may potentially develop into an MAH through things such as domino effects. When using a QRA to investigate different design alternatives it is important to start with qualitatively evaluating the advantages and disadvantages of different design options, and looking into how these will impact the QRA model. If they have no impact, then the QRA model will not be able to aid the decision-making process or in the worst case, even lead to the wrong conclusion, proposing that there is no difference between the different design options.
Challenges the end-user of a QRA shall be aware of
Before a QRA can be applied for decision making the decision maker must understand the uncertainties of the QRA. QRA, as well as all other forms of mathematical modelling of reality (economic modelling, epidemics, etc.), is far from an exact science and is associated with a significant level of uncertainty.
Several studies have indicated that there can be significant differences in the risk predicted by QRAs when performed by different risk consultants for the same plant. For offshore oil and gas QRAs, a study indicated that the risk levels predicted by different consultants could vary from a factor of 2 to 3. This indicates the vast uncertainty that is involved in the process of QRA modelling. For onshore QRAs, the difference may be smaller, as often more standardized and simplified modelling techniques are applied e.g. the Purple Book. This means that it is more likely that different risk consultants will produce similar results through QRAs, however, this does not mean that the results produced are more accurate than offshore QRAs.
Based on these studies, some people may conclude that QRA is a useless tool as the results are too uncertain and unreliable, but this is far from the truth. As long as QRA is utilized correctly and the modeler and end user are aware of the shortcomings of the QRA, a decision can be made based on the results that are produced.
Absolute versus relative risk comparison
One important insight is that whereas the absolute risk numbers of a QRA are often very uncertain the relative risk differences of QRAs evaluating different plants or design alternatives are significantly lower provided the same fundamental QRA model is applied (QRA model based on the same assumption register). Hence the QRA methodology can be a good tool for establishing the risk differences of different design alternatives in the early project phase. QRA methodology should only be applied for evaluating design options that differ significantly e.g. where different layouts or operating conditions are applied etc. For smaller design changes the QRA methodology is not likely to aid anything in the decision-making process.
Understanding the sources of uncertainty in QRAs
To better understand the uncertainties related to QRA modelling, it is important to understand the main aspects of QRA modelling:
Hazard identification;
Frequency modelling of hazard outcomes;
Consequence modelling of hazard outcomes;
Impact modelling of consequences e.g. number of fatalities.
The uncertainties about the above steps will be discussed in the following.
Hazard identification
It is self-evident that if a hazard is not identified and is unknown to the QRA modeler, then this introduces further uncertainty to the QRA model. This can normally be avoided by performing a thorough HAZID and conducting lessons learned from similar types of plants. The risk of missing a hazard is at its highest when looking at new types of plants relying on new technology. It is usually relatively straightforward to identify the immediate potential consequences of hazards, but potential escalation and domino effects can be easily missed. Missing these may not impact the absolute risk numbers predicted by the QRA significantly, as these events tend to be very rare (very low likelihood). However, should these events occur, the consequences may be severe, potentially resulting in multiple fatalities. It is therefore extremely important that these risks are identified and understood by the decision-maker.
Frequency modelling of hazard outcomes
When it comes to risk quantification, the frequency analysis introduces by far the highest uncertainty in the risk numbers. It is not unrealistic that the frequency analysis can introduce an uncertainty in the range of a factor of 10 to 100. Take for instance a release of a flammable material in a process plant that ignites to form an explosion. To model this frequency, the modeler has to establish as a minimum:
The release frequency;
The ignition probability;
And the probability the ignition may cause an explosion.
For release frequency modelling, generic failure frequency statistics for different types of equipment are often applied. The most comprehensive source of data is most likely the release frequency statistics from the Oil and gas industry in the North Sea region. Still, it is not uncommon that when data sources are updated for the Oil and gas industry the release frequencies vary a factor of 2 from the previous assessment. Looking at ignition probabilities the uncertainty increases further, approximately by another order of magnitude. This is due to ignition events being even more unlikely than loss of containment leading to an even more uncertain statistical sample than loss of containment events.
Consequence modelling of hazard outcomes;
Consequence modelling is normally far less uncertain than frequency modelling. With the use of Computational Fluid Dynamics (CFD), it is often possible to predict consequences with a relatively high degree of certainty. Even with simpler methods as opposed to using CFD, consequence modelling can normally be performed with far less uncertainty than frequency analysis. Thus allowing the decision maker to put more trust in the consequence modelling and less in the frequency modelling i.e. there is a good understanding of events that may happen, but less certainty in how likely they are to happen.
Impact modelling of consequences e.g. number of fatalities.
Furthermore, there is also uncertainty with impact modelling, however, it is normally possible to assign a robust conservative ruleset for this which results in realistic results. It is not normally considered the main source of uncertainty.
Based on the above, the decision maker must be focused on low-frequency events with high consequences, as the QRA may underestimate how often such events can occur. It is therefore important to focus on barriers preventing (left-hand side of a bow tie analysis) and mitigating (right-hand side of a bow tie analysis) such scenarios even though a QRA may deem that the risk of such events is insignificant compared to the overall risk of the plant and risk acceptance criteria. Furthermore, special focus should be placed on the uncertainties of such events. Read our article about "How to get bow tie assessments right"
How to effectively apply QRA’s for decision making.
Note while reading this, that decision-making shall never be solely based on QRA as input. QRA should only form part of the basis for making the final decision. For a QRA model to be suitable for decision-making in the design phase, it needs to be ready to be used in the design phase and provide results while the design is ongoing. This may seem like a self-evident requirement, but there are many real-life examples where complex QRA modelling has been performed, which was not concluded until after the detailed design was finished. This also emphasizes the fact that sometimes a simple and fast QRA model will provide more value to the decision-making process than a state-of-the-art complex QRA model. It is all about balance and compromise. It is also important that the QRA model is easy to update to match design changes and can provide relatively fast answers to the design team about proposed design changes. This is often a problem on projects where the design team needs to make decisions before knowing the consequences e.g. assigning blast load rating of structures etc.
Intermediate frequency and consequence results provide at least as much information as the calculated end risk metrics (personnel risk) used to check if the plant meets the risk acceptance criteria. This is especially true in light of the great uncertainties in the QRA results discussed earlier. Still, it is not uncommon that such data is left out of QRA reports. The data is relevant for assessing what is the worst that can happen, emergency planning, investigating near misses in the operational phase of the plant, etc. It is also vital information required if it is needed to update the QRA during the lifecycle of the plant, otherwise, you will be at the mercy of the original QRA provider.
As an end user of a QRA, it is important to interrogate the QRA modeler about how Quality Assurance and Quality Control have been achieved and make some sanity checks of both intermediate and results.
