In light of the recent Epsilon data breach, it seems appropriate to chat briefly about the realities of balancing information risk. First and foremost, we need to make sure that we understand this thing called “risk.” In our context, risk is defined as “the probable frequency and probable magnitude of future loss” (based on Jack Jones’ FAIR definition). Put into practical terms, risk is the likelihood that we’ll experience a negative event. We then balance that out against the cost of defending against various scenarios (i.e., trying to reduce or transfer that risk), with the goal being to optimize cost vs. benefit. Let’s look at a couple practical examples.

Read the rest of this entry »

Unless you’ve been living under a rock for the past week, then you undoubtedly know that Japan was rocked a few days ago by an 8.9 magnitude earthquake (the 3rd largest in the past decade and top 10 overall – also check out the NYT’s before & after shots) and a subsequent tsunami that exponentially compounded the ill effects of the disaster. Coming out of that incident, one of the most hyped “news” items has been the aftermath at the Fukushima nuclear power generation facility. It turns out (unsurprisingly) that much of this coverage has been faulty, inappropriately throwing around talk of “melt downs” when, in fact, things are under control.

For a great, detailed description of the entire incident, check out Barry Brook’s post “Fukushima Nuclear Accident – a simple and accurate explanation” over on the Brave New Climate blog. It’s an excellent discussion of the accident, which highlights several poignant points that can be directly applied to information security and information risk management (also see this post, which dispels one inaccuracy in Brook’s post – that there is not, in fact, a “core catcher” installed – and provides even greater assurance that things are well in-hand).

Specifically, there are 5 take-away points to consider:
Read the rest of this entry »

News flash: Those so-called “risk” labels/ratings included in pentest and vuln scan reports are NOT actually “risk” representations.

I was in attendance at the OWASP Summit 2011 a couple weeks back, and the topic of “risk metrics” and labels came up during one session. As a result, I led a break-out session on what risk really looks like in the macro sense, in accordance with formal methods, and where these various scan/test results really fit in. The session had great conversation and highlighted for me a need to expand risk analysis training to a broader audience.

Below is a picture of the taxonomy of factors that make up a FAIR (Factor Analysis of Information Risk) risk analysis. Putting aside the discussion of how one generates the value ranges that go into each factor, let’s look at where pentest/scan results fall. Looking at the taxonomy below, note that there are two key halves: Loss Frequency and Loss Magnitude. As you peruse the factors that roll up to those halves, think about where your pentest/scan results might fit.

In order to properly estimate “risk” based on the results of a pentest or vuln scan, you need to understand the business impact in a number of structured scenarios. Simply understanding the Loss Frequency is not sufficient to estimate risk. However, that said, there is certainly valuable IF you process those findings accordingly. The main factor reflected by these test results will be “Resistance Strength,” which is to say that you will now have a better understanding of how much effort would be required to compromise your organization (given: compromise given enough time and resources is inevitable).

The next time you get a report from a vendor that talks about “risk,” please challenge their assertion. Ask them to explain how they estimated the financial impact of various weaknesses on your organization. Unless they interviewed your business management team to understand how a weakness could impact the business if exploited, then I submit that they’re not providing you with an actual risk rate. Instead, what you’re getting a snapshot reading of your Resistance Strength in a given context, as well as receiving some hints on what opposing Threat Capability might be. You’re still left needing to estimate the other factors under Loss Frequency, and you almost certainly need to fill in the blanks under Loss Magnitude (something that can be accomplished independently by developing loss tables for key systems, and then either providing those to 3rd parties performing the assessments, or combining them with the results on your own).

As an aside it’s worth noting that some 2nd generation “GRC” platforms are starting to integrate risk analysis capabilities like FAIR that can be leveraged to merge in scan/pentest results and to generate a reasonable risk analysis.

There has been much criticism of risk assessment and analysis over the past few years that amount to much ado about nothing. Why is it much ado about nothing? Well, because, quite simply, people oftentimes don’t understand what it is they’re criticizing, especially in the case of quantified risk analysis methods.

Before we get into risk measurement, let’s first make one thing clear: risk analysis is nothing more than a decision-analysis (or decision-support) tool. It helps provide reasonably accurate data points that decision-makers can use when make decisions. It is not a panacea for all things risk or infosec, nor is it some sort of special magic-sauce voodoo with no grounding in reality (at least not in terms of well-considered quant methods). Clear? Crystal, I’m sure.

When performing a risk analysis, we need to start with the basics. Here at Gemini we subscribe to the Factor Analysis of Information Risk (FAIR) methodology for performing quantified risk analysis. FAIR defines risk as “the probable frequency and probable magnitude of future loss.” What this means in real terms is that FAIR reduces “risk” into two main components: Loss Event Frequency and Loss Magnitude. Both are estimates that are created using Douglas Hubbard’s calibration technique, as advocated in his book How to Measure Anything.

Read the rest of this entry »

Risk assessment gets a bad rap these days, thanks in large part to a checkered past colored by qualitative analyses. Historically, risk assessments have been fuzzy, at best, and down-right inaccurate and misleading at worst. You know the ones I’m talking about: some hot shot consultant comes in, pokes around, maybe runs a couple scans, and then churns out a report with a bunch of High, Medium, and Low findings. However, as you dig into the results – particularly the so-called “High Risk” findings – you start finding extreme squishiness with no connection to reality, rational thought, or logic. And this is what we’re supposed to use to “better manage” security? Don’t think so…

Enter Factor Analysis of Information Risk (FAIR), a different sort of beast altogether, created by Jack Jones of Risk Management Insight (RMI). FAIR is a decision support tool that provides a means for performing a quantitative risk analysis around a given scenario. It allows you to conduct deep analysis into given asset+threat scenarios, digging into the business to arrive at accurate estimates (via ranges) for probabilities and expected losses. Loss events are divided between primary and secondary, wherein primary losses are often fairly well known (e.g. how much it costs to replace a server), whereas secondary losses can vary widely.

For an excellent introduction to FAIR, the RMI white paper “An Introduction to Factor Analysis of Information Risk (FAIR)” is highly recommended. In it, you’ll start to see the breakdown of FAIR into its component pieces. Overall, within the context of FAIR we define risk as a derived value measuring “the probable frequency and probable magnitude of future loss.” There is much that can be said about this definition and overall approach, but I’ll leave that for another day. In the meantime, I encourage anybody with an interest in risk analysis to take a deeper look at FAIR.

My article, “Maddening Methods: Fundamentals of Risk Assessment and Analysis,” was published in the July 2010 edition of The ISSA Journal. It covers some of the key concerns around risk assessment today, including addressing common arguments posited against risk assessments and risk management. From the abstract:

Considerable confusion exists in the security industry around the effectiveness of risk assessment and analysis methodologies. Points of contention often focus on specific attributes of a given method, such as data quality, statistical analysis, or a qualitative versus quantitative approach. There are reasonable, viable answers to these points of contention that resolve most of these concerns.

I hope that you’ll find this piece informative and enjoyable.