Audit Drips

I was catching up on the podcast backlog today. I listened for the first time to the Risk Hose, which had a meaty midsection on the internal auditing profession, and whether and how internal auditors assess, analyze and otherwise manage and misconstrue risk.
(A couple caveats. I speak as an internal auditor, with a background in food service and deckhanding. I'm ISACA Platinum, which is more like Centruum Silver than American Express Gold, i.e., it is bestowed upon age. I'm an autodidact when it comes to information risk analysis, but I'm trying to learn.)

Firstly, the standards. The Red Book, or more correctly, the International Professional Practices Framework, includes the following standard (2010 A1)

The internal audit activity’s plan of engagements must be based on a documented risk assessment, undertaken at least annually. The input of senior management and the board must be considered in this process.

So, every internal audit shop has to perform a risk assessment annually, and use it to plan which audits will be performed in the next year.
This type of risk assessment evaluates "audit risk," defined in Sawyer's Internal Auditing (from my raggedy 4th edition, Part 3 Scientific Methods* Chapter 8 "Risk Assessment") as the following:

Audit Risk = Inherent Risk x Control Risk x Detection Risk

A heavy dose of "professional judgment" (also known as "the gut") is used in this method.   The output of this assessment prioritizes the auditable units (chunks of business functions which make up the audit universe), and crank them through the cycle to maintain "coverage."  Purchasing on even years, Accounts Payable on odd, et cetera.  Area with weak controls and lots of potential loss should probably float to the top.  This method is old fashioned even for the conservative internal audit profession, but has the backing of some of the AICPA's more ancient Statements of Auditing Standards.   The resulting assessment is used  internally for audit's planning purposes, and, from talking to my peers in industries without a regulatory mandate to perform risk assessment, it may be the only organization-wide assessment that gets performed.   The methods vary, as do the results.

The recent revisions to the Red Book standards state that internal auditors  "must evaluate the effectiveness and contribute to the improvement of risk management processes."  So a shop that follows standards will be in the business of whoever is performing the "risk management" function, including "information systems."   Internal auditors can't manage risk, but can help assess.

 From my perspective, a lot of internal auditors have a lot of experience in an old fashioned style of risk assessment, and end up with a gut quantification exercise.  There may be some bet hedging, vindictiveness and four tons of politics involved in the process (see above as to who must have input into it), and, in the end, the board will get what it wants.  Quality and sophistication of boards will vary widely, and if they want red, yellow, and green heat maps, by gum they are going to get it.  If they want quant analysis, they'll get that too, especially if there is overlap between the Audit Committee and the Risk Committee.

Personally, it is approaching risk assessment season for my shop, and, with Hubbard and FAIR in hand,  I'm working with our CAE to get together at least some quantitative analysis.  Gotta start somewhere.  I'll get the blame regardless.









*I think I hear a head exploding somewhere.

Risk a Harm?

Interesting post and comments on privacy risk from Solove at Concurring Opinions.  Despite being raised by a pack of feral solicitors, I can't claim to understand all the legal theories involved.  I'm attracted to the liquidated damages idea for a number of reasons, including the ability to build a reserve or get underwriting to mitigate potential incidents.  

Harms at Risk

On the other hand, this is where the disclosure rules suck.   For example, an organization loses track of a hunk of physical media that contains a couple hundred thousand records that contain personally identifiable information (but not financial information - no bank or credit card account number).   In this example, there is a very high probability that the media was subsequently destroyed.  Are the individuals identified on the media well served by being notified?  

Imagine there was a method to calculate the likelihood of financial damage to the individual due to the loss of the media.  Lets imagine that there is less than 1% chance that the information will be used in a crime in the next 2 years, and it decreases by half every year that follows.  However, if it is used in a crime, it is likely that the crime will be of a significant impact - a genuine fraud involving a false credentials that would take more than $100,000 for the victim to unravel.   Is notifying the victim of the risk, and making him feel uneasy (since humans perceive risk differently than equations) responsible?  

Or is this just an excuse for me to illustrate a post with a picture of Harms at risk?  

Non Fiction: Risk

From Alex Roy's The Driver:

"Our second hour of 150 mph or more inspired a highly unscientific analysis of the actual danger we faced. I concocted what I called The Danger Coefficient (DC). I guessed the average NASCAR driver, in a thirty-six race season including practice, probably drove 15,000 miles -- with a safety cage and onboard active fire suppression -- on highly prepared tracks, with hospitals less than 14 minutes away by choppers on standby. Assuming this represented a DC of ten, Gumball's 3,000 miles meant our DC was two.... until factoring our relative safety deficiencies. High speeds over potholes had to triple our DC to six. Civilian traffic doubled it again, to twelve. Time and distance to medical help? Double again, to twenty-four. Lack of roll cages, harnesses and HANS devices? My guesses ended when I realized Gumball -- at least the way I did it -- was at least five times more dangerous than NASCAR."

From Wright and Decker's Burglars on the Job:

They referred to this process as "burning bread on yourself."

"Thieves got a thang they say [about getting caught,] "If you think about thangs like that, you burnin' bread on yourself" So you don't think about it... Just go for it. [No. 011]

Several of the subjects found it difficult to speak about the risk of apprehension, fearing that such talk would jinx their future illegal activities.
...
Some of the offenders also tried not to think about getting caught because such thought generated an uncomfortably high level of mental anguish. They believed that the best way to prevent this from happening was to forget about the risk and leave matters to fate.

From Rothman, an article at CSOnline discusses Moody's infosec risk rating service.

I personally dig this quote:

The idea for such an at-a-glance rating is appealing to risk executives such as Andre Gold, head of security and risk management for ING’s U.S. Financial Services business... Last year Gold oversaw reviews of 176 new technology vendors; his team visited sites as far away as South Africa to conduct security assessments. “It’s a service that we must do, but I think it’s a non-value-add service,” he says.

A non-value-add service? To quote Michael Scott, that's what she said.


photo from Dwight K. Schrute.

White Knuckles

This looks interesting, in the context of cultural cognition of risk. Entertaining legal wonking on the issue at Concurring Opinions and Volokh.

Amazing the lack of agreement as to when "Yee haw!" becomes "Holy Crap!" while behind the wheel.





Photo courtesy Marie Rose Ferron / Flickr

Die Doing Something You Love

"To die doing something you love."
I encountered variations of this phrase three times Saturday.

1. In Chris Jonnum's biography of the Haydens, the on track death of flat-tracker Will Davis. Davis was a hero of Nick Hayden's. Mourning his death, Nick said that there is no tragedy if you die doing something you love. Nick did run his next road racing victory lap backwards in Davis' honor.

2. On the DVD of The Race to Dakar, Andy Caldicott died doing the thing he loved, as described by Charlie Boorman. No one will be permitted to die this way this year, since ASO has cancelled the Dakar race due to threats for terrorism. (You can die doing what you love, not what Al Qaeda loves.)

3. Andy Olmstead states in his posthumous blog post that he died doing the job he loved.

If you love your job, you can accept any level of risk.

Confusion In My Eyes That Says It All

I figured I'd wait until after my paternity leave was over before I started thinking seriously about words like "control" and "compliance," but I felt the need to say something after reading Bejtlich's post "Controls are Not the Solution to Our Problem."

He illustrates through citing an example of a control, and identifying ways that it fails to achieve total effectiveness. The control may not work and could be superfluous. His alternate approach is a system of assessments, tests and monitoring coupled with a rigorous set of metrics.

If someone describes an asset as "secure," "safe" or "reliable," my job as an auditor is to ask the question "How do you know?" The answer is a control. Bejtlich's "field-assessed" approach is another set of controls, mostly detective rather than preventative. What happens when his approach is codified into a government procedure or a vendor contract? A security practitioner with a preventative approach could grouse about how these pen tests and honeynets don't address the security needs in his shop (due to scale of operations or type and level of risk).

Tossing out controls is also just not an option. Effective or not, compliance keeps you out of jail. I don't always feel that on some roads a 55 mph limit is a necessary control to prevent accidents, but that will mean I am not breaking the law when I speed.

I'm not as big a proponent of metrics as a control solution, but I'll leave that to another post.

Compliance for Road and Track

My Alfa, a 72 GTV coupe, like all GTVs of its approximate vintage, has a recessed panel in the headliner over the back seats. It has proven to be a mystery to passengers in the car, looking like the cruelest joke of a sun roof for the rear passengers who are otherwise treated poorly by the car's design. So cruel, in fact, that a sticker was placed on the rear windows by Alfa. When viewed from the outside, it read:

ALFA ROMEO "2000" GT VELOCE
GROUP 2 TOURING CHAMPION 1971
EUROPEAN MANUFACTURES SERIES

This side of the sticker explains in part the pseudo-sun roof. The GTV raced in the sedan class. To comply with sedan class regulation, there had to be a specific number of inches of headroom for the two passengers in the back seat. Hence the "cheat" of recessing a spot in the headliner, because the seats were as low as they could go. So you can race, and sometimes beat, Minis, BMW 2002s and Datsun 510s.

From the inside, the other story of compliance was visible. From a knees-to-chin head-ducked position, the contorted rear seat passenger could read the obverse:

REAR SEATS ARE NOT DESIGNATED
TO BE OCCUPIED BY PASSENGERS
WHILE VEHICLE IS IN MOTION

The other set of regulations the GTV had to comply with were written by the US Department of Transportation, that defined of sports cars and sedans. Being classified by the DOT as a two-seater would require less modification of Alfa's aging (yet still stylish) design - less in the way of bumper protection for the would be passengers. Actually taking the seats out and putting in a package shelf (a la 911) would make it race in an uncompetitive class. Hence the sticker forbidding rear seat passenger, which attempts to serve both masters. (I'm guessing the seat belts back there are for securing cases of Chianti and bundles of pastrami.)

The different approaches in compliance reflect the different levels of enforcement. Perhaps Alfa felt it could convince the DOT that, really, who would ever be so silly as to sit back there? This is a sporting coupe, not a sedan. However, Alfa knew that the sanctioning bodies for the racing series they participated in would be out there with tape measures and calipers before every single race for tech inspection. Alfa's compliance would be challenged by every other team on the track.

I don't believe it would be too far off the mark to say that an implementation of a control, especially a compliance control that may not have a palpable financial return, will be as effective as the perceived enforcement.

(Read the story of the 2.5 liter Trans-Am at 1971 Laguna Seca for more sad stories of compliance. The Datsun version, the Alfa version. "Oversize fuel lines" vs "expanding gas tanks." )

(sticker image courtesy Papajam at the AlfaBB)

Market Fresh

A curious discussion of terror risk, and a terror prediction futures market by some GMU economist types and at the Chronicle's Footnoted blog.

I don't know enough to about econ to assess the value of such a market, but I do wish that some one would set up a Privacy Breach Futures Market so we could make the security analystas put their magic quadrants where their mouths are. (Or vice versa: whichever would be more unpleasant.) Viz, the TJX OMG!!1! MILLIONS IN PWNAGE!! NO!!BILLIONS! analysis found on Computerworld. Maybe something more along the lines of buying squares in a football pool would offer as much predictive value as the collective voices of these cats.


Photo courtesy The Prodigal Son.


And yes, this is the second consecutive post with a Broken Social Scene related title. Because Broken Social Scene are one of my top five most favorite things that are Canadian.

Motoprox

Yesterday I was barreling down the concrete slab choked with tractor-trailers and nitro-burnining funny trucks laden with oily 2x4s and spent joint compound jugs, I was engaging my left brain in random problem solving ("Resolved: The world is as random as it is not.") and engaging my right wrist in focussed throttle control on my Triumph Bonneville. I hate the road - a stretch of oversubscribed interstate that at an unfamiliar time (around 3:00 pm) and was unfamiliar with how the traffic would be flowing. The part of the brain that controls motorcycle function became increasingly engaged.

Fortunately, it didn't come out of nowhere: some set of clues were processed so I was pretty sure the black sedan was going to dart into the part of highway I was occupying. I braked as much as I could, as the pickup behind was riding my exhaust, and I moved as far to the left of the lane as I could. Just as his door was nearing my knee, the driver of the sedan spotted me, and made a panic swerve back to his lane. No harm, no foul, just a cortex soaked in adrenaline. People pay good money for that.

Which led me to my thought. Do near misses count?

UK Civilian Aviation Authority Airprox Board thinks so. They are dealing with potential accidents, however, with an not unreasonable assumption that neither party wishes a collision. There is no attacker, so it is easier to get both sides of the story, and a clearer, truer account of the incident, and quality information to improve the process. In a security incident, you will rarely get the other side of the story, so the account is skewed to what the defender has observed, and the attacker has failed to hide.

The Risk Management and Decision Process Center at the Wharton School has this brief description of its Near Miss Management study.

It may be nothing useful, but I'm wondering how "near miss" security incidents are handled. How are the elements of "luck" and "skill" (i.e, controls, response,etc.) allocated? Since the bullet was dodged, is there a increase in comfort in the level of security, even though it may have just been luck, or the actions of the attacker, that made it a "miss"?

I don't know, but I've been hyperaware of traffic lately, and my head is encased in Shoei and my body in Tourmaster. (And for more on motorisks, see Chandler's post from last September.)


Hot Honda on Duck action courtesy PhillC.

Waffle are Just Pancakes with Little Squares On 'Em

I've been working on something, but I don't know if it will make by race time in Shanghai.

In the meantime, the most important part of internal auditing is "production value." And we know what that means.

The Red, Yellow and Green Legos of Judgment

I'm out here in Coyote and Roadrunner land, knee deep in internal auditing. I co-presented yesterday on privacy, as a co-author of an Institute of Internal Auditing publication.

It's been a interesting couple of days, driven in part by the isolation of the location. As attractive as a golf/casino resort may sound, it's not so groovy if you don't golf, don't gamble and didn't have the foresight to rent a car. I can meditate on the cacti, and read. I packed a couple of books to get me in and out of the Internal Auditing mindset: The Digital Person by Daniel Solove (highly recommended), a Kierkegaard anthology (because what is auditing but fear, trembling, and sickness unto death?) and Nassim Nicholas Taleb's The Black Swan (I've been alternately writing "YES!" and "BULLSH*T!" in the margins. (It's my policy to keep the margins safe for work.))

But this morning I had my own inverse Damascus moment, as Bill Power (if that is his real name) of the PCAOB was giving the assembled throng his information technology application auditing method, as demonstrated through a manufacturing case study. It was interesting enough as analysis of manufacturing financial systems go (yes, exactly that interesting), but at the end of his case study it seemed to me that he just plopped Red, Yellow and Green Legos into the risk spaces in his spreadsheet, and chalked it up to judgment. In fact, one of the slides read something like "RISK ASSESSMENT IS ALL JUDGEMENT" (I'd quote directly, but his presentation is not on the conference CD-ROM. I do remember he spelled "Judgment" with two "E"s.)

O.k. Sure. Risk assessment without judgment is pretty worthless. And auditors have an obligation to use their judgment to assess risk. Nonetheless, it doesn't seem worthwhile to go through all this spreadsheetin' and flowchartin' just to get to the point where you pull red, yellow and green Legos out of your velvety Audit Sack of Judgment and snick-snack them on financial information systems and processes master control grid. How about the stuff you don't understand well enough to apply judgment? I'm getting the idea that it's called "Out of Scope."

At what point does "judgment" intersect with "caprice"?

Go ahead, call me naive (if you haven't already). But it's getting dark, and I'm going to see if the cows come back to the hotel parking lot again tonight. This time I'll be ready.



Photo courtesy of The Bill.

Invincible

New York Magazine article "The Young Invincibles: A Generation Uninsured" discusses the way uninsured 20-30 year olds in New York deal with health risks (link and commentary from Concurring Opinions.)

The article is interesting study of people who do not participate in the most common health risk management strategy: insurance. Unable to afford it, or "rationally" choosing to be uninsured, they have created their own strategies to minimize exposure. Curtailing snowboarding activities (only the half pipe), daily brushing, and yoga are balanced with careers as bike messengers and retailers. There is a wide range of risk appetites: the bike messenger who feels that "helmets are cumbersome," and artist who eschews bicycling completely. Maintenance and prevention are expensive or inconvenient, so the Invincible's focus is on the severe or catastrophic cases.

Are there corporations out there that believe themselves to be "invincible"? Is this the sort of attitude that prevents real security from becoming embedded into a corporate culture? No doubt possible. Also likely is the false sense of security associated with "compliance" as a risk mitigation technique. SOX is like a bicyclist's helmet ("too cumbersome"). PCI is like brushing your teeth every day. No one condemns daily brushing, but it won't help when you get a kick in the teeth.


(I recall my own period of "invincibility." Working without insurance as a deckhand on a towboat on the Ohio, Tennessee and Cumberland Rivers, I didn't see the dangers of hopping from barge to coal soot covered barge, lugging 90-lb ratchets and wire, all risk mitigated by my Redwings and a bump hat. Not until a near death experience while epoxying the inside of a fresh water tank did I think "Hey, what if I get crushed? What if my brain is actually damaged, and no one will ever get my jokes?" Then I sought less perilous employment. With a health and dental plan. So I found my way to the Guild of the Green Eyeshade.)

Men's 8-inch work boot with metatarsal guard courtesy Redwing.

Auditing Privacy Part 2 - Risk Assessment of Data Loss

The easy way to assess privacy risks is to focus on the impact of data theft to the organization by including the private data as a corporate asset. There are well documented methods to identify the vulnerabilities in means of collecting, storing and sharing the data. Similarly, there are methods to identify and list the data's threats (hackers, "insiders," and negligent loss). The impacts will likely shake out along the lines of direct costs (postage, call center, other incident response costs), potential legal and regulatory actions and reputation damage. (For an example, Protegrity assessed the TJX data breach at $1.7 billion, though TJX was not strictly a privacy issues, it has parallels*).

This would be the easy way, but may not result in the most accurate results. The problem lies in identifying the impacts of a privacy breach. The attribute of “privacy” assigned to the data is what makes the data valuable, and worthy of protection. However, "privacy" is not an attributed that belongs to the corporation, but to the individual the data describes. So an assessment of risk to the corporation of privacy loss should start at looking at the impact of the loss to the individual.

Why do many corporations, when disclosing losses of tremendous amounts of data, appear to suffer only short term damage to their reputation. I posit that the potential damage to a corporation is proportional to the actual real damage to privacy of the individuals described in the lost damage. (See Guin v Brazos)

The real impact of a privacy incident on individuals has been hidden behind a cloud of security vendor fear mongering and media induced panic. The common problems with the data is equating data loss with a privacy breach. Identity theft properly defined is likely a higher impact, lower frequency event than is commonly reported.

The SB1386-style disclosure laws have been a boon to identifying the frequency of data loss, but the information that has to be disclosed does little to help identify the impact. An auditor concerned strictly with compliance would have to place equal risk to any loss of private data. But the auditor should take the risk assessment to the next step and focus on the individuals, identifying the risks that lead to actual harm to the privacy of individuals. Compliance risk is equivalent for the loss of a laptop carrying an encrypted database of private data and the same databases being heisted off a web server unencrypted by a criminal with the intent to exploit the identities. The real risk to the privacy of the individuals described in the database is clearly different.

Beyond the risk of a data loss, the auditor should also consider the equally important risks of the collection of private data and the dossier-ification of data. More on that later.

*Why the high risk to TJX? Though not strictly a privacy issue, the damages related are an issue of a loss to a third party - the banks - rather than TJX itself.

"Some would call this good fortune" from s2art

Too important to be left to the generals

Interesting discussion on the secret language of security.

Which shovetails nicely into a panel discussion I saw yesterday. An assortment of CSOs and a Forrester analyst discussed the future of security. Essentially all the tech stuff is being outsourced, and the head of security is being molded into a Risk Officer. I can infer from this that the tech stuff (firewalls, antivirus, and the three letter acronyms) can scale. But the risk cannot. Risk is corporation's own, to be honed, polished and cherished like a treasured logo that no can quite figure out what it means. Risk is the new black, a point made elsewhere, and with more vigor.

One of the CSOs also mentioned that privacy will be shoved aside as a compliance thing, over with the lawyers. I stifled my desire to spring up and shout "HERESY!" for fear that it would awake my CEU seeking comrades from their deep and well deserved slumber.

I don't give a damn about my bad reputation

No. No. Not me.

I was meditating on reputation risk the other day, and behold, the Daily Dave belches forth the documents I sought. (I remembered something on Emergent Chaos on this topic, but hadn't dug deep enough into their archives.)
The study I remembered and cited by Adam Shostack was "Is There a Cost to Privacy Breachs? An Event Study."


The salient quote:

"[Privacy breach] impact is statistically significant and negative, although it is
short-lived."

Which is supported by anecdote (check out the TJX stock price).

So how do you convince your management to follow privacy principles? Appeal to the better angels of their nature? Start eavesdropping and pretexting them and see how they like it? (HP probably did as much good as the CDT, EFF or ACLU as far as advancing the privacy agenda in Congress).

I'm guessing the shift, as a result of the "privacy fatigue" and the "identity theft fatigue" should be to the high risk transactions, that expose the data's subject to verifiable risks, not just the lost computer tape or missing laptop. But I need data to support that, dagnabit. Else:

An' everyone can say what they wanna say, it never gets better anyway.

Grackles in a Pancake Mine!

This morning, city officials decided to shut down a significant portion
of Austin's central business district due to the discovery of a covey of
dead birds.

Meanwhile, Gotham panics when confronted with a strange pancake smell.

I'm not going to second guess the response to the pile of avian rats on
Congress Ave. Nor will I try to determine which eldritch spell summoned
from the Permian Basin was used to extinguish these fowl lives.

I will however, try to figure out under what circumstances a bunch of
dead birds would require the closing of a central business district.
What sort of risk assessment process went on here?

1. PANDEMIC! O.k., the birds may have had a virulent version of avian
cedar fever. Some of the carcasses have been sent to our Aggie brethren
to be tested for bird flu. We'll get the results in a week or so. Then
we will close Congress Ave. again? If the folks in the hazmat outfits
scooped up the carcasses, pureed them, placed them in 3 oz bottles and placed them in one quart zip top bags, what is the risk?

2. NERVE GAS ATTACK! Then these truly were the grackles in the
coalmine, who gave their lives for us. Only the bad guys released the
gas at 3:00 am on a Monday morning. He should at least wait until the
Lege is in session, so as to terminate some Bees as well as birds.

3. A DISTURBING MESSAGE IS BEING SENT! - Homeland Security necromancers
find an ancient passage in the code of federal regulations that speaks
ominously of the scents of phantom flapjacks aligning with the mass
suicide of capital city trash birds. Maple Alert!

4. JUST ANOTHER GRACKLE MUNDY - My just-don't-have-to-work day.

5. YUPPIE TERROR - Rich fella or fellette from out-of-state, encountering the foul stench of grackle fecal splatter, sets out a Williams-Sonoma bowl of hand-tooled Vermont pigeon poison. Problem solved. (A real Austinite, or any grad of University of Texas would use a shotgun, just like the pros.)

So what did we learn? I'll have to think on that some more.

photo courtesy of Ikayama