Now That's a Complaint.....

From Concurring Opinions (and elsewhere), a paper by Chris Hoofnagle "Measuring Identity Theft at Top Banks." Hoofnagle is asking the question: How does a consumer or regulator measure the incidence of identity theft from a financial institution? In an attempt to answer, Hoofnagle took the number of identity theft complaints collected by the FTC and matched them up with institutions listed on the complaint, with the intent of coming up with a score that could be used by consumers to judge how well the institution protects identity.

Call me crazy if I'm wrong, but Mr. Hofnagle seems to be pushing the data way beyond its utility.
Is a complaint to the FTC via a web form a reliable indicator of fraud controls at an institution? In my past experience as an investigator, I handled many cases of identity theft. I'd estimate that at least half, if not two thirds of the allegations of "identity theft" were not, in fact, identity theft. A suspicious charge on a bill, a bad skiptrace, or even a breach disclosure notice could result in complaint of "identity theft." Crime statistics that involve prosecutions of actual criminals may provide an underreported, but more reliable measure.

Hoofnagle mentions that he believes the number of FTC complaints may be low, due to historic underreporting of identity theft to criminal authorities. Again, according to my experience, which may be non-representative, I'd say that people will fill out a web form that belongs to the FTC sooner than they'd call the police. The FTC is more analogous to the Better Business Bureau than law enforcement.

I was going to write something about my frustration with the publicity that the FTC complaint statistics were receiving. Complaints are easy to count and a handy metric. But I don't think that they mean much without some evaluation of the validity of the complaint. That is, what is interesting is hard to find out.

Right before I read Hoofnagle's paper, I read this post from the Microsoft Security Development Lifecycle blog. The author makes the following statement regarding using vulnerability counts as a measure of software security:

"Measuring security is a real challenge, and while we may debate the
merits of vulnerability counts, right now it's the only concrete metric
we have."

I guess I'm saying that the only concrete metric one may have may be misleading, inaccurate, or irrelevant. Concrete isn't synonymous with valid. I may have issues with "metrics" but I love Metric. Need less, use less, we're asking for too much I guess, cause all we get is...

Stupid, powerless, uneducated.

Infoworld on a session at RSA: The Cybercrime Blame Game.
Although a conference center ballroom may not be conducive to rational discourse (see: US Political Party Conventions), this discussion appears a bit over the top:

• More people complaining about identity theft does not necessarily mean there is more identity theft. I'm sure there was a dramatic increase in complaints about anthrax without a corresponding increase in anthrax attacks. (See the corresponding stat later in the article citing an 11.5% decrease in dollar losses due to identity theft.)
  

• FTC Gorman is right: Calling people stupid doesn't solve anything. I've never been a fan of Winkler's ideas nor his rhetorical method.
  

• The job of an ISP is to move packets, not to sit in loco parentis for everyone with a broadband connection. (Why was this applauded? Were all the NANOG guys still in Toronto?)

• What makes an empowered consumer is not education, but power. Give the consumer the right and responsibility to take care of their own data. Not the credit bureau, federal law enforcement, the ISPs or Wal-Mart. The consumer. Build an infrastructure around that idea. The consumer isn't stupid, he just doesn't care and when he does care, he has no standing. Maybe the empowered consumer idea is just too European.