EUIST

Since 2001 we have been addressing security metric issues:

- Best Practice – Benchmarks – Metrics – Ten Worst Security Practices

- LIB- NIST – Pub 800-55 – Using Metrics to Measure Security Controls, Processes and Procedures

- Week 33 – Lib 1 – NIST Guidelines – Security Metrics that Work?

Recently we have picked it up again in a more systematic way as outlined below:

- CyTRAP Labs – guide – developing IT security metrics that work for you

- 4 Tips for building an effective Early Warning System – organizational and human resource issues

Performance metrics are also getting ever more popular with defence agencies around the globe. Apparently a powerpoint graphic was leaked from US Central Command, The graphic (see below – Tracking U.S. Army’s trajectory in Iraq) stracks tracks changes and events on a peace….chaos meter. (where emergent chaos fits in is not made clear). Changes are categorized

- routine,
- irregular,
- significant, or
- critical.

” In fashioning the index, the military is weighing factors like the ineffectual Iraqi police and the dwindling influence of moderate religious and political figures, rather than more traditional military measures such as the enemy’s fighting strength and the control of territory….”

A number of secondary indicators are also taken into account, including

- activity by militias,
- problems with ineffective police,
- the ability of Iraqi officials to govern effectively,
- the number of civilians who have been forced to move by sectarian violence,
- the willingness of Iraqi security forces to follow orders, and
- the degree to which the Iraqi Kurds are pressing for independence from the central government.

Unfortunately, the graphic leaves much open to question regarding the direct measures and indirect measures used by the military to arrive at their composite type of index (e.g., the reliability and validity of various measures used and how they were aggregated).

Aggregation and Disaggregation is a tricky business indeed and unless it is carefully explained and makes sense, the metric may have little meaning at all.

_PS 1. The shifting index was seen by some officials as a stark warning about the difficult course of events in Iraq, and mirrored growing concern by some military officers.
_PS 2. A spokesman for the Central Command declined to comment on the index or other information in the slide when asked, saying: “We don’t comment on secret material.