Just another ComMetrics – social media monitoring, best metrics, marketing metrics weblog

Early Warning System – taking 8 steps toward developing key performance indicators that make sense

April 4th, 2007 · No Comments ·

These days much interest is focusing on benchmarks. As a result, early warning systems are trying to demonstrate that their services are needed. In turn, this helps them demonstrate to funding agencies that the latter are getting their money’s worth.

In fact, most early warning systems look great when one reads their publications and on-line profile. However, when one starts benchmarking these early warning systems against some Key Peformance Indicators or KPIs, things might look different as adddressed here:

4 Tips for building an effective Early Warning System – organizational and human resource issues

The eight myths about Early Warning Systems

A whole other issue is how to develop security metrics that help clients making sense out of things. For instance, what should be measured that managers can use in their efforts to use the internal control system and meet risk management requirements (see Sarbanes-Oxley, COSO).

The article below tackles this issue and tries to come up with key performance indicators (KPIs) that help demonstrate that the early warning system adds value for clients (e.g., home users and SMEs) as well as public-policy decision-makers.

Hence, for something like the European Information Sharing and Alerting System (EISAS) this means that one has to define what outputs and services such as system has to deliver. For instance, some people fail to understand that EISAS may have various types of clients (e.g., government departments vs micro type of organizations or self-employed telecommuters). These clients groups do, naturally, have different needs regarding information about threats and vulnerabilities against their data and information.

As a result, EISAS has to deliver a range of services in order to satisfy vastly different expectations and needs of these stakeholders or interest groups. Until we define what EISAS has to supply its clients with (e.g., type of information, content, frequency, technical level, etc.) unfortunately, it will be an impossible challenge to figure out how to assess its performance down the line (e.g., has it delivered the goods it was supposed to and if so, was the quality such that it helped improve security for the clients). Sounds easy but do not be fooled, it is a challenge.
The paper below addresses the above issues in some more detail. It also presents an eight point checklist that one can follow to develop KPIs for an early warning system. These KPIs are grounded in security metrics that provide decision-makers with quality information that, most importantly, facilitate efforts for achieving better security and protection for SMEs and citizens.


The 3-page short version can be found here:

ENISA Quarterly – Jan-March 2007 pp. 6-8


→ No CommentsTags: 2007 · enisaquarterly · longversion