As we all have learned, getting attention (and budget) from top executives for such efforts as risk and security mitigation is a challenge, see here:
CyTRAP Labs guide to effective IT risk management – being conceptually thorough while keeping it simple
CyTRAP Labs – guide – developing IT security metrics that work for you
Managing risks while getting your CEO’s attention – communication matters
Security metrics – do you know what your boss wants?
But what quality does the information we collect with the help of security metrics provide to C** level managers, in turn, helping them make the decision in the best interest of stakeholders?
With the help of security programs and metrics we are able to obtain volumes of data every day. But as we have addressed previously (see above links), unless such security metrics make sense to C** level executives, they might do little if not damage our efforts to improve the firm’s security posture.
Here are some examples about what might help top management but only if the focus of the security metric is right:
- Incident Response Plan reporting (IRP) – is there a an incident response plan and if so, what is the quality of the IRP (e.g., services provided in case of a critical incident and does it ultimately fix the root of the problem)?
- IRP Exercised – does this mean response procedures and mechanisms were tested using exercises (e.g., similar to a fire drill) and how effective was the team in responding satisfactorily (was satisfactorily defined before the exercise, such as respond in what timeframe, what cost, what quality, etc.)?
- Contingency Plan (CP) developed – does the organization have such a plan and, most importantly, has it been tested (e.g., electricity blackout of 12 hours duration – how well did the CP work out)
But even if we gather the above information appropriately, this may not tell us what top management needs to know in conjunction with these operations and maybe, even an internal CERT.
C** level management wants to hear about a few security metrics only to help it make the strategic and policy-related decisions it must make. This means, we should avoid from making the following errors:
1) there is no clear link of the security metrics with the strategic objectives,
2) numerous numbers of attack-based metrics as well as process-based ones are generated and reported to C** level executives (PS. a maximum of 2-4 metrics for top management to gauge IT security and risk efforts does not mean that IT security engineers and CIO – Chief Information Officer cannot use more metrics to do their jobs),
3) a set of industry benchmarks are being used without making sure that maybe data used to arrive at the benchmark have limited value (e.g., here is an example comparing apples and oranges – Oracle and Microsoft SQL servers using CVEs as security metric for determining which database server is more secure),
4) the impact of business interruption on operations, the cost of business interruption as well as the risk assessment of business interruption are not worked out succinctly demonstrating how these things are interrelated,
5) how metrics help in supporting the organization’s effors for improving quality of products, services and customer relations is vague or non-existent,
6) how metric help improving the business continuity plan and, most importantly, its successful implementation and testing is not spelled out, and finally,
7) the metrics’ importance for improving legal compliance, accountability and the better protection of the firm’s assets is not addressed.
The above sins indicate that the IT security or risk management executive defines his or her business plan and the performance of resources and services around clearly articulated measures. These measures must be aligned with core business strategy and priorities. We have previously reported on how executives evaluate the importance of various security metrics, based on their relevance to business drivers such as:
- managing costs and risks,
- focusing on return on investment,
- complying with the law and company policies, and
- protecting the lives and safety of employees.
Effective use of metrics that matter to top management and the board must, most importantly, demonstrate the value of security operations. This wins one capital and the resources needed to improve the enterprise’s security posture.
If we gather the right information, we generate unique and informative data. Nonetheless, measuring something just because it can be measured easily or is used by others (e.g., certain benchmarks) does not mean the metric is useful for your organization. Accordingly, avoiding the seven deadly sins of security metrics should result in risk and security information type of security metrics that help the organization achieve its objectives (e.g., legal compliance, greater market share). So please, beware and take care.
_More resources_
No Responses to “CyTRAP Labs – guide – the seven deadly sins of security metrics”
Trackbacks/Pingbacks
[...] – CyTRAP Labs – guide – the seven deadly sins of security metrics [...]