Related

-Security metrics and Small- and Medium-Sized Enterprises (SMEs): Quo Vadis

Previousl we discussed how security-related risks must be categorized to get a better handle on them. We proposed a graphic framework that could be used here:

- Early Warning System (EWS) – Categorizing the risks

Here we expand upon this framework and develop a graphic overview in somewhat more detail.  Nonetheless, the starting point is the general schemata outlined above so you should look at it.

In the above story we pointed out that SMEs play a vital role in most if not all economies. Unfortunately, much of the the risk and IT security literature seems to ignore this fact. In other words, support for this type of firm is limited if non-existant for all practical purposes.

Unfortunately, employing less than 10 staff could mean that neither does security metrics get the attention it should nor are the human resources in place (i.e. security engineer) to take care of these matters.

Based on the above it seems that it is ever more important to provide checklist and tools about security metrics that can be applied easily and quickly by SMEs.

WHERE SHOULD WE START?

To start with, the framework outlined here should be used to get order in a not necessarily nicely structured problem:

- Early Warning System (EWS) – Categorizing the risks

As well, at this state it is important to reiterate that metrics are a a system of measurement. In this case, metrics are a way for measuring security, specifically measuring an organization’s security posture.

While there may be some guidelines or even standards that outline how security efforts impact upon security posture can be measured, ideally security metrics should be adjusted and tuned to fit a specific organization or situation.

Naturally, a micro enterprise with 7 employees and 1 Mio Euro turnover will require different security metrics compared to an SME with about 200 employees and possibly 40 Mio Euro turnover. Below we tried to provide you with a schemata to arrive at security risk indicators and security metrics for your enterprise.

If you cannot see the above graph, click here – CyTRAP Labs framework for security metrics that works

We started with risk examples that might be applicable in most enterprises including SMEs. In fact, the five we list are those that are considered the Minimum Essential (“Fundamental Five”) Practices published by the Corporate Information Security Working Group Report of the Best Practices and Metrics Teams (Revised January 10, 2005) Subcommittee on Technology, Information Policy, Intergovernmental Relations and the Census Government Reform Committee United States House of Representatives.

From there we identified the risk causes. for instance, identity management mechanisms can be a risk and this could be caused by improper authentication, authorization and access control procedures. A risk indicator that should be watched and a metric that allows measuring the indicator is the number of user accounts that may be still active on the fifth of the coming month even though those individual left at the end of the previous month.

The above provides a simple framework and the key risk indicators used to the right of the schemata need to be adjusted according to the organizational characteristics and its environment (country, business field, etc.).

But one has to start somewhere to get a handle on these risks. Unfortunately, without writing these down and putting them on paper it is sometimes difficult to convince other stakeholders about t how critical it is for the firm’s success to get control over the information security issues identified. But we should remember that if we use more than 7 security metrics, it will become difficult to focus on the matters that are critical.

- Security metrics – what affects business continuance – focus on impact?

Instead, too many things that might be important but not critical detract us from focusing on growing the business.

See also:

- CyTRAP Labs – guide – the seven deadly sins of security metrics

- Security metrics – what affects business continuance?

SUBSCRIPTION

To make it more convenient for you to take advantage of CyTRAP Labs’ offerings, just provide us with your e-mail address below. You can personalize your subscription to make it suit your needs.

Recently we have addressed various issues regarding securitymetrics, such as:

- CyTRAP Labs – guide – developing IT security metrics that work for you

- Managing risks while getting your CEO’s attention – communication matters

- Security metrics – do you know what your boss wants?

- CyTRAP Labs – guide- the seven deadly sins of security metrics

Security metrics is important but sometimes we seem to be doing it less than perfect and, as importantly, not only may we be too lazy to check the statistics (is it kosher?) but maybe even worse is the case where people outright misquote studies. Here is an example of how things can go wrong.

February 2005, Mike Nash, corporate vice president of the Security Business & Technology Unit at Microsoft quoted the National Computer Security Alliance (NCSA) as follows:

“The National Cyber Security Alliance estimates that two-thirds of the home computers in the United States do not have any activated firewall, and the same percentage is operating without current anti-virus software

As we all have learned, getting attention (and budget) from top executives for such efforts as risk and security mitigation is a challenge, see here:

CyTRAP Labs guide to effective IT risk management – being conceptually thorough while keeping it simple

CyTRAP Labs – guide – developing IT security metrics that work for you

Managing risks while getting your CEO’s attention – communication matters

Security metrics – do you know what your boss wants?

But what quality does the information we collect with the help of security metrics provide to C** level managers, in turn, helping them make the decision in the best interest of stakeholders?

With the help of security programs and metrics we are able to obtain volumes of data every day. But as we have addressed previously (see above links), unless such security metrics make sense to C** level executives, they might do little if not damage our efforts to improve the firm’s security posture.

Here are some examples about what might help top management but only if the focus of the security metric is right:

- Incident Response Plan reporting (IRP) – is there a an incident response plan and if so, what is the quality of the IRP (e.g., services provided in case of a critical incident and does it ultimately fix the root of the problem)?

- IRP Exercised – does this mean response procedures and mechanisms were tested using exercises (e.g., similar to a fire drill) and how effective was the team in responding satisfactorily (was satisfactorily defined before the exercise, such as respond in what timeframe, what cost, what quality, etc.)?

- Contingency Plan (CP) developed – does the organization have such a plan and, most importantly, has it been tested (e.g., electricity blackout of 12 hours duration – how well did the CP work out)

But even if we gather the above information appropriately, this may not tell us what top management needs to know in conjunction with these operations and maybe, even an internal CERT.

C** level management wants to hear about a few security metrics only to help it make the strategic and policy-related decisions it must make. This means, we should avoid from making the following errors:

1) there is no clear link of the security metrics with the strategic objectives,

2) numerous numbers of attack-based metrics as well as process-based ones are generated and reported to C** level executives (PS. a maximum of 2-4 metrics for top management to gauge IT security and risk efforts does not mean that IT security engineers and CIO – Chief Information Officer cannot use more metrics to do their jobs),

3) a set of industry benchmarks are being used without making sure that maybe data used to arrive at the benchmark have limited value (e.g., here is an example comparing apples and oranges – Oracle and Microsoft SQL servers using CVEs as security metric for determining which database server is more secure),

4) the impact of business interruption on operations, the cost of business interruption as well as the risk assessment of business interruption are not worked out succinctly demonstrating how these things are interrelated,

5) how metrics help in supporting the organization’s effors for improving quality of products, services and customer relations is vague or non-existent,

6) how metric help improving the business continuity plan and, most importantly, its successful implementation and testing is not spelled out, and finally,

7) the metrics’ importance for improving legal compliance, accountability and the better protection of the firm’s assets is not addressed.

The above sins indicate that the IT security or risk management executive defines his or her business plan and the performance of resources and services around clearly articulated measures. These measures must be aligned with core business strategy and priorities. We have previously reported on how executives evaluate the importance of various security metrics, based on their relevance to business drivers such as:

- managing costs and risks,

- focusing on return on investment,

- complying with the law and company policies, and

- protecting the lives and safety of employees.

Effective use of metrics that matter to top management and the board must, most importantly, demonstrate the value of security operations. This wins one capital and the resources needed to improve the enterprise’s security posture.

If we gather the right information, we generate unique and informative data. Nonetheless, measuring something just because it can be measured easily or is used by others (e.g., certain benchmarks) does not mean the metric is useful for your organization. Accordingly, avoiding the seven deadly sins of security metrics should result in risk and security information type of security metrics that help the organization achieve its objectives (e.g., legal compliance, greater market share). So please, beware and take care.

_More resources_

Standards and web application security

Research that matters: using the wrong security metrics for answering the question: which database server is more secure?�

We have previously addressed how difficult it is to develop metrics that are not only reliable but also valid and have a strategic focus – meaning C-level executives do care about getting such numbers that relate to matters the care about (e.g., new markets, strategy, bottom line):

- CyTRAP Labs – guide – developing IT security metrics that work for you

- Security metrics and audits – spreadsheets are full of errors as we know

- CyTRAP Labs – 10 reasons for why information security makes economic sense

As we all have learned, getting attention (and budget) from top executives such as risk
managers, CFOs, and CEOs, means creating metrics that help measure the
value of the security effort. The Conference Board (sponsored by the U.S. Dept. of Homeland Security) surveyed 213 senior corporate executives working for a broad range of U.S. enterprises. Results show:

- 64% felt that the cost of business interruption was the most helpful metric,

- 60% thought vulnerability assessments helped release resource to remedy the problem, while

- 49% felt metrics based on benchmarking the firm against industry standards,

- 43.5% thought the value of the facilities and

- 39% the level of insurance premiums helped them getting attention from C-level executives.

One always has to take these findings with a grain of salt because some things are a bit confusing. For instance, 60% of executives are willing to provide more resources for a metric that provides them with information regarding vulnerabilities. Unfortunately, this does not stop the same individual to continue using software that might be a bit more vulnerable than they will surely like:

- How do browsers stack up securitywise? Open source, others and Internet Explorer

Nonetheless, the study is another piece of research that indicates that security must be aligned on operational risks or operations to gather the necessary support to secure resources needed to address the problems. In other words, outlining how a potential mishap or disaster affects business operations makes things clearer for C-level folks and gets their attention.

_Methodology and Sample

Senior c-level execs were interviewed using an online survey between 2005-06-20 – and 2005-08-31. No invitations were sent to people involved with security and risk management such as CIOs or Risk Officers since the intent was to find out how receptive other managers were to security concerns. 213 participated (the study does not provide the response rate so any halo effects are unkown).

The study does not provide you with a list of questions (e.g., in an Appendix) which would have been helpful to assess the study a bit better. Also, it might not tell many of you many new things but confirm, instead that you are not alone out there with how you see the world.

Thomas E. Cavanagh (October 2006). Navigating Risk ­ The Business Case for Security. New York, NY: The Conference Board. ISBN O. 0-8237-0885-3

The above study might tell you little if anything new, nonetheless, it is an interesting read and indicates that there is still much work to be done.

Since 2001 we have been addressing security metric issues:

- Best Practice – Benchmarks – Metrics – Ten Worst Security Practices

- LIB- NIST – Pub 800-55 – Using Metrics to Measure Security Controls, Processes and Procedures

- Week 33 – Lib 1 – NIST Guidelines – Security Metrics that Work?

Recently we have picked it up again in a more systematic way as outlined below:

- CyTRAP Labs – guide – developing IT security metrics that work for you

- 4 Tips for building an effective Early Warning System – organizational and human resource issues

Performance metrics are also getting ever more popular with defence agencies around the globe. Apparently a powerpoint graphic was leaked from US Central Command, The graphic (see below – Tracking U.S. Army’s trajectory in Iraq) stracks tracks changes and events on a peace….chaos meter. (where emergent chaos fits in is not made clear). Changes are categorized

- routine,
- irregular,
- significant, or
- critical.

” In fashioning the index, the military is weighing factors like the ineffectual Iraqi police and the dwindling influence of moderate religious and political figures, rather than more traditional military measures such as the enemy’s fighting strength and the control of territory….”

A number of secondary indicators are also taken into account, including

- activity by militias,
- problems with ineffective police,
- the ability of Iraqi officials to govern effectively,
- the number of civilians who have been forced to move by sectarian violence,
- the willingness of Iraqi security forces to follow orders, and
- the degree to which the Iraqi Kurds are pressing for independence from the central government.

Unfortunately, the graphic leaves much open to question regarding the direct measures and indirect measures used by the military to arrive at their composite type of index (e.g., the reliability and validity of various measures used and how they were aggregated).

Aggregation and Disaggregation is a tricky business indeed and unless it is carefully explained and makes sense, the metric may have little meaning at all.

_PS 1. The shifting index was seen by some officials as a stark warning about the difficult course of events in Iraq, and mirrored growing concern by some military officers.
_PS 2. A spokesman for the Central Command declined to comment on the index or other information in the slide when asked, saying: “We don’t comment on secret material.

As you might have expected, there is no consensus on what security metrics should be used for measuring security effectiveness and benchmarking the enteprise.

2004-04-06 The Robert Frances Group reported in CSO magazine that the companies it surveyed used these metrics to assess security effectiveness:

We can wholeheartedly agree that IT executives must ensure that the metrics they collect are useful and understandable. But when looking at the above numers, how can we link them to bottom-line and strategic issues. Put differently, to better manage the costs and resources invested in this process, top management has to understand and know how such metrics relate to their task of enterprise risk management and profitability.

While centralizing these metrics and automating their analysis can be helpful, unless these metrics help in better managing the strategic focus of the enterprise, the metrics might be of limited use.For this purpose we have developed a brief that outlines:

- CyTRAP Labs – developing effective IT security metrics(click on Login as a Guest for free access)
The above checklist is illustrated using the malware and virus infection ratios that are also used in the above Table. Applying the checklist illustrates that unless some hard-nosed decisions are being made and a careful and systematic analysis is used before a IT security metrics is approved the firm ends up with:

1) too many metrics (see above Table) that
2) help little in better managing risks and strategic objectives

Check it out, you will be surprised.

SUBSCRIPTION

To make it more convenient for you to take advantage of CyTRAP Labs’ offerings, just provide us with your e-mail address below. You can personalize your subscription to make it suit your needs.