EUIST

In the past we have pointed you in the direction of good reasearch, such as:research that matters – identity theft

We have also pointed out why some findings can be misconstrued or not tell us the whole story in other places, such as work conduced by PWC here:

3 ENISA – awareness raising study – better prevention thanks to data crunching

Today we focus on another story that suggests that IT valuations are another important activity that we should focus on besides so many others, of course.

Figures don’t lie, liars don’t figure

This is an important study for all Chief Information Officers and managers in general. It focuses on an asset that is becoming an ever greater junk of capital equipment and software one needs to run today’s and tomrrow’s business.

When checking all the documentation we could collect about this study it appears to us that the task for doing this research was divided up as follows:

Soumitra Dutta, INSEAD – designed the study and analyzed data and was quoted in the press – see Financial Times below

Kevin Withall who is the co-founder of Vanson Bourne – seems that he was responsible for the sample selection and data collection and

Giles Fraser – Brands2Life co-counder – was in charge of the people who created the PR for this study

The public relations folks did a wonderful press release, hyping up the study and emphasizing the right the so-called key findings. All in order and an example of professionals doing exactly what they got paid for to do:

the press release that was used by Financial Times reporters to put together their story

Their effectiveness was illustrated by the Financial Times using the press release for a major story. It seems that the journalists did not investigate much beyond the press release in front of them. This is shown by comparing the press release above with the article subsequently published in an issue of the Financial Times (see below):

Financial Times reporters Pan Kwan Yuk in Paris and Philip Stafford in London – 2 journalists who seem to have not gone and read much beyond the press release or get

FT story from Yahoo!

But we do know that valuating software is an extremely difficult job to do. Moreover, most bosses demand some type of cost-benefit analysis for most activities. Naturally, this includes IT and software assets amongst many other types of assets. In fact, Jem Eskenazi (an Insead alumni) pointed his finger at some of the weaknesses in the study. He even went to the trouble to write a letter to the editor of the Financial Times focusing on the study’s problems, when looking at the findings from a manager’s point-of-view:

Get the FT story AND Jem Eskenazi’s Letter to the FT Editor – valuation of software assets study – as pdf file

So what are the problems. Remember when in 2003, Eliot Spitzer was extracting a $1.4bn settlement from Wall Street as punishment for biased research?

Well we feel that the formulation of the questions used in these telephone interviews make a careful interpretation of the findings not an easy task. Below we list some of the questions that were asked of managers in the study mentioned. We also provide you with the scaling that was used for coding the answers received during these telephone interviews. In some cases and just for your convenience, we entered our concerns right below the question that seemed to raise some warning lights….. but see for yourself.

CyTRAP Labs take on this issue
In the above list of questions we decided to enter some of the reported findings and put a clarifying question forward since we had a very hard time making sense out of the findings. In fact, we got the results as they were given to the public relations people who wrote the press release. Unfortunately, we were not given permission to publish these findings to make things more obvious for you that this study may have serious methodological problems and be outright biased.And no, we could not get answers to the clarifying questions we listed for your convenience in the table above.

Finally, a concern is that some of these questions appear rather vague and make it difficult for an expert to answer. In turn, how meaningful can the findings be?

Naturally, the author stated that valuation of IT assets is critical. Considering that Micro Focus is the sponsor of this study, no surprise there. But we wonder if the way these data were collected (no concise information available since database is proprietary – apparently owned by Vanson Bourne). Does the sample meet the usual requirements put upon scientific research before starting to generalize from the data? For instance, the data sources do neither seem random nor do they meet the requirements for a well designed set of stratified data.

Best practice and proper research methodology would require that concise and precise information about the sample used for data collection is provided. Unfortunately, what we could get even after having placed several requests for further clarification, nothing more was forthcoming.

One of the things we do wonder about is how a top business school like Insead can let itself be used for such a PR stunt paid for by Micro Focus?

Could it be that there is an inherent danger for academic institutions being instrumentalized for marketing purposes if they conduct contract research for business with vested interests? Furthermore, should we be willing to accept such research without being given what is generally accepted as minimum type of information regarding sample selection and statistics?

Without such information researchers cannot review and assess the reliability and validity of such work? In turn, how useful this kind of management research is for CIOs must be questioned. Unfortunately, the journalists were unable to see the writing on the wall.
Unfortunately, this is just one of the many examples one can find every week in the media. Worst is that I still do not know for sure if valuation of IT assets makes as much sense as the author of the study claims. So we are back to the drawing board….. and NO, I am not sure if software valuation matters in today’s and tomorrow’s enterprise based on this ‘study’
PS.

We published this study a while after the Financial Times had its article about it. One reason was that it took us nearly two weeks after the press release was published to get the information we needed from the parties mentioned….

Than we were no longer sure if we should even write anything about this. Than we thought, we could publish early on in 2008. This way it can serve as an example of our hopes for this year that not too many studies of this kind will be financed and published. They surely do not to clarify if and how I should valuate software assets at our company….

============>

If this post was helpful to you, please consider stumbling it or subscribing to feeds from CyTRAP Labs. Cheers.

============>

Might be of interest as well

1 research methodology – ecrime drop in Russia

============>