The value of information

I've probably said something like this before, but it's worth saying again.

This was part of a post by Amelia Kassel of MarketingBase - on the AIIP member mailing list.

I recall someone in a workshop I gave about using the Internet for CI about 10 years ago. I introduced the concept of fee-based databases and a young fellow from a business analyst firm raised his hand in front of group of more than 30 participants to stop me from proceeding. He didn't want to hear or learn about fee-based databases. He had tried them once and they were too hard to use. I asked what he did when couldn't find information he was looking for on the Internet and he didn't have an answer but said it didn't really matter.

I've also come across attitudes such as this - why pay for information when you can find it for free. That would be true and valid if the time required to find the information, and the work required to put it into a usable format, was the same. In reality this is rarely the case. The advantage of paying for information from services such as Factiva, Dialog and several other similar services is that you can save a lot of time. The information purchased will be formatted consistently - so it becomes much easier to edit for a report.

Further, relevant information is collected together so there is no need to check hundreds of potential sources. These services index thousands of sources in a way that users of the free services, including Google, can only dream about. As an example, on Factiva, you can specify that search terms appear in the first 50 (or 100 or whatever) words of an article, or within so many words of another term. They support full Boolean searching and wildcard searching far beyond what even the advanced search in Exalead offers.

If that was all such services offered then there could be an argument that with today's budgetary constraints, good researchers would first focus on the free sources. However many sources held won't even be available on the free web, as their publishers only make them available on a pay-to-use basis or don't keep full online archives. This means that unless the researcher has accounts with a multitude of publishers they won't get the material they need for decision making.

I think part of the secret of being a good researcher is knowing when to use free sources and when to use fee sources. I'm sure that a proportion of the information that is available on pay-to-use sources could be found for free - IF you looked long and hard enough. However employers pay you for your time - and just because something is free doesn't make it really free if you have had to spend a day finding it when you could have got it within 15 minutes by paying. Then there's the risk factors of NOT finding something at all!

People who feel it doesn't matter - that you can justify not paying for information - are actually high-risk employees. They may provide information that allows correct decision making to be made 80% of the time. Unfortunately the Pareto effect comes into play - and that 20% of the time they get it wrong represents 80% of the risk. Decisions made on inadequate data are likely to lead to serious consequences when they are wrong. Saying that you only did a Google search because Factiva cost too much won't save you or your company in such situations - as it will be too late.

CI versus corporate espionage: thoughts on an ABC News story

I read this news item from ABC news 'James Bond' Tactics Help Companies Spy on Each Other" and had only one thought: that guy is totally unethical and wrong.

A few years ago, an Israeli colleague commented to me that in his experience, most of the ex-secret service operatives who try and enter the commercial world of CI fail. The reason he said is that they don't know the boundaries of what is legitimate competitive intelligence collection and what is corporate espionage, and illegitimate. He also said that in many cases, they also have no real idea of budgets and what is valuable to a company strategically versus the cost of obtaining it. Most never had a budgetary role when working for the various national security services and so could not do a cost-benefit analysis effectively.

This story shows both examples. Purchasing the garbage from an organisation is not only unethical but strikes me as wasteful. Garbage is thrown away for a reason - it's not wanted and valueless. The majority of companies today have shredders and routinely shred anything that would be seen as highly sensitive. True, the mid-level material may be chucked, but not the high-level stuff. (And those that don't shred deserve what they get - I'd be surprised that any Fortune 500 companies don't have shredding contracts!)

As for the other shenanigans implied - any company that employed a consultant to use such techniques deserves to get sued and end up paying more than they gained. The trouble is some do - and the list of companies that learned the hard-way that espionage doesn't pay is still growing.

So let me make it clear: espionage is wrong, while CI is a legitimate practice that uses only ethical means to collect intelligence.

This involves declaring your identity and NOT collecting information that would be classed as secret or confidential. As Issur Harel the Israeli spy-chief responsible for capturing the Nazi war murderer, Eichmann, is reported to have said:

We do not deal with certainties. The world of intelligence is the world of probabilities. Getting the information is not usually the most difficult task. What is difficult is putting upon it the right interpretation. Analysis is everything. James Bond is not the real world.

Attacking a castle - or a competitor!

The leading management guru, Rosabeth Moss Kanter's, latest blog post discusses ways to attack a castle: Four Ways to Attack the Castle — And Get a Job, Get Ahead, Make Change.

Although the article is talking about job-seekers and change agents, the same applies to competitive intelligence and strategy, and I've sometimes used the same analogy in my training courses.

So how does attacking a strong fortress compare to competitive intelligence collection. Well - the approach that some still seem to think the best approach - is the full frontal attack. Go for the key contact and hope that they will speak to you. The problem is that these people tend to be surrounded by gatekeepers, guards and you may not even get their name, never mind getting to speak to them. This is the corporate equivalent of having hot oil poured down upon you.

Moss Kanter describes four other approaches that can also be used for CI collection.

1) Find other doors.
Rather than target the main entrance with your battering ram, look for a door that's not guarded. If you want to interview somebody, don't call switchboard and ask for the purchasing manager - as switchboard will ask what it's about and you will find yourself in an interminable voice-mail loop ending with a "send an email to suppliers@companyname.com". Instead, use networking tools - such as LinkedIn - to find the name of anybody involved in purchasing within the target company and ask to speak to them directly. Knowing the name means you get put through and bypass the switchboard gatekeeper.

2) Befriend the fringes.
Be polite. Switchboards get fed up with rude callers - so be friendly. Chat - and treat the operator with respect. They may know more than you think and you may get a name that way.

You won't get put through to the CEO or CFO or any C-level executive directly. Instead, you'll end up speaking to their personal assistant - the guard and gatekeeper for your source. Like the guards and gatekeepers of old, these people know who passes by, and what goes on. So rather than insist on the C-contact, be nice to the PA and chat to them instead. You may well find that all you need to know comes from them instead.

3) Go underneath
Often, going to the top won't help. If the information you require is sensitive, the people at the top know the sensitivity - including their PAs. They won't talk and you will get nothing. Rather, consider the people who report to them, or who have managers who report to them. Such people may not know the whole picture - but speak to several and you soon will. Each interviewee will feel flattered that you view their knowledge as important - and won't realise that the small bits of information they know, when combined with other small bits, can reveal the secrets the higher-ups would like to keep hidden.

4) Go around the castle
Rather than trying to contact the organisation directly, look for people who are now outside but know what goes on inside. These include ex-employees, obviously. However others may also know information - and be willing to share if asked in the right way. These can include your customers, your competitor's customers, their suppliers, as well as industry consultants, trade association staff and many more similar sources.

Collecting competitive intelligence doesn't always depend on looking for the obvious source. Like attacking castles, often the secret is to find the weaknesses that allow you to gain entrance, gather what you need to know and leave without anybody even noticing your visit.

A tale of two countries

I've just read the story of James Bain who was freed after DNA evidence proved that he was innocent. Bain was apparently convicted on the evidence of a line-up despite other evidence not linking him to the crime he was accused of. Of course Bain is not white - and so the US justice system - certainly that from 35 years ago - was prejudiced against him.

Contrast this to the cries against the Italian justice system that has just convicted an American citizen and her Italian boyfriend of a brutal murder - also based on DNA evidence. In the case of Amanda Knox and Rafaelle Sollecito it wasn't just DNA evidence either. There were bloodied footprints, erratic behaviour post-murder with changing and inconsistent stories, evidence of antagonism between Knox and her victim, plus more.

Too often, the American legal system seems prejudiced against those least able to defend themselves - look at the differences in the ethnicity of people sentenced to death versus those who escape a capital sentence.

Now contrast this to the Italian legal system where a white, affluent Italian, and his white affluent girl-friend were treated equally to Rudy Guede, a black Ivory Coast accused - sentenced earlier.

The only bias here is not the Italian legal system but the Americans who believe that you can only get justice in the USA, and that the Italian legal system and the judgement is flawed.

If this was the only example of hypocrisy emanating from the US it could be excused - but unfortunately it isn't. Another example is the US "justice" system's hounding of Gary McKinnon - an Asperger sufferer who has been accused of exposing weaknesses in US military computers and so has to be sentenced so that those who failed to protect the system can get off free.

So what has this to do with competitive intelligence? Maybe nothing - on the surface. However if you think about, a lot. As it shows how important it is to remain objective - unlike with Gary McKinnon; to avoid prejudice - as was shown with the Bain case and also the objections to the Knox case; and to ensure that there are multiple lines of evidence before coming to a conclusion and making a decision - as in the Knox case but not in the Bain case.

Yauba - Big Brother isn't watching you

Sixty years after George Orwell published 1984 many of the ideas have, unfortunately, become commonplace. There are speed cameras watching how fast you drive, and CCTV monitoring many UK towns. On the Internet, search engines such as Google monitor your searches - keeping the data for months. They know what operating system you use. AWARE doesn't record this information, despite showing some in our top bar, but many sites, and most search engines do).

Yauba bucks the trend by proudly announcing that it respects user privacy. Its privacy policy proudly states:

We do not keep any personally identifiable information.

Period.

Following the Iranian elections (June 2009) many Iranian dissidents and protesters have switched to Yauba, according to the searchengine blog site, Pandia.

"Ahmed Hossain, CIO of Yauba, tells Pandia: “Our traffic from Iran has jumped 300% over the past several days, as many of them are using the Yauba Search Engine and the anonymity proxy filter to access blocked sites and get news from foreign sources.”

Anonymity may be important for some people. However for most, it's search results that count. Although Yauba claims to be able to search semantically, differentiating between Java the island, Java the coffee and Java the computer language is this a meaningless boast?

In other words is Yauba worth using for those not looking to hide their identity.

The short answer is yes. Yauba searches various types of content - which are separated. As such it enables you to quickly find Acrobat files, Word documents, PowerPoint presentations, news, blogs, images, video, etc. in a single search. Each are kept distinct - and this is an interesting differentiator between it and other search engines. It also presents ways of refining queries and where there are alternative meanings it shows these - allowing users to pick the one they want.

Rather than use the search they suggest i.e. Java I put in Apple. The three meanings I thought of were

1. The fruit
2. The computer company
3. The music company founded by the Beatles

In fact, there are several more - as Yauba shows:

apple can mean:

Apple Inc. (formerly Apple Computer, Inc.), a consumer electronics and software company Apple Bank, an American bank in the New York City area Apple Corps, a multimedia corporation founded by The Beatles Apple (album), an album by Mother Love Bone Apple (band), a British psychedelic rock band Apple Records, record label founded by The Beatles Apple I, Apple II series, Apple III, etc., various personal computer models produced by Apple, Inc and sold from 1976 until 1992. Ariane Passenger Payload Experiment, an Indian experimental communication satellite launched in 1981 Apple (automobile), an American automobile manufactured by Apple Automobile Company from 1917 to 1918 Billy Apple, artist Fiona Apple, a Grammy award winning American singer-songwriter R. W. Apple, Jr., an associate editor at The New York Times

Clicking on Apple (automobile) gives a number of results - not all directly relevant but some which were. There is also a brief encyclopedia type entry at the top of the page:

The Apple was a short-lived American automobile manufactured by Apple Automobile Company in Dayton, Ohio from 1917 to 1918. Agents were assured that its $1150 Apple 8 model was "a car which you can sell!". Sadly for the company, it would seem that the public did not buy.

On the right of the screen are various suggestions for alternative searches. For example, a search for apple gives:

Compare the clarity of this to the same search on google. (Admitedly the search is not sophisticated and a competent searcher would refine the term - but for testing, it's good enough)


It means that amateur searchers are more likely to find resuls for complex searches - fulfilling Yauba's claim to allow people to search without a knowledge of Boolean logic.

Also interesting is that a component of each search includes a real-time element - from Twitter and social news from Digg. The real time search element is useful as it provides another option to scoopler.


Sponsored ads appear to come from the Google network. There are also options to filter searches (although there is currently no information on what is being filtered) and a Lite version which seems to remove the refinement options and the top-level definitions (i.e. making it more Google like in its results presentation).

There is also an option to refine searches - alongside the search box.

Selection of one of the options allows further search refinement either by keyword

or domain

Overall I like Yauba. The interface is clean (and the black background makes a change from competitors).


Currently the site says it's only an early Beta / Late Alpha preview release so more work / changes can be expected. Hopefully these will include Help files explaining what the Lite search is supposed to do and what a Filtered search actually filters. Also, what syntax is acceptable - to refine searches. Does Boolean searching actually work, for example? On my brief tests it seemed to - as did phrase searching i.e. putting search terms in quotes. What about other options - could any of the advanced search options from Exalead be included. And will the site cover more countries, than the current small number (Italy, France, UK, India, Brazil, Russia and the .com site)? Yauba promises to cover more countries - I'm just surprised that there is no Chinese or German version as I would have expected these before the Italian version. I guess the Yauba team have Italian speakers but currently no Chinese speakers.