Business intelligence is a wonderful marketing expression that conjures up images of magically unlocking secret corporate knowledge. But even though everyone from policemen to publicans can take advantage of what BI offers, the brutal reality is that a BI project in any company is a major undertaking requiring considerable thought and attention to detail. What is more, evidence is emerging that companies are finding it difficult to figure out what to do with their burgeoning data stores.
According to David Burman of Butler Group: "There has been no clear notion of how data could be used to (a firm's) advantage. This is a problem that has confronted companies regardless of their size."
On the surface BI seems easy - a request from marketing to segment last year's sales according to quarter, by region and major product group looks like a snap. But no.
If the underlying data is contained in more than one database, if the request leads to customer analysis or if the data resides in more than one system, then the IT department is going to have a major data extraction and transformation on its hands. And that is the good news.
The bad news is that as industry moves forward to include Internet technologies and as companies seek to integrate more systems, the problem of pulling information into a coherent form gets worse. The BI presentation tools vendors make light of the difficulties, concentrating instead on the benefits to be derived from having snazzy graphics that anyone can use.
That is the glitz and glamour bit vendors use to impress unwary decision takers. But if one takes BI to its logical conclusion, then it should be viewed as an infrastructure project that embraces storage networks, On-Line Analytical Processing (OLAP) databases, data warehousing and unstructured data.
Data integration creates vendor difficulty because it is frequently seen as a differentiator and so encourages lock-in at a time when that is not always appropriate.
That is the thinking behind Seagate Software's enterprise information management strategy. Over the last year, it has made a number of announcements regarding infrastructure products, culminating in last month's plan to dump 2.5 million free CDs containing Seagate Analysis - a series of bits and pieces based on its high-end Info 7.0 product . Seagate's unabashed attempt at market domination met with derision. For example, Ian Maclachlan ,of Accurate Business Solutions, comments: "I can't help seeing Seagate's free software promotion as another case of Microsoft-induced panic reaction.
Giving away millions of copies of even quite competent software is an act of desperation."
Big money claims
Some analysts estimate the BI market is worth about $4 billion (£2.6 billion) this year, but it is horribly fragmented and riven with claim and counterclaim. Seagate believes it is the largest BI vendor, having shipped five times more than its nearest rival, BusinessObjects. That was a surprise to Cognos, which claims it never sees Seagate in deals but does line up against BusinessObjects in 90% of bids.
Hyperion/Arbor, with revenues last quarter of $101 million (£65 million), can reasonably claim to be the largest BI vendor by revenue, a view shared by independent OLAP analyst Nigel Pendse. But then Pendse is looking at the market, which from an OLAP standpoint in itself causes confusion because OLAP is but a subdivision in a market that spans data mining, forecasting and analysis and even knowledge management.
All vendors are deeply concerned about consolidation and the ferocious competition precipitated by Microsoft's entry into the market with its OLAP Services, bundled with SQL Server 7.0.
Hyperion is already feeling the pinch and in late May unceremoniously dumped John Dillon as CEO following three dismal quarters' results. According to Pendse: "True year-on-year revenue growth was just 8%, and net income crashed by 63%. Note that Hyperion's press release talks of a revenue growth of 15%, rather than the much worse true figure of 8%, because of the strange (legal, but misleading) rewriting of accounting history that occurs when companies with different year-ends merge."
Shortly afterwards, rumours circulated that Dillon and others from Hyperion were planning a bid, but those have fizzled out while the market waits to find out just how bad things are later this month.
The Microsoft effect cannot be underestimated. Pendse has long predicted there would be a vendor shake-up, he said: "Despite denials from Hyperion, Microsoft OLAP Services is now known to be hurting Essbase sales and prices, so this is an extremely bad time for Hyperion to have such severe self-imposed organisational problems that distract it from combating dangerous external threats."
Keeping a secure base
Hyperion's Essbase has long enjoyed a secure position as the database of choice in the financial analysis, consolidation and reporting markets, but up until the latest release it consumed considerable support costs because database changes required technical tuning.
That problem has largely gone away, but MS OLAP Services is much simpler to configure. Reports indicate it is both robust and genuinely scalable to enterprise requirements.
Last year, Cognos changed its product strategy from a concentration on front-end tools like PowerPlay and Impromptu to a position as enterprise application vendor. This has been done by concentrating on server development.
That was predictable because Cognos, a longtime close partner of Microsoft, could see the way the wind was about to blow.
What is more, Cognos had failed to get the mega-seat deals it needs to drive revenues. It is this last element that has dogged most of the well-known tools vendors because they have found selling 50-seat deals to be unprofitable.
Rob Zalums, Cognos' UK managing director, recently admitted that up to December last year, Cognos UK sales were flat although things have improved sharply since then.
"We've done a lot of work revitalising the sales people and getting them focused," he says. Which is just as well, because in 1996-98, Cognos only met with limited success in the large-seat deals with its PowerPlay product.
There are several reasons for this, but much seems to rest on the fact that BI as a concept was not well understood. A US survey of finance chiefs conducted last year showed that two-thirds of respondents saw no need to provide information to those outside the finance department. It can be argued the same holds true today, largely because the spreadsheet remains the principal analytical tool, even though numerous studies consistently demonstrate high calculation error rates.
HBR, for example, reported in October 1996 that "80% of all spreadsheet systems contain errors - specifically each system had errors in between 1% and 5% of all cells." It quoted the Fidelity Magellan fund, which in 1994 omitted the minus sign on a loss of $1.3 billion (£0.84 billion) and publicly declared a $4.32 (£2.79) dividend that had to be withdrawn.
More recently, Halifax Building Society was said to have overpaid for a mortgage fund it acquired due to spreadsheet errors, and KPMG consistently warns its clients about the dangers of sticking with the spreadsheet.
Leafing through old sales literature, vendors have done little more than rebadge analytic or financial consolidation applications over the last couple of years.
Today, the emphasis is on democratising BI so it reaches a wider audience.
At one time, Comshare looked as though it had it right with Arthur, an easy-to-use sales analysis and forecasting tool. But, true to a recent history of shooting itself in the foot, Comshare sold Arthur, just as demand for sales specific software was rising. Today, it is much more difficult to see who will lead the market, making product choices risky for buyers.
In an atmosphere where vendors are struggling or going through major change and where the market is being redefined at the server level, differentiation and business advantage are harder to spot.
Hammer beats the drum
Kay Hammer, CEO of data transformation specialist ETI, has been making the European rounds in an effort to drum up support for new meta data standards. There is a lot of merit in her argument because a workable standard will simplify the database integration work that all too often dogs a BI project.
However, she admits this is a tough job and acknowledges there will be compromises. "The meta data coalition hopes to deliver about 80% of what people will need," she says. Microsoft is part of the coalition, which Hammer believes is critical to success. "With Microsoft in, people start taking notice," she says.
Hammer is not concerned about Microsoft's reputation for standards wrecking: "Microsoft knows it can't make it in the enterprise at the moment, so it needs to be seen as a collaborative partner on a project that is dedicated to solving enterprise problems. Its people are aware they need to add value the rest of us have already addressed, so for them it is part of playing catch-up." In her view, the brand leaders at the desktop - Cognos, Brio and BusinessObjects - will have to move their products forward again.
"As standards come in, there is a risk the value these vendors offer becomes eroded," she adds.
Michael Brill, VP product marketing at WhiteLight, agrees. "Only one vendor has had any measurable impact on the structure of the BI industry and that is Microsoft. Regardless of the actual technology, Microsoft is not a company that many want to take on head on as, ultimately, Microsoft will lead the OLAP server market and within a couple of years own the ad-hoc analysis tools market through Excel and the rest of Office," he says. If Brill is correct, then it would seem the BI market is entering an end-game situation.
Oracle slows down
Even Oracle, which competed with Express, has pretty well given up the ghost. It had only really been successful selling Express as part of a much larger Oracle transaction and recently dramatically curtailed investment in Express, eliminating its direct-sales force and putting Express into a very small corner where it will likely wither.
However, some disagree. Kim Dennis, business development director at UK newcomer Sylvon, says: "There's no evidence from where we sit that the bigger vendors are intruding on the smaller ones or the more recent entrants in the market. Where there are changes, they are strategic and technical. Established vendors are adapting their businesses using three distinctly different strategies - core strength enhancement, general applications or niche solutions."
Companies like SAS Institute, for example, are doing tremendous business.
SAS has a strong offering and had the good sense to make sure it has a skills qualification that is credible. Cognos, on the other hand, recently introduced Visualizer, which "graphically paints a picture anyone can understand about the vitality of an organisation," according to Rob Rose, Cognos vice president of product marketing.
In development for two years, Cognos believes Visualizer broadens the appeal of its core products. Broadbase has successfully pulled away from general database development to applications based on Microsoft technology, but it is a hard path to tread with high risks. Gentia has always had terrific technology and was first to market with a sensible balanced scorecard package. But a series of strategic and tactical errors has left it cash starved and in real danger of going out of business.
MicroStrategy, which for a long time relied on selling expensive server technology to large enterprises, has diverted development attention to the Internet, providing broadcast technology that gets information to many users inexpensively.
WhiteLight has introduced the concept of "integrated decision processing" to describe how its product enables a company to integrate all of the structured and unstructured information sources and application logic that exists inside and outside the company and then analyse this with Web-based applications.
WhiteLight has enjoyed success in the financial services industry. Dennis McGinn, the company's CEO, says: "As applications expand, the Achilles' heel has always been consistency and end user trust. We've solved those problems."
According to McGinn, users want to take application ownership so they can add value by using their own ways of thinking. "We've provided the means to get different cuts of the views very easily," he says.
Implementing a successful BI strategy is far from easy. Customers have to use more than one tool, think strategically yet reduce potential complexity.
Tom Scampion, business development manager at Trillium, believes: "The biggest issue with making business intelligence effective is that many companies don't think they have a problem in the first place." Chai Tang, marketing manager at SPSS, warns: "Before considering BI, people need to ask themselves what they want to achieve for their business over the long term (two years plus). It's not a short-term solution, so if they want to achieve something like hitting sales targets this quarter, it's not going to work."
Nigel Geary, MineShare's international vice president, adds: "Most organisations use many tools. We believe we can help enterprises to bring that number down substantially by replacing OLAP, reporting and query tools with a tool that has a familiar metaphor yet is safe - like Excel on steroids."
A changing process
But even if tools can reduce complexity, it is important to understand that BI is not a solution but a process where data and needs change over time.
Given the complexities and vendor uncertainty, is it a safe bet to go back to one's accounting application or ERP supplier and see if they can offer an integrated tool?
Independent analyst Richard Creeth is in two minds: "It depends on what the vendor is doing. Many have relationships with well-known tools vendors, but then the integration isn't always that deep. On balance, I would say think best of breed."
The big question then is should one go for a solution now or wait? Geoff McClure, director of the information strategy team at Foundation Systems, is emphatic. "It is said that soon the only competitive advantage will be the ability to put new information to work quickly," he says. "BI can deliver the first part of this equation. By not moving now the company may loose out to its competitors."
So that is sorted out - but recognise there is no one-size-fits-all solution, that strategy is all and that market changes are creating uncertainty.
As Roopinder Singh, marketing director at Ardent Software points out: "Buyers should be extremely careful in what they buy and from whom. In some cases you might want to give up some leading-edge functionality to ensure long-term safety from proven vendors that look like remaining viable."
Golden rules for BI success
- Think strategically but act tactically
- Analyse and document requirements
- Establish returns on investment on a per-project basis
- Use consultants to roadmap projects
- Consider industry- or application-specific solution vendor for rapid results
- Each user class will have a different need - therefore you need multiple vendors
- Avoid vendor lock-in
- Don't get wowed on acronyms, the technical sales pitch or the glitz of a snazzy front end
Things you should look for in BI tools
- Support for major transaction databases
- A front end that can access Microsoft's OLAP Services
- It must be Web enabled
- Functional richness
- Ease of use
- A range of products for both ordinary and power users
- Pre-built ERP or accounting systems interfaces
- A portal through which the user is automatically notified of relevant changes.
Cotton seedling freezes to death as Chang'e-4 shuts down for the Moon's 14-day lunar night
Fortnite easily out-earns PUBG, Assassin's Creed Odyssey and Red Dead Redemption 2 in 2018
Meteor showers as a service will be visible for about 100 kilometres in all directions
Saturn's rings only formed in the past 100 million years, suggests analysis of Cassini space probe data
New findings contradict conventional belief that Saturn's rings were formed along with the planet about 4.5 billion years ago