Information Systems Management | My Assignment Tutor -

Full Terms & Conditions of access and use can be found athttp://www.tandfonline.com/action/journalInformation?journalCode=uism20Information Systems ManagementISSN: 1058-0530 (Print) 1934-8703 (Online) Journal homepage: http://www.tandfonline.com/loi/uism20The Measurement of Business IntelligenceAntti Lönnqvist & Virpi PirttimäkiTo cite this article: Antti Lönnqvist & Virpi Pirttimäki (2006) The Measurementof Business Intelligence, Information Systems Management, 23:1, 32-40, DOI:10.1201/1078.10580530/45769.23.1.20061201/91770.4To link to this article: https://doi.org/10.1201/1078.10580530/45769.23.1.20061201/91770.4Published online: 21 Dec 2006.Submit your article to this journalArticle views: 2372Citing articles: 84 View citing articles32 W W W . I S M – J O U R N A L . C O MW I N T E R 2 0 0 6THE MEASUREMENT OFBUSINESS INTELLIGENCEAntti Lönnqvist and Virpi PirttimäkiBased on a literature review, this article identifies and assesses measurement approaches fortwo different purposes: determining the value of BI and managing the BI process within anorganization.N TODAY’S RAPIDLY CHANGING BUSIness environment, the need for timely andeffective business information is recognized as essential for organizations notonly to succeed, but even to survive. In this article, business intelligence (BI) refers to a managerial philosophy and a tool used to helporganizations manage and refine business information with the objective of making more effective business decisions (Ghoshal and Kim,1986; Gilad and Gilad, 1986). The term BI canbe used to refer to:1. Relevant information and knowledgedescribing the business environment, theorganization itself, and its situation in relation to its markets, customers, competitors,and economic issues2. An organized and systematic process bywhich organizations acquire, analyze, anddisseminate information from both internaland external information sources significant for their business activities and fordecision making.Several related terms include competitiveintelligence (CI), market intelligence, customer intelligence, competitor intelligence, strategic intelligence, and technical intelligence. InNorth American literature, the term CI is frequently used and the external environment andexternal information sources are emphasized(e.g., Cottrill, 1998; Fuld, 1995; Kahaner, 1996;Vibert, 2004). In European literature, the termBI is considered a broad umbrella concept forCI and the other intelligence-related termsmentioned above. Nevertheless, almost all thedefinitions share the same focus, even if theterm has been defined from several perspectives (Casado, 2004), and they all include theidea of analysis of data and information.The purpose of BI is to aid in controllingthe vast stocks and flow of business information around and within the organization by firstidentifying and then processing the information into condensed and useful managerialknowledge and intelligence. As such, the BItask includes little that is new and addressesvery old managerial problems; it is one of thebasic tasks of many management tools; that is,analyzing the complex business environmentin order to make better decisions. As Gilad andGilad (1986) have stated, organizations have:collected information about their competitors since the dawn of capitalism.The real revolution is in the efforts to institutionalize intelligence activities.BI presents business information in a timelyand easily consumed way and provides the ability to reason and understand the meaning behind business information through, forexample, discovery, analysis, and ad hoc querying (Azoff and Charlesworth, 2004).The BI literature suggests that much benefitcan be derived from using BI (e.g., see Thomas,2001). However, applying BI takes resources,and the benefits actually occurring in practiceIANTTI LÖNNQVISTis a senior researcherat Tampere Universityof Technology,Finland. He recentlycompleted his doctoraldissertation on themeasurement ofintangible successfactors. He can bereached atantti.lonnqvist@tut.fi.VIRPI PIRTTIMÄKIworks for TampereUniversity ofTechnology as a projectmanager andresearcher. She has aMaster of Sciencedegree in engineeringand is a postgraduatestudent in theuniversity’sinformation andknowledgemanagement program.BUSINESS INTELLIGENCEI N F O R M A T I O N S Y S T E M S M A N A G E M E N T 33W I N T E R 2 0 0 6BUSINESS INTELLIGENCEare not always clear. This article examines themeasurement of BI for assessing the effects ofBI activities as well as for assessing an organization’s BI process.The measurement of business performancehas long traditions in organizations. In the BIliterature, authors have identified BI measurement as an important task (Solomon, 1996;Viva Business Intelligence Inc., 2000), but acommon view among scholars is that it is difficult to carry out (Gartz, 2004; Hannula andPirttimäki, 2003; Simon, 1998).According to arecent survey, only a few organizations haveany metrics in place to measure the value of BI(Marin and Poulter, 2004).The objectives of this article are:❚ To determine the major purposes of BI measurement❚ To identify what types of measures are beingused❚ To evaluate the current measures and suggest how the measurement of BI could beimproved.The literature reviewed for this study includes publications on BI (and CI) and also literature on business performance measurement.An overall finding is that, in practice, the wayBI is applied in different companies varies a lot.For one company BI may mean single-marketresearch projects, whereas for another company it could mean a continuous process that produces various BI products and services that areused by several people. Case-specific issues arealso likely to have an effect on how BI can bemeasured. In this article, BI is understood to bea systematic activity producing BI products andservices. When there is a need to actively improve BI activities or to determine the usefulness of BI products and services, measurementis a potential tool.WHY DO ORGANIZATIONS MEASUREBUSINESS INTELLIGENCE?An important issue in determining how andwhat to measure is knowing the purpose of themeasurement (Brooking, 1996; Sveiby, 1997).According to Simons (2000), performancemeasurement can be used for the followingpurposes: decision making, control, guidance,education and learning, and external communication. The user of the measures should alsobe taken into account.According to the literature, BI measurements serve two main purposes. The first andmost common reason for measuring BI is toprove that it is worth the investment (e.g., Sawka, 2000). Davison (2001) points out that CImanagers need measures to justify their department’s existence. Similarly, executives need toknow whether it is rational for them to investin BI, because it is still a rather new managerialdiscipline. Moreover, the BI literature includesa lot of unverified assumptions about the effects of BI. For example, Kelly (1993) obtainedempirical evidence regarding the value of BI asestimated by practitioners. According to hisstudy, the estimated average payback of all BIprojects is 310 percent of cost, which seemsquite high.The second main purpose for the measurement of BI activities is to help manage the BIprocess; that is, to ensure that the BI productssatisfy the users’ needs and that the process isefficient (Herring, 1996). Namely, a BI processcan be costly if the information gathered is notaccurate or does not match the informationneeds. The users of a BI process measurementare likely to be the BI professionals in an organization and the typical measurement intent(e.g., guiding activities and learning) is to continually improve the BI products and services.Table 1 provides a summary comparison ofBI measurement for these two different purposes. TABLE 1 Two Types of BI MeasurementPurpose forMeasurementMain Users of MeasurementInformation Expected BenefitsDetermining the value of BI Executives justifying BI investmentsBI professionalsBI service providersResearchersAbility to cost-justify BI services anddemonstrate the actual effects of BIIncreased credibility of BI as amanagerial toolImproved rigor in BI researchManaging the BI process BI professionalsBI service providersContinuous improvement of BIproducts and services ccordingto a recentsurvey, onlya feworganizationshave anymetrics inplace tomeasure thevalue of BI.A34 W W W . I S M – J O U R N A L . C O MW I N T E R 2 0 0 6BUSINESS INTELLIGENCECURRENT MEASUREMENTAPPROACHESMeasures for Determining the Value of BIBefore discussing measuring the value of BI,we present a few observations on the conceptof value in this context. First, one may pose thequestion: value for whom? The perceived valueof some intelligence product, for example, islikely to vary depending on the subjective appreciation and need of the person(s) for whomthe question is addressed. In the following discussion, value is assessed from the viewpointof a company using BI (e.g., improved profit)or of the user of the intelligence (perceivedusefulness). Second, it may even be suggestedthat BI has no value at all as such — that is, thevalue is created as a result of using the intelligence, by carrying out actions based on it.Kelly (1993) has also recognized the conditional nature of the value of BI: as one of his surveyrespondents stated, information must be integrated into a decision in order for its value to beassessed.Keeping in mind the discussion above, theassessment of the value of BI requires that twomain questions be answered:1. How much does it cost to apply BI?2. What are the benefits of applying BI?A BI process takes operational resources; inaddition, an initial capital investment is likelywhen BI is first implemented. Calculating thecost of BI (Davison, 2001) requires calculatinglabor costs, information purchases, and otherexpenses related to the BI activities. A methodcalled total cost of ownership (TCO) can beused to identify all relevant costs related to anactivity (Degraeve et al., 2004).Measuring the benefits of BI is not as simpleas measuring the cost. Many of the effects thatBI is assumed to create consist primarily ofnonfinancial, and even intangible, benefitssuch as improved quality and timeliness of information (Hannula and Pirttimäki, 2003;Nelke, 1998). Although the nonfinancial effectsshould lead to financial outcomes (e.g., costsavings), there may be a time lag between theproduction of the intelligence and the financialgain. Therefore, measurement in practice isquite difficult.A typical method for assessing the monetary value of any investment is to calculate thereturn on investment, ROI. The problem in calculating the ROI for BI is that the output of theBI process is intelligence; in other words, somekind of processed information. The value of information is very difficult to assess (Kilmetzand Bridge, 1999). Other methods for measuring financial value (such as net present value orpayback period) also require determining thevalue of BI output, which is problematic.In spite of the difficulties, Davison (2001)has developed a measurement model called CIMeasurement Model (CIMM) that can be usedto calculate the return on CI investment(ROCII). The value of CI is assessed for individual CI projects and the CI output is measuredby assessing effects such as objective fulfillment and decision-maker satisfaction. The incremental value CI adds to a decision should beassessed in monetary terms. The input is calculated as a cost of carrying out the project. Thus,ROCII can be calculated using the followingformula:ROCII = (CI outputs – CI inputs) / CI inputsThe CIMM identifies various nonfinancialmeasures of tactical and strategic outputs,which can be considered quite useful in determining how successful the different aspects ofa CI project have been. For example, Davison(2001) suggests that a measure of a CI project’soutput could be simply a comparison of whether the targets set at the beginning of the projecthave been met. However, that is a measure ofthe project — not of the effects of the project.Furthermore, the fact that the value of CI outputs in the ROCII formula is based on qualitative assessments suggests that the ROIcalculation can be unreliable.Herring (1996) has identified four measuresof the effectiveness of CI: time savings, costsavings, cost avoidance, and revenue enhancement. However, it is not clear how these effectscan be measured. For example, it may be difficult to distinguish which cost savings or newrevenues result from BI and which result fromsome other, unrelated, managerial actions.According to Sawka (2000), the effectiveness of BI is best measured by evaluating thecontribution of BI to a specific decision or action and then looking at the benefit or detriment this specific decision brought to thecompany. His four measurable benefits are similar to those of Herring. First, BI can help inavoiding unnecessary costs regarding, for example, product development investments. Second, decisions based on good BI may lead toenhanced revenues. Third, BI information mayhelp in improving resource allocation decisionsand thus maximize investments into the mostprofitable purposes. Fourth, the direct link between a BI decision and business performance(e.g., stock price or customer satisfaction) couldhe CIMMidentifiesvariousnonfinancialmeasures oftactical andstrategicoutputs, whichcan beconsideredquite useful indetermininghow successfulthe differentaspects of a CIproject havebeen.TI N F O R M A T I O N S Y S T E M S M A N A G E M E N T 35W I N T E R 2 0 0 6BUSINESS INTELLIGENCEalso be measured, although it may be very difficult to do so.Sawka’s examples of how to measure the effects mentioned above are in the form of shortcase descriptions of specific projects. For example, in one pharmaceuticals company asmuch as $600 million of additional revenueswere derived from more effective marketingstrategies designed in part through good BI.This example illustrates the importance of thedecisions BI is used for. However, it is also evident that the contribution BI brings is difficultto determine. One might even argue that theabove-mentioned example says little or nothingabout the effects of the BI process — it is possible that a similar, better, or worse result wouldhave been achieved if BI had not been utilized.An alternative approach that may reveal theeffects of BI more accurately is subjective measurement of effectiveness. It is based on theconcept of perceived customer (here, the decision maker) satisfaction (Davison, 2001). Inpractice, the users of BI products are askedquestions regarding the effectiveness of theproducts. The questions may relate, for example, to how much the confidence of the decision makers has increased as a result of theadditional information provided by BI and theusers’ satisfaction regarding the insightfulnessor the timeliness of the intelligence. A positiveaspect of subjective measurements is that theresults show how effective the users considerthe intelligence products. However, subjectivemeasurements do not provide evidence of anymonetary value of the effects of BI.The results of a survey carried out by Marinand Poulter (2004) include descriptions of howcompanies actually measure the value of theirCI activities. One company compares the costof consultants to the results obtained by the CIdivision. Another company quantifies the strategic deals that the CI team has been involvedin and compares the win/loss ratios to thosedeals where they were not involved. In onecase, the use of CI was measured using statistics on requests for information from a database of competitor information.Measures for Managing the BI ProcessAs a managerial tool, performance measurement can be used in many ways. Traditionally,measurements have been used to show whether an organization’s various processes meet thepredetermined quality and efficiency criteria.Measures are also used to guide employees tofocus their efforts on areas identified as mostimportant. Based on these descriptions, performance measurement can be characterized as aversatile tool for improving the quality of business processes. Therefore, it is rational to alsoconsider using performance measurement inthe context of managing the BI process.In the literature, measures for managing theBI process have not been discussed as much asmeasuring the effects of BI. Of course, the effects of BI are created as a result of the BI process and thus are related. Of the measurespresented in the previous section, Herring’sfour measures of the effectiveness of BI, totalcost of ownership, and subjective measurements of effectiveness also seem useful formanaging the BI process. The main differencesare that, in the case of BI process management,the BI professional is the main user of the measurement information and the goal is to efficiently produce valuable intelligence for thespecific needs of the users. Therefore, amongthe issues that are the most important are theefficiency of the BI personnel, effective allocation of available resources, quality of the BIproducts produced, and the satisfaction of theusers.Similarly, Davison’s (2001) CIMM framework is mainly intended to assess the effects ofBI, but the measures proposed can also be useful from the point of view of managing the BIprocess. For example, the measurement of usersatisfaction regarding quality, relevance, timeliness, action ability, and the accuracy of the information produced provides insight into thequality of the BI products and services (produced by the BI process). Looking back at suggestions and predictions made based on BI, andassessing how accurate they have been, makesit possible to assess the reliability of the BIproducts (assessment of prediction rate). Further, the CIMM framework, which assesses thesuccess of individual BI projects, provides information on which types of projects havebeen successful and why, and vice versa. Thisinformation can be used as a learning toolwhen designing future BI projects.Information Builders, a provider of BI products and services, suggests that three characteristics of intelligence should be measured(Information Builders, 2004): deploying ability,scalability, and usability. These characteristicsmainly describe the properties of the BI software being used; for example, how peoplewith different needs and skills can utilize thesoftware.Williams and Williams (2004) have presented a method called “BI readiness assessment,”nalternativeapproach thatmay reveal theeffects of BImoreaccurately issubjectivemeasurementofeffectiveness.A36 W W W . I S M – J O U R N A L . C O MW I N T E R 2 0 0 6BUSINESS INTELLIGENCEwhich can be used to determine an organization’s ability to utilize BI; for example, continuous improvement culture, information oranalytics culture, and technical readiness. Thedata is collected with a qualitative survey. Other ways of measuring intelligence utilizationseem easy to generate. For example, it seemspossible to measure the proportion of managers that use BI tools or measure how often theBI tools are reviewed. According to a survey byMarin and Poulter (2004), some organizationshave undertaken efforts to capture user accessof competitive intelligence that is gathered anddistributed by electronic means.Measurement can also be used as a tool inguiding the execution of individual BI projectswhile they occur. Hoadley (2004) has proposed a method he names the “Hoadley suite”for determining whether a necessary amountof data has been captured and for evaluatingthe cost of additional data collection. His method is based on assessing the completeness ofthe data (have all relevant aspects been covered?) and the timeliness of the data (is the datacurrent?). Combining these two viewpointsmakes it possible to assess the degree to whichthe data intended to be captured has alreadybeen captured. The method relies on qualitative assessments made separately regarding different data sources. This facilitates focusing BIactivities on areas where much of the data hasnot yet been captured. Hoadley (2004) assumes that as the percentage of captured dataincreases, the cost of carrying out additionaldata collection increases in a linear relationship. Thus, the cost realized at a certain pointof the project can be used to estimate howmuch the additional data collection will cost.BUSINESS PERFORMANCEMEASUREMENTBalanced View of PerformanceBusiness performance measurement is an active research field, yielding a large number ofpublications annually (Neely, 1999). New practical solutions for applying measurements invarious situations are also constantly introduced. Considering BI to be a similar activity toor a process like any other business processmakes it possible to also discuss the application of the methods of business performancemeasurement in the context of BI.The definition of performance is a goodstarting point when discussing performancemeasurement in the context of BI. Most authors in the field consider performance to be acomplex issue in which a measurement object’s performance is examined from severaldifferent points of view (Kaplan and Norton,1996; Neely et al., 2002). Performance may differ depending on the perspectives from whichit is examined.The traditional way of measuring only financial phenomena is nowadays considered as providing lagging information that isnot actionable. In addition, with purely financial measurement, issues that are important formany stakeholders (e.g. customers and employees) are neglected.The balanced performance measurementframeworks can be used to identify the factorsto be measured and, at the same time, to definethe components used to determine performance. The main principles are usually similarin different balanced measurement frameworks (Lönnqvist, 2004; Tuomela, 2000). First,performance measures are chosen based onthe organization’s vision and strategy. Second,success factors are chosen from several perspectives (e.g., the shareholder’s and customer’s) to provide a balanced and holistic view ofthe organization and other factors affecting itssuccess. Third, measurement is focused on alimited number of critical success factors.Fourth, the measurement system is designed insuch a way that there are causal relationshipsbetween the success factors. Fifth, the measurement system can be used as a tool in communicating and implementing strategy.The most commonly used balanced performance measurement framework is the BalancedScorecard. Usually, the four measurement perspectives of the framework include financial,customer, process, and learning and growthperspectives (Kaplan and Norton, 1996). Another, more recent, measurement framework isthe Performance Prism. The prism shape represents the complexity and the different aspectsof organizational performance. The Performance Prism of Neely et al. (2002) consists offive facets. The top and the bottom of the prismfocus on the organization’s stakeholders. Thequestions regarding stakeholder satisfactionand contribution are (1) Who are our key stakeholders and what do they want and need? and(2) What do we want and need from our stakeholders on a reciprocal basis? After these questions have been answered, the next step is toconsider what strategies are required to satisfythe needs and achieve the contribution of thestakeholders. The next phase is to considerwhat processes must be put in place to enablethe execution of the strategies. Finally, oneeasurement can alsobe used as atool in guidingthe executionof individualBI projectswhile theyoccur.MI N F O R M A T I O N S Y S T E M S M A N A G E M E N T 37W I N T E R 2 0 0 6BUSINESS INTELLIGENCEshould consider what capabilities are requiredto allow the operation of the processes.It also seems possible to use the idea of balanced performance measurement in the context of BI. For example, Herring (1996)suggests the Balanced Scorecard approach butdoes not go into detail regarding how to do it.There are many ways to design a balanced performance measurement system for BI, depending on the framework chosen. In practice, themeasurement systems should be tailored according to the needs of the specific situation.Thus, only a general presentation of one possible approach will be presented in the followingsection.Performance of a BI Process — WhatShould Be Measured?The literature presents several BI process models. According to Pirttimäki and Hannula(2003), the biggest differences among them arethe number of phases, structure of cycles,sources of information, storage methods of information, and type of information gathered.A typical example of a four-phase BI processmodel includes the following related phases:1. Identification of information needs2. Information acquisition3. Information analysis4. Storage and information utilizationIn the first phase, the organization mustfind out what kind of business information isnecessary to resolve different problems and tomake successful decisions. This ensures thatonly relevant information is utilized in decisionmaking. The second phase, information acquisition, is driven by the business information needsand is considered a complex function becausethere are many different sources of informationboth inside and outside an organization.In the third phase, acquired information isanalyzed and then packaged into different information products and services. The productsand services are aimed at an organization’s different user groups and information needs. Concerning the storage aspect of the last phase, thegoal of a BI system is to make it possible for thedecision makers to find the required businessinformation as quickly as possible. Before theprocessed information can be utilized, information must be communicated to the critical decisions makers and disseminated at the righttime with suitable tools. The main goal of thefourth phase is to share the knowledge analyzedin decision-making processes. In addition, theutilization phase cannot be effective if one ofthe earlier phases in the BI cycle has failed.Therefore, feedback is critical to optimize eachphase of the BI cycle.To define the performance of the BI process and find out which aspects of BI should bemeasured, the Performance Prism approach isapplied to the BI process in four steps below.Step 1: Stakeholders’ Satisfaction andContribution. The first questions to ask are:Who are the key stakeholders of the BI process? and What do they want and need? Another question to ask is: What does the BI processneed from these stakeholders on a reciprocalbasis? The two key stakeholders of a BI processinclude the users of the intelligence and the financial sponsor or manager of the BI activity(e.g., the chief financial officer). User satisfaction requires that users have useful informationthat is accurate, relevant, and timely. An effective BI process requires the definition of information needs by the users. In addition, to beable to derive any benefits from BI, users mustactively utilize the intelligence. The sponsor ofthe BI activity wants concrete benefits (e.g.,cost savings) to be achieved using BI and coststo be low. On the other hand, the BI process requires adequate resources and managerial support in order to function.Step 2: BI Strategy. The second step toconsider is what strategies are required to satisfy the needs and ensure the contribution of thestakeholders identified. This deals with choosing how to try to achieve the goals related tothe users of the intelligence. The followingsteps are determined based on the decisionsmade in this phase. For the purposes of this example, we shall assume that the four-phase BIprocess described above will be used as thestrategy for achieving the goals of BI.Step 3: BI Process. The next step is toconsider how to run the process to achieve thegoals. It is possible to assign objectives to eachphase of the BI process. For example, the criteria for the “identification of information needs”phase include efficiency of the identificationprocedure and relevance, amount, and necessity of needs identified. Similarly, the criteria ofan efficient and effective “information acquisition” phase include costs of gathering information, reliability and quality of informationcompiled, and scope and timeliness of the information collection procedure. For “information analysis,” they include reliability and38 W W W . I S M – J O U R N A L . C O MW I N T E R 2 0 0 6BUSINESS INTELLIGENCEaccuracy of analysis, elimination of useless information, quality of information analyzed, andtime and costs associated with the analysis. Finally, the criteria for an efficient and effective“storage and information utilization” phase include accuracy of the knowledge, usability andcosts of the knowledge storage tools, efficiencyand cost of knowledge dissemination, and timesavings and benefits achieved by the BI process.Step 4: Capabilities. The final step to consider is which capabilities are required to be inplace to allow the operation of the processes.Capabilities could include the resources available, the BI personnel’s competencies, and theavailability of suitable information technology.Naturally, the capability requirements arestrongly related to the particular phase of theprocess in question; for example, the amount oftime and money or the suitability of technologiesand skills available for information acquisition.It is also possible to identify various successfactors of the BI process phases and then define the performance measures of each factor.This is discussed in the following section. Itshould be noted that some phases have overlapping success factors and not all of the factors should necessarily be measured. Forexample, in balanced business performancemeasurement, only the most important factorsare measured (see Kaplan and Norton, 1996).Designing BI Performance MeasuresAfter we have identified the BI process successfactors to be measured, we can define performance measures. Performance measures areusually designed for a specific situation, although some measures are quite standard (e.g.,ROI). Performance measures should be valid,reliable, relevant, practical, and well suited tothe particular measurement situation (Hannula, 1999; Lönnqvist, 2004; Neely et al., 2002).According to Kaydos (1999):anything can be measured to a usefuldegree, especially in a business environment. The real question is not whethersomething can be measured, but whether it is worth the effort and money to doso.There are also different ways to measure aparticular factor. For example, Figure 1 showsdifferent ways of measuring the “Effects of Intelligence Produced.” The first and probablymost common way is to use a direct measurement approach and objective measures. Oneexample is to quantify the strategic deals thatthe BI (CI) team has been involved in and compare the win/loss ratios to those deals in whichthey were not involved — an example mentioned earlier. A second way would be to designa direct and subjective measure, such as by asking managers to assess the effect of BI on theirdecision making.However, direct measurement is not alwaysfeasible. Indirect measurement (Kaydos, 1999)can be carried out by first identifying a factorthat is somehow associated with the primaryfactor and then measuring it. For example, “Utilization of Intelligence” is an indirect factor related to the “Effects of Intelligence Produced”— utilization of the intelligence is a key precondition of the effects of BI. Also, “Utilizationof Intelligence” can be measured both objectively and subjectively. An example of an indirect and objective measure is calculating theuser activity on some key BI database. An indirect and subjective measure of the utilization ofthe intelligence can be formed using a surveyin which users are asked how often they utilizethe intelligence in their work.All of the performance measures presentedin Figure 1 are nonmonetary (or nonfinancial).However, monetary performance measures arealso quite common and have many positivecharacteristics. For example, it is possible tocompare measurement results of different issues because the same measurement unit (i.e.,a currency such as the euro) is used. However,as discussed earlier, in many situations it is difficult to apply monetary measures becausesome of the issues are nonfinancial in nature.SUMMARY AND CONCLUSIONSThere are two main purposes for measuring BI:to prove BI is worth the effort and to help manage a BI process. Many measures can be usedfor these two purposes. However, the purposesare partially overlapping. A large number of thecurrent measures found in the literature focuson justifying the value of BI. This is an important issue when the usefulness of BI is underinitial consideration and also later when thereis a need to determine if BI continues to provide valuable results. Many measures of the effects of BI seem problematic. However, thereare also many that seem useful.Measures intended for managing the BI process are applicable for the continuous improvement of the process. The literature discussesfewer measures for the BI process than for the effects of BI. In many cases, certain success factorshe firstand probablymost commonway is to usea directmeasurementapproach andobjectivemeasures.TI N F O R M A T I O N S Y S T E M S M A N A G E M E N T 39W I N T E R 2 0 0 6BUSINESS INTELLIGENCEof the BI process have been identified, but theactual performance measures have not beenpresented. However, metrics related to the different phases of a BI process seem easier tomeasure than the effects. Therefore, it wouldseem to be easy to design new measures forthis purpose.The measurement approaches presented inthe recent business performance measurementliterature also seem quite useful for measuringBI. A balanced performance measurement system could cover both the effects of a BI processas well as the important factors of the process.However, we currently lack case studies to assess whether balanced performance measurement could be applied to the BI process.On the basis of our findings, we encouragecompanies to start applying the measurementof BI in practice. The balanced performancemeasurement of BI is proposed as a potentialmeasurement approach. Future researchers areencouraged to report practical experiences forthe purposes of learning about possible problems and improving on the measurement ofBI. ▲ReferencesAzoff, M., Charlesworth, I. (2004), The NewBusiness Intelligence. A European Perspective,Butler Group, White Paper.Brooking, A. (1996), Intellectual Capital. CoreAssets for the Third Millennium Enterprise,International Thomson Business Press, London.Casado, E. (2004), Expanding Business IntelligencePower with System Dynamics, in Raisinghani,M. (Ed.), Business Intelligence in the DigitalEconomy: Opportunities, Limitations andRisks, Idea Group Publishing, Hershey, PA.Cottrill, K. (1998), Turning Competitive Intelligenceinto Business Knowledge, Journal of BusinessStrategy,Vol. 19, No. 4, pp. 27–30.Davison, L. (2001), Measuring CompetitiveIntelligence Effectiveness: Insights from theAdvertising Industry, Competitive IntelligenceReview, Vol. 12, No. 4.Degraeve, Z., Labro, E., Roodhooft, F. (2004), TotalCost of Ownership Purchasing of a Service: TheCase of Airline Selection at Alcatel Bell,European Journal of Operational Research,Vol. 156, No. 1, pp. 23–40.Fuld, L. (1995), New Competitor Intelligence. TheComplete Resource for Finding, Analyzing,and Using Information about YourCompetitors, Wiley, New York.Gartz, U. (2004), Enterprise InformationManagement, in Raisinghani, M. (Ed.), BusinessIntelligence in the Digital Economy:Opportunities, Limitations and Risks, IdeaGroup Publishing, Hershey, PA.Ghoshal, S., Kim, S. K. (1986), Building EffectiveIntelligence Systems for Competitive Advantage,Sloan Management Review, Vol. 28, No. 1, pp.49–58.Gilad, B., Gilad, T. (1986), SMR Forum: BusinessIntelligence — The Quiet Revolution, SloanManagement Review,Vol. 27, No. 4, pp. 53–61.Hannula, M. (1999), Expedient Total ProductivityMeasurement,Acta Polytechnica Scandinavica,Industrial Management and BusinessAdministration Series, No. 1.Hannula, M., Pirttimäki, V. (2003), BusinessIntelligence — Empirical Study on the Top 50Finnish Companies, Journal of AmericanAcademy of Business, Cambridge, Vol. 2, No. 2,pp. 593–599.FIGURE 1 Different Ways to Measure a Success Factor of a BI Process(Amended from Kemppilä and Lönnqvist, 2003)Success factor(effects of intelligencePhenomenon produced)Direct(effects ofintelligence)Indirect(utilization ofintelligence)Objective(BI’s effect on win/loss ratios ofstrategic deals)Subjective(manager’sassessment)Subjective(survey amongusers)Objective(databaseusage)Data qualityMeasurementapproach40 W W W . I S M – J O U R N A L . C O MW I N T E R 2 0 0 6BUSINESS INTELLIGENCEHerring, J. (1996), Measuring the Value ofCompetitive Intelligence: Accessing &Communicating CI’s Value to YourOrganization, SCIP Monograph Series,Alexandria, VA.Hoadley, G. (2004), Measuring the Unknown,Competitive Intelligence Magazine, Vol. 7, No.4, pp. 25–28.Information Builders (2004), Internet site ofInformation Builders, Visited September 9th2004, [http://www.informationbuilders.com].Kahaner, L. (1996), Competitive Intelligence. Howto Gather, Analyze, and Use Information toMove Your Business to the Top, Simon &Schuster, New York.Kaplan, R. S., Norton, D. P. (1996), The BalancedScorecard. Translating Strategy into Action,Harvard Business School Press, Boston,Massachusetts.Kaydos, W. (1999), Operational PerformanceMeasurement. Increasing Total Productivity,St. Lucie Press, Boca Raton, Florida.Kelly, M. (1993), Assessing the Value of CompetitiveIntelligence, Journal of AGSI, November.Kemppilä, S., Lönnqvist, A. (2003), SubjectiveProductivity Measurement, Journal ofAmerican Academy of Business, Cambridge,Vol. 2, No. 2, pp. 531–537.Kilmetz, K., Bridge, R. S. (1999), Gauging theReturns on Investments in CompetitiveIntelligence: A Three-Step Analysis for ExecutiveDecision Makers, Competitive IntelligenceReview, Vol. 10, No. 1.Lönnqvist, A. (2004), Measurement of IntangibleSuccess Factors: Case Studies on the Design,Implementation and Use of Measures,TampereUniversity of Technology, Publication 475,Tampere.Marin, J., Poulter, A. (2004), Dissemination ofCompetitive Intelligence, Journal ofInformation Science,Vol. 30, No. 2, pp. 165–180.Neely, A. (1999), The Performance MeasurementRevolution: Why Now and What Next?International Journal of Operations &Production Management, Vol. 19, No. 2. pp.205–228.Neely, A., Adams, C., Kennerley, M. (2002), ThePerformance Prism. The Scorecard forMeasuring and Managing Business Success,Prentice Hall.Nelke, M. (1998), Knowledge Management inSwedish Corporations. The Value ofInformation and Information Services,Swedish Association for Information Specialists,Documentation, Stockholm.Pirttimäki, V., Hannula, M. (2003), Process Modelsof Business Intelligence,Frontiers of e-BusinessResearch 2003, Conference Proceedings,Cityoffset Oy, Tampere, pp. 250–260.Sawka, K. (2000), Are We Valuable? CompetitiveIntelligence Magazine, Vol. 3, No. 2.Simon, N. J. (1998), Determining Measures ofSuccess, Competitive Intelligence Magazine,Vol. 1, No. 2, pp. 45–48.Simons, R. (2000), Performance Measurement &Control Systems for Implementing Strategy,Prentice Hall, New Jersey.Solomon, M. (1996), The Intelligence Asset-BuildingProcess, Competitive Intelligence Review, Vol.7, No. 4, pp. 69–76.Sveiby, K.-E. (1997), The New OrganizationalWealth: Managing and Measuring KnowledgeBased Assets, Berrett-Koehler Publishers Inc.,San Francisco.Thomas Jr., J. H. (2001), Business Intelligence —Why? eAI Journal, July, pp. 47–49.Tuomela, T.-S. (2000),Customer Focus and StrategicControl. A Constructive Case Study ofDeveloping a Strategic PerformanceMeasurement System at FinABB, Publicationsof the Turku School of Economics and BusinessAdministration, Series D-2:2000.Vibert, C. (Ed.), (2004), Competitive Intelligence, aFramework for Web-Based Analysis andDecision Making,Thomson South-Western.Viva Business Intelligence Inc. (2000), Measuringthe Benefits of Business Intelligence, Pro-HowPaper, Helsinki, Vol. 2.Williams, S., Williams, N. (2004), Capturing ROIthrough Business-Centric BI DevelopmentMethods, DM Review,August 2004.