In this article I wrote for SearchDataManagement.com, I offer advice for organizations trying to settle on BI analytics tools: Figure out which features are must-haves, versus nice-to-haves.

There are four articles in this series: Understanding BI analytics tools and their benefits, Business use cases can determine the right BI analytics tool, How to evaluate and select the right BI analytics tool and Comparing BI analytic tools from the leading vendors.

How to evaluate and select the right BI analytics tool

Selecting the business intelligence analytics tool that’s the best fit for your enterprise is critical to the success of your BI project. This process includes gathering and prioritizing BI requirements, as well as determining use cases and tool categories and styles.

In this article, we will define the key features and functions used to evaluate and select a BI analytics tool. The criteria can be utilized when it’s time to create your request for proposal (RFP).

BI analytics tool selection and evaluation criteria

Although industry analyst product reviews can be a good source of introductory research, particularly if you aren’t familiar with the overall market, these reviews are often oriented toward selecting the product with the most features. Your organization should instead select the BI analytics tool that’s the best fit for its use cases, meets its budget and can be implemented given its resources and skills. To simplify the process, you may wish to classify the features and functions to consider as must-haves, nice-to-haves and will-not-use:

• Must-haves. This classification should be unambiguous. In other words, if the product doesn’t have this particular feature, it’s eliminated from further consideration.
• Nice-to-haves. Although nice-to-have features aren’t required, they’re often the differentiators in selecting a product.
• Will-not-use. Many BI analytics products have a laundry list of features that your company may never use. In that case, don’t waste time examining those aspects of products during the evaluation process.

Caveat: Although a product may have the features that meet your criteria, there may be special considerations for how those features are obtained. For example, in order for a BI analytics tool to provide these features, are any of the following required?

1. Custom coding.
2. The purchase of an add-on product from a third party.
3. A specific product edition, such as an enterprise versus a basic edition.

These conditions all mean additional time and expense. To ensure an objective evaluation and avoid surprises if this product is selected, you need to determine how to identify and factor the additional time and cost into product comparisons.

Overall BI features: The must-haves

The following are often must-have features for organizations:

Data sources. Access to various databases and file types such as comma-separated values file, text, Excel and XML are basic staples of all BI products. Increasingly, BI analytics tools are providing access to specific applications such as Salesforce or NoSQL databases. Your specific needs will determine if these features are must-haves.

Data filters and drill-down. The product should enable the contents in a tabular report or visualization to be filtered by data values. Filtering is provided by features such as pull-down lists, search filters and slicers. The product should also allow the user to drill-down from summarized to more detailed data and then drill up (i.e., go back to where they started); this is essential in both tabular reports and visualizations.

Web-based client user interface. The product’s client user interface for the BI consumer-role should be Web-based. This has become an industry best practice, as it’s more cost- and resource-effective for administration, support and deployment than desktop-based interfaces. It’s a nice-to-have feature if the BI application creator and administrator roles are also Web-based versus a desktop-based client application.

Independent and interconnected mash-ups. When the BI style enables multiple visualizations, including tabular reports, to be displayed on a single screen, the software should allow for these visualizations to be either independent of each other or interconnected. If they’re interconnected, data filters and selection affect all visualizations; for example, if a particular attribute is selected, all visualizations share that attribute.

Visualizations. The BI analytics tool must provide bar, line, pie, area and radar chart types, as well as the ability to mix and match various combinations.

Security. All BI products must require both user and user role-based security, designating who can create, modify, publish, use and administer the BI applications. You may require the BI product to integrate with operating system or other pre-existing security applications. BI security often involves using the product’s security along with a combination of mechanisms from operating systems, networks, databases and the source system.

Microsoft Office Data Exchange. The product must be able to import and export data with Microsoft Office products, especially Microsoft Excel.

Print and export. The product must allow for print visualizations and tabular reports to be exported to PDF or other graphics. Tabular reports need to be exportable to text files at a minimum and, preferably, to spreadsheets.

Must-have features specific to self-service BI uses cases

There are several must-have features that are specific to self-service BI use cases. These are unique because they provide more data management functionality for the business person creating an analytical application than for an information consumer who is relying on pre-built BI applications with pre-built integrated data. These features include:

Select data for analysis. The BI analytics tool must enable the user to select the data used in analysis and present it as a pivot table-style interface where dimension attributes are placed in rows and columns, measures are selected and filters are applied.

Data blending. The product must permit the user to blend data from various data sources. This includes accessing the data and mapping or creating relationships with data from multiple sources.

Create measures. The product must enable the user to create and save measures or calculations for use in analysis. These are also referred to as performance measures or key performance indicators.

Create hierarchies. The product must allow the user to create dimensional hierarchies, such as by geography or product, to group and summarize data. This establishes the drill-down paths.

Save queries and analysis. The product should enable the BI user to save the data filters, selections and drill-down paths used in an analysis so they can be reused.

Overall BI features: The nice-to-haves

These features often are the criteria that become the differentiators in selecting BI products:

Create and publish by business users. The product enables the user to save and share his or her analysis with other BI consumers.

Context-based filters. Filters will list only choices that have values given the current selection of facts and dimensions.

Context-based visualizations. Only visualizations or chart types that are relevant to the data selected will be listed as options.

Advanced visualizations. More advanced visualizations include heat maps, scatterplots, bubble charts, histograms, geospatial mapping and combinations of each of these, such as bubbles on a map. The best mapping capabilities will leverage city, state and country attributes in your data, rather than force the inclusion of longitude and latitude.

Collaboration and social interaction: The BI analytics tool enables a business community that can share and discuss their analysis. This would include annotating analysis to share observations and social media enabling discussion threads or chats.

Storyboarding: Business analysis often involves a process or workflow of analyzing different data from different perspectives. Storyboarding enables a series of reports or visualizations to be tied together in a workflow that can be shared.

Microsoft Office real-time integration. Beyond simple import and export, the product should provide real-time integration with Microsoft Office products, which enables business people to embed analytics from the BI analytics tool into a PowerPoint or Excel presentation, for example, and refresh it automatically as the data is updated.

Mobile version. The BI analytics tool should be able to differentiate between viewing BI applications on a Web browser on a mobile device versus a mobile BI application.

In-memory analytics. The product should pull data into an in-memory or locally cached data store. In-memory columnar is an increasingly popular feature enabling very fast analytics once the data is loaded.

Offline updates. The BI analytics tool, when it stores its own copy of the source data in an OLAP cube or in-memory columnar data store, should allow users to schedule automatic data updates.

Performance monitoring. BI products that monitor report and data usage enable a BI group to improve analytical performance for the business, eliminating bottlenecks and assessing infrastructure needs.

BI platform administration. Although all BI tools should provide code and version management, there are many application development features, such as team development and user administration, that are useful for larger BI deployments.

Other considerations

Establishing the scope of your BI project in terms of how many people will use it and what data will need to accessed is the foundation for creating the selection criteria. Monetary considerations such as the anticipated budgets for the initial project, sustaining the BI program and expansion the following year also are key factors in selection criteria. Although you wouldn’t go to an architect and just ask to design a house without talking about the size, type of rooms and budget, too many BI evaluation projects have started without any scoping or budgetary boundaries, resulting in time wasted in examining BI products that don’t fit their need or budget.

The following are often included in evaluation criteria, but since they’re very subjective, it’s important to provide clear definitions of what the comparison will be based on:

• Ease of analytical use. There should be different criteria defined for each type of user: information consumer, business analyst and IT.
• Ease of creating BI applications. There should be different criteria for each type of analytics creator: business analysts and IT.
• Speed of access. Query performance will vary based on the complexity of queries and amount of data involved. Dashboards with multiple visualizations will need to get query results from many queries. The best practice is to create several pre-built query scenarios and compare how each product performs on these specific examples. The worse practice is to just have evaluators arbitrarily rate the speed.
• Scalability. The best practice is to establish a testing environment to determine scalability in terms of both the number of concurrent users and data metrics such as volumes, variety and veracity.
• Platform. Do you prefer on premises versus cloud, open source versus commercial software, operating systems or other infrastructure options?
• Training. There should be separate criteria for BI user versus administration training. Training may include in-person classes, online classes (live or pre-recorded) or Web recordings for specific features or processes.
• Documentation. There should be separate criteria for BI user online help, versus technical documentation.

Once you’ve created your evaluation criteria, it’s time to select a shortlist of product candidates and proceed with your RFP process. To help you create a candidate pool from which to choose, the final article in this series will examine the leading BI analytics tools on the market.