case studies — a telecommunications firm

In late 2001, a leading telecommunications company ("Telco") approached OpinionLab about delivering an online customer satisfaction measurement system that continuously monitored the performance of its website improvement process as well as benchmarked its website's user experience against other websites.

Telco had investigated other vendors, but selected OpinionLab's OnlineOpinion system because it:

implementation

Telco began by implementing OnlineOpinion's Standard data collection system (see Exhibit 1) across its entire website. Telco chose this implementation because it placed a very low burden on its Web visitors, thereby maximizing the volume of quantitative data Telco received. Telco wanted as much data across as many Web pages as possible so it could easily identify and prioritize where user issues and opportunities existed.

A few months after its initial implementation, Telco decided it had adequately identified and prioritized the pertinent user issues and opportunities, so it chose to switch to OnlineOpinion's Direct to Comment Card version of the data collection system (see Exhibit 2). Telco hoped to increase the volume of qualitative feedback collected and planned to use these comments to diagnose user issues and opportunities more granularly. (See OpinionLab's paper: Which "O" is Right for You?)

With both of these implementations, Telco chose to employ OnlineOpinion's standard Comment Card (see Exhibit 3) so it could benchmark its content, design, usability and overall performance against OpinionLab's standardized norms.

results

Telco immediately began to see a wide dispersion of user opinions across its Web pages (see Exhibit 4). "Of those issues identified by the OnlineOpinion system, approximately 70% were issues already known by us," said Telco's Market Research Group Manager. "However, approximately 30% were brand new issues. And with these new issues, we found that the comments collected through OnlineOpinion were very detailed and constructive. The comments were what helped us address the issues and opportunities our users faced."

Telco soon began to see patterns in its data (see Exhibit 5). "Each time we launched new functionality on our website, we saw a downward slope in OpinionLab's Site Indices," said Telco's Marketing Programs Manager for eCustomer Management. "We believe this was because our visitors were a bit taken aback by the changes. But as users learned the new functionality and become comfortable with it, the Site Indices begin to slope upwards." Telco's objective is to ensure that its post redesign indices exceed its pre-redesign baseline.

"In the end, our users are pleased with all the changes we have made to our website because they know we are launching new functionality based on their opinions," said Telco's Marketing Programs Manager for eCustomer Management.

Lastly, Telco did notice a substantial increase in non-website oriented comments when it switched from the Standard version of the OnlineOpinion data collection system (Exhibit 1) to the Direct to Comment Card version (Exhibit 2). (See OpinionLab's paper: Which "O" is Right for You?) Although the initial increase caught Telco a bit off-guard, it quickly established the appropriate internal procedures to respond to the volume. More importantly, by analyzing the non-website oriented comments, Telco found an effective means to quickly learn of issues in offline areas of the business that needed to investigated.

going forward

Telco currently has a number of business units employing the OnlineOpinion system. OpinionLab has begun to deliver cross business unit comparisons for Telco. Additionally, Telco has begun to leverage OpinionLab's Dynamic Comment Card to further explore page-specific issues (see Exhibit 6).

typical monthly activity

	Timeframe	March 1, 2003 - March 31, 2003
	Total ratings collected during timeframe	900 ratings
	Number of pages rated	83 pages
	Number of content, design, and usability ratings	711 C, D and U ratings (79% of ratings)
	Number of ratings with comments	383 comments (43% of ratings)
	Analysis tools used	24/7 Metric Toolkit, O-Trend and Page Viewer