Measuring Customer Satisfaction – Creating a Reliable Survey

White Papers

Download White Paper: Measuring Customer Satisfaction – Creating a Reliable Survey

Abstract

Every for-profit, for-cause and even governmental organization cares about customer satisfaction. When someone wants to measure customer satisfaction the natural instrument is a survey, but we all know that surveys can be problematic (more about this later). This case study documents a process for measuring customer satisfaction based on an implementation project with a financial-services Client. The survey we created was robust, had a high response rate, and provided valuable insight to the management team. Best of all, the CEO is the first person to get the data and is responsible for follow-up; he did not delegate to the manager of client service. This sent a powerful message to clients and employees that client satisfaction starts at the top.

Background

Our Client is a mid-size financial institution that prides itself on client service. They feel they are differentiated in this aspect and charge a premium for this service. However, their high opinion of their great service was all based on anecdotal feedback from clients. The CEO wondered if he only received the good news and if they were as good as they thought.

They originally contacted Supply Velocity to create an annual survey that would go out once a year to all clients. We discussed what they wanted to measure and realized that this instrument would not be effective.

What is wrong with surveys

The “annual survey” has many problems, some are rather obvious and others require an understanding of survey theory. The obvious problems include the measurement occurring during some relatively short period prior to the survey, the typically low response rates and the mindless or blank responses due to the number or type of questions. Then, and most importantly, the survey results are usually looked at, discussed in management meetings and then set aside, with no definable actions coming from the data. (We would argue that this is probably best anyway, because the data is often unreliable!)

The second problem is related to two concepts that are critical when designing surveys; measurement error and non-response bias. Measurement error means that different people with the same opinion could answer differently. This can occur because the question is vague or the scale is not clear. (Have you ever been confused by what “above average” means?) Non-response bias is the concept that the people that don’t respond may feel differently (be biased negatively or positively) than the people that responded. With response rates of less than 10%, the company sending out a survey should probably worry about how the other 90% feel.

We attempted to solve all of these problems using the process described in the next section.

Creating a robust and easy-to-complete survey

  • Surveying should be a continuous process
  • Determine the high level question you are trying to answer
  • Less is more when it comes to surveys
  • Carefully word the questions
  • Define the scale in very specific terms
  • Act on the data

Monthly phone surveys

When implementing a customer satisfaction survey we feel that the best method is to survey continuously. This requires surveys to go out to customers every month. You can differentiate customers based on size and survey your “A” customers quarterly, “B” customers twice a year and “C” customers once a year. Or you can divide your entire customer base up into 12 equal groups so every customer gets surveyed once a year. We recommend using a phone-survey service. The internet is revolutionary, but for surveys it is too easy to ignore. Our phone-survey partner has representatives that call everyone on the customer list, usually leaving one voicemail and then trying a defined number of times to make contact. (Another benefit of this process is you can find out that your key contact may have left the company, or you don’t have their correct phone number. It also requires you to really think about who from your customer should be answering the survey; the owner, CEO, CFO, etcetera). Our financial institution Client decided to survey all of their clients once a year.

What question are you trying to answer?

So often companies start the survey process by throwing out lots of questions they want answered. It is important to first think about the top-level question. If there was a single question you wanted answered by your customers, what would it be?

Our Client brainstormed the following 10 questions:

  1. Do clients think we are a great financial intuition?
  2. Are clients thrilled with our service?
  3. Are we better than competitors?
  4. Are we as good as we think we are?
  5. What can we do better (blind spots)?
  6. Can we elevate the client experience?
  7. Are we like Nordstrom?
  8. Are we worth a premium? 1
  9. Is it the people or the institution?
  10. Do we have a high level of quality in our service?

To simplify the next step the team voted this list down to two questions:

  • What can we do better (blind spots)?
  • Are we worth a premium?

We felt that the “blind spots” would come out in the process, so we focused on “are we worth a premium?” (Remember that this may not be applicable to your firm. Our Client prided itself on client service and being a “high touch” service provider.)

Carefully word the questions (and less is more)

If you are going to be contacted by phone, and asked survey questions, the phone call better be fast. Therefore, we set the maximum number of questions at five.

The first step was to create questions or statements that that helped us define what a premium is worth. The team came up with the following five definitions of premium financial services:

  • A trusted advisor
  • High quality service
  • Few errors
  • Timely response for info/service, availability/access to people/team, proactive
  • Resolve problems quickly and painlessly
  • Clients feel special
  • Total experience is wonderful

These statements turned into the five questions on the survey:

  • Rate your primary contact
  • Rate your experience with our firm
  • How many errors have you experienced over the last year?
  • If we made an error how have we done resolving the problem?
  • What can we do better?

Define the scale in very specific terms

One reason that firms get inconclusive results from surveys is the vague definition of the scale (the words associated with 1 through 5). Often 1 = poor, 2 = below average, 3 = average, 4 = above average and 5 = outstanding. This type of scale creates tremendous opportunities for measurement error. As asked earlier, “what exactly does above average mean?”

The team took two days to determine a very specific scale for our four questions. Below is the result. As you observe the scales you will quickly notice the care that we took to be as specific as possible. We used “below average” as our worst response, but made the others real words that people use to describe relationships with their financial services provider. Instead of “outstanding” we used the words “trusted advisor”, “strategic partnership”, “zero errors”, and “fast and painless”. Note that we ended up using a four-point scale (versus the normal five-point scale). This was because we felt the words that describe and 3 and a 4 were too similar and could confuse clients. Keeping the words clear and specific was more important than trying to use the symmetrical five-point scale.

What describes your primary contact at our firm?

  1. Below average
  2. No different than others
  3. Helpful
  4. A trusted advisor

How would you describe your client experience?

  1. Below average
  2. No different than others
  3. Helpful
  4. Strategic partnership

How many errors have you experienced over the last year?

  1. Three or more
  2. Two
  3. One
  4. None

If you had an error, how would have we done resolving these errors?

  1. It is painful
  2. Some hassles, could do better
  3. Acceptable
  4. Fast and painless

What can we do better? (This question was open-ended and didn’t have a scale.)

Act on the data

Each month 1/12 of this firm’s clients receive a call from the phone-survey provider. Approximately 70% of their clients respond to the survey. The results go directly to the CEO. In addition, if any client answers any question with a 1 or 2, that client goes on a “priority” list for the CEO to contact immediately and help resolve a problem or prevent that client from going to a competitor. The results are graphed and shared with all employees, so negative trends, if they occur, can be acted on quickly.

Learn more about our Business Process Improvement Services.

Client Voice

What Clients Say

View All Testimonials

"If you are looking for someone to help drive improvements, create repeatable processes and mentor younger staff in your facility then reach out to Supply Velocity. Your expertise and dedication have driven significant im..."

Tim Donohoe

COO, Chemline

"I am thrilled to provide this testimonial for Supply Velocity and their outstanding work in implementing Lean Warehouses and processes at Crescent Parts & Equipment through the COVID pandemic. With their data-first focus..."

Josh Cole

Director of Supply Chain, Crescent Parts & Equipment

"For several years we have worked with Supply Velocity to support us with their expertise on Lean Operations and Supply Chain Management. Supply Velocity has helped us implement Lean, improve our inventory systems, and ed..."

Lorenza Pasetti

CEO, Volpi Foods

"Supply Velocity is driving instrumental change in our inventory management processes. This is critical for us to be competitive in a supply chain environment with numerous disruptions. They are making change happen, whic..."

Jim Carroll

Executive Vice President Operations, Schaeffer Manufacturing

"In a time of volatile supply chain disruption, Supply Velocity is helping us develop Demand and Supply Planning processes to proactively tackle these new challenges. They are genuine partners, working with our team, faci..."

Jane Thrasher

Vice President of Supply Chain, Horizon Hobby

"Mitch Millstein and his team helped guide our shop fabrication division in the re-layout of our custom pipe and steel fabrication facility when we moved into a new building. It is not only the results but how he helped...."

Geoff Gross

President, Gross Mechanical

"In 2015 we began working with Dr. Mitch Millstein to optimize our inventory locations supporting e-commerce and in-store inventory needs. From this work we developed a new omni-channel warehousing and inventory plan that..."

Rob Bowers

Vice President of Strategy, Total Hockey

"Myerson engaged Supply Velocity, specifically Ray Davis to visit our plant in Trinidad to conduct a two day assessment of our production procedures and provide us feedback on areas for improvement and where applicable, a..."

Lee Hartwell CPA

Plant Manager, Myerson Tooth

"Supply Velocity has provided the technical expertise and political capital to move our project forward. They have just the right amount of push and the right amount of support. Supply Velocity has helped us make real cha..."

David Walters

President, Hy-C

"Our experience with Supply Velocity was one of the best values we have ever had from a consulting project. Cyril Narishkin brought a structured lean methodology, invaluable experience and engaging facilitation skills to..."

Mike Howard

CEO, Aspeq

"Supply Velocity has helped Clean succeed in a number of ways. First, they provided the education that kicked off our Lean Process Improvement journey. They also facilitated a number of large supply chain and process impr..."

Jeff Lazaroff

Senior Vice President, Clean Uniform

"We engaged with Supply Velocity to help us embed process improvement at all levels of the business. Our team learned from Mitch to let the data drive decisions, to use Lean tools to help us see our processes critically a..."

Dionne Dumitru

COO, Weekends Only

"Supply Velocity gave us the tools to analyze our business and processes based on the facts and numbers versus our perceptions. Our common quote was "Let the numbers lead us". The key for our organization was how quickly..."

Ned Lane

President, CeeKay Supply

"We used Supply Velocity to rethink our sales process. By analyzing the entire process we found wasted time in our Sales, Admin and Operations departments. Streamlining this process created extra time for each Sales Rep,..."

Jeff Reitz

Vice President, Central States Bus

"Supply Velocity created visibility within our Assurance Services Group... visibility of performance, Client-service, employee satisfaction and processing time. Using the Supply Velocity System, Audit Report Cycle time is..."

Fred Kostecki

Partner-In-Charge, Assurance Services, Rubin Brown

"Supply Velocity's Lean Six Sigma System has given us new tools to help improve our sales performance. We have learned new ways to analyze our business which makes problem solving more accurate and has made us better lead..."

Rachel Andreasson

Executive Vice President - Marketing, Wallis Companies

"The role of the Erie Insurance Marketing Department has been evolving over the past several years - from a support role to a more critical role of driving growth in our organization. Because of our increased workload and..."

Karen Rugare

Director of Marketing, Erie Insurance

"Isolating a problem, finding short, and long term solutions with measurable results is what was promised and results is what was delivered by Supply Velocity. Upon launch of the Lean Six Sigma Selling System, we knew mor..."

Mark A. Presker

General Manager, Architectural Millwork of St. Louis

"In the spring of 2003, the St. Louis Area Chapter of the American Red Cross engaged Supply Velocity, Inc. to perform a study and make recommendations to streamline office processes, maximize cash flow in purchasing and w..."

Joe White

CEO, American Red Cross - Saint Louis Chapter

"We are using Supply Velocity's Lean Six Sigma methods to analyze a variety of processes including rationalizing SKUs (stock-keeping-units). By using math to evaluate SKUs we took some of the emotion out of our decisions...."

Mark Kelso

Director of Process Improvement, Save-A-Lot

"Anheuser-Busch Precision Printing had been implementing Lean Operations on our own for the past two years. We needed to move faster and partnered with Supply Velocity, Inc. Through Supply Velocity, Inc.'s mathematical wo..."

Rich Lavosky

General Manager, Anheuser-Busch Precision Printing

"C&R was struggling with labor productivity. The construction crews were often missing materials that they needed to do their work. This caused significant idle time. Supply Velocity, Inc. and C&R used value stream mappin..."

Tom Kuthe

V.P. Construction Operations, C&R Mechanical

"Your process encourages this group to work together, better communicate and have fun doing it."

George Edinger

President, C&R Mechanical

"Closure Medical recently completed a major reorganization in order to enhance our ability to rapidly create innovative medical devices. We hired Supply Velocity to help us map out the process and service flows of the new..."

Gabe Szabo

Vice President, Product Development, Closure Medical - A Division of Johnson & Johnson

"We are pleased that Essex selected Supply Velocity, Inc. as our Lean Implementation Partner. At one facility, we have saved over $350,000 in work-in-process inventory, reduced throughput time from 2 weeks to minutes and..."

Terry Etter

Vice President of Operations, Essex Medical Systems

"In thirty years of hiring consultants, Supply Velocity, Inc. was the first to tell me what they were going to do, set a price they stuck to and substantially exceed my expectations. I have recommended them to friends and..."

Bill Gilbert

President, Fusion Coatings

"Many thanks to all three Supply Velocity presenters. I think you guys took a very difficult time for learning and capitalized on everyone's time and training needs. I truly hope this helps your business as this was a con..."

Mark Holdinghausen

VP of Operations, DEMA Engineering

"We used Supply Velocity on our Warehouse optimization project. One of the key characteristics of Supply Velocity is that they listened to our requirements and provided a clear path for our Warehouse processes using lean..."

Haris Tokalic

President, Grand Rock, Inc.

"I’d recommend Supply Velocity to organizations looking for a hands‑on, data‑driven warehouse assessment with actionable outputs - particularly those dealing with operational change, new systems, or scaling challenges."

Jayson Wickenkamp

Senior Vice President, Image Solutions