Download White Paper: Measuring Customer SatisfactionCreating a Reliable Survey

Abstract

Every for-profit, for-cause and even governmental organization cares about customer satisfaction.  When someone wants to measure customer satisfaction the natural instrument is a survey, but we all know that surveys can be problematic (more about this later).  This case study documents a process for measuring customer satisfaction based on an implementation project with a financial-services Client.  The survey we created was robust, had a high response rate, and provided valuable insight to the management team.  Best of all, the CEO is the first person to get the data and is responsible for follow-up; he did not delegate to the manager of client service.  This sent a powerful message to clients and employees that client satisfaction starts at the top.

Background

Our Client is a mid-size financial institution that prides itself on client service.  They feel they are differentiated in this aspect and charge a premium for this service.  However, their high opinion of their great service was all based on anecdotal feedback from clients.  The CEO wondered if he only received the good news and if they were as good as they thought.

They originally contacted Supply Velocity to create an annual survey that would go out once a year to all clients.  We discussed what they wanted to measure and realized that this instrument would not be effective.

What is wrong with surveys

The “annual survey” has many problems, some are rather obvious and others require an understanding of survey theory.  The obvious problems include the measurement occurring during some relatively short period prior to the survey, the typically low response rates and the mindless or blank responses due to the number or type of questions.  Then, and most importantly, the survey results are usually looked at, discussed in management meetings and then set aside, with no definable actions coming from the data.  (We would argue that this is probably best anyway, because the data is often unreliable!)

The second problem is related to two concepts that are critical when designing surveys; measurement error and non-response bias.  Measurement error means that different people with the same opinion could answer differently.  This can occur because the question is vague or the scale is not clear.  (Have you ever been confused by what “above average” means?)  Non-response bias is the concept that the people that don’t respond may feel differently (be biased negatively or positively) than the people that responded.  With response rates of less than 10%, the company sending out a survey should probably worry about how the other 90% feel.

We attempted to solve all of these problems using the process described in the next section.

Creating a robust and easy-to-complete survey

  • Surveying should be a continuous process
  • Determine the high level question you are trying to answer
  • Less is more when it comes to surveys
  • Carefully word the questions
  • Define the scale in very specific terms
  • Act on the data

Monthly phone surveys

When implementing a customer satisfaction survey we feel that the best method is to survey continuously.  This requires surveys to go out to customers every month.  You can differentiate customers based on size and survey your “A” customers quarterly, “B” customers twice a year and “C” customers once a year.  Or you can divide your entire customer base up into 12 equal groups so every customer gets surveyed once a year.  We recommend using a phone-survey service.  The internet is revolutionary, but for surveys it is too easy to ignore.  Our phone-survey partner has representatives that call everyone on the customer list, usually leaving one voicemail and then trying a defined number of times to make contact.  (Another benefit of this process is you can find out that your key contact may have left the company, or you don’t have their correct phone number.  It also requires you to really think about who from your customer should be answering the survey; the owner, CEO, CFO, etcetera).  Our financial institution Client decided to survey all of their clients once a year.

What question are you trying to answer?

So often companies start the survey process by throwing out lots of questions they want answered.  It is important to first think about the top-level question.  If there was a single question you wanted answered by your customers, what would it be?

Our Client brainstormed the following 10 questions:

  1. Do clients think we are a great financial intuition?
  2. Are clients thrilled with our service?
  3. Are we better than competitors?
  4. Are we as good as we think we are?
  5. What can we do better (blind spots)?
  6. Can we elevate the client experience?
  7. Are we like Nordstrom?
  8. Are we worth a premium? 1
  9. Is it the people or the institution?
  10. Do we have a high level of quality in our service?

To simplify the next step the team voted this list down to two questions:

  • What can we do better (blind spots)?
  • Are we worth a premium?

We felt that the “blind spots” would come out in the process, so we focused on “are we worth a premium?”  (Remember that this may not be applicable to your firm.  Our Client prided itself on client service and being a “high touch” service provider.)

Carefully word the questions (and less is more)

If you are going to be contacted by phone, and asked survey questions, the phone call better be fast.  Therefore, we set the maximum number of questions at five.

The first step was to create questions or statements that that helped us define what a premium is worth.  The team came up with the following five definitions of premium financial services:

  • A trusted advisor
  • High quality service
  • Few errors
  • Timely response for info/service, availability/access to people/team, proactive
  • Resolve problems quickly and painlessly
  • Clients feel special
  • Total experience is wonderful

These statements turned into the five questions on the survey:

  • Rate your primary contact
  • Rate your experience with our firm
  • How many errors have you experienced over the last year?
  • If we made an error how have we done resolving the problem?
  • What can we do better?

Define the scale in very specific terms

One reason that firms get inconclusive results from surveys is the vague definition of the scale (the words associated with 1 through 5).  Often 1 = poor, 2 = below average, 3 = average, 4 = above average and 5 = outstanding.  This type of scale creates tremendous opportunities for measurement error.  As asked earlier, “what exactly does above average mean?”

The team took two days to determine a very specific scale for our four questions.  Below is the result.  As you observe the scales you will quickly notice the care that we took to be as specific as possible.  We used “below average” as our worst response, but made the others real words that people use to describe relationships with their financial services provider.  Instead of “outstanding” we used the words “trusted advisor”, “strategic partnership”, “zero errors”, and “fast and painless”.  Note that we ended up using a four-point scale (versus the normal five-point scale).  This was because we felt the words that describe and 3 and a 4 were too similar and could confuse clients.  Keeping the words clear and specific was more important than trying to use the symmetrical five-point scale.

What describes your primary contact at our firm?

  1. Below average
  2. No different than others
  3. Helpful
  4. A trusted advisor

How would you describe your client experience?

  1. Below average
  2. No different than others
  3. Helpful
  4. Strategic partnership

How many errors have you experienced over the last year?

  1. Three or more
  2. Two
  3. One
  4. None

If you had an error, how would have we done resolving these errors?

  1. It is painful
  2. Some hassles, could do better
  3. Acceptable
  4. Fast and painless

What can we do better?  (This question was open-ended and didn’t have a scale.)

Act on the data

Each month 1/12 of this firm’s clients receive a call from the phone-survey provider.  Approximately 70% of their clients respond to the survey.  The results go directly to the CEO.  In addition, if any client answers any question with a 1 or 2, that client goes on a “priority” list for the CEO to contact immediately and help resolve a problem or prevent that client from going to a competitor.  The results are graphed and shared with all employees, so negative trends, if they occur, can be acted on quickly.

"*" indicates required fields

Ask us your Lean, Six Sigma, or Supply Chain question.

We're the experts! We'll get back within 48 hrs. with a valuable response.

This field is for validation purposes and should be left unchanged.

“Supply Velocity has helped Clean succeed in a number of ways. First, they provided the education that kicked off our Lean Process Improvement journey. They also facilitated a number of large supply chain and process improvement projects including: 1) building quality into our direct-labor productivity incentive program, 2) designing the layout of our garment warehouse, and 3) streamlining our route service process.

We have integrated the methods that Supply Velocity taught us into our management and strategic planning. In the process our quality measurement has improved 22% from 2013 to 2015, we have reduced required annual labor by 2200 hours from the garment facility layout project and we’ve seen 50% decrease in error rate. Obviously the numbers speak for themselves, but just as important, Supply Velocity has been fun to work and have become true partners. They have “taught us how to fish” so our internal teams are able to implement change on their own, with the skills we learned from Supply Velocity. This relationship has been invaluable.”

Jeff Lazaroff, Senior Vice President, Clean Uniform
“Anheuser-Busch Precision Printing had been implementing Lean Operations on our own for the past two years. We needed to move faster and partnered with Supply Velocity, Inc. Through Supply Velocity, Inc.’s mathematical workflow balancing and visual management tools, dramatic improvements were achieved. The entire converting operation was rearranged based on Lean principles. The result is a 20.6% productivity improvement, enabling us to operate with 23 fewer people in production.”
Rich Lavosky, General Manager, Anheuser-Busch Precision Printing
“We are pleased that Essex selected Supply Velocity, Inc. as our Lean Implementation Partner. At one facility, we have saved over $350,000 in work-in-process inventory, reduced throughput time from 2 weeks to minutes and increased inventory turns 3 to 8 times per year. All these results are in just 6 months. Our return of investment is very high.”
Terry Etter, Vice President of Operations , Essex Medical Systems
“Isolating a problem, finding short, and long term solutions with measurable results is what was promised and results is what was delivered by Supply Velocity. Upon launch of the Lean Six Sigma Selling System, we knew more about our customers, our products, and were able to create a solid plan to increase sales of our most profitable products. Within months of implementation, our booked sales jumped 60% and our most valued customers were getting direct, active, and calculable attention.”
Mark A. Presker, General Manager, Architectural Millwork of St. Louis
“We used Supply Velocity to rethink our sales process. By analyzing the entire process we found wasted time in our Sales, Admin and Operations departments. Streamlining this process created extra time for each Sales Rep, allowing them to spend more time with Customers and increase the value we add. Gross profit margins are up 40%!

We are now using Supply Velocity to help us rethink our entire Strategic Plan.”

Jeff Reitz, Vice President, Central States Bus
“Supply Velocity is driving instrumental change in our inventory management processes. This is critical for us to be competitive in a supply chain environment with numerous disruptions. They are making change happen, which can be challenging in a 182 year old organization.”
Jim Carroll, Executive Vice President Operations, Schaeffer Manufacturing
“We engaged with Supply Velocity to help us embed process improvement at all levels of the business. Our team learned from Mitch to let the data drive decisions, to use Lean tools to help us see our processes critically and objectively, and to create a control plan to manage all of the tasks that were the outcome of the data study.

The project turned out to be very significant to the company and most importantly, our customers. We reduced our customer wait times by 40%, and cut in half the labor cost to fulfill customer orders.

Some results are not able to be measured. However, as a result of this project, we have started to build a Lean mindset and culture, which is part of our strategic mission to save our customers money. Supply Velocity has been a valued partner in this mission.”

Dionne Dumitru, COO, Weekends Only

“Supply Velocity gave us the tools to analyze our business and processes based on the facts and numbers versus our perceptions. Our common quote was “Let the numbers lead us”. The key for our organization was how quickly we moved from classroom to actual project initiation. We were able to jump in, start using the tools and see a difference right away.

The get-into-action approach was good for our culture.”

Ned Lane, President, CeeKay Supply

“I am thrilled to provide this testimonial for Supply Velocity and their outstanding work in implementing Lean Warehouses and processes at Crescent Parts & Equipment through the COVID pandemic. With their data-first focus and Mitch’s exceptional coaching and experience, they transformed our business into a more supply chain-oriented organization, enabling us to grow while prioritizing employee safety and creating a better work environment. Supply Velocity’s expertise in Lean methodologies and their comprehensive evaluation of our customers have been instrumental in optimizing our operations and increasing customer satisfaction. We highly recommend Supply Velocity to any company seeking to implement Lean processes and enhance their supply chain efficiency.”

Josh Cole, Director of Supply Chain, Crescent Parts & Equipment
“Mitch Millstein and his team helped guide our shop fabrication division in the re-layout of our custom pipe and steel fabrication facility when we moved into a new building. It is not only the results but how he helped. We were involved in every step. I personally did time studies and was able to see the non-value added steps required to manufacture in our old layout. When we created our new layout, everyone was involved, from the executive team to our direct labor force. With Mitch’s help we increased our throughput by a 3x multiple, while providing more competitive prices to our clients as a result of the efficiency improvements.

This has enabled us to not only make more money but also to expand our commercial reach and serve more, and larger customers. I would recommend Supply Velocity to any company that wants to make improvement in supply chain and operations.”

Geoff Gross, President, Gross Mechanical
“Supply Velocity’s Lean Six Sigma System has given us new tools to help improve our sales performance. We have learned new ways to analyze our business which makes problem solving more accurate and has made us better leaders.”
Rachel Andreasson, Executive Vice President – Marketing, Wallis Companies
“In thirty years of hiring consultants, Supply Velocity, Inc. was the first to tell me what they were going to do, set a price they stuck to and substantially exceed my expectations. I have recommended them to friends and acquaintances. They were true partners in assisting with the turnaround of an acquisition we had been struggling with for two years.

Their math-based technology, solid down-to-earth facilitation skills, and positive, patient and enthusiastic attitude combined to make our implementation of Lean a very rewarding experience.

We increased our production by 50% in the first month of implementation and continue to see improvements. Improvements have not only been realized in productivity, but also in quality and morale. We have increased profitability by $2M on flat sales of $10M.

Based on Supply Velocity, Inc.’s integrity and our results, I will continue to refer them to others and utilize them in the future as we expand our company through acquisitions.”

Bill Gilbert, President, Fusion Coatings
“Supply Velocity has provided the technical expertise and political capital to move our project forward. They have just the right amount of push and the right amount of support. Supply Velocity has helped us make real changes to improve efficiencies in logistics without jeopardizing our performance. We’re happy and our customers are happy.”
David Walters, President, Hy-C
“We are using Supply Velocity’s Lean Six Sigma methods to analyze a variety of processes including rationalizing SKUs (stock-keeping-units). By using math to evaluate SKUs we took some of the emotion out of our decisions. We expect significant increases in sales and productivity from reducing poor performing SKUs.”
Mark Kelso, Director of Process Improvement, Save-A-Lot
“In the spring of 2003, the St. Louis Area Chapter of the American Red Cross engaged Supply Velocity, Inc. to perform a study and make recommendations to streamline office processes, maximize cash flow in purchasing and warehousing and restructure and enhance our maintenance department. Supply Velocity, Inc.’s process was methodical, flexible, staff-oriented, inclusive and, above all, trackable.

In the last six months, our Chapter has realized expense savings of over $380,000 annually, and significant improvement in intra-company service levels has been attained. Supply Velocity, Inc. will return to the Chapter periodically throughout the next 18 months to audit our newly implemented processes. We have been pleased with our results and Supply Velocity, Inc.’s professionalism.”

Joe White, CEO, American Red Cross – Saint Louis Chapter
“Many thanks to all three Supply Velocity presenters. I think you guys took a very difficult time for learning and capitalized on everyone’s time and training needs. I truly hope this helps your business as this was a considerable undertaking on the part of Supply Velocity. Outstanding!!”
Mark Holdinghausen, VP of Operations, DEMA Engineering
“For several years we have worked with Supply Velocity to support us with their expertise on Lean Operations and Supply Chain Management. Supply Velocity has helped us implement Lean, improve our inventory systems, and educate our people. They are professionals who are always available to help us as needed.”
Lorenza Pasetti, CEO, Volpi Foods

“In a time of volatile supply chain disruption, Supply Velocity is helping us develop Demand and Supply Planning processes to proactively tackle these new challenges. They are genuine partners, working with our team, facilitating and teaching.”

Jane Thrasher, Vice President of Supply Chain, Horizon Hobby

“Your process encourages this group to work together, better communicate and have fun doing it.”

George Edinger, President, C&R Mechanical
“In 2015 we began working with Dr. Mitch Millstein to optimize our inventory locations supporting e-commerce and in-store inventory needs. From this work we developed a new omni-channel warehousing and inventory plan that entirely redefined our approach to warehousing, inventory management, store distribution and fulfillment. As a result of the analyses by Dr. Millstein we have begun the move to an improved omni-channel design by reassigning MSAs to new warehouses, greater leveraging of in-store inventories to satisfy e-commerce demands, and exploring acquisitions of new warehousing space in strategic locations. We have already seen an improvement of $300,000 from both more efficient shipping strategies due to better inventory management.”
Rob Bowers, Vice President of Strategy, Total Hockey
“C&R was struggling with labor productivity. The construction crews were often missing materials that they needed to do their work. This caused significant idle time. Supply Velocity, Inc. and C&R used value stream mapping and visual management tools to make dramatic improvements in operations. Most importantly, two years later C&R is sustaining and improving on the implementation. C&R’s return on investment was 11:1. C&R had a record year last year in both sales and profits and would not have been able to pull it off without the changes Supply Velocity, Inc. helped us make.”
Tom Kuthe, V.P. Construction Operations, C&R Mechanical
“The role of the Erie Insurance Marketing Department has been evolving over the past several years – from a support role to a more critical role of driving growth in our organization. Because of our increased workload and desire to prioritize the most critical projects, we hired Supply Velocity to teach us the skills of Lean Six Sigma.

Participants included the Promotions, Market Research and Agency Licensing sections of the Marketing Department.

We learned valuable tools to help us to prioritize based on the voice of the customer.

I firmly believe these skills made a difference in how we work every day. We are moving new projects forward, eliminating or changing ineffective processes, and we are a much stronger department. We continue to use the tools to help us with our highly-complex and time-consuming projects. Supply Velocity helped us to accomplish our goals.”

Karen Rugare, Director of Marketing, Erie Insurance

“Supply Velocity created visibility within our Assurance Services Group… visibility of performance, Client-service, employee satisfaction and processing time. Using the Supply Velocity System, Audit Report Cycle time is down over 50%. We are using his strategies to create greater Client loyalty.”

Fred Kostecki, Partner-In-Charge, Assurance Services, Rubin Brown
“We used Supply Velocity on our Warehouse optimization project. One of the key characteristics of Supply Velocity is that they listened to our requirements and provided a clear path for our Warehouse processes using lean tools and our future sales growth as objectives.”
Haris Tokalic, President, Grand Rock, Inc.
“Myerson engaged Supply Velocity, specifically Ray Davis to visit our plant in Trinidad to conduct a two day assessment of our production procedures and provide us feedback on areas for improvement and where applicable, areas for future analysis.

Put simply, we got everything we paid for and in addition to more in depth analysis, we got specific tasks that were immediately actionable. Our local management team found Ray to be engaging, highly credible and insightful based on his wide experience. In other words the cultural differences and lack of specific industry knowledge weren’t impediments to things we could implement immediately and on our own. In a nutshell it was money well spent and will pay itself back many times over.”

Lee Hartwell CPA, Plant Manager, Myerson Tooth
“Closure Medical recently completed a major reorganization in order to enhance our ability to rapidly create innovative medical devices. We hired Supply Velocity to help us map out the process and service flows of the new organization that would maximize our product development process (PDP). In four weeks, Supply Velocity helped us envision a new PDP structure, develop measures and accountability for each step, and gain consensus within the organization. Supply Velocity’s focus on speed and accountability helped us complete a critical project in a timely fashion.”
Gabe Szabo, Vice President, Product Development, Closure Medical – A Division of Johnson & Johnson
“Our experience with Supply Velocity was one of the best values we have ever had from a consulting project. Cyril Narishkin brought a structured lean methodology, invaluable experience and engaging facilitation skills to help us streamline a very complex and disjointed sales order process. Just as importantly, our team now has the knowledge and process competencies to address other business improvement opportunities going forward.”
Mike Howard, CEO, Aspeq