This month we celebrate 10 successful years of performance testing and availability monitoring for hundreds of customers over the course of thousands of projects. We are very proud of our accomplishments and we’ve made what we believe to be a real impact on the success of our customers’ business solutions. Over the years, we’ve had a multitude of exciting, frustrating, humorous and challenging “learning moments” that have helped us develop into the company we are today. I’d like to share some of the lessons we learned from these moments in hopes that you can apply them to your business solution testing process and perhaps even your professional life.

What we’ve learned about being a service company:

The value of customer service is real!
I started working for Honeywell in 1967. I went through the Management by Objectives era, the Total Quality Management era, the Shareholder Value era and plenty more. During this time and for a brief period before we started IQ Services in 1996, I was constantly exposed to the idea that the focus of all business activity should be the customer, both internally and externally. I was a believer, but sometimes it was difficult to measure the value of sincerely treating the people you worked with and the people you worked for as customers. The emphasis often seemed like it was for show and did not have real impact on business. Over the past decade, we have practiced and experienced the real value of customer service. If you treat people the way you would like to be treated, they want to do business with you. If you do a good job, you get paid—even when there are lots of problems and frustrations. As a service company, customer service is our most important “product.”

Another valuable lesson we have learned is that treating the people who you are relying on to deliver high quality customer service must also be treated as customers. People who believe you care about them and their success will treat your customers the same way.

What we’ve learned about testing contact center and communication business solutions:

1. Defining your business objectives and related test objectives is the key to the successful implementation of a business solution.

To properly test a business solution and be confident it works the way you want it to, you need to identify your overall business objectives and establish test objectives that define the required performance relative to those objectives. For example, if your business objective is to increase the customer satisfaction related to a self-service banking application and a key parameter affecting customer satisfaction is responsiveness at a host look up step, the test objective should be to measure the response time at that step relative to your design objective. This is just one example of how you can establish test objectives that demonstrate you are meeting your performance objectives for a business solution.

2. Setup is everything.

Load testing is conducted to demonstrate how a system performs at traffic levels consistent with your production performance objectives. All too often, the first time we conduct a load test it has to be stopped prematurely because some basic setup task or some unit level software test was not completed. Typical load testing is about what happens when a business solution works as designed at the unit level and then is subjected to traffic levels required in production. Load testing costs money. False starts because of poor setup are frustrating, cost money and delay schedules.

3. Simple is better than complex.

Testing process and results documentation need to be simple to implement and easy to interpret. Over the years we have tried to listen to our customers and evolve our testing process and tools so we keep them simple and effective. Straightforward processes for setting up and implementing testing along with easy-to-interpret results and reports facilitate communication and problem resolution. Simple, easy-to-use and understand tools and data bring real value to the testing sessions and results.

4. Everyone who cares needs to be involved in test planning and implementation.

Early in the life of IQ Services, we were faced with customers who would request testing where they provided test cases and asked us to run the test and send them the results. This was fairly common practice for telecommunications testing. Testers ran the test and spent hours analyzing the reported results. Sometimes it was relatively easy to evaluate results. Sometimes it was not so easy. Sometimes there were problems that terminated the testing and there was no opportunity to immediately access what happened, take some corrective action and continue testing. This frequently meant it took longer to find and fix problems than would have been the case if the people who really knew how the business solution was supposed to work observed the problems in real time. Our conclusion was that in order to get the optimum value out of testing, the people who really cared if everything worked needed to participate in the testing as it was being planned and implemented. When something is not working as expected it is immediately observed and in many cases corrective action can be implemented during a testing session. Time is not lost trying to recreate an issue because people see the issue while it occurs or immediately after it occurs. By participating in the planning, the people can help identify risk areas and the most cost-effective test scenarios.

5. Hearing is believing.

When issues with contact center business solutions are experienced, the first challenge frequently is convincing people the issue actually occurred. If the issue is not a hard failure and only occurs on some calls, one of the first questions is whether there is a problem with the test scenarios or test process. In our availability monitoring business, it became obvious that we needed a way to very quickly validate for our customers that an issue really did occur. Our solution was to start recording all of our test calls from end to end. When a customer questioned the validity of a notification that a call was not processed as expected, we played the call back. This virtually eliminated any question about whether the issue really occurred. The other observation we made was that customers sometimes would say something like, “I never thought that could happen, but now that I have heard it I have a good idea where to go look.” For us this was an “ah-ha” moment and we decided to build complete call recordings into our standard process for both availability monitoring and performance testing. Since 1998, our customers have been able to listen to full recordings of calls from both load testing and monitoring sessions. Since the advent of the Monitor Control Web Site in 2001 and ORCA in 2004, they can access the recordings online as soon as the calls are completed. We’ve truly learned that hearing is believing and can also be the key to quickly finding and fixing problems.

6. Testing must be flexible and responsive – we need to be available when people need us.

Testing is not an end in itself. It is a means to an end. When you are ready to test, you want to be able to test. If things do not work as planned, you need to have adequate time to fix problems before you test again. This does not mean that test schedules are not important and critical to project implementation. It does mean that if you feel you are really not ready to test; the tail does not wag the dog. If you are not ready, you need to be able to reschedule your testing with minimum cost and overall schedule consequences. You need to be able to expeditiously schedule testing when the business solution is ready to test. We’ve learned we must have pricing and implementation processes that are flexible and responsive, and to be available when our customers need us.

7. Testing saves time and money.

Everyone knows that it costs time and money when business solutions are not implemented on schedule. In addition to the direct cost, the opportunity cost of an eroded ROI and customer dissatisfaction due to delays or going live with problems can be very significant. Properly planned and implemented testing is one of the best insurance policies you can buy and is a key element of risk management. Performance testing may also drive early life failures out of a new system and significantly reduce post installation support cost.

8. Real time access to testing activity and results is essential to the value of testing.

In addition to having everyone who cares whether a system works involved in test planning and implementation, a key to optimizing test activity is providing immediate on-line access to test results. This requires straightforward, easy-to-interpret summaries of test results and the ability to listen to calls as described earlier. The Internet makes it possible for us to provide our customers online real time access to test results. ORCA, our Online Reporting and Charting Application, is a tool that has evolved out of this requirement and suggestions from our customers.

9. The total end user experience is what is really important.

At the end of the day, what matters most is knowing that your business solutions provide the desired end user experience. This means your testing must validate that your business solutions perform the way you expect them to from end to end. Testing must demonstrate and document the total end user experience under conditions that give you confidence your business objectives are being achieved before and after they are put into production.

The past 10 years have been ones of learning, growth, and continuous improvement. Many of you are some of the reasons we’ve learned these important lessons. It has been our pleasure to work with hundreds of customers in virtually every industry during the past decade and we look forward to learning more about how we can support you in the delivery of excellence to your customers.

Back