Testing Very Large Systems
By
Mike Burke
IQ Services

THE CHALLENGE
It is a challenging and risky endeavor to put a very large contact center solution into production.

Whether it is best in class or homogeneous, hosted or premise-based, on-shore or offshore, the dimensions of the project rapidly become overwhelming.  The communication solution is the face of the company and careers may be on the line if it does not perform the way everyone expects. 

Is the following example the kind of situation you find yourself in?  Somehow you or someone on your team convinced the CFO that it really was worth a $4M investment to provision 4,000 ports of IVR with speech recognition and text-to-speech engines; the latest in VXML browser and web services technologies; unified queuing to treat all your customers according to the form of presence they choose; and multiple locations to meet the CIO’s business continuity and disaster recovery objectives.  On top of that, thanks to the miracle of IP Telephony, your 4,000 agents are virtual officed because working from home improves morale, which in turn enables your agents to more effectively achieve overall CRM Customer Satisfaction objectives.

When you are faced with the complexity and volatility of the contact center solution environment, how can you be sure all the integrated elements will work together so you can turn it on with confidence?  And how can you be sure everything that was working the day you went into production is still performing solidly 6 or 8 months later?

You test it.

You run it through a comprehensive suite of end-to-end tests with IQ Services.

THE SOLUTION
IQ Services has been testing very large contact center and communications systems for years using both incremental and comprehensive testing approaches.  Depending on your requirements, schedule, budget and risk tolerance, we jointly define a test plan that accesses and exercises your system piece by piece, site by site, or all at once.

With a solution/system of any size there are really two testing objectives in play:

1. Does each and every element of the solution work as it should? (E.g., is each channel clean and terminated properly; is every DNIS properly programmed, is the firewall protecting you from undesired transactions, etc.)
2. Does the whole thing still work as it is supposed to when it is running at design load and volume?

YOUR CUSTOMERS
Very few of your users are going to know about carrier diversity, load leveling among locations, VDN trees, confidence intervals on speech recognition servers, VXML, or screen-pop.  But many of them know about GetHuman!  It is also likely that they have an opinion about offshore agents too.  With all the technology you’ve installed and the maintenance gyrations you’ve gone through to deliver a pleasing experience, you want to delight your customers, not disappoint them. 

So, regardless of solution capacity, IQ Services approaches testing your very large system the way your customers use it -- from the “outside-in” and from “end to end.” 

Because IQ Services configures its telephony servers to follow scripts intended to exercise your systems, we can accurately tell you what happens on individual test calls – in other words, we can tell you what the caller experience is based on real traffic.  If you think about the anatomy of a typical call that enters your contact center solution, each step in that call can be tied to a component in the architecture of the system – access/network; switch/ACD; IVR box and application; caller verification services; data bases for account number and password verification; web services; local databases or screen scrapes for “content” such as requested account information or status; and finally transfer to agent or expert.

By dialing into your solution from your customer’s perspective, we might breakdown a test call into two main segments:  (1) access and (2) self-service. 

(1) Access
How are all your calls getting to your facility?  Are all of the 6,000 channels purchased from a toll-free carrier really provisioned?  Do you know that you configured all your routes, rollovers and backups properly?  What’s the real capacity of the SIP pipes?  What if there’s music on hold while the customers are queuing for an agent?

If you only had ten trunks you’d ring them all out, right?  You wouldn’t want someone to land on a dead channel, especially a customer that spent hundreds or thousands of dollars on her last call!   So, of course, you check every channel -- it only takes a minute per channel.  But wait…now you’ve got thousands of channels and a different toll-free number for every product line, each with a DNIS all its own!  How can you thoroughly check access to this large complex system?

You leverage IQ Services’ team of experts and proven methodologies.

By taking either an incremental or comprehensive approach, IQ Services allows you to verify that access to the solution is unimpeded.  Using an incremental approach, IQ Services configures its test systems to generate traffic so you can direct calls through the system – one span, shelf, or frame at a time – according to a predefined sequence.  Every call placed by IQ Services is recorded start to finish, and thoroughly analyzed.  Between your team’s observations about the traffic flowing during the test and the results provided by IQ Services, you’ll be able to determine whether or not all trunks take traffic and sound clean as well as whether or not the handled calls met your QoS objectives.

As an alternative to incrementally testing every component of your telecom access infrastructure, IQ Services can take a comprehensive approach to the test by generating very large flows of traffic – in excess of 7,000 concurrent telephone calls from multiple LATAs around the country.  We’ll be able to tell you which calls accessed your systems and proceeded through a complete scripted interaction.  We can selectively initiate traffic from any or all of our POPs to help you determine whether or not there are issues related to initiation point of the calls.  Conducting a very large test for an extended period of time also ferrets out early-life failures in your infrastructure.  With very large systems, the sheer number of components increases the probability that something will fail the first time it is fully stressed for an extended period of time.   Why wait for the first or second day of peak busy hour traffic to find out which gateway or port card is going to be the one to succumb to the MTBF numbers game? 

(2) Self-Service
How are calls being handled when routed to your contact center solution?  Can your solution perform as expected under sustained full load as well as under peak volume conditions?  If your system has to handle 300,000 peak busy hour calls without skipping a beat, do you have enough friends, relatives and pizza to help you generate a large enough volume of test traffic? Maybe you’ve got toll-free numbers for various functions or caller types, each with its own DNIS and different IVR application menu sequence.  Your business rules may specify skill-based routing using ANI to ensure your premium customers always have a premium experience.

Everyone knows self-service is a great way to deal with many customer requirements, as long as it does not:

1. Lead the caller on a wild goose chase
2. Bog down under load
3. Prevent callers from transferring to the right agent who happens to have all the pertinent information from the self-service part of the call
4. Drop calls, etc.

Regardless of the size of your solution, there are numerous methodologies IQ Services can use to deliver testing traffic that is unique and appropriate to your self-service solution and applications.  Together, we develop a test plan and strategy to meet your business and risk management objectives as well as budget constraints.  From triggering premium call treatments to setting up test cases that mimic power users, you know how well your self-service solution fulfills customer requirements.

But self-service involves a lot more than just getting the right application to light up on an IVR port when it answers a call.  There are also all the servers supporting the process that need to be considered in a test plan including: DTMF receivers; speech recognizers; text-to-speech engines that voice the content requested by VXML browsers; local databases; remote legacy databases; and more.  And all of these components are stitched together -- sometimes by IP and hosted on the same LAN/WAN infrastructure that runs hourly batch processing in the background and provides voice connectivity between Hong Kong and New York City.  The application running on the IVR farm is undoubtedly programmed to tell the calling party exactly what’s going on – “I’m sorry, that information’s not available right now.  Please hang up…” which is exactly what you don’t want your customers to hear when they try to plumb the depths of your self-service environment.  IQ Services can configure your test to help determine whether or not all the elements of your large or small self-service solution are performing as expected.  If not, we provide you with actionable data and recordings to help you focus on the areas of the solution that need tuning.

And then you’ll know your solution is ready to face the challenge of real customer requirements and expectations.

CONCLUSION
Just because your system is very large doesn’t mean you can’t thoroughly test it.  IQ Services can help you configure incremental and/or comprehensive test strategies so you can be confident your very large contact center solutions and supporting infrastructure really work when you go into production.  And for solutions that allow customers to pick the form of presence most convenient for them – voice, Web, chat, email, callback, click-to-call, faxback, etc. – IQ Services can help you as well.  Our systems can be configured to simultaneously or separately exercise more than just the voice portion of your solution.  Let’s talk about your requirements and we’ll suggest a testing approach that meets your needs.

Proactive testing for confident customer interactions – because your customers are important!