Fix it cheaply during testing or pay a fortune later!
It's critical to find issues during the test phase, as finding them later when a system has been deployed can be very costly. Bringing a 'live' system down for maintenance can cause widespread disruption and affect productivity, costing thousands in 'downtime'.
How can you be sure you've found all the issues during testing?
Well, firstly it's impossible to fully test any system so we need to look at minimising the risk by trying to cover as much testing as possible in the time and budget available. By using an appropriate test tool our testing coverage can be good enough to isolate many major issues before going 'live', which can save thousands in rectification costs.
The effectiveness in spotting the major issues during testing has a substantial reliance on the test tool used. Unfortunately not all test tools are the same and can very considerably in their effectiveness, which can lead to some issues being undetected, leading to substantial costs in rectification later on.
1. Enterprise Class
There is no point in designing and building an enterprise system and then using a non enterprise test tool, to test it with. I've seen many systems worth millions being tested by test tools which have been cobbled together by a bunch of developers, designers or system builders, with little regard to fundamentals of testing.
It's like building a jet engine to the highest quality, with stringent processes, top notch components and incredible tolerances and then using a measuring tape (which measures only in centimetres and inches) to validate the correctness of the measurements.
Sounds crazy I know but this isn't far off the mark with how Citrix fronted environments are being tested.
2. End to End Capability
From experience most of the serious problems I've encountered in the larger organisations have not been with Citrix servers but with the associated infrastructure components. Such as poor configuration of databases, networks, directory services and so on. These also need to be tested and therefore the test tool needs an end to end capability.
Smaller businesses can have serious issues with their Citrix infrastructure. This is down to limited Citrix skills at hand and/or using substandard technical consultants who don't understand what's under hood.
3. Modular
The way the test tool is designed to run is very important. Design encompasses not only whether the test tool has a modular component structure but how these components work together.
Running a single test tool application on a Citrix server, which runs the test scripts, collects data and then alerts when it's finished, isn't an ideal test tool design.
Ideally the test tool must have a controller component, scripting component, generator component, data capture component and analysis component.
4. Non Intrusive
It's important to use a test tool which doesn't run any of it's components actually on the Citrix servers, as the overhead produced can taint the quality of results produced.
Having sessions initiated on the Citrix server side, can add several megabytes per test user and additional CPU processing. Multiply this by 50, 60, 70 test users and this needless hit on the Citrix server resources, will affect the quality of test data produced.
It's like testing the acceleration time of a Ferrari and having extra passengers occupying the seats as well as the test driver in the test. This won't give an accurate reading of the acceleration capabilities of the Ferrari.
5. Test Language
Over simplifying things means it's difficult to work out what's happening under the hood and I've found tools which haven't a scripting language as such, don't provide the granularity of control needed to test and analyse accurately. They use a black box approach of using simple controls and wizards to create tests.
I ideally look for a test tool which has it's own language specifically designed for testing, this reassures me of the pedigree of the test tool I'm evaluating. Any test tool with it's own language which has evolved to become a testing language in it's own right, scores mega points with me.
6. Scalability
Enterprise organisations can have multiple test sites and these need to be factored in during a test. Moving test apparatus from one site to another site to do the next set of tests and then moving them back, isn't practical. Test apparatus doesn't just involve test equipment but the people involved in developing, running and managing the tests.
A scalable test tool architecture needs to be able to accommodate horizontal scaling of test sessions, so additional test sessions can be added to share the work load at any particular site.
7. Usability
When I talk about usability, I'm not referring to how easy a test tool is to use. This doesn't come into the equation, as a professional tester will be able to use the test tools they're experienced with. The consideration to evaluate a test tool for the sole purposes of it being suitable for non test professionals such as Citrix designers and implementers is not recommended either.
What I'm more concerned about is whether the test tool can do what is expected. I say this, thinking back to a series of tests I did with applications delivered via Softricity's Softgrid (pre-Microsoft).
Where the majority of the test tools evaluated, failed to recognise the applications being launched and therefore none of the test scripts ran. These test tools couldn't see inside the Softgrid sandbox and therefore the applications running inside.
8. Transaction Processing
Testing isn't just about metrics, that is relying solely on performance monitoring graphs to see how well the infrastructure handles it's resources. Application usability is the key and by measuring whether an application is still usable when tested is essential.
The usability measurement for applications can be achieved by defining transactions. A good test tool will have a transaction facility built into their scripting language, so there will be no need to reinvent the wheel, to monitor transactions.
9. Scenarios
A good test tool will cater for the grouping of different test scripts, so different scenarios can be run and managed easily. Each scenario can include a series of events and a number of test users to be associated with these events. This strategy allows the testing to simulate more closely with what is expected in a 'live' environment.
10. Error Handling
To make it easier to find errors in test scripts, a test tool should have excellent debug facilities, which allow the code to be checked.
11. Scheduling
Logging on and off Citrix servers places a high demand on resources such as CPU. Additionally using products like SoftGrid, places a further burden on resources at logon, as the SoftGrid services authenticate and update application access when the user logs on. This can if not assessed create a dangerous situation where Citrix servers can temporarily stop responding if they become overloaded.
From previous experiences I've come across situations where additional servers were needed to bolster a farm, as during testing, we found the farm under test couldn't cope with the expected user logon loads. At other times, I found the servers to be underpowered, so these servers were required to be either replaced or upgraded to meet the logon demands.
To be able to test the implications of adding users to the Citrix server, the test tool needs to have the capability to schedule user logons onto servers.
12. Spike Testing
Sometimes certain steps in a program may require a large degree of processing power, memory or hard disk utilisation, which needs to be assessed.
A test tool creates spike testing by allowing test user scripts to reach a pre-defined point in the test script and then waits for all other test users to reach the same point. Once all the test users have reached the pre-defined point in the test script, only then does the rest of the test script run. This allows a coordinated run of a particular part of the script, so test users could run the spell checker if it's the next piece of code in the script.
Article Source: http://EzineArticles.com/1848095
