Storage domain testing was something that was exclusively reserved for the hardware vendors but over the course of innovation in storage technology, we are seeing more and more enterprises preferring to perform the load testing on their own. The hardware vendors were on the forefront for development of black box tools, the virtual Instruments load Dynamix or open source software will able to measure how the disk arrays actually perform in case of pressure. However as we saw increased data usage and generation the stress number from the vendors became benchmarks, however, this also resulted in errors in the benchmark numbers as the load was not stimulated with real-world situation making the off-the-shelf benchmarking irrelevant in the modern data storage requirement. As the data got bigger and applications became diverse the benchmarks and vendor numbers weren’t giving the real stress capacity for the storage. After such errors we saw a shift from the old black box technique to a more improved version of software storage stress testing solution, courtesy of Calypso Systems, the storage networking industry association (SNIA) and from the University of New Hampshire’s Interoperability Laboratory.
Load Testing Software isn’t new; it works under the technique of running a sample data through the network. In May 2018, Calypso System released a specification via SNIA, wherein the testing is performed by capturing your actual company data. The new software-based strategy testing is named as Real World Storage Workload (RWSW) performance test specification for data-center storage, the data admins in various enterprises use the system to actually discover the performance issues.
Calypso CEO Eden Kim said in a statement that “In the traditional load-testing systems, they use various synthetic benchmarks to saturate the drives and test the numbers. Data Admins realized that when you put storage in real-world applications they perform completely different. ” Using the new system the data admins will be able to define how to capture their workload and how to assess the results based on the output. He also added that the enterprises can open a file and then create an I/O stream map. It allows the users to see what actually occurs in the application. Once you can see the data and storage reaction then comes the analysis part. The network admin then can save that and automatically create the test script that would create a replay for it. Network admins can filter the data towards the application they want to test. The technique differs slightly when it comes to cloud storage, a cloud admin is required to test storage stress.
Kim added during the QnA that almost 1,500 users have downloaded the tool and uploaded the results. The results can also be put to analysis with free online tools for basic results or with Calypso’s paid software for advanced results. The analysis tool can also be built by enterprises based on their requirements. The traditional testing techniques by the UNH laboratory was only limited to storage vendors who are willing to bring their products in firsthand. Currently, the only companies that responsible for their own storage testing are hyper-scale organizations such as Amazon or Google. Lab Engineer David Woolf added that testing is as imperative as proper implementation of any storage. Enterprises need to have faith when it comes to going through the testing phase of our storage it would assist you to understand whether or not your product is giving the best performance for your workload. The enterprises would also be able to know what exactly the applications are doing, what are the traffic patterns you have and what type of infrastructure design would work best for them. For storage users who want access to such tests, there is something that we are working on expanding them.
Some of the data centers are using the Calypso System for storage stress testing as the improved functionality is assisting to test the real-time application performance effect on the storage. Woolf also added that in the future she would be more deliverance of performance along with the compliance issues. “It’s like adding more innovation to the current technology, the tool would not only tell the type of data center requirement or workload objective but also how you can improve the data.” Though testing in the synthetic workload in a controlled environment often in the vendor’s lab, such testing is quick and requires minimum investment beyond the time to pick and tests to run. However, the challenge with such techniques is the need for modeling the workload and having an interpretation for the results so that the applications act accordingly after the implementation. This is an important part of the testing, based on the situational requirements wherein testing phase is established with a true method of information gathering and analyzing this data based on the years of experience in storage array performance testing and tuning.
Conclusion
Storage performances are strongly based on the technology and live requirements of the application that can change dynamically. So performance testing has never been easy to do, given the required time limit and effort required to plan, perform and analyze such results. It’s critical that anyone looking to acquire the new storage system takes the time to choose an approach that will most effectively be able to solve the storage issues like putting the most critical application in an array wherein it can perform to best of its ability.
To know more, you can download the latest whitepapers on storage technology.
