There is often an underlying and unspoken fear with software systems: How resilient is your solution to unexpected outages and system failures? Sure, you’ve tested your solution’s functionality and performance, but have you tested against system outages? This is where chaos testing comes in.
Chaos testing is a technique that helps to identify how systems behave under unexpected or turbulent conditions. By incorporating these unpredictable disruptions into LoadRunner Professional (LRP) scenarios, you can gauge the resilience of your applications and systems.
Chaos Testing Address Issues That Other Testing Techniques Don’t
Most systems nowadays are heavily interconnected, with multiple customer access points. If even a small component were to fall over, it could lead to much larger issues.
A lot of companies choose to bury their head in the sand. Ignoring these potential issues in the hope that they won’t happen. Unfortunately, though, they do happen.
With chaos testing, you can directly address these issues, introducing system failures into your LoadRunner Professional scenarios to mimic random, real-world outages.
For more information on Chaos testing, check out What is Chaos Engineering? (and Why You Need It)
A Few Chaos Testing Examples
Terminating Server Instances
You could randomly shut down server instances to assess whether your solution can withstand the loss of any single instance without affecting the overall user experience.
Network Disruption
In network chaos testing you intentionally introduce delays, packet loss, or outages to simulate network issues and test how well your solution handles network instability.
Application Dependency Failure
You can deliberately disable access to an application’s external dependencies to understand how it responds when third-party services, APIs, or data stores go down.
Understanding Chaos Testing
When you initiate chaos tests, you simulate attacks on various system components, such as databases, web servers, CPU, or memory. Within an LRP test, disruption events are added to your scenario.
The EventsHandlerMicroservice in the Controller then communicates with the chaos platform, in this case, Steadybit, to execute these predefined disruptions. The goal is to see how these disruptions affect the system in conjunction with the LRP scenario.
A chaos test hinders typical workflows, which may slow down services and reduce successful transactions. Comparing system responses between scenarios with and without these disruptions will offer insights into the system’s resilience.
Steadybit and LoadRunner Professional
Steadybit is one of the platforms compatible with LRP, and it operates in both public clouds or on-premises through a private cloud.
Within Steadybit, you define experiments that may consist of multiple disruptions. These experiments are then added to the LRP scenarios.
To coordinate the chaos tests, LRP employs integrated Steadybit APIs. However, ensure the Steadybit agent is running on your Application Under Test (AUT) machines. It’s often beneficial to bundle several disruptions into a single experiment instead of running a series of shorter, separate experiments.
Chaos Testing From the Cloud
You can also use a cloud-based chaos testing tool, Gremlin, to introduce chaos events into LoadRunner scenarios – check out this article for more information.
Incorporating Disruption Events into LoadRunner Professional Scenarios
To utilise disruption events within your LRP scenarios:
- Setup: Initialize your chaos engineering platform (Steadybit) and outline the disruptions for the application in question.
- Connection: Before diving into the Controller, validate the connection between the Steadybit agent and the platform.
- Scenario Design: Launch your LRP scenario and navigate to the ‘Controller Design’ tab. You’ll find the ‘Disruption Events’ button under the’ Scenario Schedule’ pane.
- Event Addition: Select’ Add Event’ within the ‘Disruption Events’ dialogue. Choose ‘Steadybit’ as the chaos engineering provider type and input the necessary API Key. Check the ‘On-prem’ box for private cloud operations and provide the necessary URL.
- Event Selection: A table populated with disruption events (or experiments) from Steadybit will appear once connected. You can select the desired events and hit ‘Add’. You can even search for specific events or see a brief about the experiment.
- Scheduling: After adding the events, define the start time relative to the overall LRP scenario start time. A scheduling graph visually represents the event timelines against the overall Vusers schedule.
- Event Error Handling: From version 2023 R1, the ‘continue on error’ feature allows you to decide if the scenario should proceed even if a scheduled disruption fails.
- Save: After completing the setup, save your changes.
As the LoadRunner Professional scenario progresses, you can observe how the chaos events from Steadybit influence the monitored machines with real-time updates on the scenario graphs.
Configure and Review Disruption Event Logs
You can configure different logging levels like Trace, Debug, Information, Warning, Error, or Critical.
To review the logs for these disruption events:
- For microservice-related logs, refer to <LoadRunner root folder>\Logs\EventsHandlerSrv\events_handler.log.
- Controller-related logs can be found at %TEMP%\lr_disruption_events_err.log.
Summary
In short, chaos testing lets you see whether you solution can take an unexpected hit.
Adding chaos events into your LoadRunner Professional scenarios introduces outages into your test environment, letting you assess how your solution responds.
Chaos testing with LoadRunner and Steadybit is a straightforward process that gets your software ready for the unexpected.
