Apica For Complex Load Testing and Monitoring

Apica makes application load testing and monitoring tools.

The main distinction between Apica’s tools and things like Application Performance Monitoring (APM) tools is that Apica does a more active kind of testing that tries to break your system before users do so you can fix the problems before it impacts actual people. APM is more about watching the application to help you diagnose problems that occur while it’s running. They’re complementary activities, and while you could probably do part of both jobs with either kind of tool, you won’t do the job very well.

You’re better off having a tool that is designed to do testing and not just monitoring when testing is your goal. Let’s talk about why.

The Value of Testing

The kinds of testing that tools like Apica do are really important for complex systems with lots of moving parts. Unit tests are great for testing individual components, and integration tests or system tests are great for making sure things fit together properly and work more or less as expected. These tests are best done to exercise the pieces under both use- and misuse-cases to make sure they work as expected, but also don’t break in unexpected ways when someone tries to pass a string instead of an integer to a function expecting only integers.

Where tools like Apica come into their own is in testing more complex scenarios that mirror real-world use of the system. That can be both for functionality testing, to make sure the full user journey works as expected (both on the happy path and the very many possible unhappy paths) but also under load.

Earlier in my career I did a lot of work doing performance and capacity tuning of complex systems (like the ADSL billing system for Australia’s biggest telco, banking systems, and some government things I’m not allowed to talk about). This kind of testing was generally a major part of the acceptance process for whatever contractor won the contract/tender to supply the system, and big money was involved. You didn’t want to suddenly find out that a system didn’t work properly when hundreds of thousands of customers started to use it on day one and a bunch of them stumbled across some path through the system you hadn’t considered.

Why Apica?

For a good part of Apica’s presentation at #TFD25 I was puzzled about what it did that wasn’t available on tools like Mercury Load Runner that I used 15+ years ago, or Selenium from 10 years ago. Mostly, the answer seems to be that it’s an incremental advance on those tools to deal with the more complex deployments that people have today.

Today, complex applications are a lot more distributed. They’ll make use of cloud-based SaaS for things like taking credit card payments, for example. They generally consist of a bunch of network-connected services, many of which won’t even be running inside your firewall, and may well be spread all over the world. Services will also be encrypted a lot more than they used to be, and have more complex security and identity mechanisms that the testing software needs to be able to emulate or use. How do you do large-scale automated testing of multi-factor authentication using an external service like Okta?

That seems to be the big selling point of Apica. It can deal with all these disparate systems, and handle the complexity of the data flows and user journeys in modern applications. You could probably build something yourself using other tools, but it’s a lot of work (I would prefer to forget the time I was building SSL client certificate testing modules and reverse engineering parts of a vendor’s software to make it work) so why put yourself through all that?

Apica has a lot of nerd knobs, which is somewhat unavoidable given the kind of work it’s designed to do. Testing is complicated and you do need to configure lots of different parts to fully exercise a system. It’d be nice to have more of this automated, but there’s a limit to how much you want to trust a computer to automatically attempt to test itself. Apica does have some mechanisms that help, like saving presets for common scenarios that you can just select and save yourself time. You will still need to invest the time in configuring a set of scenarios that reflect the way your applications work, though, because they tend to be unique to each company, even when they follow some common patterns.

I was left with the impression that Apica is what I would have wanted to use back in the early 2000s but it didn’t exist yet. Now it does, so it’s worth checking out if you have some testing to do.