3 tools to manage performance testing

Choosing the right tool for performance testing is crucial for several reasons:

First of all, it must be able to carry out performance tests on the installed software base you wish to test, and therefore, offer adequate protocol coverage. Not all tools can test Citrix, SAP or Oracle Forms, for example. It also has to match the customer’s organization and way of working.

Secondly, price or subscription is also an important criterion.

Finally, ease of use, quality of support and a strategy for upgrading versions to keep pace with market developments are all important criteria.

In this article, we offer a synthetic comparison of 3 tools.

JMeter

JMeter is an open-source load-testing tool now widely used by all types of customer: public sector, major accounts, various companies.

Its graphical interface is simple and efficient. It can be used to test all Web applications, Webservices (REST, SOAP), Web 2.0, messaging services, etc…
JMeter comes with native functions (in the macro sense) for dynamic data injection and/or on-the-fly data manipulation during load testing.

Developed in JAVA, JMeter offers a large number of plug-ins to various monitoring technologies.

It can be used for testing early in the development cycle – Shift Left – i.e. API, Web services and Microservices testing – as well as for end-to-end testing. Its recording interface is efficient.
JMeter’s receiver elements enable results to be visualized in the form of graphs or basic statistics, and results to be interpreted.
It’s a complete tool which, together with a Grafana-type reporting tool and a Prometheus-type metrics capture tool, enables load tests to be carried out in a wide range of contexts.

Neoload

Neoload is a load-testing tool from Tricentis (since the acquisition of Neotys).
Neoload supports protocols not offered by Jmeter, such as Citrix, Oracle Forms, SAP, etc.

It also offers integrated functions for script variabilization and dynamic variable management.
In recent releases, NeoLoad has taken an important step towards the Shift Left approach, enabling teams to test earlier in the software development cycle, at API level, thanks to new features such as in-code performance testing and Swagger file import.
In addition to the scenario capture tool for scripting and the load injection control module, it also offers a powerful integrated reporting module, as well as a monitoring module for a comprehensive tool.

What’s more, it is interfaced with APM tools, notably Dynatrace, which make it easier to correlate response times with technical metrics, and thus bring great added value to recommendations for improving and tuning technical components.

The tool is not free, and its licensing model has recently changed to make it significantly more expensive.

Gatling

Gatling is a basic open source performance testing tool that is being used more and more frequently, particularly in a context where the user company decides to implement performance testing very early in the release delivery cycle (mainly in agile mode) and where developers are more involved in the performance commitments of their code. Gatling makes it easy to perform API and Webservice tests.
Gatling is a “developer’s tool” in the sense that its user interface and organization make it easy for developers to find their way around, whereas traditional testers (using JMeter or Neoload, for example) have trouble finding their way around (I’ve tested …). You could say it’s a performance testing tool in code form. Note that Gatling is developed in Scala and requires a basic knowledge of this development language.
For the rest, the functionalities are there, close to those of JMeter, and the 2 tools are comparable in their advantages and limitations to an editor-type tool like Neoload.

It should be noted that Gatling offers a paying version which, for example, gives access to support and is necessary for distributed testing (controller + multiple load generators).

Philippe Boudard

Passionné de performance applicative depuis plus de 20 ans, Philippe Boudard est expert sur les sujets d’optimisation des infrastructures logicielles, tout en garantissant sécurité et scalabilité pour les systèmes les plus exigeants. Il a contribué à fonder et développer SCOPTEAM, ESN spécialisée dans les services autour des tests de performance, avant de diriger pendant 8 ans l’entité Altersis Performance au sein du Groupe Altersis. Plus récemment, il a rejoint les équipes de QESTIT France, où il gère et organise le département performance applicative et sécurité, qui veille à la performance, la résilience, la robustesse et la fiabilité des applications dans des contextes critiques. Il garde également les 2 pieds sur les sujets opérationnels et réalise des prestations de conseil, d’audit et de mise en œuvre de schéma directeur sur ces sujets.

EXPERIENCE AND INSIGHTS Stay updated!

Get knowledge, news, inspiration, tips and invitations about Quality Assurance directly in your inbox.

share the article

3 tools to manage performance testing

JMeter

Neoload

Gatling

Yes, let's get inspired!

MORE USEFUL INSIGHTS ABOUT PERFORMANCE