DevOps.yoga


A DevOps Wiki

View project on GitHub

This is content about Testing tools.

List of Testing Tools

Note: This tools list is currently sourced from, and thus linked to, XebiaLabs. Much thanks to them for their valuable DevOps Toolchest.

NameIconDescription
Cucumber Cucumber is a software tool that computer programmers use for testing other software. It runs automated acceptance tests written in a behavior-driven development (BDD) style. Cucumber is written in the Ruby programming language. Cucumber projects are available for other platforms beyond Ruby.
SonarQube SonarQube is an open platform to manage code quality. Covering new languages, adding rules engines, and computing advanced metrics can be done through a powerful extension mechanism. More than 50 plugins are available.
Selenium Selenium is a portable software testing framework for web applications. Selenium provides a record/playback tool for authoring tests without learning a test scripting language (Selenium IDE). It also provides a test domain-specific language (Selenese) to write tests in a number of popular programming languages, including Java, C#, Groovy, Perl, PHP, Python and Ruby.
JMeter Apache JMeter is an Apache project that can be used as a load testing tool for analyzing and measuring the performance of a variety of services, with a focus on web applications.
Katalon Studio - Intelligent Test Automation Katalon Studio is a powerful automation toolset for web and mobile app testing. Katalon Studio revolutionizes the way software testers use Selenium and Appium with a complete test automation framework that allows testers to quickly setup, create, run, report, and maintain their automated tests. Katalon Studio is available to testers at no cost, with a free download available. Katalon Studio is packed with many essential features, including: Object Spy, Record & Playback, Cross browser execution, JIRA integration, Keyword-Driven Testing, Data-Driven Testing, and Web Service Testing.
JUnit JUnit is a unit testing framework for the Java programming language. JUnit has been important in the development of test-driven development, and is one of a family of unit testing frameworks which is collectively known as xUnit that originated with SUnit.
Cucumber.js Cucumber, the popular Behaviour-Driven Development tool, brought to your JavaScript stack.
TestRail TestRail helps you manage and track your software testing efforts and organize your QA department. Its intuitive web-based user interface makes it easy to create test cases, manage test runs and coordinate your entire testing process. Easily track and follow the status of individual tests, milestones, and projects with dashboards and activity reports. Get real-time insights into your testing progress and boost productivity with personalized to-do lists, filters, and email notifications.
HttpMaster HttpMaster is a professional web testing tool with primary focus on web API applications and RESTful web services. Features like dynamic parameters, advanced response data validations, and flexible request chaining enable thorough testing of any web application type.
Karma A simple tool that allows you to execute JavaScript code in multiple real browsers. The main purpose of Karma is to make your test-driven development easy, fast, and fun.
Jasmine Jasmine is an open source, behavior-driven testing framework for JavaScript with an easy-to-read test syntax.
TestNG TestNG is a testing framework for the Java programming language inspired by JUnit and NUnit. The design goal of TestNG is to cover a wider range of test categories: unit, functional, end-to-end, integration, etc., with more powerful and easy-to-use functionalities.
Perfecto Continuous Quality Lab The Perfecto Continuous Quality Lab is a cloud-based Platform as a Service for integrating real mobile devices and web browsers into testing activities across a DevOps pipeline. With integrations to Jenkins, REST APIs for execution and reporting, and 24-7 availability and support, it is the leader in lab management solutions for enterprise development and testing at scale.
FitNesse FitNesse is a web server, a wiki, and an automated testing tool for software. It is based on Ward Cunningham's Framework for Integrated Test. FitNesse is designed to support acceptance testing rather than unit testing in that it facilitates detailed readable description of system function.
Appium Appium is an open source test automation framework for use with native, hybrid and mobile web apps. It drives iOS and Android apps using the WebDriver protocol.
Gatling Gatling is an open-source load testing framework based on Scala, Akka and Netty. The software is designed to be used as a load testing tool for analyzing and measuring the performance of a variety of services, with a focus on web applications.
Behat Behat — a php framework for autotesting your business expectations. Behat is an open source Behavior-Driven Development framework for PHP. It is a tool to support you in delivering software that matters through continous communication, deliberate discovery and test-automation.
Sauce Labs Sauce Labs provides the world’s largest cloud-based automated testing platform for web and mobile applications. Sauce Labs helps companies accelerate software development cycles, improve application quality, and deploy with confidence across hundreds of browser / OS platforms, including Windows, Linux, iOS, Android & Mac OS X. Optimized for CI, CD and DevOps, the Sauce Labs platform has native plugins for Microsoft VSTS/TFS, IBM Bluemix, Jenkins, Team City, JIRA and other essential technologies that form the DevOps build pipeline toolchain. Sign up for a free trial @ http://saucelabs.com.
Flood.io Distributed load testing platform with open source tools including JMeter, Gatling and Selenium WebDriver. Scale out your tests for millions of users with results analyzed in real time.
Nightwatch.js Nightwatch.js is an easy to use Node.js based End-to-End (E2E) testing solution for browser based apps and websites. It uses the powerful W3C WebDriver API to perform commands and assertions on DOM elements.
TestObject TestObject is a cloud hosted service that provides mobile devices running Android and iOS for application testing. It supports a number of testing frameworks, such as Appium, Robotium, and Espresso, and can also be used for manual testing.
Cobertura Cobertura is a free Java tool that calculates the percentage of code accessed by tests. It can be used to identify which parts of your Java program are lacking test coverage. It is based on jcoverage.
Cypress.io Fast, easy and reliable testing for anything that runs in a browser. An open source, front-end testing tool, built for the modern web. Cypress makes setting up, writing, running and debugging tests easy.
TestComplete TestComplete is a functional automated testing platform developed by SmartBear Software. TestComplete gives testers the ability to create automated tests for Microsoft Windows, Web, Android (operating system), and iOS applications. Tests can be recorded, scripted or manually created with keyword driven operations and used for automated playback and error logging.
Protractor Protractor is an end-to-end test framework for AngularJS applications. Protractor runs tests against your application running in a real browser, interacting with it as a user would.
SpecFlow Specflow is a software tool that computer programmers use for testing other software. It runs automated acceptance tests written in a behavior-driven development (BDD) style. Specflow is based on Cucumber and is written in the .Net programming language.
pytest Pytest is a mature full-featured Python testing tool that helps you write better programs. Pytest provides easy no-boilerplate testing, scales from simple unit to complex functional testing, and integrates with other testing methods and tools.
Robot Framework Robot Framework is a generic test automation framework for acceptance testing and acceptance test-driven development (ATDD). It has easy-to-use tabular test data syntax and it utilizes the keyword-driven testing approach. Its testing capabilities can be extended by test libraries implemented either with Python or Java, and users can create new higher-level keywords from existing ones using the same syntax that is used for creating test cases. Robot Framework project is hosted on GitHub where you can find further documentation, source code, and issue tracker. Downloads are hosted at PyPI. The framework has a rich ecosystem around it consisting of various generic test libraries and tools that are developed as separate projects. Robot Framework is operating system and application independent. The core framework is implemented using Python and runs also on Jython (JVM) and IronPython (.NET).
Concordion Concordion brings your software delivery team together around living documentation. Implementing a Specification by Example (SBE) / Behaviour Driven Development (BDD) approach, the specifications are written in natural language using Markdown, HTML or Excel. The rich HTML output can be augmented with features such as storyboards, screenshots and logging output to present different views to different stakeholders.
Mocha Mocha is a JavaScript test framework running on node.js, featuring browser support, asynchronous testing, test coverage reports, and use of any assertion library.
Micro Focus Unified Functional Testing (UFT) Micro Focus Unified Functional Testing (UFT) (formerly known as HP Unified Functional Testing), automates functional and regression testing for applications and environments. Micro Focus UFT features a graphical user interface, integration with the Business Process Testing framework, and the ability to convert manual tests to automated ones.
Codacy Codacy is an automated code review tool that allows developers to improve code quality and monitor technical debt. Codacy automates code reviews and monitors code quality on every commit and pull request. It reports back the impact of every commit or pull request in new issues concerning code style, best practices, security and many others. It monitors changes in code coverage, code duplication and code complexity. It allows developers to save time in code reviews and tackle efficiently technical debt.
Locust Locust is an open source load testing tool that allows you to define user behavior with Python code, and swarm your system with millions of simultaneous users.
Parasoft Virtualize Parasoft Virtualize, an open automated service virtualization solution, creates, deploys, and manages simulated dev/test environments. It simulates the behavior of dependent applications that are still-evolving, difficult to access, or difficult to configure for development or testing.
Sahi Sahi is an automation and testing tool for web applications coming in an open-source and a proprietary version. The open-source version includes a basic tools set sufficient for most testing purposes (Record on all browsers, Playback on all browsers, HTML playback reports, JUnit Style playback reports, Suites and batch run, Parallel playback of tests), whereas the Pro version includes further features such as test distribution and report customization.
Ranorex Ranorex Studio offers easy-to-use test automation tools for creating reliable automated testing projects. Optimize your robust test automation frameworks with Ranorex Studio – test any desktop, web or mobile application. Ranorex is easy-to-use test automation software for developing and managing projects in teams made up of both testers and developers. A step-by-step wizard helps you to set up the test environment and quickly get started. Non-programmers can use the script-free drag & drop functionality, whereas professional programmers can use an API for C# and VB.NET to enhance their test suites and recordings.
AutoIt AutoIt v3 is a freeware BASIC-like scripting language designed for automating the Windows GUI and general scripting. It uses a combination of simulated keystrokes, mouse movement and window/control manipulation in order to automate tasks in a way not possible or reliable with other languages (e.g. VBScript and SendKeys). AutoIt is also very small, self-contained and will run on all versions of Windows out-of-the-box with no annoying “runtimes” required!
Screenster Screenster is a cloud-based platform for UI testing automation. It is primarily used for visual regression testing of web applications and websites.
Squash TM Squash TM is the test repository manager for the open source Squash suite. It manages requirements, test scenarios, and execution campaigns, in a natively multi-project context.
Tricentis Tosca Tricentis Tosca is a Continuous Testing platform that accelerates testing to keep pace with Agile and DevOps.
Parasoft Development Testing Platform Parasoft Development Testing Platform (DTP) enables Continuous Testing. Leveraging policies, DTP consistently applies software quality practices across teams and throughout the SDLC. It enables your quality efforts to shift left–delivering a platform for automated defect prevention and the uniform measurement of risk.
Qunit QUnit is a JavaScript unit testing framework. While heavily used by the jQuery Project for testing jQuery, jQuery UI and jQuery Mobile, it is a generic framework to test any JavaScript code. It supports server-side (e.g. node.js) and client-side environments.
Parasoft API Test Parasoft SOAtest automates complete end-to-end testing for business and security-critical transactions. Parasoft SOAtest is widely recognized as the leading enterprise-grade solution for API testing and API integrity. Thoroughly test composite applications with robust support for REST and web services, plus an industry-leading 120+ protocols/message types.
CodeFactor Automated Code Review for GitHub and BitBucket. SAAS service that checks code with every commit or pull request and provides actionable feedback.
Rational Quality Manager IBM's Rational Quality Manager is a collaborative hub for business-driven software and systems quality across virtually any platform and type of testing. This software helps teams share information seamlessly, use automation to accelerate project schedules and report on metrics for informed release decisions.
Parasoft Environment Manager Parasoft Test Environment Management enables development and QA to rapidly access the specific environment instances they need to “shift left” testing effort and accelerate the SDLC.
Smartbear SoapUI For REST, SOAP and other popular API and IoT protocols, SoapUI Pro provides the industry's most comprehensive and easy-to-learn functional testing capabilities. Based on open core technology proven by millions of community members, SoapUI Pro helps you ensure that your APIs perform as intended, meet your business requirements, timeframes, and team skill sets right from day one.
CasperJS CasperJS allows you to build full navigation scenarios using high-level functions and a straight forward interface to accomplish all sizes of tasks.
Rational Integration Tester IBM Rational Integration Tester is a scripting-free environment for developing tests for service-oriented architecture (SOA) messaging and business process integration projects. Tests can be built by recording existing system behaviors or from requirements, by entering the data to send and the data that is expected in return.
NUnit NUnit is a unit-testing framework for all .Net languages. Initially ported from JUnit, the current production release, version 3.0, has been completely rewritten with many new features and support for a wide range of .NET platforms.
Squash TA Squash TA (Test Automation) is an open source tool for automating functional tests and industrialization of their executions. Compatible with several open source PLCs (Ex. Selenium), it allows management of the automated tests of web applications, web services and batches. The features of Squash TA are based on developments initiated in 2009 by Henix. Realized to meet the automation needs of the TRA (Third Party Recipe Applicative) of this SSII, these developments were donated to the community Squash to constitute the base of Squash TA.
xUnit.net xUnit.net is a unit testing framework for .NET projects with full support for .NET Core and Xamarin. It is developed under the umbrella of the .NET Foundation and was started by the original developer of NUnit 2.
Karate Karate is an open-source web-API test-automation framework that can script calls to HTTP end-points and assert that the JSON or XML responses are as expected. Karate also has support for service-virtualization where it can bring up "mock" (or stub) servers which can substitute for web-services that need to participate in an integration-test. Karate's capabilities include being able to run tests in parallel, HTML reports and compatibility with Continuous Integration tools.
QF-Test QF-Test from Quality First Software is a cross-platform software tool for the GUI test automation specialized on Java/Swing, SWT, Eclipse plug-ins and RCP applications, Java applets, Java Web Start, ULC and cross-browser test automation of static and dynamic web-based applications (HTML and AJAX frameworks like ExtJS, GWT, GXT, RAP, Qooxdoo, RichFaces, Vaadin, PrimeFaces, ICEfaces and ZK). Web applications cross-browser on Chrome, Firefox, Internet Explorer, Safari and Edge with the Selenium WebDriver approach; headless browser for Chrome and Firefox.
BrowserStack BrowserStack is a cloud-based cross-browser testing tool that enables developers to test their websites across various browsers on different operating systems and mobile devices, without requiring users to install virtual machines, devices or emulators.
GenRocket GenRocket software redefines the role of complex integration testing with a platform that automates the generation of white box test code. It uses on-demand, real-time test data that scales to large and complex database environments while preserving referential integrity.
Watir Watir is an open-source cross-platform web application testing tool. It is most reliable and flexible automation tool of Ruby libraries for web browsers automation. This cross-browser testing tool interacts with a browser just like human being so it clicks links, filling out forms and validating text.
CrossBrowserTesting CrossBrowserTesting by SmartBear is the only all-in-one testing platform that allows teams to run automated, visual, and manual tests on thousands of real desktop and mobile browsers in the cloud.
Google Test Google Test is a unit testing library for the C++ programming language, based on the xUnit architecture. The library is released under the BSD 3-clause license.
Qualitia Automation Studio Qualitia is a pioneer in script-less functional test automation. Qualitia brings the power of simplicity and speed to adopt test automation in the most effective way. Qualitia has enabled businesses to transform the way software functional testing is done, by significantly optimizing cost and effort to make “ship/no-ship” decisions with high confidence. With Qualitia, large enterprises are successfully driving “Shift Left” agenda in Agile & DevOps environment.
Micro Focus LoadRunner LoadRunner is a software testing tool from Micro Focus. It is used to test applications, measuring system behavior and performance under load. LoadRunner can simulate thousands of users concurrently using application software, recording and later analyzing the performance of key components of the application
Parasoft SOATest Automate complete end-to-end testing for business and security-critical transactions. Parasoft SOAtest is an enterprise-grade solution for API testing and API integrity. Thoroughly test composite applications with robust support for REST and web services, plus a 120+ protocols/message types.
SoapUI SoapUI is an open-source web service testing application for service-oriented architectures (SOA) and representational state transfers (REST). Its functionality covers web service inspection, invoking, development, simulation and mocking, functional testing, load and compliance testing.
Test Architect With its modular approach, TestArchitect embeds a method and workflow that enables team members to organize tests well, give them a clear scope, and keep them at the level of abstraction to meet the scope. The keyword-driven actions help all team members and stakeholders, regardless of background, to understand and communicate about the tests. This works just as well for tests that target the UI, as it does for tests targeting non-UI platforms like services, embedded software, telecom, IoT, etc.
Bugwolf Transform bug tracking into competitive testing challenges that accelerate digital releases, lower customer support calls and reduce defect costs. Watch on as professional testers race against the clock to dramatically improve your app or website. In as little as six hours, you will have a full usability, functional or UX review for key user journeys. At the end of each challenge, you will receive an overview of all bug reports, prioritised by severity, complete with video and audio commentary.
CodeScene A code and developer analysis tool. Codescene is able to find the code that is hard to evolve and prone to defects and creates a prioritized list of suggested refactoring targets.
CA Test Data Manager CA Test Data Manager offers an automated solution to one of the most time-consuming and resource-intensive problems in Continuous Delivery: the creating, maintaining and provisioning of the test data needed to rigorously test evolving applications. CA Test Data Manager uniquely combines elements of data subsetting, masking, synthetic, cloning and on-demand data generation to enable testing teams to meet the agile needs of the organization.
Telerik Test Studio Telerik Test Studio empowers you to quickly and easily craft automated tests, integrate them in your CI/CD environment following your agile workflow, find defects earlier and ship a better quality software product. Automate repetitive manual QA tasks and ensure high level of software quality continuously in time without last minute surprises.

Prev: Practices | Next: Glossary