Which applications can be used to test RESTful API?

In this blog-post, I am going to look at some existing tools suitable for testing REST API. I'll try to evaluate how they do tests and what they are testing and how it can be used for testing REST API.
These tool are: soapUI, Apache JMeter, FitNesse, Fiddler 2 and JUNITs + maven build + Spring MVC REST.
I am not going to list all of the features the tools poses, just those interesting for REST testing.

soapUI 
( http://www.soapui.org/ )
licence: open source,
platform: Java,
available as: standalone app or as plugin to IntelliJ, Eclipse, NetBeans, maven.
Through graphic interface, soapUI allows you to easily and rapidly create and execute automated functional, regression, compliance, and load tests. Key features are:

• Can test SOAP- and REST-based Web services, JMS enterprise messaging layers, databases, Rich Internet Applications etc.
• It can mock services (no automatic generation for REST).
• Can record, analyze and even modify client-server traffic as it happens.
• Offering Drag and Drop Test Creation.
• Nice request builder.
• Nice manual.

There is option to buy pro version with aditional features such as Groovy Scripting, Data-Driven Testing.

Typical use is to manually create a TestCase - a set of specified HTTP requests and expected responses which soapUI evaluates.


Apache JMeter 
( http://jmeter.apache.org/ )
licence: open source,
platform: Java,
available as: standalone app.
Java application designed to load test functional behavior and measure performance. Is usable through graphic interface or command line. Key features are:

• It can make a graphical analysis of performance or test your server/script/object behavior under heavy concurrent load. Supports regression tests.
• Can test HTTP, HTTPS, SOAP, Database via JDBC, LDAP, JMS Mail.

Isn't focused on REST testing but since its capability to send and evaluate http requests we can use it this way - create a test case like with soapUI.


FitNesse 
( http://fitnesse.org/ )
licence: open source,
platform: Java,
available as: standalone app.
FitNesse enables customers, testers, and programmers to learn what their software should do, and to automatically compare that to what it actually does do. It compares customers' expectations to actual results.
You can Collaboratively define Acceptance Tests and execute them.

Typical use is to define tests by wiki pages. Then wiki pages uses binding (Java) code to execute methods of tested app. The binding code is written by you and is meant for making calls to jars of tested app. Because of that, you will have to tveak your binding code in order to call REST API and I don't think that FitNesse was designed this way.


Fiddler 2 
( http://www.fiddler2.com/fiddler2/ )
licence: freeware
platform: .NET,
available as: standalone app.
Fiddler is a Web Debugging Proxy which logs all HTTP(S) traffic between your computer and the Internet. Fiddler allows you to inspect traffic, set breakpoints, and "fiddle" (read, create, modify http responses and requests) with incoming or outgoing data. You can do this by graphic interface.


JUNITs + maven build + Spring MVC REST
This is not out of the box app for testing. It's 3 separate products which can be used together to help you create test application for REST API.

JUNIT is a simple framework to write repeatable tests for the Java programming language. It is an instance of the xUnit architecture for unit testing frameworks. Alternative to JUnit can be TestNG.

Maven build refers to Apache Maven. Maven can manage a project's build, reporting and documentation from a central piece of information.

Spring MVC REST refers to REST support in Spring's MVC framework. Spring is modular application development framework for enterprise Java. It allows you to focus on your business problem rather than the plumbing code that connects components and systems. Spring MVC REST allows you to annotate java methods (that contains just business logic) which are supposed to communicate using REST. The framework then generates plumbing code for these methods and add it to you application.
Alternative to Spring MVC REST can be Jersey.

By using these three tools you gain the big advantage of writing your testing app completely suited for your needs. That's of course the biggest disadvantage too as you get absolutely no functionality out of the box.

Summary
So that was some apps for testing. Except JUNITs + maven build + Spring MVC REST which is too general, they all have one thing in common. They just inspect and evaluate HTTP requests/responses and compare them against expected values you entered. They doesn't lead you to create truly RESTful API, nor they test if it is really RESTful.

We can ask ourselves if we can combine these apps to solve more then what single app offer. Well only particularly. SoapUI has great GUI for creating request and view responses. JMeter is shifted more to performance testing. FitNesse offers wiki based UI for inserting tests. Fiddler can control packet flows between your app and world. But even if we combine them, we still just get evaluation of response against expected response. We are still not able to test REST constraints.

So if we are going to create new testing app. Based on all these findings we can say we don't need another request/response inspector. We have to focus on testing REST constraints compliance and let other tools do the "expected response evaluation".