What is API Testing?

What is an API?

An Application Programming Interface (API) at its core is a formal specification that acts as a guaranteed contract between two separate pieces of software

Modern computer systems are generally designed using the ‘layered architecture approach’:

This means that the core functionality of the system is contained within the “business logic” layer as a series of discrete but connected business components. They are responsible for taking information from the various user interfaces (UIs), performing calculations and transactions on the database layer and then presenting the results back to the user interface.

However, in addition to communicating with human beings via the UI layer, computers systems have to be able to communicate directly with each other. For example, your mobile ride sharing application will need to communicate with the mapping service, the traffic and weather services and other specialized applications used by the drivers providing the rides. In the modern, interconnected world, we take for granted that all these different systems can speak to each other seamlessly, in reality that would not be possible without APIs.

How is An API Defined?

As mentioned above, an API at its core is a formal specification that acts as a guaranteed contract between two separate pieces of software. The API provider defines the set of operations, data formats and protocols that it expects, and the consumer of the API (called the client) will use those rules on the understanding that, as long as it follows the rules, the client will always be able to use the API without having to worry about the internals of the API itself:

Why are APIs important?

The importance of APIs is that it lets different organizations create software applications that rely on other application and services without having to constantly update their application when the internals of the dependent applications or services change. As long as the API itself remains stable, the internal implementation can change. This is an important feature of APIs, they consist of a part that doesn’t change.

Important features of an API consist of:

• The interface or contract: the part of the API that doesn’t change and specifies the operations, data formats, and behaviors.

• The implementation: the part of the API that can change as needed.

Dealing with API Changes

So, what happens when you want to change an API and expose new functionality? You basically have two choices:

• Choice 1: Change the existing API to reflect the updated version. However, this is called “breaking compatibility” and means that all clients of the API will need to be updated. Sometimes this is necessary but it should be avoided if possible. This is especially true for applications that are widely used and have many applications dependent on them.

• Choice 2: Create a New API Version and Leave the Old API. This is the recommended option where possible. You create a new API to expose the new functionality but leave the old API in place for existing clients. You may need to add a translation layer to ensure the old API behaves exactly the same as before.

So now that we have established what an API is and why APIs are critical to modern interconnected, globally distributed applications and services, it is important to understand why API testing is critical.

API Testing

API testing involves testing the application programming interfaces (APIs) directly and as part of integration testing to determine if they meet expectations for functionality, reliability, performance, and security.

Why is API Testing Important?

The move to cloud computing has highlighted the importance of Application Programming Interfaces (APIs). With the rise in cloud applications and interconnect platforms, API testing is a necessity. Many of the services that we use every day rely on hundreds of different interconnected APIs, if any one of them fails then the service will not work!

What's the Difference Between API Testing and GUI Testing?

API testing is a testing method of verifying that the web API responds to requests from the client applications in the correct way. GUI testing is a method of verifying that the user interface behaves as expected.

Since APIs lack a GUI, API testing is performed at the message layer. API testing is critical for automating testing because APIs now serve as the primary interface to application logic and because GUI tests are difficult to maintain with the short release cycles and frequent changes commonly used with Agile software development and DevOps.

When you release a new version of the system (e.g. changing some of the business components or internal data structures) you need to have a fast, easy to run set of API regression tests that verify that those internal changes did not break the API interfaces and therefore the client applications that rely on the APIs will continue to function as before.

Benefits of API testing

Time effective

API tests require less scripting efforts as compared to GUI tests. GUI test time is spent on loading and rendering web pages and UI elements that might not even be needed in the end, which can slow down the testing process. API testing reduces all these tasks to one and as a result, the testing process is faster with better coverage.

Testing cost reduction

 API testing can provide faster results with early defect diagnosis, which reduces overall testing costs.

Access without UI

API testing allows access to the application without a user interface or users interacting with the system. API tests can run without needing to experience the software application which gives an early insight into defects and errors. This allows for early issue resolution before the GUI is impacted.

Integratable testing

API testing allows for highly integrable tests. This is especially beneficial if you plan to perform functional GUI tests following your API testing.

Improves testing coverage

Testing the code-level functionality of an application provides an early evaluation of its overall build strength. This exposes small errors before they become larger problems during GUI testing. Core access enables testing in tandem with development, fostering communication and improved collaboration between both teams.

Language-independent

An API test exchanges data using XML or JSON. These transfer modes are completely language-independent, meaning that you can select any core programming language.

What are the types of API Testing?

1. Functional testing

Functional tests validate the software system against functional specifications and requirements. To test each function of the API, a tester provides appropriate input parameters in XML or JSON payload, then verifies the output against the functional requirements.

2. Security testing 

API security testing reveals any risk, vulnerability, or threat within the API. Security testing identifies all potential loopholes and API weaknesses that can result in a loss of information, revenue, and reputation if misused by an unauthorized user.

3. UI testing 

UI testing is a test of the user interface for the API and other integral parts. UI testing focuses more on the interface which ties into the API rather than the API testing itself. Although UI testing is not a specific test of API in terms of the codebase, it still provides an overview of the health, usability, and efficiency of the frontend and backend.

4. Penetration testing 

Penetration testing emulates an external attacker or malicious insider specifically targeting a custom set of API endpoints and attempting to undermine the security in order to impact the confidentiality, integrity, or availability of an organization's resources.

5. Load testing

Load testing emulates a constant steady stream of traffic at both normal and peak conditions. This ensures that the APIs do not contain memory leaks or other similar defects that might cause issues after running for a prolonged period of time.

Load testing often happens after a specific unit or the whole codebase has been completed. 

6. Runtime/ error detection testing

Runtime error detection testing is related to the actual running of the API and the universal results of utilizing the API codebase. This test focuses on monitoring, execution errors, resource leaks, or error detection.

7. Fuzz testing

Fuzz testing or noise testing sets operation parameters to unexpected values in an effort to cause unexpected behavior and errors in the API backend. This helps discover bugs and potential security issues.

8. Validation testing

Validation testing helps ensures that the software meets the business requirements and is seen as an assurance of the correct development. Validation testing occurs among the final steps and plays an essential role in the development process. It verifies the aspects of product, behavior, and efficiency.

9. Interoperability and WS compliance testing

 Web Services and Interoperability Compliance testing pertains to SOAP APIs and guarantees that they comply with Web Services Interoperability Profiles. The compliance is checked to see if the predetermined requirements are met.

API Testing Approach

API testing is performed by submitting requests to the software using the application’s programming interface of the application and then checking if it returns the expected data.

Create API testing requirements

To be able to plan API tests, first, you need to determine testing boundaries and requirements. This includes:

• the purpose of the API 

• the workflow of the application

• the functions of the API

• APIs this API interacts with

• aspects of testing

• the problems that are being tested for

• testing priorities

• the desired output

• The definition of Pass and Fail

Knowing the purpose of the API will set a firm foundation to prepare your test data for input and output.

Establish the API testing environment 

Setting up an API test environment requires the configuration of the servers, databases, and every resource the API interacts with, depending on the software requirements.

Make a trial API call 

Before diving into thorough testing, make an API call just to check that nothing is broken and the API is operational.

Define the input parameters

Plan all possible input combinations to use them in test cases, and to authenticate the results to determine whether the API performs as expected.

Create API test cases

Execute test cases and compare actual results with the expected ones. A good practice is to group them by test category. 

Examples of API test cases include:

• testing value in response to the input condition: define the input and authenticate the output, which can be any type of data or status

• checking the behavior of the API in the system when there is no return value in response

• tracking events and interruptions triggered by the output

• authenticating the effect on the system after data structure updates

• validating resources modified by the API call

Considerations for Performing API Tests

The following factors should be considered when performing API testing:

• Understand API requirements: ensure you know the purpose of the API and the workflow of the application. 

• Specify the API output status: you need to verify the response status code in API testing. All API response status codes are separated into five classes.

There are five values for the first digit: 

• 1xx (Informational): The request is received and continues to be processed

• 2xx (Successful): The request is successfully received, understood, and accepted

• 3xx (Redirection): Further action needs to be taken to complete the request

• 4xx (Client Error): The request contains the wrong syntax or cannot be fulfilled

• 5xx (Server Error): The server fails to fulfill an apparently valid request

The first digit of the code defines the class of the response. Use these codes to determine the output of your API. Determine if the code follows global standard classes or if the code is specified in the requirement. 

Analyze the validation response: during the API testing process, a request is raised to the API with the known data. This way you can analyze the validation response. While testing an API, you should consider: 

• Accuracy of data

• Schema validation

• HTTP status codes

• Data type, validations, order, and completeness

• Authorization checks

• Implementation of response timeout

• Error codes in case API returns, and

• Non-functional testing like performance and security testing

Focus on functional APIs: focus on the simple APIs that have only one or two inputs. These are necessary access points to further APIs. By focusing on these before moving on to test the others you ensure that the initial points of access work.

Common Types of API Technologies

Over the years, APIs have evolved from simple code libraries that applications could use to run code on the same computer, to remote APIs that can be used to allow code on one computer to call code hosted somewhere else.

Here is a quick list of the more common API technologies that exist in approximate chronological order:

• TCP/IP Sockets

• Remote Procedure Call (RPC)

• Common Object Request Broker Architecture (CORBA)

• Java Remote Method Invocation (RMI) and Enterprise Java Beans (EJBs)

• Microsoft Distributed Component Object Model (DCOM) – also known as ActiveX

• Web Services (SOAP then REST)

Deciding on an API Testing Tool

When looking at an API testing tool, it is important to understand which API technologies you will be using and how best to test them. Nowadays most APIs you will come across will be of the Web Service variety (either REST or SOAP), but you may come across other technologies such as Java EJBs or Microsoft DCOM/ActiveX DLLs.

Web Service API Testing

A Web service is a unit of managed code that can be remotely invoked using HTTP, that is, it can be activated using HTTP requests. So, Web Services allows you to expose the functionality of your existing code over the network. Once it is exposed on the network, other application can use the functionality of your program.

There are two broad classes of web service:

• Simple Object Access Protocol (SOAP)

• REpresentational State Transfer (REST)

Testing SOAP Web Services

SOAP web services make use of the Web Service Definition Language (WSDL) and communicate using HTTP POST requests. They are essentially a serialization of RPC object calls into XML that can then be passed to the web service. The XML passed to the SOAP web services needs to match the format specified in the WSDL.

SOAP web services are fully self-descripting, so most clients do not directly work with the SOAP XML language, but instead use a client-side proxy generator that creates client object representations of the web service (e.g. Java, .NET objects). The web service consumers interact with these language-specific representations of the SOAP web service.

However, when you are testing SOAP services as well as having a nice interface for viewing the provide services and invoking test operations, you need to always have a way to verify the raw SOAP request and response packets being sent in XML:

This feature in particular distinguishes a true SOAP solution from merely a SOAP client library. The former helps you test the service and understand failures, whereas the latter is just a way of making SOAP web service calls more easily from different programming languages.

In addition, you ideally want to be able to generate your test scripts programmatically from the invoked endpoints and automatically include validation checks:

The following features should be looked for in a SOAP web service testing tool:

• The ability to download a remote Web Service Description Language (WSDL) file and visually inspect all of its functions and data structures.

• There should be a way to graphically create the SOAP request by taking the functions and populating the required parameters and then viewing the response back from the API. It should handle both simple values (called primitives, such as integers, dates, strings) and more complex structured objects (e.g. a new user object)

• There should be a way to see the raw SOAP XML structure for both the sent request and the retrieved response. Ideally there should be a way to see the XML data nicely formatted so that it’s easier to understand the interactions.

• If possible the tool should support the different versions of SOAP (1.0, 1.1, 1.2) and also handle vendor-specific extension such as Microsoft Windows Communication Foundation (WCP) and ASP.NET Web Services (ASMX).

Testing REST Web Services

A RESTful web API (also called a RESTful web service) is a web API implemented using HTTP and REST principles. Unlike SOAP-based web services, there is no "official" standard for RESTful web APIs. This is because REST is an architectural style, unlike SOAP, which is a protocol.

Typically REST web services expose their operations as a series of unique "resources" which correspond to a specific URL. Each of the standard HTTP methods (POST, GET, PUT and DELETE) then maps into the four basic CRUD (Create, Read, Update and Delete) operations on each resource.

REST web services can use different data serialization methods (XML, JSON, RSS, etc.).

Traditionally the format used for REST Web Services was XML. This is partly because it was used extensively in SOAP web services and is therefore familiar, but also when bandwidth is not a limiting factor, it is self-describing, with the fields and data clearly described:

A common format used in web browser based APIs is JSON since it returns the data as JavaScript Object Notation (JSON) object which can be used directly in a web browser since it matches the format used by JavaScript to store arrays and objects. It is also a very compact format, making it ideal for communications on mobile networks with limited bandwidth.

When choosing a tool to perform REST web service testing, you should look for:

• The ability to prototype and preview the HTTP request, with the ability to specify the HTTP headers, body, method and standard HTTP credentials.

• The ability to see the request and response bodies in a variety of formats, including JSON and XML. The best tools offer automated formatters to make the data easier to enter and view.

• The ability to record your actions and generate a programmatic test script that can be executed automatically for regression and API testing. If possible the tools should provide a way to convert the request and responses into software “objects” that make dealing with the data easier.

• If possible, there should be a way to parameterize the REST requests so that you can create generic REST test functions that can be used in the test script in different ways without having to rewrite each time.

Microsoft DCOM/ActiveX API Testing

Microsoft's Component Object Model (COM) also known as ActiveX is a standard for communication between separately engineered software components (source). Any object with a COM interface can be created and used remotely:

var doc = new ActiveXObject("Word.Application");

doc.Documents.Open(wordFileName);

Using this approach, any API packaged as a COM or .NET accessible Dynamic Linked Library (DLLs) can be tested natively by testing tools such as Rapise.

Rapise from Inflectra provides support for testing the following different types of DLL API:

• Managed DLLs – written using the .NET Framework, Rapise provides special access classes for testing these APIs.

• Unmanaged DLLs – written using native Intel x86 code, Rapise provides a special DynamicWrapper than makes it easy to access these APIs