Performance and Load Test your REST APIs with Visual Studio and Swagger

Quality is about both building the right thing and building it right.  Building the right thing is about ensuring your application meets functional requirements.  Equally important is building it right by verifying the appropriate non-functional requirements (NFRs) for your application to meet.  NFRs are about ensuring that your application has acceptable performance, security, and maintainability to name a few.  Each application change can positively or negatively affect these NFRs so it is important that these NFRs should be verified early and throughout the lifecycle of the application.   Measuring that your application responds within the specified times and operates under load are two key NFRs that performance and load tests can validate. If you need professional application load testing, Apica Systems offers this service.

Editions of Visual Studio have included performance and load testing tools for a number of years.  Visual Studio Web Performance and Load test projects provide an excellent tool for measuring the performance of RESTful APIs with technologies such as Web API.  VS Performance and Load Testing tools predate RESTful services and JSON but there are a couple ways to effectively use VS for testing these services.  One way I found useful and efficient is using the Swagger pages to help record the service calls.  If you haven't used swagger before it is essentially a tool for documenting the APIs that is also a built in test harness.

The Swagger pages can be dynamically generated from any Web API project by adding the Swashbuckle Nuget package.  These might sound like some funny names but these two packages are quite useful for documentation and testing.  In addition to using the Swagger pages for manual testing, these pages allow you to easily use the WebTest record capabilities in Visual Studio.   Here are a few tips for creating these.

I am using simple Values controller found in the Visual Studio Web API project template.  Be sure to choose no authentication when creating the project.  This provides but sufficient examples for getting started.  We will use this for the remainder of the article.  Once you create the project, add the Swashbuckle nuget package, run the application, and navigate to http://yourapi/swagger.  The default pages will display.

The site shows you all of the operations available against the Values resource.  Let's try one to see how this is much more than just static documentation.  Click on the GET for /api/Values to expand it.  In addition to showing a sample of the return JSON, there is the "Try it out!" button.  Click this to call the API and view the result.

We can use the same steps to record our performance test and make it reusable.  Add a Web Performance Test project to the solution.  Run the Web API project and then click on record for WebTest1.  Click on a couple actions.  I recorded the Values GET and performed a POST.  The captured Urls include a combination of the Swagger pages and the actual API calls.  For performance testing, we are only concerned with load testing the APIs so the swagger pages can be removed.

Once the files have been removed, the two calls that we care about remain.  You can view the JSON payload.  You can do simple value replacement by manually inserting variables into the JSON using the {{variablename}}.  I'll share some helper functions I have created in an upcoming post.

To run the performance, simply click on the run button highlighted above.  This will replay all of the activities.  There could be some clean up.  If your client is JavaScript and supports CORS, the tests could record some OPTIONS calls.  These will fail on playback so I found removing them is the best option.  Second, some pages make dynamic dependent calls to additional resources and pages.  I have found sometimes these do not replay very well.  In these cases I will choose to mark Parse Dependent Requests to False.  If you running analytics tools like Google Analytics or App Insights, your tests will record the JavaScript calls to these services.  These could require manipulation of the JSON payload to succeed.  For example in the App Insights calls, the timestamp needs to modified so it will always pass the current date time.  Since I'm really after performance testing the server, I usually just delete these analytics calls.  Authentication can be required to call your services.  I'll do an upcoming post on testing Oauth2 based services as it outside of the scope of this.

Once you have your performance tests created, you will want to add one or more to a load test. There is nothing different about these than any other web test.

As you can see you can quickly create performance and load tests for Web APIs or any RESTful APIs.  Create these tests with Visual Studio and rerun them often to help ensure that performance issues are not introduced throughout the project and don't wait until the end of the project to run these.  Often it is too late to fix these performance issues and will either negatively affect the release date or the users' experience.

I hope you found this helpful.  Let me know if you have any questions or comments.  You can reach me at @mikedouglasdev.