At DataGrail, we build a lot of integrations with 3rd party web APIs. Some APIs are a joy to work with, while some are a nightmare. In this blog series, we will look at design and feature choices that can make or break an application programming interface. Note that we will primarily focusing on RESTful web APIs.
Authorization & Authentication
Figuring out how to properly authorize an HTTP request is generally our first interaction with a SaaS API, and it tends to set the tone for how painless or how frustrating the rest of our integration experience is going to be.
Instead of debating the security merits of common authorization & authentication patterns, let’s talk about implementation choices. Whatever approach – basic auth, bearer tokens, OAuth flows – we recommend that it be straight-forward, well documented, and not to stray from convention without a good reason.
Here our some of our thoughts on auth:
OAuth2 results in a better user experience – When asking a user to connect one of their SaaS apps to DataGrail, it is so much nicer when all that user needs to do is click a button, login to that 3rd party service, and confirm the connection by clicking ‘Accept’. Other auth types often require guiding the user through a number of additional steps in a 3rd party system to find, copy, and paste various bits of information. OAuth2 is just better for the end user.
Keep your base urls consistent – This applies to all auth types, but let’s talk about OAuth. OAuth2 requires an authorization request and a token exchange. Typically, you will hit an ‘/authorize’ and a ‘/token’ endpoint. Occasionally, we run into services that implement paths that are slightly different from convention, which is a minor inconvenience (and makes us grumble a bit). Very rarely, the base url will change between the two requests… why do this?! As a best practice, the recommendation is to keep your base url’s consistent across the API.
Custom Auth… maybe don’t do it? – If you plan on going through the trouble of designing and rolling your own auth pattern (assuming you have a really good reason for doing it), please be kind to the developers that need to use it! We have integrated with some services that decided to go with a somewhat custom HMAC pattern that was very sparsely documented. The result was a much longer development time and a lot more hair pulling than we would have liked.
Testing
Each service with which we integrate needs to be tested thoroughly before we release. It is very clear that some services, when designing their web APIs, keep the developer and their testing needs in mind.
Here are a few things to consider to make my life easier when building integrations:
- Developer Accounts – Come on, don’t make us buy a premium account just so we can test an integration with your product. Stripe has a really cool (and free) developer account that allows toggling test data on and off and other fun testing features.
- Test Endpoints – We love a `GET /ping` or a similar endpoint that is fast and returns a status 200 or 204. We just want to make sure that our request is well formed and our creds are valid! A number of services, like Okta, Sendoso, and Hubspot provide a simple `/me` or `/users/me` which we find to be great test endpoints.
- Sandboxes – These environments are awesome, and in some cases – think economic transactions or emailers – sandboxes are required. Fake data to play around with and no chance of breaking something or irreparably changing someone’s data? A+. Webhook sandboxes are especially cool if you let us simulate and fire off different events on demand. Sendgrid allows us to enable sandbox mode in the request body, so that we don’t accidentally send marketing emails to 200 people when trying to test something out.
- OAuth Redirect – For test apps, we strongly recommend allowing developers to redirect wherever we want. Redirecting to localhost to test an OAuth flow makes things so much easier. I would prefer not to deploy or live edit code on a virtual box to test it out. Hubspot lets us redirect to localhost in development… thank you!
Rate Limiting
Our perspective – it is better when 3rd party APIs do not have a rate limit. However we recommend that rate limits be reasonable and are communicated clearly in API docs.
One application (which shall remain nameless) set up a complicated scheme that looked more like a freemium currency model than rate limit documentation. Different HTTP methods and endpoints were given different scores. Different account levels could ‘spend’ higher scores each day.
On top of that weirdness, there was not a single operation that could be performed in one request-response cycle. For example, this app had a webhook design that required 3 back-and-forth requests before getting the webhook event data – “charging” you for each request…
Can’t we settle on something simpler… “5 requests/second up to 80k requests per day”?
Status Codes
This one is pretty straightforward… we recommend that the correct status codes in HTTP responses from web APIs reflect the actual status of the response.
If we ask you to create a new contact or some other record, and you say “Sure! 200 OK,” but hidden in the response body is some random Status: 1F. Now, we have to go to the API docs to figure out that 1F translates to 401 Unauthorized. Honestly though, what is the point of all of that? If the HTTP response status code was 401 to begin with, that is all we really needed!
If I go to a restaurant and order coq au vin, and the waiter comes back and says “Here is your coq au vin, sir” and he hands me a plate of human hair… I am going to be pretty upset.
API Functionality
We recommend that the application programming interface mimic the user interface as much as possible.
Imagine two users of an application, with the same permissions. All else being equal, the first user logs into the app from a browser, the second communicates with the app from the command line. Both users should have access to the same basic functionality, right?
Okay, okay. There are legitimate business and UX reasons why functionality might differ across platforms. But IDEALLY, the user should be able to get, create, edit, and delete all the same information from the UI or the API.
Sometimes asymmetrical access can be error prone – think about asking a non-technical user to configure a webhook or an OAuth application in the UI.
Sometimes, limited access makes a web API unusable. As an extreme example, one application stood up an API that only allowed DELETE requests. What are we supposed to do with that?!
Wrapping up
In part 1 of this series, we touched on high level choices that are made about how an API can be accessed, how easy it is to use, and what it should be capable of doing. Designing any interface can be difficult, but with some consideration and forethought about who will use the interface, how often, and how they plan to use it, you will be empowered to start off on the right foot. We haven’t even really touched on response models, error messages, base urls, pagination, or a host of other important design decisions. So, stay tuned for the next part in the series!