Bryan Helmig

Co-founder of Zapier, speaker, musician and builder of things.

This post is adapted from a talk I gave at APIdays SF, view slides here.

At Zapier, we sit at a unique crossroads between APIs and lots of users. Because of our position, we’re noticing a shift in API consumption. While many of our users have no idea that everything they do on Zapier is powered by an API, many of them are becoming more aware of that fact. Consequently, they’re asking us for API advice, they’re pinging their vendors about API related limitations, and they’re starting to get comfortable with the idea that an API can be a powerful tool for their toolbox.

Traditionally, you’d expect most inquiries about APIs to be very technical: How can I upload bulk attachments with multipart POST bodies? Increasingly, questions are becoming more like amateur Stack Overflow questions: How do I hook the Microsoft Access into the REST? The users behind such questions aren’t tech-illiterate, they simply have no idea how to code and aren’t familiar with the lingo. They don’t really know where to start, but they want to know about your API.

This is awesome.

So who exactly are these users and what are their skill-sets? They defy universal categorization, but here are a handful of examples:

  • The lone IT worker who has only ever replaced PC hardware and installed software before. Now they are tasked with setting up and migrating to their first cloud app.
  • The tech-savvy blogger who knows how to install WordPress on shared hosting. They can, after much trial and error, tweak the HTML and CSS of their blog to suit their style.
  • The small business owner who has cobbled together a powerful Excel spreadsheet to organize their business, but realizes it can’t scale past one or two employees.

The reason why non-coders are becoming interested in your API is actually quite simple: APIs are the only means by which they could possibly implement some vital but missing feature. It is worth noting that these missing features are also very long-tail, which means that you, as the vendor, probably won’t add that functionality in the native app. While you’ve got the 90% use case nailed down, they require that last 10%.

All of these users are tech-savvy, but don’t code. All of these users are DIY’ers that invested a lot of time into getting your SaaS set up and working almost perfectly. But they hit a wall, they started researching and without fail, they’ll find your API. Then what? The first thing they do is look at your docs just like any developer would, but then they suddenly feel one of two things: empowered or overwhelmed.

Before we can start addressing the issue of making APIs more accessible to those who cannot code, we need one more piece of information: what are the goals of these non-coders? What features are they missing? Unfortunately, because they are inherently long-tail features, there isn’t a single answer. But we can categorize them. Consider these common requests:

  • Can we copy records to another project?
  • Can we import/export/backup our records via insert format?
  • Can we move or modify a record after we apply a special tag?
  • Can we get an overview email every night/week/month?
  • etc…

In the absence of these features, the only other option is to perform this work by hand. That means wasted hours on work that could easily be automated. The features aren’t usually that complicated either, a slight remixing of basic CRUD operations usually does the trick.

We’ll call these sorts of features CRUD automation.

So now that we’ve identified these non-coders and figured out what sorts of things they want do do, we’re left with two quintessential questions:

  1. What can we do to help non-coders consume APIs?
  2. What can we do to improve APIs for CRUD automation?

Just to set your mind in the direction: The answer to the first question addresses the empowerment issue and revolves mostly around your API experience. The answer to the second question mostly revolves around reducing complexity and common API “gotchas”.

Also, these ideas can make your API more joyful to user for even the most seasoned of neck-beards.

Let’s get started.

What can we do to help non-coders consume APIs?

First up is easier documentation. Note I didn’t say better documentation, because that can be highly subjective and varies with the audience. The best documentation for an extremely talented developer might just give them the facts as fast as possible: here’s how to auth, here are the endpoints. We’re not suggesting that here.

Instead, focus on the introduction that your landing page gives to your API. This is the point where you need to be very encouraging and talk about use cases, individual success stories and other humanizing bits. Skip the jargon. Again, you want to empower the user and open their eyes to the possibilities that lay before them.

Next, provide a wedge by spoon feeding some great “hello world!” examples to the user as soon as possible. These sorts of DIYers are the kind that take chances on tech: jailbreaking their smart phone, editing their WordPress theme’s HTML & CSS, and creating crazy spreadsheet formulas. If you can get them to upload your simple PHP example to their shared hosting, they can iterate from there. With enough wind at their back, they’ll even brute force a solution. So, give them a good wedge and they’ll surprise you every time.

Finally, provide live data right in the docs. Their experience with your app so far is via the standard web UI which doesn’t always map cleanly to your API resources or endpoints. As soon as you inline some examples of real data they’ll recognize, they can make that connection much faster.

Second up is multiple auth mechanisms. This will almost always be the first thing a user has to grok and implement before they can even get started. So it might be the most vital of technical details. If they can get past this, things are looking good!

The best thing you can do is skip OAuth and provide API key or basic authentication. Don’t get me wrong, OAuth is wonderful for platforms, and if you are doing a platform where users can publish apps you need to do OAuth. But for your average lone wolf, access tokens and refresh tokens are rather annoying and confusing, especially when they expire! At the very least generate access tokens and don’t expire them, but you’d be better served to just do API keys or basic authentication too.

Additionally, make your example code snippets live code. That means you hand them a loaded gun in the form of code that will do something on their account when ran. Insert the real tokens right into the the code.

Third up is useful error messages, which is not the same as descriptive error messages. This just means you shouldn’t just tell the user what went wrong, but what they should do to fix it.

Consider a 401: Not Authenticated error, that alone doesn’t really help a really green user move forward. Instead, 401: Could not authenticate this request. Try adding an API key to the URL like this: https://:current-url:?api_key=123abc gives a much better next step for the users. Of course, if they just paste in the example (and invalid) key, tell them where to get the key next!

Also, always serialize your errors the same way. This may seem like common sense, but I’ve seen these three errors more than I’d like to admit:

  1. Sometimes a list of errors, sometimes a single string message.
  2. No matching URL routes giving HTML 404s instead of JSON/XML.
  3. 500 errors giving HTML error pages instead of JSON/XML.

The final piece of advice to help non-coders with APIs is a little out there: don’t do what I say, do what I mean. Basically, if you can write a sufficiently intelligent and prescriptive error message, you could have probably just done what the user had meant to do in the first place. It can get tricky, but as you write the really nice error messages, just think: could I just do this for them? You can only do this in some special cases; you can’t bend the rules for something like authentication.

What can we do to improve APIs for CRUD automation?

A common theme here will be complexity reduction. Any non-coder is going to have a hard time managing complexity, especially around data manipulation and traversing. Remember, even syntax is going to trip them up quite a bit, so the more your API can abstract away, the more they can skip coding and continue playing.

First up are flexible endpoints which are a must for enabling efficient automation. The reasoning is fairly straight forward: filtering, paging, and merging data from an API in code is a cumbersome and error prone process even in the best of situations.

Also, the long-tail nature of these automation opportunities mean they won’t be easily predicted. So if you can provide a slew of query-string arguments that help a user only get at the data they want, you’ll make their lives much easier. Besides, an extra WHERE (for filter) or JOIN (for inlining) is much more efficient than paging through every page of results or merging extra resources from extra API calls. Your servers will thank you.

As a final point around flexibility: a majority of users won’t know a lot about big O notation. That means they are a few nested for-loops away from attempting to make more API calls than there are atoms in the known universe. Just inline the related resources.

Second up are real time endpoints. At first glance, a real time enabled API seems to add complexity, but that is only because you are looking at it through your own lens. Real time (if done correctly) is vastly simpler for API consumers. The complexity is all on your side.

The reason for all the complexity is because naive read operations involve polling if you want to identify new data. Polling means infrastructure (via crons or queues) and deduplication logic of some sort. Worse, it is basically hidden complexity; a user will get started and be suddenly blindsided when they realize they need to filter out data they’ve already seen.

No matter how you slice it, polling sucks. It wastes resources, adds complexity, and is slow. Let’s just agree to kill polling once and for all.

The solution has been with us for a long time: webhooks. Not only should you be able to create them in the GUI (and as many as you want, not just one), you should be able to create them via the API too. Plus webhooks are dead simple: point them at some PHP script and capture the POST body. They make for very nice “Hello world!” examples.

Just skip long-polling and websockets for now, they just add complexity and infrastructure on both sides. And until node.js or event based programming is de-facto on shared hosting it is unlikely to change.

Third up and final point for making APIs easier to automate, provide PATCH and upsert capabilities.

PATCH simply removes the null value gotcha if you forget to include an original value during a PUT update. The particularly nasty bit about a PUT nulling values is that the discovery is often delayed and the rogue PUT is hard to identify as the source. Lots of APIs already do this with PUT, but it is nice to be explicit about it no matter what you decide on.

Upsert removes the complexity around managing uniqueness in an API. It sort of wraps the whole get or create pattern that is fairly common in ORMs: it will either insert or update a record based on some included unique field (often email).

Again, we want to handle the complexity of insert or updating records so the user doesn’t have to.

Who is at the end of your API requests?

Before we wrap up, its worth considering that even if a brilliant coder did the grunt work of building a feature based on an API, it is possible a less skilled coder will attempt to expand on or will inherit that body of work.

Also, it is unlikely that the end-user of said API enabled feature is a coder either, so when things break, the above tips can be the difference between “it doesn’t work” and “ah, I think I can fix that”.

As a parting thought, there are a lot more non-coding but tech-savvy users out there than there are coders. This deviation towards the mean is something to celebrate and embrace.

Remember, your API consumers aren’t always who you think they are.

Posted June 24, 2013 @ 1:24 pm under Startups, Tech.