6 November 2018

API documentation

by mo


I recently had to create API documentation for an API that I have been building.

I wanted a system that would:

  • display API request headers
  • display API request body
  • display API response headers
  • display API response body
  • provide example curl requests
  • automatically generated using the rspec test suite so that it stays up to date.

I decided to use a combination of tools to accomplish this goal. The tools I used were:

The application is a ruby on rails application. The layout of the application is:

も tree -L 3
.
├── app
│   └── ...
├── config
│   ├── ...
│   ├── jekyll.yml
│   └── ...
├── config.ru
├── db
│   └── ...
├── doc
│   ├── ...
│   ├── _includes
│   │   ├── curl.erb
│   │   ├── oauth-dynamic-client-registration.html
│   │   └── ...
│   ├── index.md
│   └── _posts
│       ├── 2018-10-28-oauth-dynamic-client-registration.markdown
│       └── ...
├── lib
│   └── ...
├── log
│   ├── ...
├── package.json
├── package-lock.json
├── public
│   ├── ...
│   ├── doc
│   │   ├── oauth
│   │   └── ...
│   └── ...
├── Rakefile
├── spec
│   ├── documentation.rb
│   └── ...
├── tmp
│   ├── ...
│   ├── _cassettes
│   │   ├── oauth-dynamic-client-registration.yml
│   │   └── ...
│   └── ...
└── ...

The three most important folders are:

  • app: The rails application code.
  • doc: The location of the jekyll source files.
  • spec: The rspec test suite code.

The API documentation _includes are generated using the following command.

も rspec spec/documentation.rb

When the tests in that file run, VCR is configured to record cassettes in /tmp/_cassettes. At the end of the test suite, the recordings are converted to jekyll _includes using an erb template named curl.erb.

An example of a VCR recording:

---
http_interactions:
- request:
    method: post
    uri: http://127.0.0.1:45155/oauth/clients
    body:
      encoding: UTF-8
      string: '{"redirect_uris":["https://harvey.name","https://brown.ca"],"client_name":"Kandra Treutel","token_endpoint_auth_method":"client_secret_basic","logo_uri":"https://osinskipouros.name","jwks_uri":"https://wiegand.info"}'
    headers:
      Accept:
      - application/json
      Content-Type:
      - application/json
      User-Agent:
      - net/hippie 0.1.9
      Accept-Encoding:
      - gzip;q=1.0,deflate;q=0.6,identity;q=0.3
  response:
    status:
      code: 201
      message: Created
    headers:
      Cache-Control:
      - no-cache, no-store
      Pragma:
      - no-cache
      Content-Type:
      - application/json; charset=utf-8
    body:
      encoding: UTF-8
      string: '{"client_id":"dccee95b-3647-4748-b3f8-2a936cd4def7","client_secret":"u657JJNEci9a92ewkjMpTmR3","client_id_issued_at":1541440251,"client_secret_expires_at":0,"redirect_uris":["https://harvey.name","https://brown.ca"],"grant_types":["authorization_code","refresh_token","client_credentials","password","urn:ietf:params:oauth:grant-type:saml2-bearer"],"client_name":"Kandra Treutel","token_endpoint_auth_method":"client_secret_basic","logo_uri":"https://osinskipouros.name","jwks_uri":"https://wiegand.info"}'
    http_version: 
  recorded_at: Mon, 05 Nov 2018 17:50:51 GMT
recorded_with: VCR 4.0.0

The VCR recording is converted to an html jekyll include using the following erb template. (I had to remove leading backticks to get it to render on this page)

<% @configuration['http_interactions'].each do |interaction| %>
#### <%= interaction['request']['method'].upcase %> <%= interaction['request']['uri'].gsub(/\h{8}-\h{4}-\h{4}-\h{4}-\h{12}/, ':id') %>

Example curl request:
<% headers = interaction['request']['headers'].map { |(key, value)| "-H \"#{key}: #{value[0]}\"" } %>
``bash
$ curl <%= interaction['request']['uri'] %> \
  -X <%= interaction['request']['method'].upcase %> \
  -d '<%= interaction['request']['body']['string'] %>' \
  <%= headers.join(" \\\n  ") %>
``
Request:
``text
<%= interaction['request']['headers'].map { |(key, value)| "#{key}: #{value[0]}" }.join("\n") %>
``
``json
<%= JSON.pretty_generate(JSON.parse(interaction['request']['body']['string'])) rescue nil %>
``
Response:
``text
<%= interaction['response']['status']['code'] %> <%= interaction['response']['status']['message'] %>

<%= interaction['response']['headers'].map { |(key, value)| "#{key}: #{value[0]}" }.join("\n") %>
``
``json
<%= JSON.pretty_generate(JSON.parse(interaction['response']['body']['string'])) rescue nil %>
``
<% end %>

The generated includes looks like.

POST http://127.0.0.1:45155/oauth/clients

Example curl request:

$ curl http://127.0.0.1:45155/oauth/clients \
  -X POST \
  -d '{"redirect_uris":["https://harvey.name","https://brown.ca"],"client_name":"Kandra Treutel","token_endpoint_auth_method":"client_secret_basic","logo_uri":"https://osinskipouros.name","jwks_uri":"https://wiegand.info"}' \
  -H "Accept: application/json" \
  -H "Content-Type: application/json" \
  -H "User-Agent: net/hippie 0.1.9" \
  -H "Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3"

Request:

Accept: application/json
Content-Type: application/json
User-Agent: net/hippie 0.1.9
Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
{
  "redirect_uris": [
    "https://harvey.name",
    "https://brown.ca"
  ],
  "client_name": "Kandra Treutel",
  "token_endpoint_auth_method": "client_secret_basic",
  "logo_uri": "https://osinskipouros.name",
  "jwks_uri": "https://wiegand.info"
}

Response:

201 Created

Cache-Control: no-cache, no-store
Pragma: no-cache
Content-Type: application/json; charset=utf-8
X-Request-Id: 3ba9a945-e78a-4dee-8889-c6f5ea8a5eca
Transfer-Encoding: chunked
{
  "client_id": "dccee95b-3647-4748-b3f8-2a936cd4def7",
  "client_secret": "u657JJNEci9a92ewkjMpTmR3",
  "client_id_issued_at": 1541440251,
  "client_secret_expires_at": 0,
  "redirect_uris": [
    "https://harvey.name",
    "https://brown.ca"
  ],
  "grant_types": [
    "authorization_code",
    "refresh_token",
    "client_credentials",
    "password",
    "urn:ietf:params:oauth:grant-type:saml2-bearer"
  ],
  "client_name": "Kandra Treutel",
  "token_endpoint_auth_method": "client_secret_basic",
  "logo_uri": "https://osinskipouros.name",
  "jwks_uri": "https://wiegand.info"
}

Then in the jekyll site these partials are referred to using liquid syntax.

 {\% include oauth-dynamic-client-registration.html \%}

I can include any additional information that I like on each page.

How?

Most of the magic happens in rspec before/after blocks.

# frozen_string_literal: true

ENV['RAILS_ENV'] ||= 'test'
require File.expand_path('../config/environment', __dir__)
require 'rspec/rails'
require 'vcr'

$server = Capybara::Server.new(Rack::Builder.new do
  map "/" do
    run Rails.application
  end
end.to_app)

RSpec.configure do |config|
  config.include(Module.new do
    def server
      $server
    end
  end)

  config.before :suite do
    puts "Booting"
    $server.boot
    print "." until $server.responsive?
    FileUtils.rm_rf(Rails.root.join('tmp/_cassettes/'))
    VCR.configure do |x|
      x.cassette_library_dir = "tmp/_cassettes"
      x.hook_into :webmock
    end
  end

  config.after :suite do
    erb = ERB.new(IO.read('doc/_includes/curl.erb'))
    Dir["tmp/_cassettes/**/*.yml"].each do |cassette|
      @configuration = YAML.safe_load(IO.read(cassette))
      result = erb.result(binding)
      IO.write("doc/_includes/#{File.basename(cassette).parameterize.gsub(/-yml/, '')}.html", result)
    end
  end
end

RSpec.describe "documentation" do
  let(:hippie) { Net::Hippie::Client.new }
  let(:scheme) { 'http' }
  let(:host) { server.host }
  let(:port) { server.port }
  let(:url_prefix) { "#{scheme}://#{host}:#{port}" }

  specify do
    body = {
      redirect_uris: [generate(:uri), generate(:uri)],
      client_name: FFaker::Name.name,
      token_endpoint_auth_method: :client_secret_basic,
      logo_uri: generate(:uri),
      jwks_uri: generate(:uri),
    }
    VCR.use_cassette("oauth-dynamic-client-registration") do
      response = hippie.post("#{url_prefix}/oauth/clients", body: body)
      expect(response.code).to eql('201')
    end
  end
end

I like this approach because it allows me to write tests that are used to generate API documentation examples.

Like the following:

💎