How difficult can it be to “just” get some data from somewhere. This post is copied almost verbatim from our largest codebase, where we try to write code that makes sense to both computers and humans. The file it’s copied from lives in the same directory as the modules it’s about, giving it low proximity and high usability.

Familiarity with TypeScript will probably make this easier to read. Agreeing that using try and catch for control flow is bad, or at least clunky, will make this a more pleasant read.

Integrations

This aims to be the home of all modules that abstract external systems. We call these modules integrations.

There are some patterns we like to use here to make sure that the caller can use any module in a safe and pleasant manner, following the concept that

Programs must be written for people to read, and only incidentally for machines to execute.

― Harold Abelson, Structure and Interpretation of Computer Programs

Specifying Outcomes

The caller mustn’t be required to care about the communication mechanisms that integrations use (often JSON over HTTP).

What matters is the outcome of an invoked action. We model this by returning { outcome } where outcome is an enum.

Mapping HTTP status codes to a well named outcome is common.

Example of a fictitious member.ts module:

import { memberApiUrl } from "api-urls";
import { authHeaders } from "auth";

export enum GetMemberOutcome {
  authenticationFailure,
  success,
  unknown,
}

export const getMember = async (
  token
): Promise<{ outcome: GetMemberOutcome }> => {
  const response = await fetch(memberApiUrl, { headers: authHeaders(token) });

  if (response.ok) {
    return { outcome: GetMemberOutcome.success };
  }

  if (response.status === 401) {
    return { outcome: GetMemberOutcome.authenticationFailure };
  }

  return { outcome: GetMemberOutcome.unknown };
};

Note that the unknown outcome is explicit. We often deal with APIs that are undocumented or documented poorly. Being able to indicate that we have no idea what happened is often useful.

The obove works fine when we don’t need to receive any data, but we are often requesting data and for that we add a value.

Adding Values

Let’s extend our previous return type with a value so we can pass data back to the caller.

type Member = {
  id: string;
};

export const getMember = async (
  token
): Promise<{ outcome: GetMemberOutcome; value: Member }> => {
  // Implementation removed for brevity
};

This seems to be fine, but what if the outcome is not success? We could make value optional, but that would require the caller to do null checking on value, which is not a very nice burden to put on someone. Let’s return multiple types instead, and have TypeScript help us with narrowing.

export const getMember = async (
  token
): Promise<
  | { outcome: GetMemberOutcome.success; value: Member }
  | { outcome: Exclude<GetMemberOutcome, GetMemberOutcome.success> }
> => {
  const response = await fetch(memberApiUrl, { headers: authHeaders(token) });

  if (response.ok) {
    return {
      outcome: GetMemberOutcome.success,
      value: (await response.json()) as Member,
    };
  }

  if (response.status === 401) {
    return { outcome: GetMemberOutcome.authenticationFailure };
  }

  return { outcome: GetMemberOutcome.unknown };
};

Now we’ve let our consumers, and our compiler, know that they can expect either a successful result with a value, or an outcome that is every GetMemberOutcome except GetMemberOutcome.success.

This lets our consumers do nice things like this:

const memberResult = await getMember(token);

switch (memberResult.outcome) {
  case GetMemberOutcome.success: {
    // memberResult.value is Member here
  }
  case GetMemberOutcome.authenticationError: {
    // memberResult.value does not exist here
  }
}

This becomes even better when different types of data is returned for different outcomes.

Let’s say that authentication failures return useful data that can be used further:

type MemberAuthenticationError = {
  reason: string;
};

export const getMember = async (
  token
): Promise<
  | { outcome: GetMemberOutcome.success; value: Member }
  | {
      outcome: GetMemberOutcome.authenticationError;
      value: MemberAuthenticationError;
    }
  | {
      outcome: Exclude<
        GetMemberOutcome,
        GetMemberOutcome.authenticationError | GetMemberOutcome.success
      >;
    }
> => {
  const response = await fetch(memberApiUrl, { headers: authHeaders(token) });

  if (response.ok) {
    return {
      outcome: GetMemberOutcome.success,
      value: (await response.json()) as Member,
    };
  }

  if (response.status === 401) {
    return {
      outcome: GetMemberOutcome.authenticationFailure,
      value: (await response.json()) as MemberAuthenticationError,
    };
  }

  return { outcome: GetMemberOutcome.unknown };
};

Our consumers can switch nicely as before (or narrow in other ways), and be sure that the type of value is always what’s expected relative to the outcome:

const memberResult = await getMember(token);

switch (memberResult.outcome) {
  case GetMemberOutcome.success: {
    // memberResult.value is Member here
  }
  case GetMemberOutcome.authenticationError: {
    // memberResult.value is MemberAuthenticationError here
  }
  case GetMemberOutcome.unknown: {
    // memberResult.value does not exist here
  }
}

The above works fine, but there is no runtime guarantee that data received actually is of a certain type. We guarantee type safety at runtime with decoders.

Decoding Data

When receiving data from external sources, we put a lot of trust in their documentation when making our types.

Something as simple as this could come back to bite us if we blindly trust it:

type Member = {
  email: string;
  id: string;
};

What happens if email is not a string? Maybe it’s null, undefined, or missing? API documentation stating that something is not nullable when it in reality is, happens often.

Since TypeScript types are compiled away, we can’t use TypeScript to help us check data types at runtime. We instead use the ts.data.json ↗️ library to do this. Using this library to specify types looks a little different than doing the same with TypeScript.

import { JsonDecoder } from "ts.data.json";

// Pretend we already have `member` from a successful API call

const memberDecoder = JsonDecoder.object("Member", {
  email: JsonDecoder.string,
  id: JsonDecoder.string,
});

const memberResult = memberDecoder.decode(member);

if (memberResult.isOk()) {
  // We, and the compiler, now know that `memberResult.value` is `Member`
  return { outcome: MemberOutcome.success, value: memberResult.value };
} else {
  // We, and the compiler, now know that `memberResult.error` contains type errors
  return { outcome: MemberOutcome.decoderError };
}

In the case where email turns out to be optional, we can make it optional like so:

import { JsonDecoder } from "ts.data.json";

const memberDecoder = JsonDecoder.object("Member", {
  email: JsonDecoder.optional(JsonDecoder.string),
  id: JsonDecoder.string,
});

optional handles the value of email being null or undefined, and the property itself being missing.

We have our own module json-decoder that exports helpers that make our lives a little easier. One of the most useful exported types is DecoderReturnType which allows us to extract a TypeScript type from a decoder.

export type DecoderReturnType<C extends JsonDecoder.Decoder<any>> =
  C extends JsonDecoder.Decoder<infer T> ? T : unknown;

import { DecoderReturnType } from "json-decoder";

const memberDecoder = JsonDecoder.object("Member", {
  email: JsonDecoder.optional(JsonDecoder.string),
  id: JsonDecoder.string,
});

type Member = DecoderReturnType<typeof memberDecoder>;

We can then use Member like it was a TypeScript defined type, if we need to. TypeScript’s type inference is often more than enough, and we find that extracting the return types of decoders isn’t always necessary.

Outro

This is how we have come to ensure strongly typed and runtime type checked integrations, making our programs both safer and more pleasant to maintain. It might seem like too much extra code and unnecessary abstractions at first, but the benefits for us are too great to dismiss.

Using any external API is now never scary, and we don’t need to trust poor documentation or infer types from logged responses. Refactoring is also a breeze, although this is mostly thanks to TypeScript.