Decoding JSON Structures with Elm

I’ve been working with Elm for a couple weeks now, and one aspect I struggled with early on was parsing structured JSON into my own types.

Let’s take a similar journey, beginning with flat data structures, and ending on parsing deeply-nested JSON structures. We’ll be using Elm’s Json.Decode library, which ships with Elm core, as well as the elm-json-extra library for the |: operator.

Installing `elm-json-extra`

If you haven’t installed an elm package yet, great! Let’s start with adding elm-json-extra by cding into the directory where your Elm application is located and running:

elm package install circuithub/elm-json-extra

The installer will prompt you for approval, which you can accept.

Parsing Basic Types

Let’s start with a very basic JSON structure:

{
  "name": "Awesome place to meet"
}

And a Lobby:

-- DotsAndBoxes/Model.elm
module DotsAndBoxes.Model (Lobby, nullLobby) where

type alias Lobby =
  { name: String }

nullLobby : Lobby
nullLobby = { name = "" }

To decode the JSON payload, I started with:

-- DotsAndBoxes/Decode.elm
module DotsAndBoxes.Decode (decodeLobby) where

import DotsAndBoxes.Model exposing (..)
import Json.Encode as Json
import Json.Decode.Extra exposing ((|:))
import Json.Decode exposing (Decoder, decodeValue, succeed, string, (:=))

decodeLobby : Json.Value -> Lobby
decodeLobby payload =
  case decodeValue lobby payload of
    Ok val -> val
    Err message -> nullLobby

lobby : Decoder Lobby
lobby =
  succeed Lobby
    |: ("name" := string)

Let’s first look at decodeLobby; it accepts a payload : Json.Value and returns a Lobby. We attempt to decode the value using decodeValue and pattern-match on the result (which is of the type Result String Lobby)

Let’s look at Result‘s type signature and documentation to understand the significance of Ok and Err:

type Result error value
    = Ok value
    | Err error

A Result is either Ok meaning the computation succeeded, or it is an Err meaning that there was some failure.

The signature for decodeValue is:

decodeValue : Decoder a -> Json.Value -> Result String a

With that in mind, let’s look at decodeLobby again:

decodeLobby : Json.Value -> Lobby
decodeLobby payload =
  case decodeValue lobby payload of
    Ok val -> val
    Err message -> nullLobby

This means that if decoding is successful (the Ok case), we return the properly decoded Lobby; otherwise, we return a nullLobby (the Err case). This intentionally hides parsing errors from the person interacting with the site; you may want to bubble this error up somehow.

Our lobby : Decoder Lobby is where we’ll actually describe the decoder. succeed transforms a Lobby into a Decoder Lobby; each subsequent line applies additional decoders to “fill in the blanks”.

Let’s break the line down:

   |:  ("name" :=  string)
-- [1]   [2]   [3] [4]

-- [1] applies the resulting decoder for this line (provided by elm-json-extra)
-- [2] field from the JSON structure
-- [3] infix operator to decode the object if it has the correct field ("name")
-- [4] the decoder to use on the data from the "name" field (string comes from Json.Decode)

Json.Decode ships with decoders oriented towards decoding objects, where the object decoder is coupled to the number of fields. From the Json.Decode object3 documentation:

job: Decoder Job
job =
  object3 Job
    ("name" := string)
    ("id" := int)
    ("completed" := bool)

As the number of fields grows, however, the decoder (object3) will need to change to object4, object5, and so on, versus the more flexible succeed and |::

job: Decoder Job
job =
  succeed Job
    |: ("name" := string)
    |: ("id" := int)
    |: ("completed" := bool)

This seems like a clear win to me in terms of maintainability and ease of use.

Parsing Union Types

With lobby configured to handle name, let’s dig into parsing strings into union types.

{
  "name": "Awesome place to meet",
  "status": "not_started"
}

Our data model contains a union type GameStatus, which can be one of Unknown, NotStarted, or Started.

-- DotsAndBoxes/Model.elm
module DotsAndBoxes.Model (Lobby, GameStatus, nullLobby) where

type GameStatus = Unknown | NotStarted | Started

type alias Lobby =
  { name: String
  , status: GameStatus
  }

nullLobby : Lobby
nullLobby = { name = "", status = Unknown }

Here, the value for "status" is a string ("not_started"), but we need to turn it into NotStarted.

Let’s talk through what we want to do: “read status as a string, and then convert it to a GameStatus.” lobbyDecoder doesn’t need to change, but we do need to modify lobby:

-- DotsAndBoxes/Decode.elm
module DotsAndBoxes.Decode (decodeLobby) where

-- ...imports and decodeLobby

lobby : Decoder Lobby
lobby =
  succeed Lobby
    |: ("name" := string)
    |: (("status" := string) `andThen` decodeStatus)

decodeStatus : String -> Decoder GameStatus
decodeStatus status = succeed (lobbyStatus status)

lobbyStatus : String -> GameStatus
lobbyStatus status =
  case status of
    "not_started" -> DotsAndBoxes.Model.NotStarted
    "started" -> DotsAndBoxes.Model.Started
    _ -> DotsAndBoxes.Model.Unknown

Parsing "status" reads similarly to how we described it above; andThen will pass the string value to decodeStatus where we convert the value to an actual type, and we capture a wildcard (the _) because the JSON could theoretically be any string value.

Parsing Nested Data Structures

With Lobby’s basic structure outlined, let’s dig into nested data structures by introducing a Game with the concept of a current player, and a list of all the players:

{
  "name": "Awesome place to meet",
  "status": "not_started",
  "game": {
    "current_player": null,
    "players": [
      {
        "id": "2d9bbd9c-7fdb-43e7-8c46-54ec2d271741",
        "active": true,
        "name": "Joe"
      }
    ]
  }
}

-- DotsAndBoxes/Model.elm
module DotsAndBoxes.Model (Lobby, GameStatus, Guid, Player, Game, nullLobby, nullPlayer, nullGame) where

type GameStatus = Unknown | NotStarted | Started

type alias Guid = String

type alias Player =
  { name: String
  , active: Bool
  , id: Guid
  }

type alias Game =
  { current_player: Player
  , players: List Player
  }

type alias Lobby =
  { name: String
  , status: GameStatus
  , game: Game
  }

nullPlayer : Player
nullPlayer = { name = "", active = True, id = "" }

nullGame : Game
nullGame = { players = [], current_player = nullPlayer }

nullLobby : Lobby
nullLobby = { name = "", status = Unknown, game = nullGame }

I opted for a Player leveraging a null object for Game’s current_player over Maybe Player; I don’t necessarily know if this is correct, and could probably be convinced otherwise. At any rate, let’s move onto the updated decoder:

-- DotsAndBoxes/Decode.elm
module DotsAndBoxes.Decode (decodeLobby) where

-- ...imports and decodeLobby
-- update the import to include oneOf, null, list, bool
import Json.Decode exposing (Decoder, decodeValue, succeed, string, oneOf, null, list, bool, (:=))

lobby : Decoder Lobby
lobby =
  succeed Lobby
    |: ("name" := string)
    |: (("status" := string) `andThen` decodeStatus)
    |: ("game" := game)

game : Decoder Game
game =
  succeed Game
    |: ("current_player" := oneOf [player, null nullPlayer])
    |: ("players" := list player)

player : Decoder Player
player =
  succeed Player
    |: ("name" := string)
    |: ("active" := bool)
    |: ("id" := string)

We introduced a few new concepts here.

First, we’ve written game : Decoder Game, which can be used directly in lobby. This decoder will be used to handle the nested structure of "game" within the JSON payload, and I named it game to feel similar to Json.Decode’s methods for decoding other types (e.g. int, string, bool). Second, we use oneOf and null to handle decoding a person when a value exists; if the JSON payload of "current_player" is null (in JavaScript), null nullPlayer will handle that case and default to nullPlayer. Had Game’s current_player had the type signature Maybe Player, the game decoder would look like:

game : Decoder Game
game =
  succeed Game
    |: ("current_player" := maybe player)
    |: ("players" := list player)

Maybe is a way to wrap a present value or the concept of “nothing”. In our Game record, the data type for current_player would be either Just Player (when a player is present) or Nothing, and the maybe player would handle both cases appropriately.

Handling Optional JSON Keys

Finally, let’s introduce a Score, where the JSON structure may or may not include a "winners" key; if it does, it’ll be an array of players. Either way, Score has a field winners: List Player, meaning we’ll have to handle both when the data is served and when it’s missing by assigning an empty list to winners.

With winners:

{
  "name": "Awesome place to meet",
  "status": "not_started",
  "game": {
    "current_player": null,
    "players": [
      {
        "id": "2d9bbd9c-7fdb-43e7-8c46-54ec2d271741",
        "active": true,
        "name": "Joe"
      }
    ],
    "score": {
      "winners": [
        {
          "id": "2d9bbd9c-7fdb-43e7-8c46-54ec2d271741",
          "active": true,
          "name": "Joe"
        }
      ]
    }
  }
}

And without:

{
  "name": "Awesome place to meet",
  "status": "not_started",
  "game": {
    "current_player": null,
    "players": [
      {
        "id": "2d9bbd9c-7fdb-43e7-8c46-54ec2d271741",
        "active": true,
        "name": "Joe"
      }
    ],
    "score": {}
  }
}

-- DotsAndBoxes/Model.elm
module DotsAndBoxes.Model (Lobby, GameStatus, Guid, Player, Game, Score, nullLobby, nullPlayer, nullGame) where

type GameStatus = Unknown | NotStarted | Started

type alias Guid = String

type alias Player =
  { name: String
  , active: Bool
  , id: Guid
  }

type alias Score =
  { winners: List Player }

type alias Game =
  { current_player: Player
  , players: List Player
  , score: Score
  }

type alias Lobby =
  { name: String
  , status: GameStatus
  , game: Game
  }

nullScore : Score
nullScore = { winners = [] }

nullPlayer : Player
nullPlayer = { name = "", active = True, id = "" }

nullGame : Game
nullGame = { players = [], current_player = nullPlayer, score = nullScore }

nullLobby : Lobby
nullLobby = { name = "", status = Unknown, game = nullGame }

Let’s dig into the decoder:

-- DotsAndBoxes/Decode.elm
module DotsAndBoxes.Decode (decodeLobby) where

-- ...imports, decodeLobby, previously defined decoders
-- update the import to include maybe
import Json.Decode exposing (Decoder, decodeValue, succeed, string, oneOf, null, list, bool, maybe, (:=))

game : Decoder Game
game =
  succeed Game
    |: ("current_player" := oneOf [player, null nullPlayer])
    |: ("players" := list player)
    |: ("score" := score)

score : Decoder Score
score =
  succeed Score
    |: ((maybe ("winners" := list player)) `andThen` decodeWinners)

decodeWinners : Maybe (List Player) -> Decoder (List Player)
decodeWinners players =
  succeed (Maybe.withDefault [] players)

Here, we handle decoding "score" as we have other nested structures; however, "winners"’s presence requires a bit more work. (maybe ("winners" := list player)) states that if "winners" is provided in the JSON payoad, it’ll be a list of players, but it’s wrapped in a maybe, meaning we’ll need to handle the Nothing case in decodeWinners.

decodeWinners takes a List Player wrapped in Maybe and returns a proper Decoder (List Player), ensuring that if the list of winners isn’t provided, it results in an empty list (List Player).

Wrapping Up

At this point, we’ve touched on some of the more challenging aspects of JSON decoding in Elm. Elm’s docs for the various Decoders within Json.Decode are solid and provide some very helpful examples for parsing various structures, and we’ve seen how to decode nested structures as well as improve decoding with elm-json-extra.

If you have other tips or pointers for how you’ve been decoding JSON structures with Elm, let us know in the comments!

You can view this code on Share Elm.

Installing elm-json-extra