---
title: How HTTP Cookies Work
teaser: 'How do browser cookies even work? Let''s find out together in this story
  with a lucky ending.

  '
tags: lucky,web,http
author: Edward Loveall
published_on: 2018-12-21
---

[Lucky],
a type-safe web framework written in Crystal by thoughtbot,
recently released version 0.12.
Along with many other changes,
it includes a new cookie system.
When starting this work,
I knew very little about how HTTP cookies actually worked.
We'll explore what I learned about cookies
and how they are implemented.

[Lucky]: https://luckyframework.org

## Back and Forth

Let's talk about how cookies are transferred between the browser
and the server.
Cookies use two headers:
`Set-Cookie`
and `Cookie`.
When a server responds to a browser request,
it can send down a `Set-Cookie` header with one
or many cookies:

```cookie
Set-Cookie: user_id=5; Expires=Fri, 5 Oct 2018 14:28:00 GMT; Secure; HttpOnly, remember_me=true
```

Each cookie is separated by a comma `,`
and each cookie attributes are separated by semicolons `;`.
The two values required are the first `name=value` pair
which are always string values.
The remaining attributes that set other parameters of a cookie
are optional
and set other parameters of a cookie.

To send a cookie back to the server,
the browser uses the `Cookie` header:

```cookie
Cookie: user_id=5; remember_me=true
```

Each cookie is separated by a semicolon `;`.
Don't confuse this with the `Set-Cookie` header
which confusingly uses a `,` to separate multiple cookies.
Notice that each cookie only contains the `name=value` pair.
The browser cannot send other attributes of a cookie back to the server.

Although many programming languages
and frameworks will abstract the parsing
and creation of these cookie headers for you
([Crystal],
[Ruby],
[PHP],
[Phoenix],
[Node.js],
[Python]),
it is occasionally good to know how it all works behind the scenes.

[Crystal]: https://crystal-lang.org/api/0.26.1/HTTP/Cookies.html#from_headers%28headers%29%3Aself-class-method
[Ruby]: https://ruby-doc.org/stdlib-1.9.3/libdoc/webrick/rdoc/WEBrick/Cookie.html#method-c-parse
[PHP]: https://secure.php.net/manual/en/reserved.variables.cookies.php
[Phoenix]: https://hexdocs.pm/plug/Plug.Conn.html#fetch_cookies/2
[Node.js]: https://nodejs.org/dist/latest-v8.x/docs/api/http.html#http_response_setheader_name_value
[Python]:
https://docs.python.org/3/library/http.cookies.html

## Expiration and Removal

Now that we know how cookies are set let's look at how they are un-set.

Cookies can expire.
A cookie with no expiration date specified will expire when the browser is closed.
These are often called session cookies
because they are removed after the browser session ends
(when the browser is closed).
Cookies with an expiration date in the past will be removed from the browser.

To remove a cookie,
[you must] set it's set its expiration date in the past.
This will signal to the browser that the cookie should be removed.
For cleanliness,
it's also a good idea to set its value to an empty string.

[you must]: https://tools.ietf.org/search/rfc6265#page-8

```cookie
Set-Cookie: user_id=; Expires=Fri, 5 Oct 2018 14:28:00 GMT;
```

## Other Goodies

Cookies have a few other interesting attributes
that are used to restrict
or permit them from certain locations:

* `Secure`: This will ensure that cookies can only be sent to HTTPS servers.
* `Domain`: A list of hosts that a cookie can be sent to.
* `Path`: Similar to `Domain` but restricts the cookie from being sent to URLs
  that do not include the `Path`.

I won't cover all these in detail.
You can read more about them on the
[Mozilla's web resource].

[Mozilla's web resource]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies

Also not covered here
will be the maximum amount of data you can store on a cookie.
For the most part,
you can assume that you're fine if you're under 4k worth of data.
In practice,
it's [more complicated].

[more complicated]: http://browsercookielimits.squawky.net

## Sessions

Sessions are less straightforward.
A session represents the currently logged in user.
This might be done by storing something as simple as the `user_id`,
but there is no standard.
As far as HTTP is concerned
there is no such thing as a session.
We must come up with our own way to store this information.

One common method is to use a cookie's value to store the session:

```cookie
Set-Cookie: _myapp_session={"user_id": "5"};
```

An app can now look at the cookie with the name of `_myapp_session`,
read
and parse the JSON,
and use it for things like setting the `current_user` on the request.
However,
the method above as-is is extremely easy to hack.
It's just plain text!

## Lock It Down

A good session is encrypted.
A more real-world example would be:

```cookie
Set-Cookie: _myapp_session=zjMvwPnfH7BSRrVIppsUI41eCimOtMOcMwjhAupZntBY1KEML%2FY0i5KYswzB54Mr; Expires=Fri, 1 Jan 2020 00:00:00 GMT; Secure; HttpOnly
```

This is a properly encrypted session.
The session is first converted to a JSON string.
The string of JSON is encrypted using the
[AES 256 standard]
which turns it into garbled bytes.
It is then base64 encoded so it is an ASCII string,
since the underlying HTTP protocols expect to work with ASCII.
That base64 encoded string becomes the value of the cookie.

[AES 256 standard]: https://en.wikipedia.org/wiki/Advanced_Encryption_Standard

When cookies are sent back to the server,
they are read,
(base64)
decoded,
decrypted,
JSON parsed,
and stored in memory as key/value pairs.

This is how sessions work in Lucky.
The session will be a cookie with a name like `_myapp_session`.
The value of the cookie is an encrypted JSON string
that can only be decrypted by a server with the session key.
Storing it as JSON allows us to have a key/value like store
but using a single cookie instead of multiple.

There are other ways to store session data,
such as a key-value store like Redis.
But even this requires a cookie to identify which values to retrieve.

## A Side Note About The Flash

The flash is a one-off message displayed to the user after they do something[.]
For example,
after you sign in you might see the message:

[.]: https://thoughtbot.com/blog/fourteen-hours-to-save-the-earth

> Welcome back Edward!

If you refresh the page
or navigate elsewhere,
this message disappears.

Flash messages are stored as two separates parts:
messages for the current request
and messages to be displayed on the next request.
These parts are called `now`
and `next`.
At the start of the request,
the existing data is read into an internal `now` hash which is read
and displayed to the user.

When the flash is set during a request/response cycle,
it's stored in an internal `next` hash.
At the response stage,
`next` is converted to JSON
and stored in the session.
During the following request,
`next` is read into `now`
and the cycle continues.

## Feeling Lucky?

And that's how cookies,
sessions,
and the flash works in Lucky!
We've made a nice,
type-safe wrapper around all things cookies can represent.
For example,
if you wanted to set a cookie with an expiration date that is HTTP Only,
you could write it like this:

```crystal
cookies
  .set(:current_user_id, 123)
  .expires(1.year.from_now)
  .http_only(true)
```

Because Lucky is type safe,
you can only pass a Date to the `expires` method
and only a boolean to the `http_only` method.
The same is true for all the other methods for setting cookie attributes.

This is only the start of what's new in the latest version of Lucky.
Check out the
[Lucky Framework]
if you'd like to play around with any of these concepts
or just try out a new framework!

[Lucky Framework]: https://luckyframework.org

If you want to read more about HTTP Cookies,
I highly recommend the
[MDN page]
on the topic.

[MDN page]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies