Chromedriver/Chrome 80 cookie domain bug

At thoughtbot we often use Chromedriver and headless Chrome to run integration tests on our web applications. We encountered an interesting challenge upon trying to upgrade one of our projects to Chrome 80. I could not find anything on the web when I searched for this error message, so I’m documenting this in the hope that it helps somebody out!

Every time we’d attempt to upgrade our CI environment to Chrome 80 we would see the following error in our CI logs:

Selenium::WebDriver::Error::InvalidCookieDomainError: invalid cookie domain
  (Session info: headless chrome=80.0.3987.122)
from #0 0x56495152bd29 <unknown>

This error led us down a long and fruitless path of looking at the domain we were trying to set cookies for and making sure it matched the domain in the URL of the page. They matched. So what was going wrong?

We run our integration specs over TLS, with a self generated CA certificate that we add to the system certificate store in order for the browser to trust a TLS certificate signed with that CA.

It turns out that Chrome 80 no longer trusts the CA certificate we were adding to the system certificate store. The solution then is to pass an additional option to the Chromedriver–allow-insecure-localhost–in order to sidestep this.

Capybara.register_driver :headless_chrome do |app|
  capabilities = Selenium::WebDriver::Remote::Capabilities.chrome(
    "goog:chromeOptions": {
      args: %w(headless disable-gpu window-size=1280,2000 no-sandbox allow-insecure-localhost),
    },
  )

  Capybara::Selenium::Driver.new app,
    browser: :chrome,
    desired_capabilities: capabilities
end

I know what you’re thinking; why is the reported error about the cookie domain if it turned out to be a TLS trust issue? Read on to find out!

We started this journey by stepping through the cookie setting code to figure out where the error was being raised. Selenium communicates with the webdriver over an HTTP protocol and it was this request that was raising the error.

The next step was to start following the webdriver code, and see where this error actually came from. That led us to this section of Chromedriver’s ExecuteAddCookie function.

Status ExecuteAddCookie(Session* session,
                        WebView* web_view,
                        const base::DictionaryValue& params,
                        std::unique_ptr<base::Value>* value,
                        Timeout* timeout) {
  // ...

  std::string url;
  Status status = GetUrl(web_view, session->GetCurrentFrameId(), &url);
  if (status.IsError())
    return status;
  if (!base::StartsWith(url, "http://", base::CompareCase::INSENSITIVE_ASCII) &&
      !base::StartsWith(url, "https://",
                        base::CompareCase::INSENSITIVE_ASCII) &&
      !base::StartsWith(url, "ftp://", base::CompareCase::INSENSITIVE_ASCII))
    return Status(kInvalidCookieDomain);

  // ...
}

The error itself is raised in the return line near the bottom of that block of code. This is the only place in the entire codebase where this particular error is raised.

But what is it actually doing? It’s checking to ensure that the URL of the page starts with either http://, https:// or ftp:// and raising the kInvalidCookieDomain error if it doesn’t.

The first thing we noticed here is that the error the code raises does not seem appropriate for the check that is being made. The second thing we wondered was “why does our URL not start with http://, https:// or ftp://?”

To get to the bottom of this, we needed to find out what Chromedriver thought the URL was at this point in the code. The code above is calling GetUrl() to determine this, so let’s take a look at that function:

Status GetUrl(WebView* web_view, const std::string& frame, std::string* url) {
  std::unique_ptr<base::Value> value;
  base::ListValue args;
  Status status = web_view->CallFunction(
      frame, "function() { return document.URL; }", args, &value);
  if (status.IsError())
    return status;
  if (!value->GetAsString(url))
    return Status(kUnknownError, "javascript failed to return the url");
  return Status(kOk);
}

Take a close look at the fifth line. It’s getting the URL from the browser by running a snippet of Javascript; return document.URL;. Well, we can do that!

> browser.execute_script("return document.URL;")
=> "chrome-error://chromewebdata/"

Well, well, lookie here! It seems we have some kind of error happening in the browser. The most likely culprit is TLS.

The problem turned out to be that Chrome 80 has changed the scope of what it trusts as a CA certificate (at least on Linux). It’s no longer enough just to add the CA cert to the system’s certificate store. Google Chrome has its own CA certificate store that resides in a cert9.db file in the users home directory. Adding the certificate to this database file was enough to get regular desktop Chrome to trust the CA, but sadly seemed to be ignored by Chrome running in headless mode. Adding the allow-insecure-localhost option to our Chromedriver config was enough to sidestep this.

And with that, the issue was resolved, or at least neatly worked around. What a fun journey through the bowels of Chromedriver!