At thoughtbot we often use Chromedriver and headless Chrome to run integration tests on our web applications. We encountered an interesting challenge upon trying to upgrade one of our projects to Chrome 80. I could not find anything on the web when I searched for this error message, so I’m documenting this in the hope that it helps somebody out!
Every time we’d attempt to upgrade our CI environment to Chrome 80 we would see the following error in our CI logs:
Selenium::WebDriver::Error::InvalidCookieDomainError: invalid cookie domain
(Session info: headless chrome=80.0.3987.122)
from #0 0x56495152bd29 <unknown>
This error led us down a long and fruitless path of looking at the domain we were trying to set cookies for and making sure it matched the domain in the URL of the page. They matched. So what was going wrong?
We run our integration specs over TLS, with a self generated CA certificate that we add to the system certificate store in order for the browser to trust a TLS certificate signed with that CA.
It turns out that Chrome 80 no longer trusts the CA certificate we were adding
to the system certificate store. The solution then is to pass an additional
option to the Chromedriver–allow-insecure-localhost–in order to sidestep this.
Capybara.register_driver :headless_chrome do |app|
capabilities = Selenium::WebDriver::Remote::Capabilities.chrome(
"goog:chromeOptions": {
args: %w(headless disable-gpu window-size=1280,2000 no-sandbox allow-insecure-localhost),
},
)
Capybara::Selenium::Driver.new app,
browser: :chrome,
desired_capabilities: capabilities
end
I know what you’re thinking; why is the reported error about the cookie domain if it turned out to be a TLS trust issue? Read on to find out!
We started this journey by stepping through the cookie setting code to figure out where the error was being raised. Selenium communicates with the webdriver over an HTTP protocol and it was this request that was raising the error.
The next step was to start following the webdriver code, and see where this
error actually came from. That led us to this section of Chromedriver’s
ExecuteAddCookie
function.
Status ExecuteAddCookie(Session* session,
WebView* web_view,
const base::DictionaryValue& params,
std::unique_ptr<base::Value>* value,
Timeout* timeout) {
// ...
std::string url;
Status status = GetUrl(web_view, session->GetCurrentFrameId(), &url);
if (status.IsError())
return status;
if (!base::StartsWith(url, "http://", base::CompareCase::INSENSITIVE_ASCII) &&
!base::StartsWith(url, "https://",
base::CompareCase::INSENSITIVE_ASCII) &&
!base::StartsWith(url, "ftp://", base::CompareCase::INSENSITIVE_ASCII))
return Status(kInvalidCookieDomain);
// ...
}
The error itself is raised in the return line near the bottom of that block of
code. This is the only place in the entire codebase where this particular error
is raised.
But what is it actually doing? It’s checking to ensure that the URL of the
page starts with either http://, https:// or ftp:// and raising the
kInvalidCookieDomain error if it doesn’t.
The first thing we noticed here is that the error the code raises does not
seem appropriate for the check that is being made. The second thing we wondered
was “why does our URL not start with http://, https:// or ftp://?”
To get to the bottom of this, we needed to find out what Chromedriver thought
the URL was at this point in the code. The code above is calling GetUrl() to
determine this, so let’s take a look at that function:
Status GetUrl(WebView* web_view, const std::string& frame, std::string* url) {
std::unique_ptr<base::Value> value;
base::ListValue args;
Status status = web_view->CallFunction(
frame, "function() { return document.URL; }", args, &value);
if (status.IsError())
return status;
if (!value->GetAsString(url))
return Status(kUnknownError, "javascript failed to return the url");
return Status(kOk);
}
Take a close look at the fifth line. It’s getting the URL from the browser by
running a snippet of Javascript; return document.URL;. Well, we can do that!
> browser.execute_script("return document.URL;")
=> "chrome-error://chromewebdata/"
Well, well, lookie here! It seems we have some kind of error happening in the browser. The most likely culprit is TLS.
The problem turned out to be that Chrome 80 has changed the scope of what it
trusts as a CA certificate (at least on Linux). It’s no longer enough just to
add the CA cert to the system’s certificate store. Google Chrome has its own CA
certificate store that resides in a cert9.db file in the users home directory.
Adding the certificate to this database file was enough to get regular desktop
Chrome to trust the CA, but sadly seemed to be ignored by Chrome running in
headless mode. Adding the allow-insecure-localhost option to our Chromedriver
config was enough to sidestep this.
And with that, the issue was resolved, or at least neatly worked around. What a fun journey through the bowels of Chromedriver!
