Shadow DOM in Selenium

I’ve seen numerous bugs reported for how Chrome v96 has changed the shadow root return values for Selenium. This is a feature, not a bug! Here’s how to work with the shadow DOM in Selenium 4.


TL/DR

To access Shadow DOM elements in Selenium 4 with Chrome v96+ use the new getShadowRoot() method:

WebElement shadowHost = driver.findElement(By.cssSelector("#shadow_host"));
WebElement shadowContent = shadowHost.getShadowRoot().findElement(By.cssSelector("#shadow_content"));

What happened in v96 is that Chrome has made its shadow root values compliant with the updated W3C WebDriver specification, which now includes definitions getting an element’s shadow root and locating elements in a shadow root. Microsoft Edge will support this functionality in v96, and we can expect Firefox to add support soon. Additionally, the Selenium team is working to get support added to WebKit, so eventually we’ll see this in Safari.

The way people are used to accessing shadow DOM elements is with JavaScript. If you are already working with shadow roots in Chrome, Edge or Safari, most likely your code looks like this:

WebElement shadowHost = driver.findElement(By.cssSelector("#shadow_host"));
JavascriptExecutor jsDriver = (JavascriptExecutor) driver;

WebElement shadowRoot = (WebElement) jsDriver.executeScript("return arguments[0].shadowRoot", shadowHost);
WebElement shadowContent = shadowRoot.findElement(By.cssSelector("#shadow_content"));

This code works for Chrome before v96, Edge before v96 and Safari. The issue is that Chrome v96 breaks this code because the script getting executed is no longer returning a value that can be parsed by the WebElement interface. The cast error looks like this:

java.lang.ClassCastException: class com.google.common.collect.Maps$TransformedEntriesMap cannot be cast to 
class org.openqa.selenium.WebElement (com.google.common.collect.Maps$TransformedEntriesMap and 
org.openqa.selenium.WebElement are in unnamed module of loader 'app')

The fix for this in Selenium 4 is to change the cast from the WebElement interface to the SearchContext interface. The ShadowRoot class itself is kept as package-private because we want people to code against the SearchContext API directly. This change is backwards compatible with what you’ve been doing, since WebElement extends SearchContext.

SearchContext shadowRoot = (SearchContext) jsDriver.executeScript("return arguments[0].shadowRoot", shadowHost);

Note that not all the Selenium bindings considered this use case, so getting a shadow root object instance from a script execution might not be able to work in non-Java languages until Selenium 4.1.

The especially tricky issue here is how to deal with a shadow root in Chrome v96 while still using Selenium 3. If you are using Selenium 3, seriously, please update to Selenium 4; issues like this highlight why it is worth investing time to upgrade. You can still get a WebElement using JavaScript in Chrome v96 using Selenium 3, it’s just really hacky:

WebElement shadow_host = driver.findElement(By.cssSelector("#shadow_host"));
Object shadowRoot = ((JavascriptExecutor) driver).executeScript("return arguments[0].shadowRoot", shadow_host);

String id = (String) ((Map<String, Object>) shadowRoot).get("shadow-6066-11e4-a52e-4f735466cecf");
RemoteWebElement shadowRootElement = new RemoteWebElement();
shadowRootElement.setParent((RemoteWebDriver) driver);
shadowRootElement.setId(id);
WebElement shadowContent = shadowRootElement.findElement(By.cssSelector("#shadow_content"));

So far, I haven’t mentioned Firefox— that’s because it is special. Until Firefox implements W3C-compliant shadow root support, you have to get the shadow DOM elements from the executed script directly with the children property, then loop through the elements to find the one you want to work with:

WebElement shadowHost = driver.findElement(By.cssSelector("#shadow_host"));
JavascriptExecutor jsDriver = (JavascriptExecutor) driver;

List<WebElement> children = (List<WebElement>) jsDriver.executeScript("return arguments[0].shadowRoot.children", shadowHost);

WebElement shadowContent = null;
for (WebElement element : children) {
    if (element.getAttribute("id").equals("shadow_content")) {
        shadowContent = element;
        break;
    }
}

Selenium 4 with Chrome 96 provides a much cleaner API for working with Shadow DOM elements without needing to use JavaScript. Please update your code to take advantage of this.


Follow me if you found this article interesting,
or answer one of these questions in the comments or on Twitter: