This is similar to an issue I ran into several years ago, and was working until just the other day. I'm assuming it's due to changes made on the site I am working with.
I am using version 2.53.0
For reference, here is what was discussed on Stack Overflow a few years ago: https://stackoverflow.com/questions/53037782/htmlunit-dynamic-content-not-found
Here is the sample code. Essentially no anchors are found nested under "react-drop-test".
try (final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_78)) {
webClient.getOptions().setThrowExceptionOnScriptError(false);
HtmlPage page = webClient.getPage(url);
webClient.waitForBackgroundJavaScript(50000);
System.out.println("----------------");
System.out.println(page.asNormalizedText());
HtmlDivision div = (HtmlDivision) page.getElementById("react-drop-test");
List<HtmlElement> anchors = div.getElementsByTagName("a");
for (DomElement anchor:anchors) {
if ((anchor.getAttribute("class").contains("data-export"))) {
HtmlAnchor leftyAnchor = (HtmlAnchor) anchor;
Page p = leftyAnchor.click();
System.out.println();
System.out.println("----------------");
System.out.println(p.getWebResponse().getContentAsString());
break;
}
}
} `
Pay now to fund the work behind this issue.
Get updates on progress being made.
Maintainer is rewarded once the issue is completed.
You're funding impactful open source efforts
You want to contribute to this effort
You want to get funding like this too