I just want to get a string from the ipinfo.io site, but when parsing the page using HtmlUnit, the list is returned empty for some reason, although the element code on the site shows that it is not empty. Why is this happening and how to fix it?
My java code:
LogFactory.getFactory().setAttribute("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.NoOpLog");
java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("org.apache.commons.httpclient").setLevel(Level.OFF);
final WebClient webClient = new WebClient(BrowserVersion.EDGE);
webClient.getOptions().setThrowExceptionOnScriptError(false);
webClient.getOptions().setAppletEnabled(true);
webClient.getOptions().setCssEnabled(true);
webClient.getOptions().setJavaScriptEnabled(false);
webClient.getOptions().setFetchPolyfillEnabled(true);
webClient.getCookieManager().setCookiesEnabled(true);
webClient.getOptions().setUseInsecureSSL(true);
final HtmlPage page1 = webClient.getPage("https://ipinfo.io/");
HtmlForm form = page1.getForms().get(0);
List<DomElement> elements = StreamSupport.stream(form.getChildElements().spliterator(), false).collect(Collectors.toList());
HtmlTextInput textField = form.getInputByValue("");
textField.setValueAttribute("8.8.4.4");
HtmlPage page2 = elements.get(1).click();
HtmlDivision div = (HtmlDivision) page2.getElementById("tryit-data");
HtmlUnorderedList list = (HtmlUnorderedList) StreamSupport.stream(div.getChildElements().spliterator(), false).collect(Collectors.toList()).get(0);
System.out.println(list.asXml())
this is what I get when i run the code
How it actually looks on the site
Pay now to fund the work behind this issue.
Get updates on progress being made.
Maintainer is rewarded once the issue is completed.
You're funding impactful open source efforts
You want to contribute to this effort
You want to get funding like this too