I have checked that HtmlUnit does not load part of this page:
https://www.milanuncios.com/mis-anuncios/
When inspected with a browser, the section:
<div class="ma-LayoutBasicMainContent">
Has into a lot of contents. But when loaded by HtmlUnit... it is empty!
I have tried various webClient switches, including
webClient.setAjaxController(new NicelyResynchronizingAjaxController());
webClient.getOptions().setDownloadImages(true);
webClient.getOptions().setCssEnabled(true);
webClient.getOptions().setJavaScriptEnabled(true);
webClient.setJavaScriptTimeout(10000);
But always with same result: The "ma-LayoutBasicMainContent" section is not loaded. This is the code I use:
import com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.*;
class MarnvHtmlUnitTest {
public static void main(String[] args) {
WebClient webClient = null;
try {
final long javascriptTimeout = 10000;
webClient = new WebClient();
webClient.setAjaxController(new NicelyResynchronizingAjaxController());
webClient.getOptions().setDownloadImages(true);
webClient.getOptions().setCssEnabled(true);
webClient.getOptions().setJavaScriptEnabled(true);
webClient.setJavaScriptTimeout(10000);
String loginURL = "https://www.milanuncios.com/mis-anuncios/";
System.out.println("Connecting to " + loginURL + " (" + webClient.getBrowserVersion() + ")");
HtmlPage page = webClient.getPage(loginURL);
System.out.print(" Waiting for Javascript to complete...");
long millis = System.currentTimeMillis();
webClient.waitForBackgroundJavaScript(javascriptTimeout);
System.out.println(System.currentTimeMillis() - millis + " milliseconds");
if (!page.asText().contains("gestiona tus anuncios")) {
System.out.println("ERROR!");
System.out.println(page.asXml());
System.out.println("EXITING. " + webClient.getWebWindows().size());
return;
}
System.out.println("OK");
} catch (Exception e) {
e.printStackTrace();
}
finally {
if (webClient != null)
webClient.close();
}
}
}
In case of correct page loading, the page should contain the text "gestiona tus anuncios".
Note that the call to "waitForBackgroundJavaScript" returns inmediately, which for me it's strange... it normaly waits some seconds until the page is completely loaded.
I am using HtmlUnit 2.36.0.
Thank you in advance for your help.
Pay now to fund the work behind this issue.
Get updates on progress being made.
Maintainer is rewarded once the issue is completed.
You're funding impactful open source efforts
You want to contribute to this effort
You want to get funding like this too