Failing to get javascript rendered web pages even when .setJavaScriptEnabled(true) is called
I can't get htmlunit to return javascript-rendered web pages even when I ensure webClient.getOptions().setJavaScriptEnabled(true) has been called.
For example, calling .getPage() to get the NemWatch web page for live supply and demand figures for national energy (https://ausrealtimefueltype.global-roam.com/expanded) returns the following web page:
<body>
<noscript>
<strong>We're sorry but LiveSupplyDemand doesn't work properly without JavaScript enabled. Please enable it to continue.</strong>
</noscript>
<div id="app">
</div>
<script src="/js/chunk-vendors~6e8b5f81.0ebb7f26.js">
</script>
<script src="/js/chunk-vendors~e9d21a69.b2c45f87.js">
</script>
<script src="/js/chunk-vendors~5565c913.6026e631.js">
</script>
<script src="/js/chunk-vendors~660d1c5c.06f990bc.js">
</script>
<script src="/js/chunk-vendors~f903c053.f9aad10d.js">
</script>
<script src="/js/chunk-vendors~4a37d647.a99d60f5.js">
</script>
<script src="/js/chunk-vendors~f525df99.63625d91.js">
</script>
<script src="/js/chunk-vendors~6cebe388.3d43d774.js">
</script>
<script src="/js/chunk-vendors~6596a6a1.93144171.js">
</script>
<script src="/js/chunk-vendors~73c03416.1749b9e9.js">
</script>
<script src="/js/chunk-vendors~d147297f.39d1e44a.js">
</script>
<script src="/js/chunk-vendors~0a1e6715.2858b6da.js">
</script>
<script src="/js/chunk-vendors~5ea1a303.21cac99f.js">
</script>
<script src="/js/chunk-vendors~567ab0f1.b7cb39ef.js">
</script>
<script src="/js/chunk-vendors~7e4cda95.eb423a5e.js">
</script>
<script src="/js/chunk-vendors~987e6011.5423d464.js">
</script>
<script src="/js/chunk-vendors~eddad35c.c4440657.js">
</script>
<script src="/js/chunk-vendors~678f84af.9f4a9b90.js">
</script>
<script src="/js/app~d0ae3f07.f9d50f26.js">
</script>
</body>
The website doesn't seem to recognise that htmlunit is running with JavaScript enabled. As this example shows, the website returns a warning that it doesn't think the client browser can handle JavaScript, then simply returns the JavaScript unprocessed with all of the tabular information about energy missing.
This seems to happen for every website I try that requires JavaScript to run for the website's data values to display.
Am I doing something wrong? Or does htmlunit only ever return web pages like the one above, never managing to return a fully JavaScript-rendered web page equivalent to what you get in a manual browser?
How does funding with Polar work?
Pay now to fund the work behind this issue.
Get updates on progress being made.
Maintainer is rewarded once the issue is completed.
Backer
You're funding impactful open source efforts
Contributor
You want to contribute to this effort
Maintainer
You want to get funding like this too