Tested with htmlunit 2.40.0 . The URL is "https://optout.aboutads.info/" and may be difficult to get working, but I wanted to report the problem attempting to get the raw content of the page while disabling the logging of htmlunit errors (with the expectation of typical javascript problems). The URL does some javascript processing, providing a progress bar and eventually getting to a window with a continue button. The content htmlunit returns shows that it is unable to get to the window (with various text information) with a continue button (unable to click it to continue). Was using the BrowserVersion.BEST_SUPPORTED
for the request.
The XML that is currently returned is below:
<?xml version="1.0" encoding="UTF-8"?>
<html ng-controller="MainController" class="js flexbox no-flexboxlegacy canvas canvastext webgl touch geolocation postmessage no-websqldatabase no-indexeddb hashchange history draganddrop websockets rgba hsla multiplebgs backgroundsize borderimage borderradius boxshadow textshadow opacity cssanimations csscolumns cssgradients no-cssreflections csstransforms no-csstransforms3d csstransitions fontface no-generatedcontent video audio localstorage sessionstorage webworkers applicationcache svg inlinesvg smil svgclippaths ng-scope" ng-app="naibcApp">
<!-- <![endif]--> <head>
<style type="text/css">
[uib-typeahead-popup].dropdown-menu{display:block;}
</style>
<style type="text/css">
.uib-time input{width:50px;}
</style>
<style type="text/css">
[uib-tooltip-popup].tooltip.top-left > .tooltip-arrow,[uib-tooltip-popup].tooltip.top-right > .tooltip-arrow,[uib-tooltip-popup].tooltip.bottom-left > .tooltip-arrow,[uib-tooltip-popup].tooltip.bottom-right > .tooltip-arrow,[uib-tooltip-popup].tooltip.left-top > .tooltip-arrow,[uib-tooltip-popup].tooltip.left-bottom > .tooltip-arrow,[uib-tooltip-popup].tooltip.right-top > .tooltip-arrow,[uib-tooltip-popup].tooltip.right-bottom > .tooltip-arrow,[uib-tooltip-html-popup].tooltip.top-left > .tooltip-arrow,[uib-tooltip-html-popup].tooltip.top-right > .tooltip-arrow,[uib-tooltip-html-popup].tooltip.bottom-left > .tooltip-arrow,[uib-tooltip-html-popup].tooltip.bottom-right > .tooltip-arrow,[uib-tooltip-html-popup].tooltip.left-top > .tooltip-arrow,[uib-tooltip-html-popup].tooltip.left-bottom > .tooltip-arrow,[uib-tooltip-html-popup].tooltip.right-top > .tooltip-arrow,[uib-tooltip-html-popup].tooltip.right-bottom > .tooltip-arrow,[uib-tooltip-template-popup].tooltip.top-left > .tooltip-arrow,[uib-tooltip-template-popup].tooltip.top-right > .tooltip-arrow,[uib-tooltip-template-popup].tooltip.bottom-left > .tooltip-arrow,[uib-tooltip-template-popup].tooltip.bottom-right > .tooltip-arrow,[uib-tooltip-template-popup].tooltip.left-top > .tooltip-arrow,[uib-tooltip-template-popup].tooltip.left-bottom > .tooltip-arrow,[uib-tooltip-template-popup].tooltip.right-top > .tooltip-arrow,[uib-tooltip-template-popup].tooltip.right-bottom > .tooltip-arrow,[uib-popover-popup].popover.top-left > .arrow,[uib-popover-popup].popover.top-right > .arrow,[uib-popover-popup].popover.bottom-left > .arrow,[uib-popover-popup].popover.bottom-right > .arrow,[uib-popover-popup].popover.left-top > .arrow,[uib-popover-popup].popover.left-bottom > .arrow,[uib-popover-popup].popover.right-top > .arrow,[uib-popover-popup].popover.right-bottom > .arrow,[uib-popover-html-popup].popover.top-left > .arrow,[uib-popover-html-popup].popover.top-right > .arrow,[uib-popover-html-popup].popover.bottom-left > .arrow,[uib-popover-html-popup].popover.bottom-right > .arrow,[uib-popover-html-popup].popover.left-top > .arrow,[uib-popover-html-popup].popover.left-bottom > .arrow,[uib-popover-html-popup].popover.right-top > .arrow,[uib-popover-html-popup].popover.right-bottom > .arrow,[uib-popover-template-popup].popover.top-left > .arrow,[uib-popover-template-popup].popover.top-right > .arrow,[uib-popover-template-popup].popover.bottom-left > .arrow,[uib-popover-template-popup].popover.bottom-right > .arrow,[uib-popover-template-popup].popover.left-top > .arrow,[uib-popover-template-popup].popover.left-bottom > .arrow,[uib-popover-template-popup].popover.right-top > .arrow,[uib-popover-template-popup].popover.right-bottom > .arrow{top:auto;bottom:auto;left:auto;right:auto;margin:0;}[uib-popover-popup].popover,[uib-popover-html-popup].popover,[uib-popover-template-popup].popover{display:block !important;}
</style>
<style type="text/css">
.uib-datepicker-popup.dropdown-menu{display:block;float:none;margin:0;}.uib-button-bar{padding:10px 9px 2px;}
</style>
<style type="text/css">
.uib-position-measure{display:block !important;visibility:hidden !important;position:absolute !important;top:-9999px !important;left:-9999px !important;}.uib-position-scrollbar-measure{position:absolute !important;top:-9999px !important;width:50px !important;height:50px !important;overflow:scroll !important;}.uib-position-body-scrollbar-measure{overflow:scroll !important;}
</style>
<style type="text/css">
.uib-datepicker .uib-title{width:100%;}.uib-day button,.uib-month button,.uib-year button{min-width:100%;}.uib-left,.uib-right{width:100%}
</style>
<style type="text/css">
.ng-animate.item:not(.left):not(.right){-webkit-transition:0s ease-in-out left;transition:0s ease-in-out left}
</style>
<style type="text/css">
@charset "UTF-8";[ng\:cloak],[ng-cloak],[data-ng-cloak],[x-ng-cloak],.ng-cloak,.x-ng-cloak,.ng-hide:not(.ng-hide-animate){display:none !important;}ng\:form{display:block;}.ng-animate-shim{visibility:hidden;}.ng-anchor{position:absolute;}
</style>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta charset="utf-8"/>
<meta name="viewport" content="width=device-width"/>
<base href="/"/>
<title translate="global.title" class="ng-scope">
WebChoices: Digital Advertising Alliance's Consumer Choice Tool for Web US
</title>
<link rel="P3Pv1" href="/w3c/p3p.xml" type="text/xml"/>
<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"/>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Oswald:400,300,700"/>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Gudea:400,700,400italic"/>
<link rel="stylesheet" href="/styles/f2e2d05b.vendor-home.css"/>
<link rel="stylesheet" href="/styles/5351c859.home.css"/>
<!--[if lte IE 8]>
<style>.navbar-collapse{ display: none !important; }</style>
<!--<![endif]--> <!--[if lte IE 8]>
<script src="/scripts/3b40df82.shiv-home.js"></script>
<!--<![endif]--> <!-- Determine the theme on the server and set a variable to determine it at runtime. --> <script async="" src="//www.google-analytics.com/analytics.js">
</script>
<script type="text/javascript" src="/theme">
</script>
</head>
<body class="client-daa" ng-class="{'is-admin': isAuthorized(userRoles.admin)}">
<div class="loading-overlay" ng-hide="hideOverlay">
</div>
<div class="navbar navbar-default" role="navigation">
<div class="container">
<div class="navbar-title">
<a href="/">
<span translate="" class="ng-scope">
WebChoices: Digital Advertising Alliance's Consumer Choice Tool for Web US
</span>
</a>
</div>
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar-collapse">
<span class="sr-only">
Toggle navigation
</span>
<span class="icon-bar">
</span>
<span class="icon-bar">
</span>
<span class="icon-bar">
</span>
</button>
<span class="language-menu language-menu__mobile">
<!-- ngIf: client === 'daa' --> <span ng-if="client === 'daa'" class="dropdown nav navbar-nav navbar-right ng-scope">
<a data-toggle="dropdown">
<span class="glyphicon glyphicon-globe language-menu__icon" aria-hidden="true">
</span>
</a>
<ul class="dropdown-menu">
<li>
<a ng-hide="theme==='daac' && lang==='fr'" href="" ng-click="changeLanguage('daac', 3,'fr')" class="">
Fran?ais Canadien
</a>
<a ng-hide="theme==='daac' && lang==='en'" href="" ng-click="changeLanguage('daac', 3,'en')" class="">
Canadian English
</a>
<a ng-hide="theme==='apda'" href="" ng-click="changeLanguage('apda', 4,'es')" class="">
Espa?ol Latinoamericano
</a>
<a ng-hide="theme==='daa' && lang==='en'" href="" ng-click="changeLanguage('daa', 2, 'en')" class="">
US English
</a>
<a ng-hide="theme==='daa' && lang==='es'" href="" ng-click="changeLanguage('daa', 2, 'es')" class="">
Espa?ol de Estados Unidos
</a>
</li>
</ul>
</span>
<!-- end ngIf: client === 'daa' --> </span>
<a class="navbar-brand" ng-href="https://optout.aboutads.info/" href="https://optout.aboutads.info/">
<img ng-src="/images/daa/nav_logo.png" alt="DAA Logo" src="/images/daa/nav_logo.png"/>
</a>
</div>
<div class="collapse navbar-collapse" id="navbar-collapse" ng-switch="authenticated">
<ul class="nav navbar-nav navbar-right external-links">
<!-- ngSwitchWhen: false --> <li ng-switch-when="false" class="dropdown ng-scope">
<button type="button" class="dropdown-toggle" data-toggle="dropdown">
<span class="sr-only">
Toggle navigation
</span>
<span class="icon-bar">
</span>
<span class="icon-bar">
</span>
<span class="icon-bar">
</span>
</button>
<ul class="dropdown-menu">
<li>
<a translate="" href="https://youradchoices.com/pmc" target="_blank" class="ng-scope">
Protect My Choices
</a>
<!-- ngIf: theme!=='apda' --> <a ng-if="theme!=='apda'" translate="" href="https://youradchoices.com/choices-faq#jr02" target="_blank" class="ng-scope">
Understanding How Online Advertising Works
</a>
<!-- end ngIf: theme!=='apda' --> <!-- ngIf: theme!=='apda' --> <a ng-if="theme!=='apda'" translate="" href="https://youradchoices.com/choices-faq" target="_blank" class="ng-scope">
Frequently Asked Questions
</a>
<!-- end ngIf: theme!=='apda' --> <a translate="" href="http://www.youradchoices.com" target="_blank" class="ng-scope">
Visit YourAdChoices
</a>
<a translate="" href="https://youradchoices.com/principles" target="_blank" class="ng-scope">
DAA Principles
</a>
<a translate="" href="https://youradchoices.com/learn#zone-preface3-wrapper" target="_blank" class="ng-scope">
Report a Problem
</a>
<!-- ngIf: theme!=='apda' --> <a ng-if="theme!=='apda'" translate="" href="https://youradchoices.com/choices-faq#jr10" target="_blank" class="ng-scope">
Help with WebChoices
</a>
<!-- end ngIf: theme!=='apda' --> <a translate="" href="http://youradchoices.com/privacy-policy?language=en" target="_blank" class="ng-scope">
DAA Privacy Policy
</a>
<a translate="" href="http://youradchoices.com/terms-of-use" target="_blank" class="ng-scope">
DAA Terms of Service
</a>
</li>
</ul>
<span class="language-menu language-menu__desktop">
<!-- ngIf: client === 'daa' --> <span ng-if="client === 'daa'" class="dropdown nav navbar-nav navbar-right ng-scope">
<a data-toggle="dropdown">
<span class="glyphicon glyphicon-globe language-menu__icon" aria-hidden="true">
</span>
</a>
<ul class="dropdown-menu">
<li>
<a ng-hide="theme==='daac' && lang==='fr'" href="" ng-click="changeLanguage('daac', 3,'fr')" class="">
Fran?ais Canadien
</a>
<a ng-hide="theme==='daac' && lang==='en'" href="" ng-click="changeLanguage('daac', 3,'en')" class="">
Canadian English
</a>
<a ng-hide="theme==='apda'" href="" ng-click="changeLanguage('apda', 4,'es')" class="">
Espa?ol Latinoamericano
</a>
<a ng-hide="theme==='daa' && lang==='en'" href="" ng-click="changeLanguage('daa', 2, 'en')" class="">
US English
</a>
<a ng-hide="theme==='daa' && lang==='es'" href="" ng-click="changeLanguage('daa', 2, 'es')" class="">
Espa?ol de Estados Unidos
</a>
</li>
</ul>
</span>
<!-- end ngIf: client === 'daa' --> </span>
</li>
<!-- end ngSwitchWhen: --> <!-- ngSwitchWhen: true --> <!-- ngSwitchWhen: true --> <!-- ngSwitchWhen: true --> </ul>
</div>
</div>
</div>
<div class="container">
<!--[if lt IE 9]>
<div class="browserupgrade">
<p translate="global.browsehappy">You are using an <strong>outdated</strong> browser. Please <a href=\"http://browsehappy.com/?locale=en\">upgrade your browser</a> to improve your experience.</p>
</div>
<![endif]--> <div id="nojavascript" style="display: none;">
<h1>
JavaScript Not Detected
</h1>
<p>
The site requires that you enable JavaScript for your browser. Please turn on JavaScript for your browser and click "Try Again" to proceed.
</p>
<p>
<a href="" class="btn btn-primary" role="button">
Try Again
</a>
-
<a href="" class="btn btn-info" role="button">
Learn More
</a>
</p>
</div>
<!-- ngView: --> <div class="footer">
</div>
</div>
<script type="text/javascript">
//<![CDATA[
// Fake noscript tag to avoid Chrome bug
document.getElementById('nojavascript').style.display = 'none';
//]]>
</script>
<script src="/scripts/600fbcb3.home.js">
</script>
</body>
</html>
Pay now to fund the work behind this issue.
Get updates on progress being made.
Maintainer is rewarded once the issue is completed.
You're funding impactful open source efforts
You want to contribute to this effort
You want to get funding like this too