dead lock occur use WebClient.waitForBackgroundJavaScript
dead lock occur when i use
com.gargoylesoftware.htmlunit.WebClient.waitForBackgroundJavaScript(WebClient.java:1985)
each thread has its own WebClient in my code
version of htmlunit
<dependency>
<groupId>net.sourceforge.htmlunit</groupId>
<artifactId>htmlunit</artifactId>
<version>2.36.0</version>
</dependency>
thread dump just like follows
"siteSubtaskContainer-5" - Thread t@30
java.lang.Thread.State: TIMED_WAITING
at java.lang.Object.wait(Native Method)
- waiting on <18a767b5> (a com.gargoylesoftware.htmlunit.javascript.background.JavaScriptJobManagerImpl)
at com.gargoylesoftware.htmlunit.javascript.background.JavaScriptJobManagerImpl.waitForJobs(JavaScriptJobManagerImpl.java:195)
at com.gargoylesoftware.htmlunit.WebClient.waitForBackgroundJavaScript(WebClient.java:1985)
at com.xxx.stream.site.service.helper.WebClientHelper.sleep(WebClientHelper.java:70)
at com.xxx.stream.site.service.SiteServiceImpl.waitForPageAfterClick(SiteServiceImpl.java:734)
at com.xxx.stream.site.service.SiteServiceImpl.clickElements(SiteServiceImpl.java:680)
at com.xxx.stream.site.service.SiteServiceImpl.crawlPage(SiteServiceImpl.java:557)
at com.xxx.stream.site.service.SiteServiceImpl.handleSiteSubtask(SiteServiceImpl.java:318)
at com.xxx.stream.site.jms.SiteSubtaskConsumer.onMessage(SiteSubtaskConsumer.java:35)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761)
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076)
at java.lang.Thread.run(Thread.java:748)
Locked ownable synchronizers:
- None
"siteSubtaskContainer-3" - Thread t@28
java.lang.Thread.State: TIMED_WAITING
at java.lang.Object.wait(Native Method)
- waiting on <3801cbc2> (a com.gargoylesoftware.htmlunit.javascript.background.JavaScriptJobManagerImpl)
at com.gargoylesoftware.htmlunit.javascript.background.JavaScriptJobManagerImpl.waitForJobs(JavaScriptJobManagerImpl.java:195)
at com.gargoylesoftware.htmlunit.WebClient.waitForBackgroundJavaScript(WebClient.java:1985)
at com.xxx.stream.site.service.helper.WebClientHelper.sleep(WebClientHelper.java:70)
at com.xxx.stream.site.service.helper.WebClientHelper.sleep(WebClientHelper.java:84)
at com.xxx.stream.site.service.helper.WebClientHelper.sleep(WebClientHelper.java:84)
at com.xxx.stream.site.service.helper.WebClientHelper.sleep(WebClientHelper.java:84)
at com.xxx.stream.site.service.helper.WebClientHelper.sleep(WebClientHelper.java:84)
at com.xxx.stream.site.service.helper.WebClientHelper.sleep(WebClientHelper.java:84)
at com.xxx.stream.site.service.helper.WebClientHelper.sleep(WebClientHelper.java:84)
at com.xxx.stream.site.service.helper.WebClientHelper.sleep(WebClientHelper.java:84)
at com.xxx.stream.site.service.SiteServiceImpl.waitForPageAfterClick(SiteServiceImpl.java:734)
at com.xxx.stream.site.service.SiteServiceImpl.clickElements(SiteServiceImpl.java:680)
at com.xxx.stream.site.service.SiteServiceImpl.crawlPage(SiteServiceImpl.java:557)
at com.xxx.stream.site.service.SiteServiceImpl.handleSiteSubtask(SiteServiceImpl.java:318)
at com.xxx.stream.site.jms.SiteSubtaskConsumer.onMessage(SiteSubtaskConsumer.java:35)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761)
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076)
at java.lang.Thread.run(Thread.java:748)
Locked ownable synchronizers:
- None
"siteSubtaskContainer-1" - Thread t@26
java.lang.Thread.State: TIMED_WAITING
at java.lang.Object.wait(Native Method)
- waiting on <e378b4c> (a com.gargoylesoftware.htmlunit.javascript.background.JavaScriptJobManagerImpl)
at com.gargoylesoftware.htmlunit.javascript.background.JavaScriptJobManagerImpl.waitForJobs(JavaScriptJobManagerImpl.java:195)
at com.gargoylesoftware.htmlunit.WebClient.waitForBackgroundJavaScript(WebClient.java:1985)
at com.xxx.stream.site.service.helper.WebClientHelper.sleep(WebClientHelper.java:70)
at com.xxx.stream.site.service.helper.WebClientHelper.sleep(WebClientHelper.java:84)
at com.xxx.stream.site.service.helper.WebClientHelper.sleep(WebClientHelper.java:84)
at com.xxx.stream.site.service.helper.WebClientHelper.sleep(WebClientHelper.java:84)
at com.xxx.stream.site.service.helper.WebClientHelper.sleep(WebClientHelper.java:84)
at com.xxx.stream.site.service.helper.WebClientHelper.sleep(WebClientHelper.java:84)
at com.xxx.stream.site.service.helper.WebClientHelper.sleep(WebClientHelper.java:84)
at com.xxx.stream.site.service.helper.WebClientHelper.sleep(WebClientHelper.java:84)
at com.xxx.stream.site.service.helper.WebClientHelper.sleep(WebClientHelper.java:84)
at com.xxx.stream.site.service.helper.WebClientHelper.sleep(WebClientHelper.java:84)
at com.xxx.stream.site.service.SiteServiceImpl.waitForPageAfterClick(SiteServiceImpl.java:734)
at com.xxx.stream.site.service.SiteServiceImpl.clickElements(SiteServiceImpl.java:680)
at com.xxx.stream.site.service.SiteServiceImpl.crawlPage(SiteServiceImpl.java:557)
at com.xxx.stream.site.service.SiteServiceImpl.handleSiteSubtask(SiteServiceImpl.java:318)
at com.xxx.stream.site.jms.SiteSubtaskConsumer.onMessage(SiteSubtaskConsumer.java:35)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761)
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076)
at java.lang.Thread.run(Thread.java:748)
Locked ownable synchronizers:
- None
my code of WebClientHelper.sleep just show by follows
private static final long SLEEP_UNIT = 800L;
public static void sleep(WebClient webClient, long timeMillis) {
long until = System.currentTimeMillis() + timeMillis;
timeMillis = timeMillis > SLEEP_UNIT ? SLEEP_UNIT : timeMillis;
if (webClient != null) {
int jobCount = webClient.waitForBackgroundJavaScript(timeMillis);
if (jobCount == 0) {
return;
}
} else {
LockSupport.parkNanos(timeMillis * 1000_000L);
}
if (Thread.currentThread().isInterrupted()
|| (GlobalVariable.APP_CONTEXT != null && GlobalVariable.APP_STATUS >= GlobalConstant.APP_STATUS_STOPPING)) {
// avoid cannot exit when kill
throw BusinessException.newUntraceInstance("thread is interrupted");
}
long currentTimeMillis = System.currentTimeMillis();
if ((timeMillis = until - currentTimeMillis) > 0) {
sleep(webClient, timeMillis);
}
}
on the beginning of each consumer thread, i create a new WebClient
WebClient wc = new WebClient();
WebClientOptions options = wc.getOptions();
setDefaultOptions(
// enable ajax
wc.setAjaxController(new NicelyResynchronizingAjaxController());
wc.getCookieManager().setCookiesEnabled(true);
I will keep the thread dump file for two weeks, and you can contact my email[[email protected]] if necessary.
HtmlUnit / htmlunit
HtmlUnit is a "GUI-Less browser for Java programs".
871
How does funding with Polar work?
1
Pay now to fund the work behind this issue.
2
Get updates on progress being made.
3
Maintainer is rewarded once the issue is completed.
FAQ
Backer
You're funding impactful open source efforts
Contributor
You want to contribute to this effort
Maintainer
You want to get funding like this too