red5/redbox rtmpt deadlock

Post your questions and bug-reports about the audio/video streaming add-on based on Red5 Media Server.

Moderators: Lapo, Bax

projuri
Posts: 7
Joined: 25 May 2010, 01:25

red5/redbox rtmpt deadlock

Postby projuri » 26 May 2010, 00:36

Hi,

We have switched to RTMPT protocol to traverse firewalls and after ~36 hours redbox has stopped receive requests on RTMPT port.

Those are the last messages from redbox's log file:

Code: Select all

Acceptor1 SelectChannelConnector@0.0.0.0:8080:( org.mortbay.log.invoke ) Out of threads for org.mortbay.thread.BoundedThreadPool@27e32382
DefaultQuartzScheduler_Worker-7:( org.red5.server.net.rtmp.RTMPConnection.execute ) Closing RTMPTConnection from 58.107.144.100:64373 to 114.141.198.176 (in: 4132, out: 7331624) due to too much inactivity (60006).
DefaultQuartzScheduler_Worker-5:( org.red5.server.net.rtmp.RTMPConnection.execute ) Closing RTMPTConnection from 60.229.13.140:60689 to 114.141.198.176 (in: 3610, out: 665852) due to too much inactivity (64996).


jstack has detected 8 deadlocks:

Code: Select all

Found one Java-level deadlock:
=============================
"btpool0-455 - /send/651773844/146":
  waiting to lock monitor 0x000000005422f350 (object 0x00002aaab42c8958, a org.red5.server.stream.PlaylistSubscriberStream$PlayEngine),
  which is held by "pool-1-thread-10"
"pool-1-thread-10":
  waiting to lock monitor 0x0000000054213438 (object 0x00002aaab42c8828, a java.util.ArrayList),
  which is held by "btpool0-455 - /send/651773844/146"

Java stack information for the threads listed above:
===================================================
"btpool0-455 - /send/651773844/146":
        at org.red5.server.stream.PlaylistSubscriberStream$PlayEngine.stop(PlaylistSubscriberStream.java:1267)
        - waiting to lock <0x00002aaab42c8958> (a org.red5.server.stream.PlaylistSubscriberStream$PlayEngine)
        at org.red5.server.stream.PlaylistSubscriberStream.stop(PlaylistSubscriberStream.java:298)
        at org.red5.server.stream.PlaylistSubscriberStream.removeAllItems(PlaylistSubscriberStream.java:361)
        - locked <0x00002aaab42c8828> (a java.util.ArrayList)
        at org.red5.server.stream.StreamService.play(StreamService.java:210)
        at org.red5.server.stream.StreamService.play(StreamService.java:238)
        at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.red5.server.service.ServiceInvoker.invoke(ServiceInvoker.java:217)
        at org.red5.server.net.rtmp.RTMPHandler.invokeCall(RTMPHandler.java:187)
        at org.red5.server.net.rtmp.RTMPHandler.onInvoke(RTMPHandler.java:414)
        at org.red5.server.net.rtmp.BaseRTMPHandler.messageReceived(BaseRTMPHandler.java:143)
        at org.red5.server.net.rtmpt.RTMPTHandler.messageReceived(RTMPTHandler.java:117)
        at org.red5.server.net.rtmpt.RTMPTServlet.handleSend(RTMPTServlet.java:387)
        at org.red5.server.net.rtmpt.RTMPTServlet.service(RTMPTServlet.java:471)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:362)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
        at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
        at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
        at org.mortbay.jetty.Server.handle(Server.java:309)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506)
        at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:844)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:644)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)
        at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:396)
        at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
"pool-1-thread-10":
        at org.red5.server.stream.PlaylistSubscriberStream.hasMoreItems(PlaylistSubscriberStream.java:408)
        - waiting to lock <0x00002aaab42c8828> (a java.util.ArrayList)
        at org.red5.server.stream.PlaylistSubscriberStream$PlayEngine.stop(PlaylistSubscriberStream.java:1273)
        - locked <0x00002aaab42c8958> (a org.red5.server.stream.PlaylistSubscriberStream$PlayEngine)
        at org.red5.server.stream.PlaylistSubscriberStream$PlayEngine.pullAndPush(PlaylistSubscriberStream.java:1407)
        - locked <0x00002aaab42c8958> (a org.red5.server.stream.PlaylistSubscriberStream$PlayEngine)
        at org.red5.server.stream.PlaylistSubscriberStream$PlayEngine.access$400(PlaylistSubscriberStream.java:786)
        at org.red5.server.stream.PlaylistSubscriberStream$PlayEngine$PullAndPushRunnable.run(PlaylistSubscriberStream.java:2032)
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
        at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown Source)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown Source)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)


SmartFoxServer 1.6.6 is being used.

Is there any fix for that problem?

Thank you
User avatar
Bax
Site Admin
Posts: 4609
Joined: 29 Mar 2005, 09:50
Location: Italy
Contact:

Postby Bax » 26 May 2010, 07:21

Actually that seems to be a problem with the Red5 server. Maybe you should Google-around to see if you can find specific support on this topic.
Also, the Red5 version which comes with SmartFoxServer is quite old. You may try to upgrade it to the latest version.
Paolo Bax
The SmartFoxServer Team
rock217
Posts: 1
Joined: 26 May 2010, 14:58

Postby rock217 » 26 May 2010, 15:00

Seeing this myself a lot as well, very irritating. Google-ing yields lots of users with the same issue and zero resolutions.

PLZ HALP US!
projuri
Posts: 7
Joined: 25 May 2010, 01:25

Postby projuri » 30 May 2010, 22:52

Red5 upgrade has solved the issue.

Thank you, bax!
User avatar
Bax
Site Admin
Posts: 4609
Joined: 29 Mar 2005, 09:50
Location: Italy
Contact:

Postby Bax » 31 May 2010, 07:02

projuri,
could you create a new post describing the upgrade process step-by-step? This will help other users. Thank you.
Paolo Bax
The SmartFoxServer Team

Return to “RedBox”

Who is online

Users browsing this forum: No registered users and 19 guests