sfs2x stops accepting udp

Post here your questions about the Unity / .Net / Mono / Windows 8 / Windows Phone 8 API for SFS2X

Moderators: Lapo, Bax

creat326
Posts: 87
Joined: 13 Jun 2010, 09:50

sfs2x stops accepting udp

Postby creat326 » 03 Nov 2012, 15:21

Hi

I've been using sfs2x for a while and i'm pretty familiar with it (in fact I ported the client side code to windows 8). I'm posting here so maybe Thomas can help out.
It seems there is a bug on the server side on regards to UDP connections. It has been working perfectly before I changed from 2.0 to 2.3 so i'm blind and it is driving my players nuts.

smartfox 2.3... after running for a few hours it begins disconnecting people. I rebooted the server and begins happening again, eventhough there is only 1 connection on the server (mine). It seems the udp server side begins to fail for whatever reason.

it doesn't seem to do it for any particular reason but it's certainly UDP related since the TCP connection is fine (I connect to the game), it's only after I get the UDP error that it fails. UDP is simply not able to connect. What's odd is that it begins happening after a while, so it runs fine for 3 hours and then it fails. I'm using port 443 which should be open for everything. Rebooting the smartfox server fixes it (which makes it clear is a server side problem), but I can't be rebooting the servers every 2 or 3 hours... could it be possible that is due to the license side messing up port 443 for udp? I really don't know what could happen.


This is the log on the client:

UDP error: unknown error
UnityEngine.Debug:Log(Object)
SmartFoxManager:Log(Object) (at Assets/Scripts/SmartFox.cs:464)
SmartFoxManager:OnUdpInit(BaseEvent) (at Assets/Scripts/SmartFox.cs:1280)
Sfs2X.Core.EventDispatcher:DispatchEvent(BaseEvent) (at Assets/Scripts/SFS2XClient/Core/EventDispatcher.cs:47)
Sfs2X.SmartFox:ProcessEvents() (at Assets/Scripts/SFS2XClient/SmartFox2x.cs:1487)



and the logfile on the server side is:

bitswarm.sessions.DefaultSessionManager | | Session removed: { Id: 1545, Type: DEFAULT, Logged: Yes, IP: 83.45.180.72:55813 }



03 Nov 2012 | 09:36:25,390 | INFO | Scheduler1-thread-1 | v2.entities.SFSZone | | User: zuperman was disconnected.



03 Nov 2012 | 09:36:25,390 | INFO | Scheduler1-thread-1 | v2.api.SFSApi | | User disconnected: { Zone: Dogfight }, ( User Name: zuperman, Id: 1247, Priv: 2, Sess: 83.45.180.72:55813 ) , Type: UnityPlayer::, SessionLen: 18472



and the config file:




<serverSettings>

<socketAddresses>

<socket address="127.0.0.1" port="9933" type="TCP"/>

<socket address="127.0.0.1" port="9933" type="UDP"/>

<socket address="127.0.0.1" port="443" type="TCP"/>

<socket address="127.0.0.1" port="443" type="UDP"/>

<socket address="myip" port="9933" type="TCP"/>

<socket address="myip" port="9933" type="UDP"/>

<socket address="myip" port="443" type="TCP"/>

<socket address="myip" port="443" type="UDP"/>

</socketAddresses>

<ipFilter>

<addressBlackList>

<string>1.2.3.4</string>

</addressBlackList>

<addressWhiteList>

<string>127.0.0.1</string>

<string>myip</string>

</addressWhiteList>

<maxConnectionsPerAddress>10</maxConnectionsPerAddress>

</ipFilter>

<flashCrossdomainPolicy>

<useMasterSocketPolicy>true</useMasterSocketPolicy>

<policyXmlFile>crossdomain.xml</policyXmlFile>

</flashCrossdomainPolicy>

<systemControllerThreadPoolSize>1</systemControllerThreadPoolSize>

<extensionControllerThreadPoolSize>3</extensionControllerThreadPoolSize>

<systemControllerRequestQueueSize>20000</systemControllerRequestQueueSize>

<extensionControllerRequestQueueSize>20000</extensionControllerRequestQueueSize>

<schedulerThreadPoolSize>3</schedulerThreadPoolSize>

<protocolCompressionThreshold>1024</protocolCompressionThreshold>

<protocolMode>BINARY</protocolMode>

<useBinaryProtocol>true</useBinaryProtocol>

<remoteAdmin>

<administrators>

<adminUser>

<login>root</login>

<password>password</password>

</adminUser>

</administrators>

<allowedRemoteAddresses>

<string>127.0.0.1</string>

</allowedRemoteAddresses>

<adminTcpPort>9933</adminTcpPort>

</remoteAdmin>

<bannedUserManager>

<isAutoRemove>true</isAutoRemove>

<isPersistent>true</isPersistent>

<customPersistenceClass></customPersistenceClass>

</bannedUserManager>

<mailer>

<isActive>true</isActive>

<mailHost>smtp.1and1.com</mailHost>

<mailUser>dogfight@myemail</mailUser>

<mailPass>password</mailPass>

<smtpPort>25</smtpPort>

<workerThreads>1</workerThreads>

</mailer>

<webServer>

<isActive>true</isActive>

<cfgFile>config/jetty/jetty.xml</cfgFile>

<blueBoxPollingTimeout>26</blueBoxPollingTimeout>

<blueBoxMsgQueueSize>40</blueBoxMsgQueueSize>

</webServer>

<webSocket>

<isActive>false</isActive>

<bindAddress>127.0.0.1</bindAddress>

<tcpPort>8888</tcpPort>

<validDomains/>

</webSocket>

<startExtensionFileMonitor>true</startExtensionFileMonitor>

<useDebugMode>true</useDebugMode>

<extensionRemoteDebug>false</extensionRemoteDebug>

<useFriendlyExceptions>true</useFriendlyExceptions>

<sessionMaxIdleTime>50</sessionMaxIdleTime>

<userMaxIdleTime>300</userMaxIdleTime>

<licenseCode></licenseCode>

<licenseEmails></licenseEmails>

<ghostHunterEnabled>false</ghostHunterEnabled>

</serverSettings>
User avatar
Lapo
Site Admin
Posts: 23009
Joined: 21 Mar 2005, 09:50
Location: Italy

Re: sfs2x stops accepting udp

Postby Lapo » 04 Nov 2012, 13:46

Moved in the correct section. Thanks
Lapo
--
gotoAndPlay()
...addicted to flash games
ThomasLund
Posts: 1297
Joined: 14 Mar 2008, 07:52
Location: Sweden

Re: sfs2x stops accepting udp

Postby ThomasLund » 04 Nov 2012, 16:19

I'm still running 2.2 on my production setup, so not run into this one yet.

First up though - what client version are you running, since the line numbers dont match up against mine (latest SVN - v 1.0.4)

Anyways - Marco - any changes server side in 2.3 that could influence UDP?

/T
Full Control - maker of Unity/C# and Java SFS API and indie games
Follow on twitter: http://twitter.com/thomas_h_lund
User avatar
Lapo
Site Admin
Posts: 23009
Joined: 21 Mar 2005, 09:50
Location: Italy

Re: sfs2x stops accepting udp

Postby Lapo » 04 Nov 2012, 21:15

No changes on that front and no other similar reports.
Thanks
Lapo

--

gotoAndPlay()

...addicted to flash games
ThomasLund
Posts: 1297
Joined: 14 Mar 2008, 07:52
Location: Sweden

Re: sfs2x stops accepting udp

Postby ThomasLund » 04 Nov 2012, 21:22

Thx Marco - expected as much.

No change either in client side that should have this effect.

Do you by any chance have the opportunity to try to run the server on default ports and test?

Is there a repro case that Marco and I can get access to that will work in our ends showing the problem?

/Thomas
Full Control - maker of Unity/C# and Java SFS API and indie games

Follow on twitter: http://twitter.com/thomas_h_lund
creat326
Posts: 87
Joined: 13 Jun 2010, 09:50

Re: sfs2x stops accepting udp

Postby creat326 » 08 Nov 2012, 17:16

Well, it's random but I did a service that checks every minute if UDP on the server stopped working and voila! caught the exception that is hit when all heaven breaks loose. This is what's on the log on the server:
08 Nov 2012 | 11:29:37,048 | INFO | com.smartfoxserver.v2.controllers.ExtensionController-3 | api.response.SFSResponseApi | | Room Recipients: [{ Id: 176, Type: DEFAULT, Logged: Yes, IP: 174.254.1.36:14106 }, { Id: 197, Type: DEFAULT, Logged: Yes, IP: 77.248.193.144:51001 }, { Id: 125, Type: DEFAULT, Logged: Yes, IP: 78.133.25.159:23826 }, { Id: 198, Type: DEFAULT, Logged: Yes, IP: 180.26.17.134:38094 }, { Id: 199, Type: DEFAULT, Logged: Yes, IP: 76.168.136.19:35256 }, { Id: 207, Type: DEFAULT, Logged: Yes, IP: 24.127.120.214:33142 }, { Id: 200, Type: DEFAULT, Logged: Yes, IP: 213.205.233.168:55737 }, { Id: 230, Type: DEFAULT, Logged: Yes, IP: 77.245.6.55:38363 }]
08 Nov 2012 | 11:29:38,148 | WARN | SocketReader | bitswarm.core.SocketReader | | Problems in SocketReader main loop: java.lang.ClassCastException: sun.nio.ch.DatagramChannelImpl cannot be cast to java.nio.channels.SocketChannel, Thread: Thread[SocketReader,5,main]
08 Nov 2012 | 11:29:38,148 | WARN | SocketReader | bitswarm.core.SocketReader | |
java.lang.ClassCastException: sun.nio.ch.DatagramChannelImpl cannot be cast to java.nio.channels.SocketChannel
com.smartfoxserver.bitswarm.core.SocketReader.closeTcpConnection(SocketReader.java:351)
com.smartfoxserver.bitswarm.core.SocketReader.readIncomingSocketData(SocketReader.java:236)
com.smartfoxserver.bitswarm.core.SocketReader.run(SocketReader.java:172)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
java.lang.Thread.run(Unknown Source)

So something on socketreader is messed up. It is odd because it seems it's passing Datagram (udp) but it comes from Tcp connection.

any clues?
ThomasLund
Posts: 1297
Joined: 14 Mar 2008, 07:52
Location: Sweden

Re: sfs2x stops accepting udp

Postby ThomasLund » 25 Nov 2012, 16:39

There were some recent UDP fixes in the latest server release. Did this fix it for you?

/Thomas
Full Control - maker of Unity/C# and Java SFS API and indie games

Follow on twitter: http://twitter.com/thomas_h_lund
creat326
Posts: 87
Joined: 13 Jun 2010, 09:50

Re: sfs2x stops accepting udp

Postby creat326 » 27 Nov 2012, 00:37

nope, the exception message is gone but the udp stops receiving connections. So basically it seems someone on the code hide the exception now but does not fix it.
User avatar
Lapo
Site Admin
Posts: 23009
Joined: 21 Mar 2005, 09:50
Location: Italy

Re: sfs2x stops accepting udp

Postby Lapo » 27 Nov 2012, 09:05

This is not correct, the exception is not hidden at all.
What was happening previously is that an unhandled UDP error would trigger a method used for closing TCP connections causing a cast exception. This has been fixed (v 2.4) and the UDP error is now handled properly.
The UDP exception indicates an unrecoverable IO error between client and server which will result in the inability of the client to communicate via UDP.

In no way such error can cause the server to stop or becoming unresponsive. If this is what you are experiencing then please provide a description of what's happening:

1. check the logs for errors
2. can you reach the server via admin tool?
3. grab a thread dump of the server while unresponsive ( http://www.smartfoxserver.com/forums/vi ... 18&t=14458 )

thanks
Lapo

--

gotoAndPlay()

...addicted to flash games
creat326
Posts: 87
Joined: 13 Jun 2010, 09:50

Re: sfs2x stops accepting udp

Postby creat326 » 27 Nov 2012, 11:38

yeah it stops completely receiving udp connections, but there is no error on the log file (no exceptions) only disconnect/reconnect messages when it happens.
It's very very hard to get the dump which I've been trying to do for over a week right now. The issue is that there is no way to know when or why it happen and since nothing on the log says anything I can't pinpoint it unless I try to connect myself and fail to do so.

What I've done right now is create a service that tries to connect every minute. If it fails to do so, it will reboot smartfox. If there is an automatic way, or a .bat script that I could use to get the JVM needed info for you guys, it would be great so I can just add it to my service before rebooting it.
User avatar
Lapo
Site Admin
Posts: 23009
Joined: 21 Mar 2005, 09:50
Location: Italy

Re: sfs2x stops accepting udp

Postby Lapo » 27 Nov 2012, 12:14

yeah it stops completely receiving udp connections, but there is no error on the log file (no exceptions) only disconnect/reconnect messages when it happens.

We need to understand what's happening:

1) Are all clients unable to use the UDP protocol after this problem happens?
2) Are you able to connect via TCP? i.e. via the AdminTool?

It's very very hard to get the dump which I've been trying to do for over a week right now. The issue is that there is no way to know when or why it happen and since nothing on the log says anything I can't pinpoint it unless I try to connect myself and fail to do so.

You might want to create a small app that keeps monitoring the udp connection and sends an alert.

What I've done right now is create a service that tries to connect every minute. If it fails to do so, it will reboot smartfox. If there is an automatic way, or a .bat script that I could use to get the JVM needed info for you guys, it would be great so I can just add it to my service before rebooting it.

Grabbing a thread dump requires a bit of manual work, but it's just a two steps operation so I am sure that you could make a little .bat file that can be executed before restarting.

In any case before using this automation please make sure to check of the Server's connectivity for both UDP and TCP.

Thanks
Lapo

--

gotoAndPlay()

...addicted to flash games
creat326
Posts: 87
Joined: 13 Jun 2010, 09:50

Re: sfs2x stops accepting udp

Postby creat326 » 27 Nov 2012, 12:33

yes, you can connect via the admin panel. TCP works just fine too.
UDP is the one not working. UDP init fails on the client side.
creat326
Posts: 87
Joined: 13 Jun 2010, 09:50

Re: sfs2x stops accepting udp

Postby creat326 » 27 Nov 2012, 13:46

also the documentation about how to generate a thread dump is obsolete. I can't find anywhere how to do it. The link on the previous link does not work for windows.
I found the jconsole manual on a different site but it says nothing about generating a dump, so what's the best way to do it?
http://docs.oracle.com/javase/6/docs/te ... nsole.html
User avatar
Lapo
Site Admin
Posts: 23009
Joined: 21 Mar 2005, 09:50
Location: Italy

Re: sfs2x stops accepting udp

Postby Lapo » 27 Nov 2012, 14:45

You can use the jstack utility from Windows as well, only thing you will need to obtain the process id from the Windows Task Manager:
https://blogs.oracle.com/pcmreddy/entry ... on_windows
Lapo

--

gotoAndPlay()

...addicted to flash games
User avatar
Lapo
Site Admin
Posts: 23009
Joined: 21 Mar 2005, 09:50
Location: Italy

Re: sfs2x stops accepting udp

Postby Lapo » 27 Nov 2012, 15:00

EDIT: Actually I've verified that JPS works under Windows just like on Unix/Linux, so you can use them same system.
Run JPS to see the process id of the server, then use JSTACK <pid> to obtain the thread dump.
Lapo

--

gotoAndPlay()

...addicted to flash games

Return to “SFS2X C# API”

Who is online

Users browsing this forum: No registered users and 23 guests