Hanging on getConnection()

[jurekf]

Ever since we upgraded to RC3, we've had hung instances a couple of times.

When connecting, the server shows the "Session created: ... <port> <---> <port> " line but no longer proceeds to call our login handler.

We managed to debug it once; what happens is that the server hangs on getConnection(). Like this:

Code: Select all

      // Get password from DB
      IDBManager dbManager = getParentExtension().getParentZone().getDBManager();
      Connection connection;
      
        try {
         connection = dbManager.getConnection(); // <--- Hangs here
       

We have no synchronized locks in our server, so we're very likely not deadlocking anywhere in our code.

No users can log in anymore at this stage.

We've checked our database (MySQL) through phpmyadmin and it's still responsive, so we're not sure where to look for the problem.

[Lapo]

We didn't change anything about login procedures. I don't know how RC3 can be the cause of the issue.
Please check your logs and admin tool (like server queues) to see what's going on. At the moment we don't have any clue and we haven't been reported of anything like this.
We also use it in production and for tests and didn't see any problem with logins...

[jurekf]

It's not an issue with the login procedure itself; it's only hanging there because we're using a custom login that includes a getConnection() call.

It seems that prior to this all our getConnection() calls hang, and it's only most noticeable because when we log back in we can't. I'm guessing the default settings only use one thread for the login, so once that hangs none of the subsequent login event are dispatched.

It's possible this is due to the mysql driver included with SFS2X; I think we've only just experienced this recently, and it might coincide with our retrieving/storing large fields in the database (an 18KB node graph).

I've seen this happen on both our test Windows machine and our Linux instance, so I suspect the mysql database driver as the only remaining common link. Worst case I'll just use flat files for our data.

[Lapo]

What do you mean by hang?
Nothing in the logs?
Did you measure the performance of a DB access from the Extension?
Please provide more details, otherwise it will be difficult to be of help

thanks

[jurekf]

Hang as in getConnection() never returns and the login function never exits.
I confirmed it by single stepping through our login function as that's the easiest place to catch it. Prior to this, there were symptoms that other getConnection calls were also hanging (we noticed some data wasn't being saved).

We're using stock configuration for the number of threads, so I'm assuming there's only one thread handling login events, and once that hangs all logins stop and the clients connecting just time out and disconnect.

In the server logs there are no errors, just session connected info messages that are never retrieved by the login thread. The clients will probably eventually just drop the session and I think that gets logged but we've set our client timeouts high nowadays for testing so they rarely do.

DB access from the extension is fast prior to this as we're currently testing it over localhost and it happens with very light access, just one user, albeit one currently retrieving relatively large database fields. We can't reliably reproduce it yet though.

This may be the cause of the other threads people have posted regarding sudden mass-dropped connections and the server accepting messages but no longer responding as the end symptoms are similar.

[falcan]

n't have any clue and we haven't been reported of anything like this.

But Lapo I'm reporting this with other users for weeks in another thread(just posted in it today), trying to get answer here on the forums This thread is very helpful, at least I'll check my login code it has a similar issue like yours or not jurek. For my customer and his game this thing is a major pain, he has to restart the server every 8 hours, I'm really not sure what I messed up but no error reports and no ability to login is really not cool to debug. I have the same symptoms - connection is fine, but the login is never catched - can't login to admin panel even.

[falcan]

After reading through this again, could this be related to the database then after all? If the connection between MySQL and SmartFox is broken, it doesn't meant you wouldn't access it with phpMyAdmin I think - MySQL would still work, but it wouldnt work with SFX because lack of connection, I guess. For some weird reason I can't understand though, the getConnection call doesn't report error but just hangs?

If what you are saying is true(there is one thread servicing the login), that explains why logins to admin wouldn't work...(and that is what I was wondering about most).

[Lapo]

@falcan: my reply was not directed at you. You didn't start this thread, right? Please do not cross post. If you have started your topic we'll deal with it.
Thanks

[falcan]

Sorry Lapo but 1) I was kind of desparate because I posted to another thread(about 3x) with the same issue, and didn't get any answer for few weeks 2) I believe users can post to threads if they are seeing similar issue? Sorry if I offended you

[falcan]

I solved my issue, going to post a thread shortly - thanks to everyone posting hints!

[jurekf]

Yup, that looks about right; we were missing one finally block.
Odd that getConnection hangs for so long instead of throwing an exception though.

Thanks a lot falcan!