[.NET] Found issues and fixes

Post here all your questions related with SmartFoxServer .Net/Unity3D API

Moderators: Lapo, Bax

at
Posts: 19
Joined: 17 Oct 2008, 08:43

[.NET] Found issues and fixes

Postby at » 27 Oct 2008, 10:59

Hi all.

I'm using SF Client API in Unity 3D and I have a problem with receiving XML-messages.
My client recieves XML-messages from the server to synchronize player's position like this:

Code: Select all

<msg t='xt'><body action='xtRes' r='-1'><![CDATA[<dataObj><var n='_cmd' t='s'>SendTransform</var><obj o='tr' t='a'><var n='z' t='n'>810.759399414062</var><var n='rz' t='n'>0</var><var n='rx' t='n'>0</var><var n='w' t='n'>0.831018149852753</var><var n='t' t='n'>1.22510742586314E12</var><var n='ry' t='n'>0.556245446205139</var><var n='y' t='n'>26.5978050231934</var><var n='x' t='n'>1376.97448730469</var></obj><var n='id' t='n'>1</var></dataObj>]]></body></msg>


But sometimes I receive from the server something like:

Code: Select all

<msg t='xt'><body action='xtRes' r='-1'><![CDATA[<dataObj><var n='_cmd' t='s'>SendTransform</var><


So the message is corrupted. It happens not very often, usually when one client are already in a room and another client joins this room, then the first client (which was already in the room) receives this broken message.

I think it's a bug. Let's look on the HandleSocketData method

Code: Select all

   
private void HandleSocketData(IAsyncResult ar)
        {
            int BytesRead;
            try
            {
                // Finish asynchronous read into readBuffer and return number of bytes read.
                BytesRead = socketConnection.GetStream().EndRead(ar);
                if (BytesRead < 1)
                {
                    // if no bytes were read server has close. 
                    throw new Exception("No bytes could be read. Server connection disconnected");
                }
                // Add the received byte message to the messageBuffer, so we can cut up that one                 
                messageBuffer += Encoding.ASCII.GetString(byteBuffer, 0, BytesRead);

                // Cut up and handle each message separately
                Regex findNumMessagesRegEx = new Regex("\0");
                int numMessages = findNumMessagesRegEx.Matches(messageBuffer).Count;

                if (numMessages != 0)
                {
                    char[] delimChar = { '\0' };
                    string[] messages = messageBuffer.Split(delimChar);
                    for (int strCount = 0; strCount < messages.Length; strCount++)
                    {
                        // If this is the last string and its null, then we send all - nothing left for the buffer
                        // place rest in buffer else
                        if (strCount == messages.Length - 1)
                        {
                            if (messages[strCount].Length != 0)
                            {
                                messageBuffer = messages[strCount];
                            }
                            else
                            {
                                //Ignore last empty one
                                messageBuffer = "";
                                break;
                            }

                        }
                        HandleMessage(messages[strCount]);
                    }
                }
                else
                {
                    // messageBuffer lives onto next socket data is received
                }

                // Start a new asynchronous read into readBuffer.
                byteBuffer = new byte[READ_BUFFER_SIZE];
                socketConnection.GetStream().BeginRead(byteBuffer, 0, READ_BUFFER_SIZE, new AsyncCallback(HandleSocketData), null);
            }
            catch (Exception e)
            {
               DebugMessage("Disconnect due to: " + e.ToString());
                HandleSocketDisconnection();
            }

        }


When working with sockets in .NET it may happen that asynchronous delegate is being called earlier than all the sent data is read. Maybe you should transmit the length of incoming message first to let the client able to check if he has read all the bytes.

Anyway I think we should not handle disconnection on every exception here. Calling the HandleSocketDisconnection() method when we are not really disconnected from server leads to incorrect behaviour of the client-side. If we have received a corrupted message we can continue working. So I have added a try-catch block in HandleMessage method like this:

Code: Select all

 
private void HandleMessage(string msg)
        {
            if (msg != "ok") {
               DebugMessage("[ RECEIVED ]: " + msg + ", (len: " + msg.Length + ")");
            }
           
            try { 

               string type = msg.Substring(0, 1);
               if (type == MSG_XML)
               {
                   XmlReceived(msg);
               }
               else if (type == MSG_STR)
               {
                   StrReceived(msg);
               }
               else if (type == MSG_JSON)
               {
                   JsonReceived(msg);
               }
           
            }
            catch (Exception e) {
               Debug.Log("Wrong message received! "+e.Message+" >> " +msg);
            }
        }


Now I'm using this as a workaround. I haven't tried to fix the socket problem yet, and I may be wrong concerning the reason of the bug. If I have time I will try to fix it, but I hope it will be fixed in the new version.

Thank you.
Last edited by at on 07 Dec 2008, 09:44, edited 3 times in total.
ThomasLund
Posts: 1297
Joined: 14 Mar 2008, 07:52
Location: Sweden

Postby ThomasLund » 27 Oct 2008, 13:05

Hi and thanks for your feedback.

The current code is very very old, and one of the things that has been rewritten in beta2.

If you want, you can drop me an PM and get the latest source code to test. I still have some issues with disconnecting properly on shutdown, so beta 2 is hanging on that one. Biggest problem being, that my development machine is currently in the repair shop (been there 3 weeks now for motherboard switch and such), and I cannot start Visual Studio on my Mac mini replacement.

Any help would be great, but should be done on the latest codebase.

Besides that, look at the post earlier in the forums by AmazingRuss - he is doing some thread-safe queuing of incomming messages from the server. That might also be a problem.
at
Posts: 19
Joined: 17 Oct 2008, 08:43

Postby at » 27 Oct 2008, 14:42

Hi, ThomasLund

First of all thanks a lot for your answer!

ThomasLund wrote:Besides that, look at the post earlier in the forums by AmazingRuss - he is doing some thread-safe queuing of incomming messages from the server. That might also be a problem.


Yes, I've already read it and I also use a thread-safe queuing in Unity. I noticed that corrupted messages appear more often on a slower connection. So maybe it's some socket or network problem.

BTW, If you have a Unity 3D on your Mac, you can develop and test the source code right on it. I haven't got a PC, so I copied all the sources of the SFS Client in a folder under my Unity project and after a little trick it works.
ThomasLund
Posts: 1297
Joined: 14 Mar 2008, 07:52
Location: Sweden

Postby ThomasLund » 28 Oct 2008, 07:53

at wrote:BTW, If you have a Unity 3D on your Mac, you can develop and test the source code right on it. I haven't got a PC, so I copied all the sources of the SFS Client in a folder under my Unity project and after a little trick it works.


Oh yeah true - DUH. Thanks for reminding me :-D
at
Posts: 19
Joined: 17 Oct 2008, 08:43

Postby at » 28 Oct 2008, 10:15

I have found the reason of the issue with the corrupted messages. When HandleSocketData method receives a buffer it splits the messages by the symbol '\0'. There is a check for the last splitted string, if it is not empty - then we have received an incomplete message, thus we should not call HandleMessage for it.
I added boolean variable called restsPartOfBuffer in order to maintain this situation.
Now everything works ok and I stopped getting exceptions in HandleMessage method.

Here the source code of the modified method:

Code: Select all

 private void HandleSocketData(IAsyncResult ar)
        {
            int BytesRead;
            try
            {
                // Finish asynchronous read into readBuffer and return number of bytes read.
                BytesRead = socketConnection.GetStream().EndRead(ar);
                if (BytesRead < 1)
                {
                    // if no bytes were read server has close. 
                    throw new Exception("No bytes could be read. Server connection disconnected");
                }
               
                // Add the received byte message to the messageBuffer, so we can cut up that one                 
                messageBuffer += Encoding.ASCII.GetString(byteBuffer, 0, BytesRead);

                // Cut up and handle each message separately
                Regex findNumMessagesRegEx = new Regex("\0");
                int numMessages = findNumMessagesRegEx.Matches(messageBuffer).Count;

                if (numMessages != 0)
                {
                    char[] delimChar = { '\0' };
                    string[] messages = messageBuffer.Split(delimChar);
                    bool restsPartOfBuffer = false;
                    for (int strCount = 0; strCount < messages.Length; strCount++)
                    {
                        // If this is the last string and its null, then we send all - nothing left for the buffer
                        // place rest in buffer else
                        if (strCount == messages.Length - 1)
                        {
                            if (messages[strCount].Length != 0)
                            {
                                messageBuffer = messages[strCount];
                                restsPartOfBuffer = true;
                            }
                            else
                            {
                                //Ignore last empty one
                                messageBuffer = "";
                                break;
                            }

                        }
                       
                        if (!restsPartOfBuffer) HandleMessage(messages[strCount]);
                    }
                }
                else
                {
                    // messageBuffer lives onto next socket data is received
                }

                // Start a new asynchronous read into readBuffer.
                byteBuffer = new byte[READ_BUFFER_SIZE];
                socketConnection.GetStream().BeginRead(byteBuffer, 0, READ_BUFFER_SIZE, new AsyncCallback(HandleSocketData), null);
            }
            catch (Exception e)
            {
                DebugMessage("Disconnect due to: " + e.ToString());
                HandleSocketDisconnection();
            }

        }
ThomasLund
Posts: 1297
Joined: 14 Mar 2008, 07:52
Location: Sweden

Postby ThomasLund » 28 Oct 2008, 17:17

Super duper!!! Thanks a bunch.

For my part I've implemented async connection today.

Here is the new connect + the async method. Replace the old connect with this:

Code: Select all

        public void Connect(string ipAdr, int port)
        {
            DebugMessage("Trying to connect");
            if (!connected)
            {
                try
                {
                    Initialize();
                    this.ipAddress = ipAdr;
                    this.port = port;

                    socketConnection = new TcpClient(AddressFamily.InterNetwork);

    IPAddress[] remoteHost = Dns.GetHostAddresses(ipAdr);
    socketConnection.BeginConnect(remoteHost, port, new AsyncCallback(ASyncSocketConnected), socketConnection);



                }
                catch (System.Net.Sockets.SocketException e)
                {
                    HandleIOError(e.Message);
                }
           }
            else
                DebugMessage("*** ALREADY CONNECTED ***");
        }

   void ASyncSocketConnected(IAsyncResult iar)
   {
      try
      {
         socketConnection.EndConnect(iar);
            // Start an asynchronous read invoking handleSocketData to avoid lagging the user
            // interface.
            socketConnection.GetStream().BeginRead(byteBuffer, 0, READ_BUFFER_SIZE, new AsyncCallback(HandleSocketData), null);
            connected = true;
            HandleSocketConnection(this, new EventArgs());

      } catch (SocketException) {
                DebugMessage("*** ERROR CONNECTING ***");
      }
   }


I am seriously hitting my head at the last known bug/feature missing.

There is something not being cleaned up correctly. This means that Firefox crashes (actually on connect as it doesnt survive an initial cleanup), as well as Safari and Unity on shutdown. Same for standalone apps with shutdown.

If you could take a peek at it also, I would be very grateful.

Spend all day today rewriting things to use sockets instead of tcpclient. While that part went well, it didnt remove the error.

I am guessing that the async methods are somehow still running and not being closed down properly on the dispose part. But its a hunch, and I simply cant get any meaningful error messages to help me.
ThomasLund
Posts: 1297
Joined: 14 Mar 2008, 07:52
Location: Sweden

Postby ThomasLund » 30 Oct 2008, 05:12

OK - dont spend time on the socket crash / disconnect problem.

I've spend 2 while days on this, and it seems to be a Unity/mono bug or a Unity peculiarity. Bug report filed with Unity, so lets see what happens.

This also means that beta 2 will not turn up until they report back on this issue.

Seems like Unity hangs onto the sockets somehow even after shutting down the actual code - and has a thread hanging that makes Unity crash. There are others on the Unity forums who had the same problems.
at
Posts: 19
Joined: 17 Oct 2008, 08:43

Postby at » 30 Oct 2008, 06:38

ThomasLund wrote:OK - dont spend time on the socket crash / disconnect problem.

I've spend 2 while days on this, and it seems to be a Unity/mono bug or a Unity peculiarity. Bug report filed with Unity, so lets see what happens.

This also means that beta 2 will not turn up until they report back on this issue.

Seems like Unity hangs onto the sockets somehow even after shutting down the actual code - and has a thread hanging that makes Unity crash. There are others on the Unity forums who had the same problems.


Hi. With the old beta version of SFS client I had almost no crashes in Unity. I always did the logout before exit application. With the new version it doesn't crashes even if I forget to logout. So, now I have no crashes at all.
Concerning Firefox, it crashes on Mac when I try to connect to the server. But Safari works ok. In Windows Firefox doesn't crash.
I think it's some compatibility problems, maybe even caused by Mono library.
ThomasLund
Posts: 1297
Joined: 14 Mar 2008, 07:52
Location: Sweden

Postby ThomasLund » 30 Oct 2008, 07:28

Oh!!! That is very interesting.

I will upload my minimal testcase. If you could try and test, that would be great!
ThomasLund
Posts: 1297
Joined: 14 Mar 2008, 07:52
Location: Sweden

Postby ThomasLund » 30 Oct 2008, 07:56

Test case is here:

http://www.fullcontrol.dk/downloads/Sma ... shTest.zip

It is as minimal as possible (more or less) while still using the same structure as the SFS beta 2.

I get guaranteed crash with Firefox, Safari, standalone Unity apps as well as the Unity editor - all when I close the binary.

Player and editor logs hint at a thread not being closed. Firefox rarely ever writes the crash in the log, but Safari does
at
Posts: 19
Joined: 17 Oct 2008, 08:43

Postby at » 30 Oct 2008, 07:57

Ok, I will test it.
ThomasLund
Posts: 1297
Joined: 14 Mar 2008, 07:52
Location: Sweden

Postby ThomasLund » 02 Nov 2008, 16:18

OK - little update. With the help from a fellow Unity guy the SFS API now uses threaded sockets instead of async sockets. And that has fixed the crash bug.

Expect a release soon :-)
at
Posts: 19
Joined: 17 Oct 2008, 08:43

Postby at » 05 Nov 2008, 09:46

Hi again. I haven't time to try your test version yet, but my collegue has the described problem with crashes in Unity with the current version of SFS client. It's nice that threaded sockets solved the problem though, in my opinion, asynchronious sockets are more convenient.

Here I would like to report another issue, this time in the SysHandler class. When user enters to an existing room, he doesn't receive the user variables of the other users, that are already present in this room.

Let's look on the part of HandleJoinOk method where the room user list are being populated.

Code: Select all

            // Populate Room userList
            foreach (XmlNode usr in XmlUtil.GetNodeList(userListXml, "./u"))
            {
                XmlUtil.Dump(usr, 0);
                // grab the user properties
                string name = XmlUtil.GetString(usr, "./n/node()");
                int id = XmlUtil.GetInt(usr, "./@i");
                bool isMod = XmlUtil.GetBool(usr, "./@m");
                bool isSpec = XmlUtil.GetBool(usr, "./@s");
                int pId = XmlUtil.GetSingleNode(usr, "./@p") == null ? -1 : XmlUtil.GetInt(usr, "./@p");

                // Create and populate User
                User user = new User(id, name);
                user.SetModerator(isMod);
                user.SetIsSpectator(isSpec);
                user.SetPlayerId(pId);

                // Handle user variables
                if (XmlUtil.GetSingleNode(usr, "./vars") != null)
                {
                    PopulateVariables(user.GetVariables(), XmlUtil.GetSingleNode(usr, "./vars"));
                }

                // Add user
                currRoom.AddUser(user, id);
            }



Here PopulateVariables method is called with XmlUtil.GetSingleNode(usr, "./vars") node. But if we look in the PopulateVariables method we see that there we take the nodes containing variables by the path "vars/var" :

Code: Select all

foreach (XmlNode v in XmlUtil.GetNodeList(xmlData, "vars/var"))


As we call method already giving to it vars node, here nothing can be found, so the lists of variables remain empty.

I have fixed this issue by changing the code of HandleJoinOk method.

The string

Code: Select all

PopulateVariables(user.GetVariables(), XmlUtil.GetSingleNode(usr, "./vars"));


changed to

Code: Select all

PopulateVariables(user.GetVariables(), usr);


Now user receives the variables of other users correctly when joining the room.
Last edited by at on 05 Nov 2008, 11:20, edited 1 time in total.
at
Posts: 19
Joined: 17 Oct 2008, 08:43

Postby at » 05 Nov 2008, 11:17

Yet another issue connected with the previous one.
On the server-side we can send 3 types of user variables: numbers, strings, and booleans. But numbers can be not only integer, they can be floating-point. So I set some floating-point values on the server-side as user variables. They are normally transmitted via xml-messages to clients.

But looking on the client-side in the PopulateVariables method:

Code: Select all

...

else if (vType == "n")
{
   variables[vName] = int.Parse(vValue);
}

...   


Here we should use double.Parse(vValue) instead of int.Parse(vValue) - then we can use any numbers correctly.

So the modified part of code looks like this:

Code: Select all

...

else if (vType == "n")
{
   variables[vName] = double.Parse(vValue);
}

...   
ThomasLund
Posts: 1297
Joined: 14 Mar 2008, 07:52
Location: Sweden

Postby ThomasLund » 05 Nov 2008, 14:46

Hey at

Thanks a bunch for testing and sending fixes. I will get them into beta 2 tonight or tomorrow morning, test and get them out with beta 2 release. Might as well!

Keep testing and please keep up the great work on also providing a fix. Makes it easy.

I would also have loved to keep the async stuff in place, but Unity/mono just doesnt like it. Only way around at the moment was to code manually created threads - a solution I dont like but have to live with.

It does mean that beta 2 will be damn near production ready, so in the end thats great.

Return to “.Net / Unity3D API”

Who is online

Users browsing this forum: No registered users and 19 guests