I have 2 problems that needs some help.
1. The server sends messages to the client in a specific order, but sometimes we see on the client that the order recieved is not the same as the order sent. This might be related to issue 2.
2. During the flow of our game, the server would send several messages at different times to the client. The issue is that on the client logs, they would sometimes receive all these messages at once instead of being spread out. Is this related to SFS using NIO or is the sendResponse really a queue in the backend? Is there a way to send a message and ensures that the client receives it?
Thx for the help.
sendResponse being received out of order or all at once
Hello,
Can you explain which messages are detected as out of order? I'd expect them to be custom extension messages? Right?
NIO does not count.
sendResponse attempts to write data immediately to the client but this is not always possible, depending on the conditions of the connection to the client. In that case the data is queued and resent as soon as the communication channel is ready for write operations.
I don't understand your last question however.
Messages are always delivered unless the client connection is highly problematic. In that case the client will start to lag badly and ultimately get disconnected.
Besides this special case, the underlying TCP protocol guarantees proper delivery of every packet being sent.
1. The server sends messages to the client in a specific order, but sometimes we see on the client that the order recieved is not the same as the order sent. This might be related to issue 2.
Can you explain which messages are detected as out of order? I'd expect them to be custom extension messages? Right?
2. During the flow of our game, the server would send several messages at different times to the client. The issue is that on the client logs, they would sometimes receive all these messages at once instead of being spread out. Is this related to SFS using NIO or is the sendResponse really a queue in the backend? Is there a way to send a message and ensures that the client receives it?
NIO does not count.
sendResponse attempts to write data immediately to the client but this is not always possible, depending on the conditions of the connection to the client. In that case the data is queued and resent as soon as the communication channel is ready for write operations.
I don't understand your last question however.
Messages are always delivered unless the client connection is highly problematic. In that case the client will start to lag badly and ultimately get disconnected.
Besides this special case, the underlying TCP protocol guarantees proper delivery of every packet being sent.
The messages that is being received out of order are custom messages from my java extension.
My last question was asking if there is a way to send a message and block until that message is sent before the server extension will proceed? The reason I need something like this is that some server messages cannot be received by the client before another, otherwise the client logic breaks.
My last question was asking if there is a way to send a message and block until that message is sent before the server extension will proceed? The reason I need something like this is that some server messages cannot be received by the client before another, otherwise the client logic breaks.
Gentleman wrote:The messages that is being received out of order are custom messages from my java extension.
This can happen when using multiple threads for the ExtensionHandler (ExtHandlerThread > 1).
Two concurrent requests might require different execution times on the server side.
For example:
ReqA retrieves some user profile from the database
ReqB returns if a game is started or not
You would expect a reply from A first and B second.
Both requests are processed concurrently, unfortunately reading data from the database is slower and while the 1st thread is waiting for the data the 2nd has already finished and is sending a response back.
Result? ResponseB gets to the user first, then ResponseA
How to solve this?
Actually in this particular case it wouldn't really matter which of the two returns first because they are two unrelated pieces of information.
If you need a sequence of messages to keep their orders you will need to send them sequentially, waiting for each one their relative response.
In other words:
Send RequestA ---> Wait ResponseA
Send RequestB ---> Wait ResponseB
Makes sense?
This is not my current use case. My issue is not a problem of multiple request from one client causing multiple responses.
The client is making one request to the server.
The server would run its logic and send back serveral messages using several calls to sendResponse(). Some of these messages to the client are control messages that cannot be received before others. My server is not using database when it is running, only when it first initialize.
The client is making one request to the server.
The server would run its logic and send back serveral messages using several calls to sendResponse(). Some of these messages to the client are control messages that cannot be received before others. My server is not using database when it is running, only when it first initialize.
ok, but how do i ensure that they are received in order if the recipient is not the sernder:
Multiplayer Game:
Player1 > sendXtMessage > create Item
Player1 > sentXtMessage > move Item
---
Player 2 needs to receive this information in the exact order, otherwise the app will break (unable to move an Item that doesn't exist).
edit: oh, i just saw that default threads == 1, so this won't be a problem...
Multiplayer Game:
Player1 > sendXtMessage > create Item
Player1 > sentXtMessage > move Item
---
Player 2 needs to receive this information in the exact order, otherwise the app will break (unable to move an Item that doesn't exist).
edit: oh, i just saw that default threads == 1, so this won't be a problem...
GentleMan:
Responses to a client are processed sequentially. Until one message is fully delivered no other message is allowed to go out.
If you really have a problem like this I would suspect that the responses entered the recipient queue in the wrong order.
In this case it could be a problem with threading coordination in your code... although I am not able to think of a specific scenario right now.
Also I would answer your question with another question.
Why do you need to send multiple separated responses to a client instead of sending them as a single packet?
If the responses are generated at the same time it would be good to pack the data into a single response object and then handle it accordingly on the client side. Unless you're using the raw protocol, you can nest response objects so you shouldn't have problems "separating" the two different messages
The client is making one request to the server.
The server would run its logic and send back serveral messages using several calls to sendResponse().
Responses to a client are processed sequentially. Until one message is fully delivered no other message is allowed to go out.
If you really have a problem like this I would suspect that the responses entered the recipient queue in the wrong order.
In this case it could be a problem with threading coordination in your code... although I am not able to think of a specific scenario right now.
Also I would answer your question with another question.
Why do you need to send multiple separated responses to a client instead of sending them as a single packet?
If the responses are generated at the same time it would be good to pack the data into a single response object and then handle it accordingly on the client side. Unless you're using the raw protocol, you can nest response objects so you shouldn't have problems "separating" the two different messages
Also I would answer your question with another question.
Why do you need to send multiple separated responses to a client instead of sending them as a single packet?
If the responses are generated at the same time it would be good to pack the data into a single response object and then handle it accordingly on the client side. Unless you're using the raw protocol, you can nest response objects so you shouldn't have problems "separating" the two different messages
The server sends messages to client for many possilble reasons other than upon client request. For example, the opponent could have done something and needs to update the other player. These event happen at different times but all the messages gets queue up on the server because the client can't receive messages for what ever reason (from my understanding of how SFS works).
When the client eventually receives these messages, sometimes that are in the wrong order. Normally, this wouldn't matter except when there are control messages that must be received before others messages. My server logic is sending these control messages in a specific order.
For example, the opponent could have done something and needs to update the other player. These event happen at different times but all the messages gets queue up on the server because the client can't receive messages for what ever reason (from my understanding of how SFS works).
Very clear.
If various events happen independently and the extension needs to tell the client, I would expect that there is no expected order in those messages. Events are asynchronous and they can happen at any time.
If there is an expected sequence in the messages the same should apply to those events. You should expect them to always fire in sequence.
Now I have hard time discussing this further in an abstract way because I have no idea of what your code does.
My only point here is that the wrong order in the messages is 99% due to how these messages go into the queue.
Try monitoring that by simply logging some data right before sending the response.
My app is a turn based game.
For example, it is Player 1(P1) turn. P1 sends messages that says it is done with the current phase. The server upon receiving this calls sendResponse() to acknowledge that it received this message and then continues with its logic to make the game advance to the next phase of the game. When this phase advancement finished, both clients gets updates for the new game states via multiple calls of sendResponse(). Player 2 received it fine, but P1 would receive all the messages except the acknowledge message is not the first thing it receives.
I have only seen this type of error when the client would received all theses messages at once. For example, the client has 60secs for a phase, if they sent the request to advance phase, they don't get anything back for about 20-30 secs and then all 5 messages would be received at once.
For example, it is Player 1(P1) turn. P1 sends messages that says it is done with the current phase. The server upon receiving this calls sendResponse() to acknowledge that it received this message and then continues with its logic to make the game advance to the next phase of the game. When this phase advancement finished, both clients gets updates for the new game states via multiple calls of sendResponse(). Player 2 received it fine, but P1 would receive all the messages except the acknowledge message is not the first thing it receives.
I have only seen this type of error when the client would received all theses messages at once. For example, the client has 60secs for a phase, if they sent the request to advance phase, they don't get anything back for about 20-30 secs and then all 5 messages would be received at once.
The server upon receiving this calls sendResponse() to acknowledge that it received this message and then continues with its logic to make the game advance to the next phase of the game.
This is an question unrelated to the problem... but, why sending a response to tell the client that server has received the data?
You seem to be re-creating what the TCP protocol already does for you behind the scenes, unless there's some logic that I am unaware of. In general this sounds unnecessary.
When this phase advancement finished, both clients gets updates for the new game states via multiple calls of sendResponse(). Player 2 received it fine, but P1 would receive all the messages except the acknowledge message is not the first thing it receives.
The problem is to guarantee that messages enter the user queue in the way you want. It's easy if you force the ExtensionHandler to work with a single thread but this could limit scalability.
It's more complex if you run multiple threads.
I have only seen this type of error when the client would received all theses messages at once. For example, the client has 60secs for a phase, if they sent the request to advance phase, they don't get anything back for about 20-30 secs and then all 5 messages would be received at once.
I don't follow here. Why the 20-30 seconds? Is it something you do on purpose or it just happens outside of your control?
This is an question unrelated to the problem... but, why sending a response to tell the client that server has received the data?
You seem to be re-creating what the TCP protocol already does for you behind the scenes, unless there's some logic that I am unaware of. In general this sounds unnecessary.
I am using this to tell both client when to start/restart their game timer.
[/quote]I have only seen this type of error when the client would received all theses messages at once. For example, the client has 60secs for a phase, if they sent the request to advance phase, they don't get anything back for about 20-30 secs and then all 5 messages would be received at once.
I don't follow here. Why the 20-30 seconds? Is it something you do on purpose or it just happens outside of your control?
This is one of the problems I have been seeing. The client would not receive any messages from the server even when there is messages on the server queue for it. It is as if the client socket cannot accept new data even when it is just sitting there waiting for me. When the client is in this state, the only way to fix it seems to be sending another message to the server, I think this does something to the socket and enables further message receiving.
-
VengantMjolnir
- Posts: 3
- Joined: 01 Mar 2010, 22:43
I am working with Gentleman on this one, I'm in charge of the client code.
The issue specifically seems to stem from the fact that the server-client communications are eratic at times. Using trace logs for the server we can verify the time stamps for when messages are sent out, and can verify this with logs from one of the connected clients. However, the other client will sometimes receive a cluster of messages in one big lump.
For example, there could be an action message from the client and a game state update from the server. Client A will receive these messages in the correct order and with up to a second in between, or almost instantly if the server logic is very light. The other client will send its action message and then trace nothing for upwards of 30 seconds. This IS a socket connection and we have verified that the client just previously sent a message and received a response within less than a second. Yet, in this case the next several messages would come in one bunch.
As gentleman has pointed out, quite often the client won't receive a message until after it has sent a message of its own. I've looked into this and found a few issues on Adobe's bug tracking site that 'may' be related. Specifically http://bugs.adobe.com/jira/browse/FP-35 ... -issue-tab
seems like it may be causing an issue. I'd like to hear your thoughts...
To address a few of your questions and to clear up the game flow, I'll comment on the game a bit more. Two clients take turns performing actions within a 'game turn' that is divided into seperate phases. Each phase has a timer that runs do to indicate how much time is available for actions. The active client has a shorter time by 15 seconds. If they take an action or let their timer run out then the server receives an action message( blank in the case of a timeout. ) Meanwhile, the inactive client is now waiting for the active one and is also ticking down its timer. this is an optomization and is done in order to leverage the clients to keep tabs on each other instead of active timers on the server. Once the server receives the communcation from the active client, it sends back updates to both machines and waits for more data. In some cases the response to the clients triggers animations that need to be played. In this case, we use the same technique to synchronize the clients by having them start an animation timer and sending a message to the server indicating when they are done animating.
Here is where one instance of the problem can be seen. Client A finishes animating and sends the message to the server, Client B however doesn't trace out the update from the server in the first place. It keeps its original timer going until it times out and then it sends a message to the server to indicate that it thinks Client A has timed out/lost connection. All of a sudden, any backed up messages related to the animations come in at once with the same timestamps on them, and some times in the wrong order. We have clear logs with time stamps indicating the server sent it in the right order and Client A has a log that backs this up.
We are running 1.6.8 with only two minor modications of the Client API, one is to add a Date object timestamp to the debug message and another is to echo the message to a custom traceOut() function instead of trace( where it is logged and uploaded at game end)
This is not an isolated event and it is not isolated to messages that are only delayed until client responds. In some instances messages just appear to be delayed by up to 30 seconds, all though the average appears to be closer to 15.( our buffer used to be 3 seconds, what we considered plenty with a socket... )
I understand flash's socket object quite well and am very familiar with the source code for SmartFoxClient, feel free to get as technical as you dare in responses.
The issue specifically seems to stem from the fact that the server-client communications are eratic at times. Using trace logs for the server we can verify the time stamps for when messages are sent out, and can verify this with logs from one of the connected clients. However, the other client will sometimes receive a cluster of messages in one big lump.
For example, there could be an action message from the client and a game state update from the server. Client A will receive these messages in the correct order and with up to a second in between, or almost instantly if the server logic is very light. The other client will send its action message and then trace nothing for upwards of 30 seconds. This IS a socket connection and we have verified that the client just previously sent a message and received a response within less than a second. Yet, in this case the next several messages would come in one bunch.
As gentleman has pointed out, quite often the client won't receive a message until after it has sent a message of its own. I've looked into this and found a few issues on Adobe's bug tracking site that 'may' be related. Specifically http://bugs.adobe.com/jira/browse/FP-35 ... -issue-tab
seems like it may be causing an issue. I'd like to hear your thoughts...
To address a few of your questions and to clear up the game flow, I'll comment on the game a bit more. Two clients take turns performing actions within a 'game turn' that is divided into seperate phases. Each phase has a timer that runs do to indicate how much time is available for actions. The active client has a shorter time by 15 seconds. If they take an action or let their timer run out then the server receives an action message( blank in the case of a timeout. ) Meanwhile, the inactive client is now waiting for the active one and is also ticking down its timer. this is an optomization and is done in order to leverage the clients to keep tabs on each other instead of active timers on the server. Once the server receives the communcation from the active client, it sends back updates to both machines and waits for more data. In some cases the response to the clients triggers animations that need to be played. In this case, we use the same technique to synchronize the clients by having them start an animation timer and sending a message to the server indicating when they are done animating.
Here is where one instance of the problem can be seen. Client A finishes animating and sends the message to the server, Client B however doesn't trace out the update from the server in the first place. It keeps its original timer going until it times out and then it sends a message to the server to indicate that it thinks Client A has timed out/lost connection. All of a sudden, any backed up messages related to the animations come in at once with the same timestamps on them, and some times in the wrong order. We have clear logs with time stamps indicating the server sent it in the right order and Client A has a log that backs this up.
We are running 1.6.8 with only two minor modications of the Client API, one is to add a Date object timestamp to the debug message and another is to echo the message to a custom traceOut() function instead of trace( where it is logged and uploaded at game end)
This is not an isolated event and it is not isolated to messages that are only delayed until client responds. In some instances messages just appear to be delayed by up to 30 seconds, all though the average appears to be closer to 15.( our buffer used to be 3 seconds, what we considered plenty with a socket... )
I understand flash's socket object quite well and am very familiar with the source code for SmartFoxClient, feel free to get as technical as you dare in responses.
Return to “Server Side Extension Development”
Who is online
Users browsing this forum: No registered users and 23 guests