Client problems while load testing

Post here all your questions related with SmartFoxServer Java API

Moderators: Bax, Lapo, rjgtav

Client problems while load testing

Postby ers » 09 Dec 2009, 15:53

Hi folks,

We're having some problems while load testing the Smartfox server (PRO 1.6.6) with the Java Client API.

We're generating load from two separate client machines to a single server. Each client tries to create 200 virtual users that perform the following actions:

* Connect
* Login
* Join room
* Walk 10 times (involves a sfs.setUserVariables() and a sfs.sendObject() )
* Logout
* Disconnect

We're having about 110 failures (out of 400) in the join action and the server seems to be fine with no dropped messages.

We're using JMeter with custom samplers to orchestrate the load generation and the client machines are quad cores with 4 GB of RAM (2 GB assigned to the JMeter heap).

We have found some of these in the log files from both client machines:

05:33:54.239 [AnonymousIoService-5] WARN i.g.u.n.x.XMLSocketClientHandler - [/XXX.XXX.XXX.XXX:9339] EXCEPTION, please implement it.gotoandplay.utils.net.xmlsocket.XMLSocketClientHandler.exceptionCaught() for proper handling:
org.apache.mina.filter.codec.ProtocolDecoderException: org.apache.mina.common.BufferDataException: Line is too long: 33564 (Hexdump: 76 61 72 73 3E 3C 2F 76 61 72 73 3E 3C 2F 75 3E 3C 75 20 69 3D 27 31 37 33 27 20 6D 3D 27 30 27 3E 3C 6E 3E 3C 21 5B 43 44 41 54 41 5B 32 37 32 5D 5D 3E 3C 2F 6E 3E 3C 76 61 72 73 3E 3C 2F 76 61 72 73 3E 3C 2F 75 3E 3C 75 20 69 3D 27 31 37 32 27 20 6D 3D 27 30 27 3E 3C 6E 3E 3C 21 5B 43 44 41 54 41 5B 35 32 32 5D 5D 3E 3C 2F 6E 3E 3C 76 61 72 73 3E 3C 2F 76 61 72 73 3E 3C 2F 75 3E 3C 75 20 69 3D 27 31 37 34 27 20 6D 3D 27 30 27 3E 3C 6E 3E 3C 21 5B 43 44 41 54 41 5B 35 32 33 5D 5D 3E 3C 2F 6E 3E 3C 76 61 72 73 3E 3C 2F 76 61 72 73 3E 3C 2F 75 3E 3C 75 20 69 3D 27 31 37 35 27 20 6D 3D 27 30 27 3E 3C 6E 3E 3C 21 5B 43 44 41 54 41 5B 32 37 34 5D 5D 3E 3C 2F 6E 3E 3C 76 61 72 73 3E 3C 2F 76 61 72 73 3E 3C 2F 75 3E 3C 2F 75 4C 73 3E 3C 2F 62 6F 64 79 3E 3C 2F 6D 73 67 3E 00)

and lots of null pointer exceptions:

05:36:09.553 [AnonymousIoService-10] WARN i.g.u.n.x.XMLSocketClientHandler - [/XXX.XXX.XXX.XXX:9339] EXCEPTION, please implement it.gotoandplay.utils.net.xmlsocket.XMLSocketClientHandler.exceptionCaught() for proper handling:
java.lang.NullPointerException
at it.gotoandplay.smartfoxclient.handlers.SysHandler.handleUserLeaveRoom(SysHandler.java:473)

It seems to be a problem with the Smartfox API not being able to process the low level communication. Could you please help us in debugging it?

Thanks a lot in advance,

Esteban
ers
 
Posts: 2
Joined: 09 Sep 2009, 13:55

Postby Lapo » 10 Dec 2009, 09:03

Hi,
it looks like the network layer of the API (based on Apache Mina) is complaining for a packet of data that exceeds its buffer capabilities. It looks like the buffer is limited around 32Kb and that's probably the origin of the problem.

I am not sure if you can re-configure that via code in the API itself. I will give it a look and get back to you
Lapo
--
gotoAndPlay()
...addicted to flash games
User avatar
Lapo
Site Admin
 
Posts: 16219
Joined: 21 Mar 2005, 09:50
Location: Italy

Postby dromos » 16 Jan 2010, 22:43

Strangely enough, I am also in the process of writing a load testing client with the SFS Java API and I believe I just ran into the exact same issue. My client successfully sends and receives dozens of messages but for some reason, when a large message is sent from the server, my client code never receives a notification for it. This has stopped me dead in my tracks and I'm not sure how to proceed.

[EDIT] - I just noticed this message was originally posted over a month ago. Did you get back to him through some other means or...?
dromos
 
Posts: 7
Joined: 07 Oct 2009, 19:45

Postby dromos » 16 Jan 2010, 23:38

For what it's worth, I verified that I am getting the same exception. Here's the stack trace...

EXCEPTION, please implement it.gotoandplay.utils.net.xmlsocket.XMLSocketClientHandler.exceptionCaught() for proper handling:
[java] org.apache.mina.filter.codec.ProtocolDecoderException: org.apache.mina.common.BufferDataException: Line is too long: 49497 (Hexdump: 69 66 69 63 61 74 69 6F 6E 20 74 6F 20 68 69 67 68 20 7A 6F 6F 6D 20 73 6E 69 70 65 72 20 6D 6F 64 65 6C 73 2E 22 2C 22 6E 61 6D 65 22 3A 22 5A 6F 72 69 6E 20 4F 70 74 69 63 73 20 53 63 6F 70 65 22 2C 22 72 61 72 69 74 79 22 3A 31 2C 22 74 68 75 6D 62 55 72 6C 22 3A 22 61 73 73 65 74 73 2F 69 63 6F 6E 73 2F 6C 61 62 2F 77 65 61 70 6F 6E 5F 63 6F 6D 70 6F 6E 65 6E 74 73 2F 6F 70 74 69 63 61 6C 5F 73 63 6F 70 65 2E 70 6E 67 22 2C 22 73 6B 75 22 3A 7B 22 77 65 61 6C 74 68 43 6F 73 74 22 3A 30 2C 22 73 74 61 74 69 6F 6E 43 61 73 68 43 6F 73 74 22 3A 30 2C 22 73 6B 75 49 64 22 3A 30 2C 22 74 79 70 65 22 3A 22 42 4C 49 4E 44 5F 42 4F 58 22 7D 2C 22 75 72 6C 22 3A 22 61 73 73 65 74 73 2F 69 63 6F 6E 73 2F 6C 61 62 2F 77 65 61 70 6F 6E 5F 63 6F 6D 70 6F 6E 65 6E 74 73 2F 6F 70 74 69 63 61 6C 5F 73 63 6F 70 65 2E 70 6E 67 22 7D 5D 2C 22 5F 63 6D 64 22 3A 22 67 65 74 5F 70 6C 61 79 65 72 5F 6C 61 62 22 2C 22 63 6F 6D 70 6F 6E 65 6E 74 5F 6D 61 78 5F 74 69 6D 65 22 3A 38 36 34 30 30 2C 22 73 75 63 63 65 73 73 22 3A 30 7D 7D 7D 00)
[java] at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:165)
[java] at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(AbstractIoFilterChain.java:299)
[java] at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilterChain.java:53)
[java] at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived(AbstractIoFilterChain.java:648)
[java] at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java:220)
[java] at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(ExecutorFilter.java:264)
[java] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
[java] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[java] at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)
[java] at java.lang.Thread.run(Thread.java:619)
[java] Caused by: org.apache.mina.common.BufferDataException: Line is too long: 49497
[java] at org.apache.mina.filter.codec.textline.TextLineDecoder.decodeNormal(TextLineDecoder.java:247)
[java] at org.apache.mina.filter.codec.textline.TextLineDecoder.decode(TextLineDecoder.java:114)
[java] at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:158)
[java] ... 9 more
dromos
 
Posts: 7
Joined: 07 Oct 2009, 19:45

Postby Lapo » 17 Jan 2010, 17:14

Hi,
the socket classes that handle the data traffic in the Java API have a limit of 32KB per message. Normally this is more than sufficient for any application. Unfortunately there is no way from code to change these limits because they are set when the protocol encoder/decoder are created and can't be changed at runtime.

The only way is that we recompile one of the API libraries with a higher limit (let's say 128-256KB or so)

Another way around this is to understand why so much data is transferred in one shot and see if you can reduce it. Maybe this is a very long room list? Or a very large extension request? In the latter case you might switch to a more compact protocol such as JSON if you haven't already done it.
Lapo
--
gotoAndPlay()
...addicted to flash games
User avatar
Lapo
Site Admin
 
Posts: 16219
Joined: 21 Mar 2005, 09:50
Location: Italy

Postby dromos » 18 Jan 2010, 02:59

Lapo wrote:Hi,
the socket classes that handle the data traffic in the Java API have a limit of 32KB per message. Normally this is more than sufficient for any application. Unfortunately there is no way from code to change these limits because they are set when the protocol encoder/decoder are created and can't be changed at runtime.

The only way is that we recompile one of the API libraries with a higher limit (let's say 128-256KB or so)

Another way around this is to understand why so much data is transferred in one shot and see if you can reduce it. Maybe this is a very long room list? Or a very large extension request? In the latter case you might switch to a more compact protocol such as JSON if you haven't already done it.


This is an extension response and we are already using JSON. The Flash client API is handling the message just fine, so this was never an issue until I tried to write a Java-based load tester. The messages are so big because we front-load a lot of data.

[EDIT] I may have misunderstood. Are you suggesting you could recompile the client library to support an increased buffer size? Is this a possibility? Perhaps you could let us override the default through a property passed to the JVM? We are planning on building a lot of games on SFS, and being able to load test them using the Java client API is really important for us.

Thanks.
dromos
 
Posts: 7
Joined: 07 Oct 2009, 19:45

Postby Lapo » 18 Jan 2010, 09:59

[EDIT] I may have misunderstood. Are you suggesting you could recompile the client library to support an increased buffer size? Is this a possibility? Perhaps you could let us override the default through a property passed to the JVM? We are planning on building a lot of games on SFS, and being able to load test them using the Java client API is really important for us.

Yes that's correct.
We are already working on it. If all goes well we'll upload an update by the end of the week. Stay tuned
Lapo
--
gotoAndPlay()
...addicted to flash games
User avatar
Lapo
Site Admin
 
Posts: 16219
Joined: 21 Mar 2005, 09:50
Location: Italy

Postby Lapo » 18 Jan 2010, 10:49

One more thing: please send us an email (click Support > Contact Us in the main menu) and we'll provide an early release that you can test.

Thanks
Lapo
--
gotoAndPlay()
...addicted to flash games
User avatar
Lapo
Site Admin
 
Posts: 16219
Joined: 21 Mar 2005, 09:50
Location: Italy

Postby dromos » 19 Jan 2010, 23:00

Lapo wrote:One more thing: please send us an email (click Support > Contact Us in the main menu) and we'll provide an early release that you can test.

Thanks


That's great, thanks! I have sent the support e-mail as requested.
dromos
 
Posts: 7
Joined: 07 Oct 2009, 19:45

Postby jsanmarco » 01 Oct 2010, 01:18

Hi,
has this been solved? we are getting the same error.

Regards,
Juan
jsanmarco
 
Posts: 1
Joined: 01 Oct 2010, 01:16

Postby Lapo » 01 Oct 2010, 06:40

Yes the problem was solved with the latest Java API version released several months ago.
We added two external settings that can be passed to the JVM to specify the message buffer size (incoming/outgoing).

Code: Select all
sfs.client.inDataMaxSize=XXX
sfs.client.outDataMaxSize=XXX


Example:
Code: Select all
java -jar <yourapp> -Dsfs.client.inDataMaxSize=64000 -Dsfs.client.outDataMaxSize=64000

The default is 32000 (32Kb)

Download: http://smartfoxserver.com/labs/API/
Lapo
--
gotoAndPlay()
...addicted to flash games
User avatar
Lapo
Site Admin
 
Posts: 16219
Joined: 21 Mar 2005, 09:50
Location: Italy

Re: Client problems while load testing

Postby GrinReaper » 15 Nov 2013, 12:16

Hi All,

I am getting same error, I tried the solution provided above but its not working, I am getting following error

WARN [AnonymousIoService-4] (SessionLog.java91) - [/127.0.0.1:9339] EXCEPTION, please implement it.gotoandplay.utils.net.xmlsocket.XMLSocketClientHandler.exceptionCaught() for proper handling:
org.apache.mina.filter.codec.ProtocolDecoderException: org.apache.mina.common.BufferDataException: Line is too long: 33455 (Hexdump: 76 61 72 20 6E 3D 27 76 69 70 4C 65 76 65 6C 27 20 74 3D 27 73 27 3E 3C 21 5B 43 44 41 54 41 5B 41 4C 4C 5D 5D 3E 3C 2F 76 61 72 3E 3C 76 61 72 20 6E 3D 27 6E 6F 4F 66 50 6C 61 79 65 72 73 27 20 74 3D 27 6E 27 3E 3C 21 5B 43 44 41 54 41 5B 30 5D 5D 3E 3C 2F 76 61 72 3E 3C 76 61 72 20 6E 3D 27 63 61 73 68 4F 72 27 73 27 3E 3C 21 5B 43 44 41 54 41 5B 4E 6F 72 6D 61 6C 5D 5D 3E 3C 2F 76 61 72 3E 3C 2F 76 61 72 73 3E 3C B 41 43 54 49 56 45 5D 5D 3E 3C 2F 76 61 72 3E 3C 76 61 72 20 6E 3D 27 72 6F 6F 6D 49 64 27 20 74 3D 27 73 27 3E 3E 3C 76 61 72 20 6E 3D 27 77 69 6E 6E 69 6E 67 27 20 74 3D 27 73 27 3E 3C 21 5B 43 44 41 54 41 5B 2D 5D 5D 3E 3C 2F 76 61 72 3E 3C 76 61 72 20 6E 3D 27 70 6F 69 6E 74 56 61 6C 75 65 27 20 74 3D 27 6E 27 3E 3C 21 5B 43 44 41 54 41 5B 31 30 30 2E 30 5D 5D 3E 3C 2F 76 61 72 3E 3C 76 61 72 20 6E 3D 27 73 65 61 74 65 64 50 6C 73 27 20 74 3D 27 73 27 3E 3C 21 5B 43 44 41 54 41 5B 5D 5D 3E 3C 2F 76 61 72 3E 3C 76 61 72 20 6E 3D 27 62 65 74 41 6D 74 27 20 74 3D 27 6E 27 3E 3C 21 5B 43 44 41 54 41 5B 38 30 30 30 2E 30 5D 5D 3E 3C 2F 76 61 72 3E 3C 76 61 72 20 6E 3D 27 74 75 72 6E 54 79 70 65 27 20 74 3D 27 73 27 3E 3C 21 5B 43 44 41 54 41 5B 4E 6F 72 6D 61 6C 5D 5D 3E 3C 2F 76 61 72 3E 3C 2F 76 61 72 73 3E 3C 2F 72 6D 3E 3C 2F 72 6D 4C 69 73 74 3E 3C 2F 62 6F 64 79 3E 3C 2F 6D 73 67 3E 00)
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:165)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(AbstractIoFilterChain.java:299)
at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilterChain.java:53)
at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived(AbstractIoFilterChain.java:648)
at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java:220)
at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(ExecutorFilter.java:264)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.mina.common.BufferDataException: Line is too long: 33455
at org.apache.mina.filter.codec.textline.TextLineDecoder.decodeNormal(TextLineDecoder.java:247)
at org.apache.mina.filter.codec.textline.TextLineDecoder.decode(TextLineDecoder.java:114)
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:158)
... 9 more

Please help..
GrinReaper
 
Posts: 12
Joined: 06 Aug 2012, 09:48

Re: Client problems while load testing

Postby Lapo » 15 Nov 2013, 12:36

The solution provided above has solved the issue for all our customer. It should work for you as well.
Lapo
--
gotoAndPlay()
...addicted to flash games
User avatar
Lapo
Site Admin
 
Posts: 16219
Joined: 21 Mar 2005, 09:50
Location: Italy

Re: Client problems while load testing

Postby GrinReaper » 18 Nov 2013, 10:56

I tried by creating jar file using following java program as a entry point for jar in MANIFEST.MF file.


import it.gotoandplay.smartfoxclient.ISFSEventListener;
import it.gotoandplay.smartfoxclient.SFSEvent;
import it.gotoandplay.smartfoxclient.SmartFoxClient;
import it.gotoandplay.smartfoxclient.data.Room;

import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Set;

public class ServerCommunication implements ISFSEventListener{

private SmartFoxClient sfs;

public ServerCommunication(){

sfs = new SmartFoxClient(false);

sfs.addEventListener(SFSEvent.onConfigLoadFailure, this);
sfs.addEventListener(SFSEvent.onConfigLoadSuccess, this);
sfs.addEventListener(SFSEvent.onConnection, this);
sfs.addEventListener(SFSEvent.onConnectionLost, this);
sfs.addEventListener(SFSEvent.onCreateRoomError, this);
sfs.addEventListener(SFSEvent.onExtensionResponse, this);
sfs.addEventListener(SFSEvent.onJoinRoom, this);
sfs.addEventListener(SFSEvent.onJoinRoomError, this);
sfs.addEventListener(SFSEvent.onLogin, this);
sfs.addEventListener(SFSEvent.onLogout, this);
sfs.addEventListener(SFSEvent.onPlayerSwitched, this);
sfs.addEventListener(SFSEvent.onRoomAdded, this);
sfs.addEventListener(SFSEvent.onRoomDeleted, this);
sfs.addEventListener(SFSEvent.onRoomLeft, this);
sfs.addEventListener(SFSEvent.onRoomListUpdate, this);
sfs.addEventListener(SFSEvent.onRoomVariablesUpdate, this);
sfs.addEventListener(SFSEvent.onUserCountChange, this);
sfs.addEventListener(SFSEvent.onUserEnterRoom, this);
sfs.addEventListener(SFSEvent.onUserLeaveRoom, this);
sfs.addEventListener(SFSEvent.onUserVariablesUpdate, this);

sfs.loadConfig("/home/user/Desktop/config.xml");
}


@SuppressWarnings("unchecked")
public void handleEvent(SFSEvent event) {
String eventName = event.getName();

try{

if (SFSEvent.onConfigLoadFailure.equals(eventName)) {

}else if(SFSEvent.onConfigLoadSuccess.equals(eventName)){

}else if(SFSEvent.onConnection.equals(eventName)){

if (event.getParams().getBool("success")) {
//Zone name - Poker
sfs.login("Poker", "player", "");
}

}else if(SFSEvent.onConnectionLost.equals(eventName)){

}else if(SFSEvent.onCreateRoomError.equals(eventName)){

}else if(SFSEvent.onJoinRoom.equals(eventName)){

Room room = (Room)event.getParams().get("room");

}else if(SFSEvent.onJoinRoomError.equals(eventName)){

}else if(SFSEvent.onLogin.equals(eventName)){

}else if(SFSEvent.onLogout.equals(eventName)){

}else if(SFSEvent.onPlayerSwitched.equals(eventName)){

}else if(SFSEvent.onRoomAdded.equals(eventName)){

}else if(SFSEvent.onRoomDeleted.equals(eventName)){

}else if(SFSEvent.onRoomLeft.equals(eventName)){

}else if(SFSEvent.onRoomListUpdate.equals(eventName)){

System.out.println((Map<Integer, Room>)event.getParams().get("roomList"));

}else if(SFSEvent.onRoomVariablesUpdate.equals(eventName)){
//System.out.println(event.getParams());

}else if(SFSEvent.onUserCountChange.equals(eventName)){

}else if(SFSEvent.onUserEnterRoom.equals(eventName)){

}else if(SFSEvent.onUserLeaveRoom.equals(eventName)){

System.out.println(event.getParams());

}else if(SFSEvent.onUserVariablesUpdate.equals(eventName)){

}else if(SFSEvent.onExtensionResponse.equals(eventName)){

}
}catch(Exception e){
e.printStackTrace();
}
}


public static void main(String[] args) {
new ServerCommunication();
}
}

And after that I rum this jar as follows:
java -jar myEng.jar -Dsfs.client.inDataMaxSize=6400000 -Dsfs.client.outDataMaxSize=6400000

but i got following exception at client side..


org.apache.mina.filter.codec.ProtocolDecoderException: org.apache.mina.common.BufferDataException: Line is too long: 33455 (Hexdump: 76 61 72 2 73 67 ............ 3E 00)
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:165)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(AbstractIoFilterChain.java:299)
at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilterChain.java:53)
at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived(AbstractIoFilterChain.java:648)
at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java:220)
at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(ExecutorFilter.java:264)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.mina.common.BufferDataException: Line is too long: 33455
at org.apache.mina.filter.codec.textline.TextLineDecoder.decodeNormal(TextLineDecoder.java:247)
at org.apache.mina.filter.codec.textline.TextLineDecoder.decode(TextLineDecoder.java:114)
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:158)
... 9 more

Please help..
GrinReaper
 
Posts: 12
Joined: 06 Aug 2012, 09:48

Re: Client problems while load testing

Postby Lapo » 18 Nov 2013, 16:20

My suspect is that you are using an older version of the Java API that did not support those runtime settings.
Please make sure to download the latest release, or at least make sure that your current version matches the latest available.

The latest is version 1.5.8.
http://www.smartfoxserver.com/download/sfsPro#p=updates

You can check the version of your current API by printing the SmartFoxClient.getVersion(); method.

Thanks
Lapo
--
gotoAndPlay()
...addicted to flash games
User avatar
Lapo
Site Admin
 
Posts: 16219
Joined: 21 Mar 2005, 09:50
Location: Italy


Return to Java2SE / Android Client API

Who is online

Users browsing this forum: No registered users and 2 guests