Java vs. AS

[Kicksome]

The performance of my SFS application in Actionscript is not good. At about 250 users I start getting up to 70% CPU spikes on a quad core machine. Since I need to support at least 1000 users - that's not good.

So I'm trying to figure out where I need to start moving my code to Java. I basically have a few different areas:

1) (Zone extension) Very large structures in action script that contain player and monster information. Probably 40 fields for each player. I have an object for each player. I have at least 5000 players in this structure. I have to loop through this object to create maps that send the player information to the clients depending on what map they are on. I only send the client limited data - but the looping seems pretty process intensive.

i.e. when I load the data when the user logs in I do...

// var player = {} in global
player[id] = {}
player[id].name = whatever
player[id].etc = whatever x 40

The loops is something like this

Code: Select all

for (var i in player)
{   
    var mapid = player[i].mapid
            if (mapsend[mapid] == undefined) {
               mapsend[mapid] = {}
               //loadallplayers
               mapsend[mapid].mapstr=player[i].x+"|"+player[i].y+"|"+player[i].playername+" ("+ player[i].explevel+")"+"|"+player[i].picnum+"|"+i+"|N;"
            } else {
               mapsend[mapid].mapstr=mapsend[mapid].mapstr+player[i].x+"|"+player[i].y+"|"+player[i].playername+" ("+ player[i].explevel+")"+"|"+player[i].picnum+"|"+i+"|N;"
            }               
}


once that loop is done I loop through the place structure again and send the updated map out to people who are logged in to that map.




2) Combat (own room). Access the player struct and does various hit calculations and deducts hitpoints from a player. Sends the results to an individual in battle.

so any idea if this would be beneficial to move to java? I so - any idea how much? It looks I need a 4x improvement or I need to look into some other options.

[thup]

I've only even worked with Java extensions, so all I can say is performance has been good with Java as long as you know what you're doing.

That said, it sounds like you are dealing with a lot of data, and your problem might be related to the way you have it organized rather than the platform you are using.

You didn't provide a lot of info, but assuming we're talking RPG stuff here (player data, NPC data, map data), I don't see who you'd have to be traversing big loops at all except perhaps when a user first enters a map.

They first enter the map, you send them one chunky update with all the map data. After that, they should only be getting tiny updates, right? Why would you need to loop through the whole map for that? An NPC moves, you tell the player that NPC moved, right? A player moves, you tell the other players that player moved. Etc. I don't see a reason to be doing big loops anywhere in that process. Probably missing something though.

Anyway, I suspect Java is generally much faster dealing with loops and iteration, since it has so many type specific structures for addressing particular problems. That said, 4x increase sounds like a tall order.

[Kicksome]

Once the map data is load - I just send player positions on the map every second or so (I believe this is more cpu effective than keeping track of each individual that has moved and then figuring out who to send the move update to - especially when there is a lot of movement. With 100+ people on a single map that's a lot of calculations ). I actually cut down the data I send by about 75% compared to the way it was done before (using http polling and coldfusion).

[Lapo]

At about 250 users I start getting up to 70% CPU spikes on a quad core machine.

Couple of questions:
1. 70% is the spike value, but what is the average?
2. It depends if we talk about 70% of all 4 CPUs or 70% of 1 CPU
For example under Linux the default hardware monitor shows individual percentages of each CPU so 70% in that case would mean 70 out of 400...
Under Windows this is probably different and you get a global value.

Suggestions:
1- The spikes may indicate that the garbage collector is working a lot, which in turn means that you're running short of heap memory. Increasing the heap memory might help.

2- Are you running many timers/threads?

Java extensions can be produce a 5x to 20x performance increase... depending on how "intense" is your server side code.

[Lapo]

Once the map data is load - I just send player positions on the map every second or so (I believe this is more cpu effective than keeping track of each individual that has moved and then figuring out who to send the move update to - especially when there is a lot of movement. With 100+ people on a single map that's a lot of calculations ). I actually cut down the data I send by about 75% compared to the way it was done before (using http polling and coldfusion).

It's not clear... you're sending every second all the player positions to every one regardless they have moved or not?

[Kicksome]

As far as CPU. It's averaging well above 50% on all 4 processors (e.g. 40%, 60%, 35%, 75% across the processors). When I look at task manager, all 4 CPUs are about the same amount utilized. So 50% average across each CPU - up to 70% spikes. At that point it was starting to show lag on the client.

As far as the player positions. We have about 50 different maps.

If you are on map #100. I send you the positions of the other VISABLE (a player doesn't show up if he is idle for over 60 seconds) players and monsters on that map every 1 second (all positions on that specific map regardless of movement). It's in STR format. About 1K raw data per string sent to each player. My network connection is only about 7% used with about 250 people on SFS (this includes chat, loading maps, combat etc...). Which is a big improvement from the http polling.

So it loops through maybe 5000 people in the player object (most are monsters). if it finds an active player and it send them the latest player positions of the map they are on. Flash then moves the people around according to the data.

So the process goes like so:

1) One single loop through all players and monsters (about 5000)
a) in that loop create a map string that is simple positions for each person/monster on each specific map
b) it will end up creating 50 different map strings - one for each map (i.e. map[id].str)

2) Another single loop through all players online and send them the map string for the map they are on.


As far as memory. I have 1 gig set aside for SFS. It was using about 350 megs max.

[Kicksome]

On a side note. I had set this in the config:

<OutQueueThreads>200</OutQueueThreads>
<ExtHandlerThreads>200</ExtHandlerThreads>

Would this have cause any type of performance problem? I just set it back to 2 each.

Additionally, when you go into combat - it creates a room with a interval set. It removes the interval though on destroy or when a battle ends. You could have 100 people in battle at a time in different rooms.

[darnpunk]

Hi Kicksome,

I may not be in a place to say this but correct me if I am wrong that you're using one interval for each battle room? If my guess is right, then that would affect the performance somehow. I used to attach one interval to each of my game room and somehow one of them got leaked and it caused a huge problem. Imagine 6000+ threads shows up in the admin tool

If you are using one interval for each game room, I suggest you start planning on a manager to handle all the timers for you. This again if I am right, each interval runs on its own thread. There were some posts on why there shouldn't be too many intervals going on. I re-coded my game extensions and have one interval working across all battle rooms which updates the time correctly and concurrently. It's working so much better now.

There's a thread regarding a ClockManager if I am not mistaken. Try searching for it. And after all that, if you're not using a new interval for each room, just ignore me All the best!

Regards,
darnpunk

[Kicksome]

I'm using an interval for each battle room. BUT I'm been using them and it works great for 100+ battles - no performance problems at all - even with 500 users logged in it's like 10% - 15% CPU time. This has worked great for the past 2 weeks.

It's when I added the maps stuff that (and changed the battle code a bit to pull from the database vs. loadvars) that I'm starting to have problems with performance.

[Lapo]

<OutQueueThreads>200</OutQueueThreads>
<ExtHandlerThreads>200</ExtHandlerThreads>



Please follow our recommendations in the advanced configuration chapter. This way you're just wasting a lot of CPU cycles.
We already designed the threading module so that it uses the smallest amount of threads ... thread creation and context switching are pretty expensive for the JVM

OutQueueThreads should be between 2 and 4
ExtHandlerThread could be any value between 1 and 4

[Lapo]

About using a thread for each game... it's okay even if there are better ways. In SmartFoxServer 1.6 we have added a super-optimized scheduler tha will help you in handling timed tasks saving tons of cpu time.

1) One single loop through all players and monsters (about 5000)
a) in that loop create a map string that is simple positions for each person/monster on each specific map
b) it will end up creating 50 different map strings - one for each map (i.e. map[id].str)

2) Another single loop through all players online and send them the map string for the map they are on.

This is one of those typical heavy-weight piece of code where Java could bring monster improvements

[Lapo]

Just to give you an idea I've setup a quick benchmark where 5000 game objects are looped and a string is built with a few properties coming from each object.

Actionscript = around 2000ms.
Java = around 7ms.

You see the difference! It's more than 200x

If we raise the number of objects from 5000 to 20.000

Actionscript = around 30.000ms. (30 seconds)
Java = around 16ms.

[Lapo]

This is the Actionscript code used in the test

Code: Select all

var getTimer = Packages.java.lang.System.currentTimeMillis

var objs = []
var MAX = 5000

function prepareData()
{
   for (var i = 0; i < MAX; i++)
   {
      var o = {}
      o.posx = Math.random() * 1000
      o.posy = Math.random() * 1000
      o.name = "obj__" + i
      o.rank = "rank__" + i
      o.color = Math.random() * 255
      o.type = "type__" + i
      o.active = Math.random() * 100
      objs.push(o)
   }
}

function mainLoop()
{
   var response = ""
   
   for (var i = 0; i < MAX; i++)
   {
      var o = objs[i]
      var s = ""
      
      if (o.active > 49)
      {
         response += "|" + o.name + "," + o.posx + "," + o.posy + "|"
      }
   }
}

prepareData()
var t1 = getTimer()
mainLoop()
var t2 = getTimer()

print ("Done in " + (t2-t1) + " ms.")

[Lapo]

This is the Java code used for the test:

Code: Select all

import java.util.ArrayList;
import java.util.Random;

public class LoopBench
{   
   public class GameObject
   {
      public int posx, posy, color, active;
      public String name, rank, type;
      
      public GameObject()
      {
         //
      }
   }
   
   Random rnd;
   ArrayList<GameObject> gameObjects;
   int MAX = 5000;
   long t1,t2;
   
   LoopBench ()
   {
      rnd = new Random();
      gameObjects = new ArrayList<GameObject>();
      
      populateData();
      
      t1 = System.currentTimeMillis();
      mainLoop();
      t2 = System.currentTimeMillis();
      
      System.out.println("Done in " + (t2-t1) + "ms.");
      
   }
   
   void populateData()
   {
      for (int i = 0; i < MAX; i++)
      {
         GameObject go = new GameObject();
         
         go.posx = rnd.nextInt(1000);
         go.posy = rnd.nextInt(1000);
         go.color = rnd.nextInt(255);
         go.name = "name__" + i;
         go.rank = "rank__" + i;
         go.type = "type__" + i;
         
         go.active = rnd.nextInt(100);
         
         gameObjects.add( go );
      }
   }
   
   void mainLoop()
   {
      StringBuilder response = new StringBuilder();
      
      for (GameObject go : gameObjects)
      {
         if (go.active > 49)
         {
            response.append("|")
            .append(go.name)
            .append(",")
            .append(go.posx)
            .append(",")
            .append(go.posy)
            .append("|");
         }
      }
      
   }
   
   public static void main(String[] args)
   {
      new LoopBench();
   }
}

[Lapo]

Besides the huge performance gap, the java code is super-optimized for the most critical part of this code, string concatenation

I believe that this in Actionscript:

Code: Select all

myString += "someText"

is pretty slow, probably slower than its equivalent in Java and if you put this type of slow code inside a long loop you will end up with not-so-good performance.

In fact by modifying the Actionscript object to use a Java StringBuilder object the performance immediately skyrockets to less than 200ms which is a good 10x!