strange network peaks

Post here your questions about SFS2X. Here we discuss all server-side matters. For client API questions see the dedicated forums.

Moderators: Lapo, Bax

Tatanan
Posts: 112
Joined: 07 Jan 2014, 12:12
Contact:

strange network peaks

Postby Tatanan » 19 May 2017, 18:59

Hi,

Suddenly we register strange network peaks the makes user not receive messages or they receive the messages after many many seconds, and the extension queue grows to strange values. It seems that during some time the server can not send any message.
Any idea? How can I monitor if an extension handler is slow?
I Attach some graphs from the Admin Tool.
screenshot network.png
(46.42 KiB) Not downloaded yet

screenshot queue.png
(52.46 KiB) Not downloaded yet

screenshot cpu.png
(40.7 KiB) Not downloaded yet
User avatar
Lapo
Site Admin
Posts: 22999
Joined: 21 Mar 2005, 09:50
Location: Italy

Re: strange network peaks

Postby Lapo » 20 May 2017, 09:37

Hi,
one instance is not enough to understand if this is a real issue or a temporary "hiccup".
The spike might have been anything, from a garbage collection cycle to something external to the JVM, such as another process suddenly reducing the available CPU and slowing everything down.

Does this happen at regular times? Or is it dependent on server load?

thanks
Lapo
--
gotoAndPlay()
...addicted to flash games
Tatanan
Posts: 112
Joined: 07 Jan 2014, 12:12
Contact:

Re: strange network peaks

Postby Tatanan » 31 May 2017, 08:52

Solved. It was a problem with the database.

Thanks you anyway and
sorry for the inconvenience
User avatar
Lapo
Site Admin
Posts: 22999
Joined: 21 Mar 2005, 09:50
Location: Italy

Re: strange network peaks

Postby Lapo » 31 May 2017, 13:45

No problem, glad you find out what caused it. :)

cheers
Lapo

--

gotoAndPlay()

...addicted to flash games
User avatar
gabrielvpy
Posts: 69
Joined: 23 Jul 2015, 20:18

Re: strange network peaks

Postby gabrielvpy » 12 Apr 2018, 14:34

I'm having the same issue. I managed to find most of the DB processes that were causing this issue and solved them. But I'm still getting some peaks. It was kind of easy to find the issues at first because all the workers (SFSWorker:Ext:1, 2, 3 and 4) stoped working if one of them was waiting for de DB to respond.

But now is different, I get peaks but there is no worker waiting for the DB to respond. Is there a another way to monitor this. What else could be causing this issue if it's not the DB?
We made an awesome GPS - MOBA.
Try it out here: Get Fhacktions
User avatar
Lapo
Site Admin
Posts: 22999
Joined: 21 Mar 2005, 09:50
Location: Italy

Re: strange network peaks

Postby Lapo » 12 Apr 2018, 15:09

What peaks do you mean exactly?
Peaks in the usage of the CPU? Can you show me a graph taken from the admin tool?

Thanks
Lapo

--

gotoAndPlay()

...addicted to flash games
User avatar
gabrielvpy
Posts: 69
Joined: 23 Jul 2015, 20:18

Re: strange network peaks

Postby gabrielvpy » 12 Apr 2018, 15:18

t like the first image Tatanan posted in the first post of this thread. It's a network peak.
We made an awesome GPS - MOBA.
Try it out here: Get Fhacktions
User avatar
Lapo
Site Admin
Posts: 22999
Joined: 21 Mar 2005, 09:50
Location: Italy

Re: strange network peaks

Postby Lapo » 12 Apr 2018, 15:38

Network peaks can be normal, depending on how frequent and wide they are.
Also they can depend on what's going on in the server's queues.

If you have screenshot we can take a look. Would be cool to actually see the screenshots of both views in the Dashboard (network and queues)

Thanks
Lapo

--

gotoAndPlay()

...addicted to flash games
User avatar
gabrielvpy
Posts: 69
Joined: 23 Jul 2015, 20:18

Re: strange network peaks

Postby gabrielvpy » 12 Apr 2018, 19:12

Here's a screenshot that we just took.

Image
Image
We made an awesome GPS - MOBA.
Try it out here: Get Fhacktions
User avatar
Lapo
Site Admin
Posts: 22999
Joined: 21 Mar 2005, 09:50
Location: Italy

Re: strange network peaks

Postby Lapo » 13 Apr 2018, 07:25

Thanks,
it's difficult to speculate here. One isolated spike in 10 minutes is not sign of anything bad, unless this happens often and with similar magnitude. There could be all kinds of reasons for that occurrence.

The fact that both the green and red lines overlap indicates that the server received a sudden "big input" (i.e. many requests) and immediately sent the relative responses (hence the overlap) which in turn suggests the server is doing its job correctly. Also the CPU does not show a higher use correlated with the spike, so everything looks fine from the SFS side.

It looks very much like the spike was generated from outside the server.
Lapo

--

gotoAndPlay()

...addicted to flash games
User avatar
gabrielvpy
Posts: 69
Joined: 23 Jul 2015, 20:18

Re: strange network peaks

Postby gabrielvpy » 04 Jul 2018, 14:02

Hi, Lapo.

I have a new question regarding this situation. Yesterday morning we were having an unusual amount of spikes and not that many players were playing at that moment. Also, we realized we were having internet issues at the office. So a question roused, could it be that the sampling in the dashboard is being done locally (our office in Paraguay) and the fact that we have an awful internet connection is what is causing these spikes in our dashboard but not on the server (AWS in Virginia).

So I went back home and opened a dashboard there and came back to the office and started monitoring from the office and from my place. Then I noticed that we still get somes spikes here in the office's dashboard but not in my place's dashboard.

My conclusion is that the sampling is done locally and that's why in my house (1 computer using a 10Mbps connection) has no spikes, unlike at the office (10 computers using a 10Mbps connection) where we do get spikes.

Is that the case?
We made an awesome GPS - MOBA.
Try it out here: Get Fhacktions
User avatar
Lapo
Site Admin
Posts: 22999
Joined: 21 Mar 2005, 09:50
Location: Italy

Re: strange network peaks

Postby Lapo » 04 Jul 2018, 16:11

gabrielvpy wrote:Hi, Lapo.

I have a new question regarding this situation. Yesterday morning we were having an unusual amount of spikes and not that many players were playing at that moment. Also, we realized we were having internet issues at the office. So a question roused, could it be that the sampling in the dashboard is being done locally (our office in Paraguay) and the fact that we have an awful internet connection is what is causing these spikes in our dashboard but not on the server (AWS in Virginia).

When you say "spikes" what you are you referring to exactly? Which parameter(s) in the server?
Generally speaking, the AdminTool simply put into graphs samples calculated by the server, based on server side parameters.

So I went back home and opened a dashboard there and came back to the office and started monitoring from the office and from my place. Then I noticed that we still get somes spikes here in the office's dashboard but not in my place's dashboard.

My conclusion is that the sampling is done locally and that's why in my house (1 computer using a 10Mbps connection) has no spikes, unlike at the office (10 computers using a 10Mbps connection) where we do get spikes.

No, there is no sampling going on in the AdminTool. The Admin just renders graphs from samples coming from the server side.

Cheers
Lapo

--

gotoAndPlay()

...addicted to flash games
User avatar
gabrielvpy
Posts: 69
Joined: 23 Jul 2015, 20:18

Re: strange network peaks

Postby gabrielvpy » 04 Jul 2018, 17:17

When I say spikes I mean spikes in outgoing and incoming data. Like the ones in the last image I posted.
We made an awesome GPS - MOBA.
Try it out here: Get Fhacktions
User avatar
Lapo
Site Admin
Posts: 22999
Joined: 21 Mar 2005, 09:50
Location: Italy

Re: strange network peaks

Postby Lapo » 05 Jul 2018, 07:10

gabrielvpy wrote:When I say spikes I mean spikes in outgoing and incoming data. Like the ones in the last image I posted.

The AdminTool adds a little bit of extra in/out traffic to the server but in the order of Kilobytes, certainly not Megabytes.

As I mentioned earlier a traffic spike (even a large one) does not represent any particular problem. It can happen for example if the garbage collector pauses the JVM for a moment (can be few ms. or hundreds of them, depending on the load). When this happens data that should be sent out is held into a queue and sent when the GC is done. This can result in a spike whose size depends on the current load of the server.

Cheers
Lapo

--

gotoAndPlay()

...addicted to flash games
User avatar
gabrielvpy
Posts: 69
Joined: 23 Jul 2015, 20:18

Re: strange network peaks

Postby gabrielvpy » 05 Jul 2018, 12:35

I don't think that's the case. As I mentioned, I'm monitoring our server from 2 seperate locations. In my house there is only 1 machine online and I see no spikes there. But in the office where we have at least 10 machines online and the same bandwith as my house we get the spikes. So our conclusion is that we are seeing this spikes because too many machines are compiting for the intenet connection.
We made another experiment. We started monitoring another server (from the office), one that has no traffic at all, and we got spikes at the same time we were having spikes in our busy server. We think that proves our theory wich is, our internet conecction is what is causing the spikes and the server is running smoothly.
I just wanted to add that in case anyone else runs into this situation.
We made an awesome GPS - MOBA.
Try it out here: Get Fhacktions

Return to “SFS2X Questions”

Who is online

Users browsing this forum: Bing [Bot], Seo-Ul-Kex and 40 guests