logging with threads

[bacuri]

Hi,

We need to submit data to a remote script for logging, we're currently using urllib2.openurl()

Because it's a time expensive operation, we are using the following model in our room extension:

1) we accumulate log events into a python list

2) Periodically ( say, every 120 seconds ) we spawn a thread ( subclass of Runnable ) to submit the accumulated data and we empty the list.

However we are concerned whether the above approach is good enough for a scenario of potentially a few hundred rooms.


In your previous posts about threads you strongly emphasize the need to avoid thread switching overhead.

In a scenario of a few hundred rooms where each spawned thread takes up to 5 seconds to process and one is spawned evry 120 seconds, it's not improbable that there could be a few dozens of these threads running simultaneously.

Is this a scenario we should be trying to avoid, considering it to run on a dual-core server?
Please advice!


Thank you.

[Lapo]

In a scenario of a few hundred rooms where each spawned thread takes up to 5 seconds to process and one is spawned evry 120 seconds, it's not improbable that there could be a few dozens of these threads running simultaneously.

Is this a scenario we should be trying to avoid, considering it to run on a dual-core server?
Please advice!

There is no problem. A few dozens of threads won't be a problem for a dual core machine.