Recording asynchronous request/response times

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Recording asynchronous request/response times

nmodin
Hi all,

I need to record the time between requests and responses for a web socker server I have and can't really figure out how to achieve this. I have a small java client that does the sending and receiving of web socket messages, so i can record the response time for individual messages as the sub protocol has message ids embedded in the message body.

Consider the following sequence (click link):
Async request/response sequence diagram

Is there any way I can do something similar to this, I.e measure the latency I get myself (in script or in Java helper) and report this explicitly to grinder ?

Cheers,
Niklas
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Recording asynchronous request/response times

Gary Mulder-3
Hi Niklas,

On 26 May 2015 at 08:16, nmodin <[hidden email]> wrote:
Hi all,

I need to record the time between requests and responses for a web socker
server I have and can't really figure out how to achieve this. I have a
 
The Grinder is more designed for pull testing than the push testing required for web sockets. If you want to test the horizontal scalability of your web sockets you need to get a lot of threads subscribed to your web socket and then wait for messages. Depending on the number of sockets, you may find that if you send a lot of messages (> 50 per Grinder process) simultaneously to your test harness then all the test threads will need to wake up at the same time, which causes thread contention and possible GC in your test harness. See http://grinder.sourceforge.net/g3/garbage-collection.html for details on how to monitor and tune GC. 

If you want to test latency and not scalability of your web sockets, you could import your small java client into The Grinder as a callable class and generate time stamps as part of the web socket payload you are injecting. On the receiving thread look at the time stamp to calculate how long it took from send to receive. You can use a custom statistic to log the latency (http://stackoverflow.com/questions/23452976/how-to-add-custom-statistics-in-grinder).

Of course you probably want to know the latency of your web sockets when you have your peak web socket connections. In the end you may spend a lot of time confirming your test harness is not the bottleneck. One solution to this is to use a lot of agents, e.g. run them in the cloud.

Regards,
Gary


------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Recording asynchronous request/response times

nmodin
Hi Gary,

Thanks for the quick response.

I guess we are in that unfortunate place where we do want to see what happens to latency (and stability) under peak horizontal loads. In this case about 500K open web sockets on the server side, so I need to reach about 50K concurrent web sockets per grinder agent server spread over 10 machines, and then
check the latency for request/response messages sent over those web sockets.

Now if we forget about reality (GC, number of machines etc), this seems to be what I need to do if I read the docs correctly:


Am I going in the right direction here or did I miss the target completely ?


Cheers,
Niklas


On Tue, May 26, 2015 at 11:56 AM, Gary Mulder-3 [via Grinder] <[hidden email]> wrote:
Hi Niklas,

On 26 May 2015 at 08:16, nmodin <[hidden email]> wrote:
Hi all,

I need to record the time between requests and responses for a web socker
server I have and can't really figure out how to achieve this. I have a
 
The Grinder is more designed for pull testing than the push testing required for web sockets. If you want to test the horizontal scalability of your web sockets you need to get a lot of threads subscribed to your web socket and then wait for messages. Depending on the number of sockets, you may find that if you send a lot of messages (> 50 per Grinder process) simultaneously to your test harness then all the test threads will need to wake up at the same time, which causes thread contention and possible GC in your test harness. See http://grinder.sourceforge.net/g3/garbage-collection.html for details on how to monitor and tune GC. 

If you want to test latency and not scalability of your web sockets, you could import your small java client into The Grinder as a callable class and generate time stamps as part of the web socket payload you are injecting. On the receiving thread look at the time stamp to calculate how long it took from send to receive. You can use a custom statistic to log the latency (http://stackoverflow.com/questions/23452976/how-to-add-custom-statistics-in-grinder).

Of course you probably want to know the latency of your web sockets when you have your peak web socket connections. In the end you may spend a lot of time confirming your test harness is not the bottleneck. One solution to this is to use a lot of agents, e.g. run them in the cloud.

Regards,
Gary


------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use



If you reply to this email, your message will be added to the discussion below:
http://grinder.996249.n3.nabble.com/Recording-asynchronous-request-response-times-tp9078p9079.html
To unsubscribe from Recording asynchronous request/response times, click here.
NAML

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Recording asynchronous request/response times

Gary Mulder-3
On 26 May 2015 at 12:34, nmodin <[hidden email]> wrote:

Now if we forget about reality (GC, number of machines etc), this seems to be what I need to do if I read the docs correctly:


Am I going in the right direction here or did I miss the target completely ?

That looks about right. You may want to also put a sequence number in the payload to uniquely identify each message if you want to also confirm that every message is received.

Also, you may want to carefully consider edge cases, i.e. when everyone is subscribed to the same message stream (one to many) as well as when everyone is subscribed to different messages streams (many to many). In the first case you may find that if the web socket server is poorly designed (e.g. it iterates serially through the subscriber list) the latency difference between the first client obtaining the message to the last client obtaining the same message is significant.

Regards,
Gary

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Recording asynchronous request/response times

nmodin
Hi,

The sub protocol the web socket service provides do have unique req/res message identifiers so I should be able to correlate the send and receive timings based on that.

The web socket should also send response messages for connections that did send the triggering request only, so that SHOULD be taken care of. If not there will be bugs filed. :)

Also just noted that I did some spelling misstakes in the sequence diagrams .. :).

Well, thanks for the info ! Greatly appreciated !

Cheers,
Niklas

On Tue, May 26, 2015 at 2:29 PM, Gary Mulder <[hidden email]> wrote:
On 26 May 2015 at 12:34, nmodin <[hidden email]> wrote:

Now if we forget about reality (GC, number of machines etc), this seems to be what I need to do if I read the docs correctly:


Am I going in the right direction here or did I miss the target completely ?

That looks about right. You may want to also put a sequence number in the payload to uniquely identify each message if you want to also confirm that every message is received.

Also, you may want to carefully consider edge cases, i.e. when everyone is subscribed to the same message stream (one to many) as well as when everyone is subscribed to different messages streams (many to many). In the first case you may find that if the web socket server is poorly designed (e.g. it iterates serially through the subscriber list) the latency difference between the first client obtaining the message to the last client obtaining the same message is significant.

Regards,
Gary

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use



------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use
Loading...