How to record aggregate latency from concurrent threads?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

How to record aggregate latency from concurrent threads?

Ahmad Alnafoosi

Hi 
I am trying to create a Grinder script that captures the following latency


1- Basic scenario is one Agent/ one thread ( Complex Scenario Multiple Agents and multiple threads )
2- Start Timer
3- For each of [Specific List of URLs] send concurrent requests
       -HTTP GET of the URL
4- When all the URLs' GET commands completes  Stop Timer. 
5- Capture the time as aggregate GET latency. 


I was not able to find any similar example in the Scripts examples http://grinder.sourceforge.net/g3/script-gallery.html  

So I attempted to use concurrency within a worker 
http://www.jython.org/jythonbook/en/1.0/Concurrency.html  

as follows: 

t1 = threading.Thread(target=lambda:ObjList[0].GET())
t2 = threading.Thread(target=lambda:ObjList[1].GET())
t1.start()
t2.start()


BUT It fails with the following error
EngineException: net.grinder.engine.common.EngineException: Must be called from worker thread
 
When I call the same method without the Thread it works sequentially (which I do not want)

Is there a better way ??

thanks

Ahmad  

------------------------------------------------------------------------------

_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use
Reply | Threaded
Open this post in threaded view
|

Re: How to record aggregate latency from concurrent threads?

Marc Holden
In order to this you can't use python threading.  Instead, you need to start a grinder worker process with a TestRunner class that has __call__ implemented in it.

We have done things similar to below in the past to mimic async requests.

grinder.startWorkerThread(GenericTestRunner.TestRunner(request=registerReq, path='registerVU&key='+self.key, verb="POST", params=None, withJson=False))

The __init__ method in the GenericTestRunner.TestRunning is below.  This takes care of managing cookies that may be required by the server.  This would need to be modified to suite your needs as well as the implementation of the __call__ method.

Hope this helps.

-Marc

def __init__(self, request, path, params, cookies=None, verb='GET', withJson = False):
        
        '''
        Constructor
        '''
        self.__request = request
        self.path = path
        self.__params = params
        
        if cookies != None:
            self.__cookies = cookies
        if cookies == None and __RUN_WORKER_IN_SEP_PROCESS:
            self.__cookies = CookieModule.listAllCookies(HTTPPluginControl.getThreadHTTPClientContext())
            self.__withJson = withJson
        self.__verb = verb
        self.isDone = False
        
        grinder.getProperties().setInt('grinder.runs',1)
            

On Tue, Aug 30, 2016 at 6:50 PM Ahmad Alnafoosi <[hidden email]> wrote:

Hi 
I am trying to create a Grinder script that captures the following latency


1- Basic scenario is one Agent/ one thread ( Complex Scenario Multiple Agents and multiple threads )
2- Start Timer
3- For each of [Specific List of URLs] send concurrent requests
       -HTTP GET of the URL
4- When all the URLs' GET commands completes  Stop Timer. 
5- Capture the time as aggregate GET latency. 


I was not able to find any similar example in the Scripts examples http://grinder.sourceforge.net/g3/script-gallery.html  

So I attempted to use concurrency within a worker 

as follows: 

t1 = threading.Thread(target=lambda:ObjList[0].GET())
t2 = threading.Thread(target=lambda:ObjList[1].GET())
t1.start()
t2.start()


BUT It fails with the following error
EngineException: net.grinder.engine.common.EngineException: Must be called from worker thread
 
When I call the same method without the Thread it works sequentially (which I do not want)

Is there a better way ??

thanks

Ahmad  
------------------------------------------------------------------------------
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use

------------------------------------------------------------------------------

_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use
Reply | Threaded
Open this post in threaded view
|

Re: How to record aggregate latency from concurrent threads?

Darren Ball
Would be really interesting to see a full working version of this script Marc.  Maybe you could add something like this to the script gallery?

On Wed, Aug 31, 2016 at 11:55 AM, Marc Holden <[hidden email]> wrote:
In order to this you can't use python threading.  Instead, you need to start a grinder worker process with a TestRunner class that has __call__ implemented in it.

We have done things similar to below in the past to mimic async requests.

grinder.startWorkerThread(GenericTestRunner.TestRunner(request=registerReq, path='registerVU&key='+self.key, verb="POST", params=None, withJson=False))

The __init__ method in the GenericTestRunner.TestRunning is below.  This takes care of managing cookies that may be required by the server.  This would need to be modified to suite your needs as well as the implementation of the __call__ method.

Hope this helps.

-Marc

def __init__(self, request, path, params, cookies=None, verb='GET', withJson = False):
        
        '''
        Constructor
        '''
        self.__request = request
        self.path = path
        self.__params = params
        
        if cookies != None:
            self.__cookies = cookies
        if cookies == None and __RUN_WORKER_IN_SEP_PROCESS:
            self.__cookies = CookieModule.listAllCookies(HTTPPluginControl.getThreadHTTPClientContext())
            self.__withJson = withJson
        self.__verb = verb
        self.isDone = False
        
        grinder.getProperties().setInt('grinder.runs',1)
            

On Tue, Aug 30, 2016 at 6:50 PM Ahmad Alnafoosi <[hidden email]> wrote:

Hi 
I am trying to create a Grinder script that captures the following latency


1- Basic scenario is one Agent/ one thread ( Complex Scenario Multiple Agents and multiple threads )
2- Start Timer
3- For each of [Specific List of URLs] send concurrent requests
       -HTTP GET of the URL
4- When all the URLs' GET commands completes  Stop Timer. 
5- Capture the time as aggregate GET latency. 


I was not able to find any similar example in the Scripts examples http://grinder.sourceforge.net/g3/script-gallery.html  

So I attempted to use concurrency within a worker 

as follows: 

t1 = threading.Thread(target=lambda:ObjList[0].GET())
t2 = threading.Thread(target=lambda:ObjList[1].GET())
t1.start()
t2.start()


BUT It fails with the following error
EngineException: net.grinder.engine.common.EngineException: Must be called from worker thread
 
When I call the same method without the Thread it works sequentially (which I do not want)

Is there a better way ??

thanks

Ahmad  
------------------------------------------------------------------------------
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use

------------------------------------------------------------------------------

_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use



------------------------------------------------------------------------------

_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use
Reply | Threaded
Open this post in threaded view
|

How to record Barrier latency for multiple threads?

Ahmad Alnafoosi
In reply to this post by Ahmad Alnafoosi

Hi 
I am trying to create a Grinder script that [records] the start to finish latency statistics for a barrier similar to the example given in 



thanks

Ahmad 


------------------------------------------------------------------------------

_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use