Quantcast

Broken Pipe when agents report to console

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Broken Pipe when agents report to console

Joel Lucuik
Hi,

Grinder version is 3-11.
4 vms, each VM has an agent, and the console shares a VM with the first agent. 10,0000 users (threads).

Problem: We are occasionally seeing broken pipes when the agent(s) report anything to the console.

My Investigation:

I have been looking at the Grinder source. There is some code which traps CommunicationException such as

AbstractSender.send() or
ClientSender.blockingSend()

but doesn't do anything with it. Such as attempt a reconnect if it's a Broken Pipe.

Has this been fixed in newer versions? More specifically if the agent tries to send stats to the console, or a status update, and encounters a broken pipe, can it reconnect?

Suggestions on which code/class/method to fix?


We have very good unix people on our team. They have already upped the OS files, so we are (fairly) sure this is not the issue.

Right now we are debating on a port to JMeter.

Any advice would be appreciated.

Joel



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Broken Pipe when agents report to console

Darren Ball
Hi Joel,

I've had a lot of issues with client sender timing out.

Here is an approach (patch below) that I've taken.  Adding a keepAlive to sender as it is created in GrinderProcess

Not sure if it will help, but it did for me in my case.  I did this as it was timing out way to early in my process and I was getting 'whilst' communicating with console errors.

Patch::

Index: grinder-core/src/main/java/net/grinder/engine/process/GrinderProcess.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- grinder-core/src/main/java/net/grinder/engine/process/GrinderProcess.java (revision ffb7bca73ea3190eaab12d1eccf73bf31464b58f)
+++ grinder-core/src/main/java/net/grinder/engine/process/GrinderProcess.java (revision )
@@ -95,6 +95,9 @@
import java.util.Map;
import java.util.Timer;
import java.util.TimerTask;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;


/**
@@ -141,6 +144,10 @@
// Guarded by m_eventSynchronisation.
private String m_shutdownReason;

+ private ClientSender sender;
+
+ private final ScheduledExecutorService m_executor = Executors.newSingleThreadScheduledExecutor();
+
/**
* Creates a new {@code GrinderProcess} instance.
*
@@ -184,13 +191,34 @@

final BarrierGroups barrierGroups;

+ sender = ClientSender.connect(
+ new ConnectorFactory(ConnectionType.WORKER).create(properties),
+ new WorkerAddress(workerIdentity));
+
+ final Runnable keepAlive = new Runnable() {
+ @Override
+ public void run() {
+ try {
+ sender.sendKeepAlive();
+ }
+ catch (CommunicationException e) {
+ // Connection is dead, terminate task.
+ throw new RuntimeException(e);
+ }
+ }
+ };
+
+ m_executor.scheduleWithFixedDelay(keepAlive,
+ 0,
+ 1,
+ TimeUnit.SECONDS);
+
+
if (m_initialisationMessage.getReportToConsole()) {
m_consoleSender =
new FirstHurdleSender(
- new QueuedSenderDecorator(
- ClientSender.connect(
- new ConnectorFactory(ConnectionType.WORKER).create(properties),
- new WorkerAddress(workerIdentity))));
+ new QueuedSenderDecorator(sender
+ ));

barrierGroups =
new ClientBarrierGroups(m_consoleSender,

On Tue, Nov 1, 2016 at 1:33 PM, Joel Lucuik <[hidden email]> wrote:
Hi,

Grinder version is 3-11.
4 vms, each VM has an agent, and the console shares a VM with the first agent. 10,0000 users (threads).

Problem: We are occasionally seeing broken pipes when the agent(s) report anything to the console.

My Investigation:

I have been looking at the Grinder source. There is some code which traps CommunicationException such as

AbstractSender.send() or
ClientSender.blockingSend()

but doesn't do anything with it. Such as attempt a reconnect if it's a Broken Pipe.

Has this been fixed in newer versions? More specifically if the agent tries to send stats to the console, or a status update, and encounters a broken pipe, can it reconnect?

Suggestions on which code/class/method to fix?


We have very good unix people on our team. They have already upped the OS files, so we are (fairly) sure this is not the issue.

Right now we are debating on a port to JMeter.

Any advice would be appreciated.

Joel



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Broken Pipe when agents report to console

Joel Lucuik
Thanks Darren.

I am about to try

1. Adding connector and address to SocketWrapper, and adding another constructor to set them.
2. Adding method SocketWrapper reconnectToSocket()
3. Calling from ClientSender.blockingSend()

Thinking about using this pattern in other places too.

That said, I think your idea sounds like it's worth a try, especially considering it's already worked for you. And I am not even sure
which methods to change!! Just kind of looking and hoping, and will do some logging to hopefully reproduce it.

Joel



On Tue, Nov 1, 2016 at 3:11 PM, Darren Ball <[hidden email]> wrote:
Hi Joel,

I've had a lot of issues with client sender timing out.

Here is an approach (patch below) that I've taken.  Adding a keepAlive to sender as it is created in GrinderProcess

Not sure if it will help, but it did for me in my case.  I did this as it was timing out way to early in my process and I was getting 'whilst' communicating with console errors.

Patch::

Index: grinder-core/src/main/java/net/grinder/engine/process/GrinderProcess.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- grinder-core/src/main/java/net/grinder/engine/process/GrinderProcess.java (revision ffb7bca73ea3190eaab12d1eccf73bf31464b58f)
+++ grinder-core/src/main/java/net/grinder/engine/process/GrinderProcess.java (revision )
@@ -95,6 +95,9 @@
import java.util.Map;
import java.util.Timer;
import java.util.TimerTask;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;


/**
@@ -141,6 +144,10 @@
// Guarded by m_eventSynchronisation.
private String m_shutdownReason;

+ private ClientSender sender;
+
+ private final ScheduledExecutorService m_executor = Executors.newSingleThreadScheduledExecutor();
+
/**
* Creates a new {@code GrinderProcess} instance.
*
@@ -184,13 +191,34 @@

final BarrierGroups barrierGroups;

+ sender = ClientSender.connect(
+ new ConnectorFactory(ConnectionType.WORKER).create(properties),
+ new WorkerAddress(workerIdentity));
+
+ final Runnable keepAlive = new Runnable() {
+ @Override
+ public void run() {
+ try {
+ sender.sendKeepAlive();
+ }
+ catch (CommunicationException e) {
+ // Connection is dead, terminate task.
+ throw new RuntimeException(e);
+ }
+ }
+ };
+
+ m_executor.scheduleWithFixedDelay(keepAlive,
+ 0,
+ 1,
+ TimeUnit.SECONDS);
+
+
if (m_initialisationMessage.getReportToConsole()) {
m_consoleSender =
new FirstHurdleSender(
- new QueuedSenderDecorator(
- ClientSender.connect(
- new ConnectorFactory(ConnectionType.WORKER).create(properties),
- new WorkerAddress(workerIdentity))));
+ new QueuedSenderDecorator(sender
+ ));

barrierGroups =
new ClientBarrierGroups(m_consoleSender,

On Tue, Nov 1, 2016 at 1:33 PM, Joel Lucuik <[hidden email]> wrote:
Hi,

Grinder version is 3-11.
4 vms, each VM has an agent, and the console shares a VM with the first agent. 10,0000 users (threads).

Problem: We are occasionally seeing broken pipes when the agent(s) report anything to the console.

My Investigation:

I have been looking at the Grinder source. There is some code which traps CommunicationException such as

AbstractSender.send() or
ClientSender.blockingSend()

but doesn't do anything with it. Such as attempt a reconnect if it's a Broken Pipe.

Has this been fixed in newer versions? More specifically if the agent tries to send stats to the console, or a status update, and encounters a broken pipe, can it reconnect?

Suggestions on which code/class/method to fix?


We have very good unix people on our team. They have already upped the OS files, so we are (fairly) sure this is not the issue.

Right now we are debating on a port to JMeter.

Any advice would be appreciated.

Joel



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Broken Pipe when agents report to console

Joel Lucuik
In reply to this post by Joel Lucuik
What is FirstHurdleSender? Did you create this? (Have Source?)

Thanks.
Joel

On Tue, Nov 1, 2016 at 3:11 PM, Darren Ball <[hidden email]> wrote:
Hi Joel,

I've had a lot of issues with client sender timing out.

Here is an approach (patch below) that I've taken.  Adding a keepAlive to sender as it is created in GrinderProcess

Not sure if it will help, but it did for me in my case.  I did this as it was timing out way to early in my process and I was getting 'whilst' communicating with console errors.

Patch::

Index: grinder-core/src/main/java/net/grinder/engine/process/GrinderProcess.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- grinder-core/src/main/java/net/grinder/engine/process/GrinderProcess.java (revision ffb7bca73ea3190eaab12d1eccf73bf31464b58f)
+++ grinder-core/src/main/java/net/grinder/engine/process/GrinderProcess.java (revision )
@@ -95,6 +95,9 @@
import java.util.Map;
import java.util.Timer;
import java.util.TimerTask;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;


/**
@@ -141,6 +144,10 @@
// Guarded by m_eventSynchronisation.
private String m_shutdownReason;

+ private ClientSender sender;
+
+ private final ScheduledExecutorService m_executor = Executors.newSingleThreadScheduledExecutor();
+
/**
* Creates a new {@code GrinderProcess} instance.
*
@@ -184,13 +191,34 @@

final BarrierGroups barrierGroups;

+ sender = ClientSender.connect(
+ new ConnectorFactory(ConnectionType.WORKER).create(properties),
+ new WorkerAddress(workerIdentity));
+
+ final Runnable keepAlive = new Runnable() {
+ @Override
+ public void run() {
+ try {
+ sender.sendKeepAlive();
+ }
+ catch (CommunicationException e) {
+ // Connection is dead, terminate task.
+ throw new RuntimeException(e);
+ }
+ }
+ };
+
+ m_executor.scheduleWithFixedDelay(keepAlive,
+ 0,
+ 1,
+ TimeUnit.SECONDS);
+
+
if (m_initialisationMessage.getReportToConsole()) {
m_consoleSender =
new FirstHurdleSender(
- new QueuedSenderDecorator(
- ClientSender.connect(
- new ConnectorFactory(ConnectionType.WORKER).create(properties),
- new WorkerAddress(workerIdentity))));
+ new QueuedSenderDecorator(sender
+ ));

barrierGroups =
new ClientBarrierGroups(m_consoleSender,

On Tue, Nov 1, 2016 at 1:33 PM, Joel Lucuik <[hidden email]> wrote:
Hi,

Grinder version is 3-11.
4 vms, each VM has an agent, and the console shares a VM with the first agent. 10,0000 users (threads).

Problem: We are occasionally seeing broken pipes when the agent(s) report anything to the console.

My Investigation:

I have been looking at the Grinder source. There is some code which traps CommunicationException such as

AbstractSender.send() or
ClientSender.blockingSend()

but doesn't do anything with it. Such as attempt a reconnect if it's a Broken Pipe.

Has this been fixed in newer versions? More specifically if the agent tries to send stats to the console, or a status update, and encounters a broken pipe, can it reconnect?

Suggestions on which code/class/method to fix?


We have very good unix people on our team. They have already upped the OS files, so we are (fairly) sure this is not the issue.

Right now we are debating on a port to JMeter.

Any advice would be appreciated.

Joel



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Broken Pipe when agents report to console

olivier merlin
In reply to this post by Joel Lucuik

Hello,
We have got the Broken pipe at startup because we have a lot of data to initialized.
The console close the connections after 30 second if there is no activity coming from the worker only before the first run. We have so recompiled the code replacing the default  value of 30 seconds by 5 minutes.
If you have the problem sometimes at startup you have this issue.
Cheers
Olivier


Le mar. 1 nov. 2016 20:26, Joel Lucuik <[hidden email]> a écrit :
Thanks Darren.

I am about to try

1. Adding connector and address to SocketWrapper, and adding another constructor to set them.
2. Adding method SocketWrapper reconnectToSocket()
3. Calling from ClientSender.blockingSend()

Thinking about using this pattern in other places too.

That said, I think your idea sounds like it's worth a try, especially considering it's already worked for you. And I am not even sure
which methods to change!! Just kind of looking and hoping, and will do some logging to hopefully reproduce it.

Joel



On Tue, Nov 1, 2016 at 3:11 PM, Darren Ball <[hidden email]> wrote:
Hi Joel,

I've had a lot of issues with client sender timing out.

Here is an approach (patch below) that I've taken.  Adding a keepAlive to sender as it is created in GrinderProcess

Not sure if it will help, but it did for me in my case.  I did this as it was timing out way to early in my process and I was getting 'whilst' communicating with console errors.

Patch::

Index: grinder-core/src/main/java/net/grinder/engine/process/GrinderProcess.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- grinder-core/src/main/java/net/grinder/engine/process/GrinderProcess.java (revision ffb7bca73ea3190eaab12d1eccf73bf31464b58f)
+++ grinder-core/src/main/java/net/grinder/engine/process/GrinderProcess.java (revision )
@@ -95,6 +95,9 @@
import java.util.Map;
import java.util.Timer;
import java.util.TimerTask;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;


/**
@@ -141,6 +144,10 @@
// Guarded by m_eventSynchronisation.
private String m_shutdownReason;

+ private ClientSender sender;
+
+ private final ScheduledExecutorService m_executor = Executors.newSingleThreadScheduledExecutor();
+
/**
* Creates a new {@code GrinderProcess} instance.
*
@@ -184,13 +191,34 @@

final BarrierGroups barrierGroups;

+ sender = ClientSender.connect(
+ new ConnectorFactory(ConnectionType.WORKER).create(properties),
+ new WorkerAddress(workerIdentity));
+
+ final Runnable keepAlive = new Runnable() {
+ @Override
+ public void run() {
+ try {
+ sender.sendKeepAlive();
+ }
+ catch (CommunicationException e) {
+ // Connection is dead, terminate task.
+ throw new RuntimeException(e);
+ }
+ }
+ };
+
+ m_executor.scheduleWithFixedDelay(keepAlive,
+ 0,
+ 1,
+ TimeUnit.SECONDS);
+
+
if (m_initialisationMessage.getReportToConsole()) {
m_consoleSender =
new FirstHurdleSender(
- new QueuedSenderDecorator(
- ClientSender.connect(
- new ConnectorFactory(ConnectionType.WORKER).create(properties),
- new WorkerAddress(workerIdentity))));
+ new QueuedSenderDecorator(sender
+ ));

barrierGroups =
new ClientBarrierGroups(m_consoleSender,

On Tue, Nov 1, 2016 at 1:33 PM, Joel Lucuik <[hidden email]> wrote:
Hi,

Grinder version is 3-11.
4 vms, each VM has an agent, and the console shares a VM with the first agent. 10,0000 users (threads).

Problem: We are occasionally seeing broken pipes when the agent(s) report anything to the console.

My Investigation:

I have been looking at the Grinder source. There is some code which traps CommunicationException such as

AbstractSender.send() or
ClientSender.blockingSend()

but doesn't do anything with it. Such as attempt a reconnect if it's a Broken Pipe.

Has this been fixed in newer versions? More specifically if the agent tries to send stats to the console, or a status update, and encounters a broken pipe, can it reconnect?

Suggestions on which code/class/method to fix?


We have very good unix people on our team. They have already upped the OS files, so we are (fairly) sure this is not the issue.

Right now we are debating on a port to JMeter.

Any advice would be appreciated.

Joel



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Broken Pipe when agents report to console

Darren Ball

Adding a keep alive means you never have to fiddle with the timeout.  IMO.


On Nov 1, 2016 3:56 PM, "olivier merlin" <[hidden email]> wrote:

Hello,
We have got the Broken pipe at startup because we have a lot of data to initialized.
The console close the connections after 30 second if there is no activity coming from the worker only before the first run. We have so recompiled the code replacing the default  value of 30 seconds by 5 minutes.
If you have the problem sometimes at startup you have this issue.
Cheers
Olivier


Le mar. 1 nov. 2016 20:26, Joel Lucuik <[hidden email]> a écrit :
Thanks Darren.

I am about to try

1. Adding connector and address to SocketWrapper, and adding another constructor to set them.
2. Adding method SocketWrapper reconnectToSocket()
3. Calling from ClientSender.blockingSend()

Thinking about using this pattern in other places too.

That said, I think your idea sounds like it's worth a try, especially considering it's already worked for you. And I am not even sure
which methods to change!! Just kind of looking and hoping, and will do some logging to hopefully reproduce it.

Joel



On Tue, Nov 1, 2016 at 3:11 PM, Darren Ball <[hidden email]> wrote:
Hi Joel,

I've had a lot of issues with client sender timing out.

Here is an approach (patch below) that I've taken.  Adding a keepAlive to sender as it is created in GrinderProcess

Not sure if it will help, but it did for me in my case.  I did this as it was timing out way to early in my process and I was getting 'whilst' communicating with console errors.

Patch::

Index: grinder-core/src/main/java/net/grinder/engine/process/GrinderProcess.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- grinder-core/src/main/java/net/grinder/engine/process/GrinderProcess.java (revision ffb7bca73ea3190eaab12d1eccf73bf31464b58f)
+++ grinder-core/src/main/java/net/grinder/engine/process/GrinderProcess.java (revision )
@@ -95,6 +95,9 @@
import java.util.Map;
import java.util.Timer;
import java.util.TimerTask;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;


/**
@@ -141,6 +144,10 @@
// Guarded by m_eventSynchronisation.
private String m_shutdownReason;

+ private ClientSender sender;
+
+ private final ScheduledExecutorService m_executor = Executors.newSingleThreadScheduledExecutor();
+
/**
* Creates a new {@code GrinderProcess} instance.
*
@@ -184,13 +191,34 @@

final BarrierGroups barrierGroups;

+ sender = ClientSender.connect(
+ new ConnectorFactory(ConnectionType.WORKER).create(properties),
+ new WorkerAddress(workerIdentity));
+
+ final Runnable keepAlive = new Runnable() {
+ @Override
+ public void run() {
+ try {
+ sender.sendKeepAlive();
+ }
+ catch (CommunicationException e) {
+ // Connection is dead, terminate task.
+ throw new RuntimeException(e);
+ }
+ }
+ };
+
+ m_executor.scheduleWithFixedDelay(keepAlive,
+ 0,
+ 1,
+ TimeUnit.SECONDS);
+
+
if (m_initialisationMessage.getReportToConsole()) {
m_consoleSender =
new FirstHurdleSender(
- new QueuedSenderDecorator(
- ClientSender.connect(
- new ConnectorFactory(ConnectionType.WORKER).create(properties),
- new WorkerAddress(workerIdentity))));
+ new QueuedSenderDecorator(sender
+ ));

barrierGroups =
new ClientBarrierGroups(m_consoleSender,

On Tue, Nov 1, 2016 at 1:33 PM, Joel Lucuik <[hidden email]> wrote:
Hi,

Grinder version is 3-11.
4 vms, each VM has an agent, and the console shares a VM with the first agent. 10,0000 users (threads).

Problem: We are occasionally seeing broken pipes when the agent(s) report anything to the console.

My Investigation:

I have been looking at the Grinder source. There is some code which traps CommunicationException such as

AbstractSender.send() or
ClientSender.blockingSend()

but doesn't do anything with it. Such as attempt a reconnect if it's a Broken Pipe.

Has this been fixed in newer versions? More specifically if the agent tries to send stats to the console, or a status update, and encounters a broken pipe, can it reconnect?

Suggestions on which code/class/method to fix?


We have very good unix people on our team. They have already upped the OS files, so we are (fairly) sure this is not the issue.

Right now we are debating on a port to JMeter.

Any advice would be appreciated.

Joel



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
grinder-use mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/grinder-use
Loading...