Hi,
In my cluster that /home is shared (from a server and some workers). The problem
is when I run my job that has some huge log data,?the disk access?as well as
network traffic is very high. I want to know is there any way so
that?the?worker?writes the log locally and?at the end, that log is copied from
worker's internal hdd to server's?hdd?(/home)?
Hope that I clearly stated the problem.
Thanks,?
// Naderan *Mahmood;
>It would depend on the application.
It uses the standard output redirection '>' which I think?every 5KB writes the
output. Can I increase that threshold?
?
// Naderan *Mahmood;
----- Original Message ----
From: Another Sillyname <[email protected]>
To: Mahmood Naderan <[email protected]>
Sent: Wed, January 19, 2011 10:05:07 PM
Subject: Re: high traffic because of loggig
On 19 January 2011 18:09, Mahmood Naderan <[email protected]> wrote:
> Hi,
> In my cluster that /home is shared (from a server and some workers). The
>problem
> is when I run my job that has some huge log data,?the disk access?as well as
> network traffic is very high. I want to know is there any way so
> that?the?worker?writes the log locally and?at the end, that log is copied from
> worker's internal hdd to server's?hdd?(/home)?
>
> Hope that I clearly stated the problem.
> Thanks,
> // Naderan *Mahmood;
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at ?http://vger.kernel.org/majordomo-info.html
>
It would depend on the application.
If the application doesn't have options to choose the path for the log
files you could set up a symbolic link to the path back to the local
drive.
>Try sticking one of these in there before the redirection: ' | dd bs=8m '
Thanks,
// Naderan *Mahmood;
----- Original Message ----
From: Lyle Seaman <[email protected]>
To: Mahmood Naderan <[email protected]>
Sent: Thu, January 20, 2011 1:30:09 AM
Subject: Re: high traffic because of loggig
Try sticking one of these in there before the redirection: ' | dd bs=8m '
On Wed, Jan 19, 2011 at 2:58 PM, Mahmood Naderan <[email protected]> wrote:
>>It would depend on the application.
>
> It uses the standard output redirection '>' which I think every 5KB writes the
> output. Can I increase that threshold?
>
> // Naderan *Mahmood;
>
>
>
> ----- Original Message ----
> From: Another Sillyname <[email protected]>
> To: Mahmood Naderan <[email protected]>
> Sent: Wed, January 19, 2011 10:05:07 PM
> Subject: Re: high traffic because of loggig
>
> On 19 January 2011 18:09, Mahmood Naderan <[email protected]> wrote:
>> Hi,
>> In my cluster that /home is shared (from a server and some workers). The
>>problem
>> is when I run my job that has some huge log data, the disk access as well as
>> network traffic is very high. I want to know is there any way so
>> that the worker writes the log locally and at the end, that log is copied
from
>> worker's internal hdd to server's hdd (/home)?
>>
>> Hope that I clearly stated the problem.
>> Thanks,
>> // Naderan *Mahmood;
>>
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
> It would depend on the application.
>
> If the application doesn't have options to choose the path for the log
> files you could set up a symbolic link to the path back to the local
> drive.
>
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>