Received-SPF: pass (google.com: domain of linux-kernel+bounces-155296-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <e082dabe-5d1c-4be0-baaa-5be30cff0016@intel.com>
Date: Tue, 23 Apr 2024 16:40:20 +0300
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V2] perf scripts python: Add a script to run instances of
 perf script in parallel
To: Ian Rogers <irogers@google.com>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>, Jiri Olsa <jolsa@kernel.org>,
 Namhyung Kim <namhyung@kernel.org>, Andi Kleen <ak@linux.intel.com>,
 linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org
References: <20240313123634.4353-1-adrian.hunter@intel.com>
 <CAP-5=fW1bH8qQkD7LrO6_3fJ3NsqoW1GrX8=s-sfaTbrvk58+A@mail.gmail.com>
Content-Language: en-US
From: Adrian Hunter <adrian.hunter@intel.com>
Organization: Intel Finland Oy, Registered Address: PL 281, 00181 Helsinki,
 Business Identity Code: 0357606 - 4, Domiciled in Helsinki
In-Reply-To: <CAP-5=fW1bH8qQkD7LrO6_3fJ3NsqoW1GrX8=s-sfaTbrvk58+A@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 11/04/24 21:19, Ian Rogers wrote:
> On Wed, Mar 13, 2024 at 5:36 AM Adrian Hunter <adrian.hunter@intel.com> wrote:
>>
>> Add a Python script to run a perf script command multiple times in
>> parallel, using perf script options --cpu and --time so that each job
>> processes a different chunk of the data.
>>
>> The script supports the use of normal perf script options like
>>  --dlfilter and --script, so that the benefit of running parallel jobs
>> naturally extends to them also. In addition, a command can be provided
>> (refer --pipe-to option) to pipe standard output to a custom command.
>>
>> Refer to the script's own help text at the end of the patch for more
>> details.
>>
>> The script is useful for Intel PT traces, that can be efficiently
>> decoded by perf script when split by CPU and/or time ranges. Running
>> jobs in parallel can decrease the overall decoding time.
>>
>> Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>


>> +
>> +       def __init__(self, cmd, pipe_to, output_dir="."):
>> +               self.popen = None
>> +               self.consumer = None
>> +               self.cmd = cmd
>> +               self.pipe_to = pipe_to
>> +               self.output_dir = output_dir
>> +               self.cmdout_name = output_dir + "/cmd.txt"
>> +               self.stdout_name = output_dir + "/out.txt"
>> +               self.stderr_name = output_dir + "/err.txt"
> 
> Why use files here and not pipes?

There is an option to pipe to another command.

>                                   Could using files cause the command
> to fail on a read-only file system?

The user chooses the output directory, so they will need the foresight
not to choose a read-only file system.