Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8;
Message-ID: <d76cafc0-572f-4a5e-b4e0-5baa060a1256@axis.com>
Date:   Wed, 11 Oct 2023 15:57:58 +0200
User-Agent: Mozilla Thunderbird
Subject: Re: [cocci] [PATCH 2/2] scripts: coccicheck: Separate spatch stdout
 and stderr
Content-Language: en-GB
To:     Julia Lawall <julia.lawall@inria.fr>
Cc:     Nicolas Palix <nicolas.palix@imag.fr>, cocci@inria.fr,
        linux-kernel@vger.kernel.org, kernel@axis.com
References: <20231003-coccicheck-v1-0-07d2d900a52a@axis.com>
 <20231003-coccicheck-v1-2-07d2d900a52a@axis.com>
 <alpine.DEB.2.22.394.2310072140340.36842@hadrien>
 <5c76da14-e34e-afbd-4265-493c66e0bc60@axis.com>
 <b37c26-154a-2db9-4944-26a8aa8af7af@inria.fr>
 <d6ab6509-9969-4f36-96fe-b6ce6bc74b73@axis.com>
 <e695187c-e83-c844-7f70-7bb9d952569@inria.fr>
From:   Anton Eliasson <anton.eliasson@axis.com>
In-Reply-To: <e695187c-e83-c844-7f70-7bb9d952569@inria.fr>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dGZnMFBCNDBTZk5qV1BLRWRJUm9ZRzV6ejZZZjRleWpodG1qZ3NscThMQ29t?=
 =?utf-8?B?bTlhOXBZakZiZHF5bTdsSXhaaFF0bmtxY3hlNG0rb09GM2NKY1BkamNuaEJM?=
 =?utf-8?B?MVIyWHBIN2ZUOCtra3hKVHY0QmdUUGVCK2FEdXQvd0thSU4xRkF1ZDhzMzJQ?=
 =?utf-8?B?UUZEajFaRWNYNEplYWt2UDhBdTF0WFg4eU82bkxieHZPSGUrRnpxTGNzMi8z?=
 =?utf-8?B?VXpEYllXeTE4ekwxc2s1TFlwaGlzbjBhemF6QVRYMXFITGtiM1VULzBrdVdK?=
 =?utf-8?B?anRwNDFBZFF2SWlNdGttNDcwRzJzZlI5T1NvNkdnd003Vm8rYVVHTVJLY1Bt?=
 =?utf-8?B?M0J1NHZEWDVKTTdmd2ZaZ2M5QXczdGJNT2ZEZGZDYnBnbEVsKzNEUkdERDc3?=
 =?utf-8?B?b21vb3JvaGcxeG1uR2M2YjNHYnJPbk9LYlpHb21vQ0sva1pxUlV5REpKQkdw?=
 =?utf-8?B?dkZpc3E1RDZwR3FnTVhGSVU2OS9qQ0JXRFhzWW9EYVBtczZFc1BjTzBtL3B3?=
 =?utf-8?B?WE1EVGdPY2oyU1d1MmZySDN4ZkNlK2NRNUV1dDdhbzhEMGJEa2RERzl3RGs4?=
 =?utf-8?B?U253dEJVUE9VaVhucWVIUzR2WXRXR1B3d25mOGRBOEpHRnE5TkNPNnhOLzl1?=
 =?utf-8?B?ektiQXJJWGU5TGp6VEt1TW1TdVRLcWJKeUFJK09NRGl5TjNQbVJGREtRbmZx?=
 =?utf-8?B?ZlNpLzd4c0JtRVJySEpuamR4OGlFeDBvdGpETjNuS3JmQSsrdEVETUY0WUo0?=
 =?utf-8?B?czR3YkRxbTF1c2F0V2RORkZmVXp2cTlHSUJmb3pFK3o5cXN1ak9Ra1c3NEFN?=
 =?utf-8?B?YmlxclpHdTE1Q09qMGM0WkdaQnNrWU9UUFhvMmYzd203VTd6ZDBkVHdFaEhH?=
 =?utf-8?B?c29UTHBOZEZPWVBnWVZHcU1kbXkvZEc0R2ZzSDJGODdhcTB5elN4NU5uanhB?=
 =?utf-8?B?anI0Yk1FVTV5RnV6TE1DQ0RsbmpoK0t2VE1scldGUzBrRzBpRSt1OXI2WFRv?=
 =?utf-8?B?TjVCN3QxanYvdFpLUGpNOVlmcEJ6eXV6QlN0YjRiLzBrb3doRHlzbTkyWTMv?=
 =?utf-8?B?RHB0OXpNZzUxMEd0b29LMGp1bzJXSVhVMjUrM21GZVNJdk9lL3NJVEZ0Z2RT?=
 =?utf-8?B?VExoNm85cjRmSWpJazlhN0s0SG55M3h5NXcxV29CVE1uUmFhZEEyeVIzSjZP?=
 =?utf-8?B?c3VSNmVlL1JML1F3bmp6Q1NqMzR4NUs5VjhQYUlRS1k2eStTV1NyaElBc2tr?=
 =?utf-8?B?cVVaeUpuNzV2SHNJcXJMWWF2QWhXRFc1azJrQmhpWmhkZVN4MllScGZnbzFp?=
 =?utf-8?B?SFlqUTNPVWZueHhQcDZGR1E4ajBzVmowNjR0eHlMWXNxOTArQXZrUGkxT3hz?=
 =?utf-8?B?VkxEcXJqdVRRN25OVDhjaUV6L3Z1UkpXaVNKeDR6SzBnMFF3blpEd0Q3dXFs?=
 =?utf-8?B?blVZY2VtUjZubWI3Q210NmZyKzdhaXltNTZCdGpVazlYTm52SU9Od1Q3VEF1?=
 =?utf-8?B?MDZIZEkvYmU4WlhweFp4blpMZW9Qa2cweFQ1OW1MRDR4dm1xSVZCdHpkVzAw?=
 =?utf-8?B?L0tTcW0xUWF3K0hNZW9Bcit1YUQ3TnlBeSs0dm4xYStMY2p2blAzOGFzQmVS?=
 =?utf-8?B?bkVNUnVXSVJrVjUwYURIUTIyMDBTeWgxR0ZaZy9Fdk55K1dJd1A3M0laRzNE?=
 =?utf-8?B?SVBWTFRtSWJNczRYeWh4MFFTd2syTHFtaWIwa3NXK3ZnazVPcURWRDR2UWhs?=
 =?utf-8?B?eWg2a2x3VWc4SGszbnlTb0ViYkRJSXgxL1R6MWMzL2tLb3RTeVVKekN1RG51?=
 =?utf-8?B?bENGb0JUQmE4RkVUdDllcytuVStxdVlzOHB0dFBZa3k2QUhsTTFGU2swbS9w?=
 =?utf-8?B?dFFtRWdFQmJPL1ZZdkZDZE00TzB5U0w5bkVnNUd2cjNUeTBvQU1qTHdReFhX?=
 =?utf-8?B?ZVdOanJzTGdEVWQyWklMbzdmNnk2MnFzUitLd0FTdEtMWmppa3NiS3U3OEVE?=
 =?utf-8?B?cUxSZmpZejZIMlhBd3lYSEFSeDk0UHNDUGdlaDdCRTFnb0tqKzNTQmRrOVlG?=
 =?utf-8?B?N2U2cFc2L0o3R2d0aE9ubDRMaVRNNmsxOFVzNVkvdlpFdVNGdWxOYlJ6THZt?=
 =?utf-8?Q?cOBI=3D?=
X-OriginatorOrg: axis.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9dd1d9bd-efac-47d2-ad7e-08dbca6214ae
X-MS-Exchange-CrossTenant-AuthSource: AS1PR02MB10348.eurprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Oct 2023 13:58:00.7350
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 78703d3c-b907-432f-b066-88f7af9ca3af
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 3bvWD00iu7xxDv99GEJ6O6iHUDSeLQmKzCI1q82AVz/XWBePl7z+YczHxA7x0JVe
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR02MB5894
X-Spam-Status: No, score=2.7 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Wed, 11 Oct 2023 06:58:20 -0700 (PDT)
X-Spam-Level: **

On 11/10/2023 15.46, Julia Lawall wrote:
>
> On Wed, 11 Oct 2023, Anton Eliasson wrote:
>
>> On 10/10/2023 18.11, Julia Lawall wrote:
>>> On Tue, 10 Oct 2023, Anton Eliasson wrote:
>>>
>>>> On 07/10/2023 21.41, Julia Lawall wrote:
>>>>> On Tue, 3 Oct 2023, Anton Eliasson wrote:
>>>>>
>>>>>> This helps automating coccicheck runs by discarding stderr and only
>>>>>> looking at the output of stdout. In report mode the only remaining
>>>>>> output on stdout is the initial "Please check for false positives"
>>>>>> message followed by each spatch warning found.
>>>>> What is getting dropped is the spatch command lines indicating the
>>>>> semantic patch.  Is this desirable?
>>>>>
>>>>> julia
>>>> It's not ideal but it's the best compromise that I have found. The problem
>>>> I'm
>>>> trying to solve is to be able to diff the output of two coccicheck runs
>>>> and
>>>> notify the developer if any new warnings were introduced. That requires
>>>> the
>>>> output to be stable. spatch is always invoked for each cocci file in the
>>>> same
>>>> order. However, the output from each spatch invocation is not stable as it
>>>> examines each source file in an arbitrary order.
>>>>
>>>> My workaround is to sort the output before diffing. The line-by-line
>>>> sorted
>>>> output only makes sense if the input is one line per warning found and
>>>> that is
>>>> why I try to discard all output except the single line per spatch warning.
>>>> While the terse output doesn't tell which semantic patch file generated
>>>> the
>>>> warning, it does usually contain the offending file, line number and a
>>>> summary
>>>> of the issue.
>>> Why does the command line pose a problem for sorting?
>>>
>>> julia
>> You're right. I was overthinking it. Since the sorted command lines will be
>> common for the two runs they will disappear after diffing.
>>
>> So at this point I don't have any need for this patch. I'll reach out to you
>> again if it turns out to be an issue after we have gotten the continuous
>> integration check in place. Thanks for the feedback and I'm sorry about the
>> noise.
> OK, thanks for the discussion.  I was also thinking about whether it could
> be possible to make the output always come out in the same order, based on
> the name of the analyzed file.  Maybe it is possible.
>
> julia

Yes, that might help to preserve some context. The difference between 
two runs could be presented as a unified diff instead of just lines 
added. That would probably contain the command line invoking the 
semantic patch that caused the warning, unless there are too many 
pre-existent warnings.


Anton

>
>>
>> Anton
>>
>>>> Anton
>>>>>> Signed-off-by: Anton Eliasson <anton.eliasson@axis.com>
>>>>>> ---
>>>>>>     scripts/coccicheck | 4 ++--
>>>>>>     1 file changed, 2 insertions(+), 2 deletions(-)
>>>>>>
>>>>>> diff --git a/scripts/coccicheck b/scripts/coccicheck
>>>>>> index 95a312730e98..7e7c44125f47 100755
>>>>>> --- a/scripts/coccicheck
>>>>>> +++ b/scripts/coccicheck
>>>>>> @@ -146,8 +146,8 @@ run_cmd_parmap() {
>>>>>>                     echo $@>>$DEBUG_FILE
>>>>>>                     $@ 2>>$DEBUG_FILE
>>>>>>             else
>>>>>> -                echo $@
>>>>>> -                $@ 2>&1
>>>>>> +                echo $@ >&2
>>>>>> +                $@
>>>>>>     	fi
>>>>>>
>>>>>>     	err=$?
>>>>>>
>>>>>> --
>>>>>> 2.30.2
>>>>>>
>>>>>>
>>