Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH 08/11] selftests: net/fcnal: Replace sleep after server
 start with -k
To:     David Ahern <dsahern@gmail.com>, Jakub Kicinski <kuba@kernel.org>,
        Shuah Khan <shuah@kernel.org>, David Ahern <dsahern@kernel.org>
Cc:     "David S. Miller" <davem@davemloft.net>,
        Ido Schimmel <idosch@nvidia.com>,
        Seth David Schoen <schoen@loyalty.org>,
        netdev@vger.kernel.org, linux-kselftest@vger.kernel.org,
        linux-kernel@vger.kernel.org
References: <cover.1633520807.git.cdleonard@gmail.com>
 <ec40bd7128a30e93b90ba888f3468f394617a010.1633520807.git.cdleonard@gmail.com>
 <43210038-b04b-3726-1355-d5f132f6c64e@gmail.com>
From:   Leonard Crestez <cdleonard@gmail.com>
Message-ID: <d6882c3f-4ecf-4b4e-c20e-09b88da4fbd6@gmail.com>
Date:   Thu, 7 Oct 2021 00:35:14 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <43210038-b04b-3726-1355-d5f132f6c64e@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk


On 06.10.2021 17:54, David Ahern wrote:
> On 10/6/21 5:47 AM, Leonard Crestez wrote:
>> The -k switch makes the server fork into the background after the listen
>> call is succesful, this can be used to replace most of the `sleep 1`
>> statements in this script.
>>
>> Change performed with a vim command:
>>
>> s/nettest \(.*-s.*\) &\n\s*sleep 1\n/nettest \1 -k\r
>>
>> Signed-off-by: Leonard Crestez <cdleonard@gmail.com>
>> ---
>>   tools/testing/selftests/net/fcnal-test.sh | 641 ++++++++--------------
>>   1 file changed, 219 insertions(+), 422 deletions(-)
>>
> 
> I have a change from January [1] that runs the tests with 1 binary -
> takes both client and server side arguments, does the server setup,
> switches namespaces as needed and then runs the client side. I got
> bogged down validating the before and after which takes a long time
> given the number of tests. The output in verbose mode is as important as
> the pass / fail. Many of the tests document existing behavior as well as
> intended behavior.
> 
> You used a search and replace to update the tests. Did you then do the
> compare of test results - not pass / fail but output?

I counted the [FAIL] or [ OK ] markers but not the output of nettest 
itself. I don't know what to look for, I guess I could diff the outputs?

Shouldn't it be sufficient to compare the exit codes of the nettest client?

The output is also modified by a previous change to not capture server 
output separately and instead let it be combined with that of the 
client. That change is required for this one, doing out=$(nettest -k) 
does not return on fork unless the pipe is also closed.

I did not look at your change, mine is relatively minimal because it 
only changes who decide when the server goes into the background: the 
shell script or the server itself. This makes it work very easily even 
for tests with multiple server instances.

--
Regards,
Leonard