Received: by 10.213.65.68 with SMTP id h4csp76103imn; Thu, 15 Mar 2018 10:00:51 -0700 (PDT) X-Google-Smtp-Source: AG47ELsUM/OSMKafu7sZNKO8Jtg/2Hn9olpIfprnNxzvyj793antHQdX4UqAbF9LIfAFkGK+9du+ X-Received: by 10.101.96.141 with SMTP id t13mr7373062pgu.427.1521133251137; Thu, 15 Mar 2018 10:00:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1521133251; cv=none; d=google.com; s=arc-20160816; b=O3+aF/5yAASJ26j/8n1vGAy4rfbNWbEAdTIkv6CrSdfbTJVKVr9YoQDhsZygVYIE7Z gWzblmTGc7zK4lqSsQNMNqWRCT45jNyreBE/LsHgfWZhFIqpIkfILX45scPryBDsH9Ea Pi6grIhks+5SMYoNMb1CTCLDUlgkrUsJtiqVcJ1BCXlNhpxhUfBIk8tKoiJNJjQwvOEU 2+Nqyied9mmBpWqb9dQMJJA0pzF1HPty6V3Ex1u6OIBa2MJaVocH+jOmDZ8G5WpqCadh /oksJRduN0TNemFGiOcELG4mlffl3yV/8UQLJQIFqZ9z8nAOeVzbsBB+p0gX6L2uGTLF 0eZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:cc:to:subject:message-id:date:from :references:in-reply-to:mime-version:dkim-signature :arc-authentication-results; bh=xR84DkNT1qoxmtbkStdF/Mb0UQU/HmNQkTfe3ug64nk=; b=DEbVRmbcrZlsOdmM2g8tHE9SRuuLdFNDVAOgGyHO7OGFOQDqJhQrCc+4JqpZ3nCmfd WFE44rBOysi2U7si3LKVoegOC0VzBbPgG40GRyzMJH4ZjC3LaMDvCFmhADtzwXs/XTkX 4kcgOP+SZBPzUQnN4Z43Ty9s/K71l09agzPrBcbbulmNJRiQh+svpbWk0Dw/kMH5WELs zI/6mO15/Cdd7iYa0eLEXit2LoIRdZD5pvVGfqf0I4tUDZxZimRO7Zx2ZzRw+IkHRD5r 1wtctW2uvARRmnw3zA2I/V7ChO0q2gUFJMpWB6u6MZCPfixfHy4L0WUvd/l7pofwX6zU BnZg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=g8odbI/z; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j15si2320497pgf.822.2018.03.15.10.00.35; Thu, 15 Mar 2018 10:00:51 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=g8odbI/z; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751945AbeCOQ6J (ORCPT + 99 others); Thu, 15 Mar 2018 12:58:09 -0400 Received: from mail-wr0-f174.google.com ([209.85.128.174]:39178 "EHLO mail-wr0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750987AbeCOQ6H (ORCPT ); Thu, 15 Mar 2018 12:58:07 -0400 Received: by mail-wr0-f174.google.com with SMTP id k3so9017320wrg.6; Thu, 15 Mar 2018 09:58:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=xR84DkNT1qoxmtbkStdF/Mb0UQU/HmNQkTfe3ug64nk=; b=g8odbI/zs0IihWpBFhHTaPUzv0uy8tMPXbuaf80KOdK6YJHgAVYG8jo6ZBzyhi/Hrt aTvOnTmK2o1MFo4/l7KrrcoC/RUws8JHtktAe5q8NakxL87ZTqZNt7/1zSgiOWuGPB5b Dqv5HadqngqtJ1KSulb3BO6+xd3zF0e798MA4HRqCYQ9Zo8iXcfyOUftzqa8FO1huS9E wDeT5v5rtCEyuhUpZ8O03OSU6gVri+gXYUHgdBNJDmg6IzD4+ZIhuf1EM54xiMwKY9xK rthBJojFmpC8VPZ4KqKiREB1DxAqHonTGX6rdVbl0WY5dP6NYOI8u4y/1pevFE6agmXt cHyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=xR84DkNT1qoxmtbkStdF/Mb0UQU/HmNQkTfe3ug64nk=; b=EANjIn6Vd1tav7MRjTgciCes4yc/Fu3FtiJkwaoDK5zuEGb2Aiaan9esvTFUS5XzVc KEMBQosXMMZojWc/YWc/ozT1+hujP6227g4sqYV/QNF9eH0NRSxONaHqv8nfYZ/qDsZ5 YAVFo8lAfCnMgkonIJhXVoTsip99uJTmrjOovw475mB/fqNuzMGsoW7OF5XAMcWgNSgQ e1hZwx0r+SAj9v+qmYs8IlzwaCWI2QmYtLas4FOG+Xv99acCgir4PnMi8XAQH8rG3QAl OrkhefrGyvdGMZMhBLvImNIq997jIHfqfbX3wTcKg6W0bQ2KXqKDuTNJWJbvZGKIKCO+ hQJw== X-Gm-Message-State: AElRT7FO8W/gbM4bwpIZrWKKuMlwTIdE7294l/bMVGjh4kfBb9XUAo3l uykYiJfGgdDYXQNICtp6ouJNxoNzOeszARhWX3S+pQ== X-Received: by 10.223.164.26 with SMTP id d26mr1205452wra.199.1521133086198; Thu, 15 Mar 2018 09:58:06 -0700 (PDT) MIME-Version: 1.0 Received: by 10.223.184.189 with HTTP; Thu, 15 Mar 2018 09:58:05 -0700 (PDT) In-Reply-To: <0175e460-3424-9838-1064-9f63dab3304f@codeaurora.org> References: <1520997629-17361-1-git-send-email-okaya@codeaurora.org> <1520997629-17361-7-git-send-email-okaya@codeaurora.org> <12150aa0-77ba-878e-31f4-d4f8d6a28ccb@codeaurora.org> <2a4f4dec64b7462ae64152f6c2df9754@codeaurora.org> <53bf7dfe-32ee-1861-e6ea-81f667590a43@codeaurora.org> <0175e460-3424-9838-1064-9f63dab3304f@codeaurora.org> From: Alexander Duyck Date: Thu, 15 Mar 2018 09:58:05 -0700 Message-ID: Subject: Re: [PATCH 7/7] ixgbevf: eliminate duplicate barriers on weakly-ordered archs To: Sinan Kaya Cc: Timur Tabi , Netdev , sulrich@codeaurora.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Jeff Kirsher , intel-wired-lan , LKML Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Mar 15, 2018 at 9:27 AM, Sinan Kaya wrote: > On 3/15/2018 12:21 PM, Sinan Kaya wrote: >> On 3/15/2018 10:32 AM, Alexander Duyck wrote: >>> We tend to do something like: >>> update tx_buffer_info >>> update tx_desc >>> wmb() >>> point first tx_buffer_info next_to_watch value at last tx_desc >>> update next_to_use >>> notify device via writel >>> >>> We do it this way because we have to synchronize between the Tx >>> cleanup path and the hardware so we basically lump the two barriers >>> together. instead of invoking both a smp_wmb and a wmb. Now that I >>> look at the pseudocode though I wonder if we shouldn't move the >>> next_to_use update before the wmb, but that might be material for >>> another patch. Anyway, in the Tx cleanup path we should have an >>> smp_rmb() after we read the next_to_watch values so that we avoid >>> reading any of the other fields in the buffer_info if either the field >>> is NULL or the descriptor pointed to has not been written back. >> >> How do you feel about keeping wmb() very close to writel_relaxed() like this? >> >> update tx_buffer_info >> update tx_desc >> point first tx_buffer_info next_to_watch value at last tx_desc >> update next_to_use >> wmb() >> notify device via writel_relaxed() >> >> I'm afraid that if the order of wmb() and writel() is not very >> obvious or hidden in multiple functions, somebody can introduce a very nasty >> bug in the future. >> >> We also have to think about code maintenance. >> > > Now that I read your email again, I think this is the reason if I understood you > correctly. > > "instead of invoking both a smp_wmb and a wmb" > > You'd need something like > > update tx_buffer_info > update tx_desc > smp_wmb() > point first tx_buffer_info next_to_watch value at last tx_desc > update next_to_use > wmb() > notify device via writel_relaxed() > > Let me work on your comments. Yes, we would be doing something like that, but we are using just a single wmb() to cover both cases since hardware will never look at the tx_buffer_info and software will never read that descriptor ring as long as the next_to_watch is NULL. By doing it this way we should have both cases covered and not need to worry The only other bit still remaining is the "maybe_stop_tx" logic which lives between the wmb and writel_relaxed. That logic has a smp_mb living in it that is triggered if we have to stop the queue. Once again though that is only viewed by software so it existing between the wmb and the writel_relaxed should not be an issue. Starting to understand why I was a bit hesitant to have us start taking on these changes now? :-) - Alex