Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755598AbcJLQEB (ORCPT ); Wed, 12 Oct 2016 12:04:01 -0400 Received: from smtp-out4.electric.net ([192.162.216.182]:50367 "EHLO smtp-out4.electric.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754567AbcJLQDt (ORCPT ); Wed, 12 Oct 2016 12:03:49 -0400 From: David Laight To: "'Alexander Duyck'" , Nikita Yushchenko CC: Eric Dumazet , David Miller , Jeff Kirsher , intel-wired-lan , Netdev , "linux-kernel@vger.kernel.org" , Chris Healy Subject: RE: igb driver can cause cache invalidation of non-owned memory? Thread-Topic: igb driver can cause cache invalidation of non-owned memory? Thread-Index: AQHSJJ3zk6gikE37GUiDR2euMAz4tKCk9wtQ Date: Wed, 12 Oct 2016 16:03:39 +0000 Message-ID: <063D6719AE5E284EB5DD2968C1650D6DB01E6F0F@AcuExch.aculab.com> References: <0b57cbe2-84f7-6c0a-904a-d166571234b5@cogentembedded.com> <20161010.050125.1981283393312167625.davem@davemloft.net> <10474d19-df1a-3b09-917e-70659be3a56c@cogentembedded.com> <20161010.075731.2449861168238706.davem@davemloft.net> <19aebfc3-5f1a-9206-4493-2255af7269f9@cogentembedded.com> In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.202.99.200] Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 X-Outbound-IP: 213.249.233.130 X-Env-From: David.Laight@ACULAB.COM X-Proto: esmtps X-Revdns: X-HELO: AcuExch.aculab.com X-TLS: TLSv1:AES128-SHA:128 X-Authenticated_ID: X-PolicySMART: 3396946, 3397078 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by mail.home.local id u9CG45tp014242 Content-Length: 1415 Lines: 32 From: Alexander Duyck > Sent: 12 October 2016 16:33 ... > > To get some throughput improvement, I propose removal of that > > sync_for_device() before reusing buffer. Will you accept such a patch ;) > > Not one that gets rid of sync_for_device() in the driver. From what I > can tell there are some DMA APIs that use that to perform the > invalidation on the region of memory so that it can be DMAed into. > Without that we run the risk of having a race between something the > CPU might have placed in the cache and something the device wrote into > memory. The sync_for_device() call is meant to invalidate the cache > for the region so that when the device writes into memory there is no > risk of that race. I'm not expert, but some thought... Just remember that the cpu can do speculative memory and cache line reads. So you need to ensure there are no dirty cache lines when the receive buffer is setup and no cache lines at all at before looking at the frame. So unless you know the exact rules for these speculative cache line reads you have to invalidate the cache after the buffer is written to by the hardware even it was invalidated when the buffer was set up. If you can 100% guarantee the cpu hasn't dirtied the cache then I think the invalidate prior to reusing the buffer can be skipped. But I wouldn't want to debug that going wrong. Might be provable safe in the 'copybreak' path. David