Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932896AbcKPO6X (ORCPT ); Wed, 16 Nov 2016 09:58:23 -0500 Received: from mail-qk0-f196.google.com ([209.85.220.196]:33992 "EHLO mail-qk0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752834AbcKPO6V (ORCPT ); Wed, 16 Nov 2016 09:58:21 -0500 MIME-Version: 1.0 In-Reply-To: <582C232F.6080205@redhat.com> References: <1479286950-21885-1-git-send-email-xlpang@redhat.com> <582C232F.6080205@redhat.com> From: Myron Stowe Date: Wed, 16 Nov 2016 07:58:20 -0700 Message-ID: Subject: Re: [PATCH] iommu/vt-d: Flush old iotlb for kdump when the device gets context mapped To: xlpang@redhat.com Cc: iommu@lists.linux-foundation.org, Joerg Roedel , Don Brace , Dave Young , kexec@lists.infradead.org, LKML , Myron Stowe , David Woodhouse Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3795 Lines: 73 On Wed, Nov 16, 2016 at 2:13 AM, Xunlei Pang wrote: > Ccing David > On 2016/11/16 at 17:02, Xunlei Pang wrote: >> We met the DMAR fault both on hpsa P420i and P421 SmartArray controllers >> under kdump, it can be steadily reproduced on several different machines, >> the dmesg log is like: >> HP HPSA Driver (v 3.4.16-0) >> hpsa 0000:02:00.0: using doorbell to reset controller >> hpsa 0000:02:00.0: board ready after hard reset. >> hpsa 0000:02:00.0: Waiting for controller to respond to no-op >> DMAR: Setting identity map for device 0000:02:00.0 [0xe8000 - 0xe8fff] >> DMAR: Setting identity map for device 0000:02:00.0 [0xf4000 - 0xf4fff] >> DMAR: Setting identity map for device 0000:02:00.0 [0xbdf6e000 - 0xbdf6efff] >> DMAR: Setting identity map for device 0000:02:00.0 [0xbdf6f000 - 0xbdf7efff] >> DMAR: Setting identity map for device 0000:02:00.0 [0xbdf7f000 - 0xbdf82fff] >> DMAR: Setting identity map for device 0000:02:00.0 [0xbdf83000 - 0xbdf84fff] >> DMAR: DRHD: handling fault status reg 2 >> DMAR: [DMA Read] Request device [02:00.0] fault addr fffff000 [fault reason 06] PTE Read access is not set >> hpsa 0000:02:00.0: controller message 03:00 timed out >> hpsa 0000:02:00.0: no-op failed; re-trying >> >> After some debugging, we found that the corresponding pte entry value >> is correct, and the value of the iommu caching mode is 0, the fault is >> probably due to the old iotlb cache of the in-flight DMA. >> >> Thus need to flush the old iotlb after context mapping is setup for the >> device, where the device is supposed to finish reset at its driver probe >> stage and no in-flight DMA exists hereafter. >> >> With this patch, all our problematic machines can survive the kdump tests. >> >> CC: Myron Stowe >> CC: Don Brace >> CC: Baoquan He >> CC: Dave Young >> Tested-by: Joseph Szczypek >> Signed-off-by: Xunlei Pang >> --- >> drivers/iommu/intel-iommu.c | 11 +++++++++-- >> 1 file changed, 9 insertions(+), 2 deletions(-) >> >> diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c >> index 3965e73..eb79288 100644 >> --- a/drivers/iommu/intel-iommu.c >> +++ b/drivers/iommu/intel-iommu.c >> @@ -2067,9 +2067,16 @@ static int domain_context_mapping_one(struct dmar_domain *domain, >> * It's a non-present to present mapping. If hardware doesn't cache >> * non-present entry we only need to flush the write-buffer. If the >> * _does_ cache non-present entries, then it does so in the special If this does get accepted then we should fix the above grammar also - "If the _does_ cache ..." -> "If the hardware _does_ cache ..." >> - * domain #0, which we have to flush: >> + * domain #0, which we have to flush. >> + * >> + * For kdump cases, present entries may be cached due to the in-flight >> + * DMA and copied old pgtable, but there is no unmapping behaviour for >> + * them, so we need an explicit iotlb flush for the newly-mapped device. >> + * For kdump, at this point, the device is supposed to finish reset at >> + * the driver probe stage, no in-flight DMA will exist, thus we do not >> + * need to worry about that anymore hereafter. >> */ >> - if (cap_caching_mode(iommu->cap)) { >> + if (is_kdump_kernel() || cap_caching_mode(iommu->cap)) { >> iommu->flush.flush_context(iommu, 0, >> (((u16)bus) << 8) | devfn, >> DMA_CCMD_MASK_NOBIT, > > _______________________________________________ > iommu mailing list > iommu@lists.linux-foundation.org > https://lists.linuxfoundation.org/mailman/listinfo/iommu