Received: by 2002:a25:c593:0:0:0:0:0 with SMTP id v141csp382481ybe; Mon, 2 Sep 2019 03:12:46 -0700 (PDT) X-Google-Smtp-Source: APXvYqzDL19x8qtXMeuy8fWY7/DCxVtVndkiaiJBAXFWbYShI8nqXNW7EjrBoyNHbMFKrmbT9ooB X-Received: by 2002:a17:90a:ac0a:: with SMTP id o10mr12413737pjq.143.1567419165929; Mon, 02 Sep 2019 03:12:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1567419165; cv=none; d=google.com; s=arc-20160816; b=fEdSMUOFa7O2B+kBY0kGS7fVmQ5xVKfUy+xKLNZ184ItCTMROzNCVQxXkJukojvDHi u1osZ4I86UN/g8ji02vvxbiZ1C8EThz9YUtvzRAiK8hVy/pPlh/DRdYoOxMVch7bTDF5 6LYBeeQ8qOs7XrGe8ST/H0fqqCUmcG7hPpt//5vzROIvWIFWk4Fj9Kqofdkj5ZUaH6ZP PUninZ4W4eFuQPjWX4Ev0cV48vkcsKRJKcKWkv7O1eSxDhZtm9rZRzAgdE2lFoVZZiiw dR7Z7R7OdYMKtE/PhIN8l2EXkyH58zQdfifnPbvFQ9XW5JU9j4BxRd91mOI9PxsgVaZ0 i77A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:organization:message-id:date:subject:cc:to :from; bh=0lFI0wU93hwfFSUw6WN4YebKpBPaL0b+kP5Zwd6fT0I=; b=Qo+Cg7rnl8hjJyeYa0jifVLfBWMSNZxGeFKmbheJ+aHPKRsgmVBRP9lQbju2/oNDZy dnG3uHAaFRq3q6Ov4dN6Zx/HTZ6s6m6JAEQBADPd/Tx5phjCxNdbDwh39QJ01lYEan3A zCKq2iA5Rcz2Oqe94NE7uTfViJqFmgKtjpYkb27ChV9vvOmIMMHDfUGHpNxurcKuhhe2 PumtoXG4EQU3/1FrgWLD/3vBE/q+LCkDZETiwb6ArxuAc5MEcfV2yNDDfnEEZcz+gYgC 2NFdBE9vm07xxQVR4nHgPvQbtheCFfmY8osxQF2lB5r3OcgfIzA+eM0qCp3XPWVwJjPm gHsA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id s17si5666423pfc.224.2019.09.02.03.12.31; Mon, 02 Sep 2019 03:12:45 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730349AbfIBIiD (ORCPT + 99 others); Mon, 2 Sep 2019 04:38:03 -0400 Received: from mga17.intel.com ([192.55.52.151]:25522 "EHLO mga17.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729756AbfIBIiD (ORCPT ); Mon, 2 Sep 2019 04:38:03 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 02 Sep 2019 01:38:02 -0700 X-IronPort-AV: E=Sophos;i="5.64,457,1559545200"; d="scan'208";a="184442382" Received: from jkrzyszt-desk.ger.corp.intel.com ([172.22.244.17]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 02 Sep 2019 01:38:00 -0700 From: Janusz Krzysztofik To: Lu Baolu Cc: David Woodhouse , Joerg Roedel , iommu@lists.linux-foundation.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, =?utf-8?B?TWljaGHFgg==?= Wajdeczko , Janusz Krzysztofik Subject: Re: [RFC PATCH] iommu/vt-d: Fix IOMMU field not populated on device hot re-plug Date: Mon, 02 Sep 2019 10:37:41 +0200 Message-ID: <1769080.0GM3UzqXcv@jkrzyszt-desk.ger.corp.intel.com> Organization: Intel Technology Poland sp. z o.o. - ul. Slowackiego 173, 80-298 Gdansk - KRS 101882 - NIP 957-07-52-316 In-Reply-To: References: <20190822142922.31526-1-janusz.krzysztofik@linux.intel.com> <3255251.C7nBVfOIaa@jkrzyszt-desk.ger.corp.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Baolu, On Thursday, August 29, 2019 11:08:18 AM CEST Lu Baolu wrote: > Hi, > > On 8/29/19 3:58 PM, Janusz Krzysztofik wrote: > > Hi Baolu, > > > > On Thursday, August 29, 2019 3:43:31 AM CEST Lu Baolu wrote: > >> Hi Janusz, > >> > >> On 8/28/19 10:17 PM, Janusz Krzysztofik wrote: > >>>> We should avoid kernel panic when a intel_unmap() is called against > >>>> a non-existent domain. > >>> Does that mean you suggest to replace > >>> BUG_ON(!domain); > >>> with something like > >>> if (WARN_ON(!domain)) > >>> return; > >>> and to not care of orphaned mappings left allocated? Is there a way to > > inform > >>> users that their active DMA mappings are no longer valid and they > > shouldn't > >>> call dma_unmap_*()? > >>> > >>>> But we shouldn't expect the IOMMU driver not > >>>> cleaning up the domain info when a device remove notification comes and > >>>> wait until all file descriptors being closed, right? > >>> Shouldn't then the IOMMU driver take care of cleaning up resources still > >>> allocated on device remove before it invalidates and forgets their > > pointers? > >>> > >> > >> You are right. We need to wait until all allocated resources (iova and > >> mappings) to be released. > >> > >> How about registering a callback for BUS_NOTIFY_UNBOUND_DRIVER, and > >> removing the domain info when the driver detachment completes? > > > > Device core calls BUS_NOTIFY_UNBOUND_DRIVER on each driver unbind, regardless > > of a device being removed or not. As long as the device is not unplugged and > > the BUS_NOTIFY_REMOVED_DEVICE notification not generated, an unbound driver is > > not a problem here. > > Morever, BUS_NOTIFY_UNBOUND_DRIVER is called even before > > BUS_NOTIFY_REMOVED_DEVICE so that wouldn't help anyway. > > Last but not least, bus events are independent of the IOMMU driver use via > > DMA-API it exposes. > > Fair enough. > > > > > If keeping data for unplugged devices and reusing it on device re-plug is not > > acceptable then maybe the IOMMU driver should perform reference counting of > > its internal resources occupied by DMA-API users and perform cleanups on last > > release? > > I am not saying that keeping data is not acceptable. I just want to > check whether there are any other solutions. Then reverting 458b7c8e0dde and applying this patch still resolves the issue for me. No errors appear when mappings are unmapped on device close after the device has been removed, and domain info preserved on device removal is successfully reused on device re-plug. Is there anything else I can do to help? Thanks, Janusz > > Best regards, > Baolu >