Received: by 2002:ab2:788f:0:b0:1ee:8f2e:70ae with SMTP id b15csp166373lqi; Wed, 6 Mar 2024 13:15:44 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWvmJzReKyYtgveRPjqwvEKsvdV4HMMuZzB+9p5A4Ae+5KgzWnanC8bEMsBnNoqgnC9XEg+ksjA+QQ3SZPg0gUzCCMyxN5CRHsZVn7XpQ== X-Google-Smtp-Source: AGHT+IHXECbY7zCDaWmlQBjwmoEwM2awE9GI7yduLtqnRioVDGIQ7aOLnEAv0e/a7M3lDzN/H6lz X-Received: by 2002:a25:b0a:0:b0:dcc:f3fe:19c with SMTP id 10-20020a250b0a000000b00dccf3fe019cmr14427860ybl.59.1709759743832; Wed, 06 Mar 2024 13:15:43 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1709759743; cv=pass; d=google.com; s=arc-20160816; b=HjVpELj6bsjfyOUrk5mre77lfukJYYekrtN17WNL+o7PS6O8LhKMzCUz5jHa3Ek45t u9jcFd6/K+UNad8FBkdKRTEP4dDnxcUur5ZgfZeEQnzBTp2gjwZXRHPAa3G5qCxIx3oo 4ieivm6hMOlzOAuo9YBoJRIVLXF0hNZatUw56CSq8Cuo5RaDCVDU/NlEH6W4L2Xc7D3J MbR0dRUWAeAtC5cAnYb5i5JKPOoQLpluz2AAVScZN3xudc6mfquFj/wbV1P0v/nmW7MN AOzE+HquOvHY1VuInZf17xRHS/ntQUq+IeIl3Kc4EBa0MyB/ybQ1kEOLVgMQoBW+LEz6 6Y2w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=RvIF0BmF0Hlx3vvmjm5B0ksVs7bzrQ0T7kjSFQLHKuY=; fh=MHJMY1O5a4FZClgQgDgk5IZb0BNqewtXeQgzD9Gumuk=; b=mpoq4cs+qXMa4slCukdlmStn9h8X8fVSctgzb42fOl9Z8oIfzYOcX+KUYG2GQ0UD3Y VrF6p9pu3ly9vQm7AycrnFpedVSf+vThjro1FCq2kEcwoWISiNfGimArJ7Ws/CiSTmUs vqfKhRFYbULBngwLtNzX4RCoRoZLu3/rEADSdsAIh/KMNkuS8sSajYQCk6xk2a587U9o F0N369hGLvKfGWbiLJEzHxqfssDjtbWKXRrsKrbogx86cNUXmBhhfTO3U7N5yAQnEcBD nQeOyRNymT1a25/OWKoUfsPYf1kkC3OPmurmvQLoOG16ioE1pZhACfSe6X4ZE9qPar1R VRaA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b="EWGPcl9/"; arc=pass (i=1 spf=pass spfdomain=redhat.com dkim=pass dkdomain=redhat.com dmarc=pass fromdomain=redhat.com); spf=pass (google.com: domain of linux-kernel+bounces-94581-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-94581-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id h15-20020a05622a170f00b0042ee54e8abdsi9439103qtk.516.2024.03.06.13.15.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Mar 2024 13:15:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-94581-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b="EWGPcl9/"; arc=pass (i=1 spf=pass spfdomain=redhat.com dkim=pass dkdomain=redhat.com dmarc=pass fromdomain=redhat.com); spf=pass (google.com: domain of linux-kernel+bounces-94581-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-94581-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 8A0381C211F0 for ; Wed, 6 Mar 2024 21:15:43 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id ED3201BC39; Wed, 6 Mar 2024 21:15:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="EWGPcl9/" Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DCAC818EAF for ; Wed, 6 Mar 2024 21:15:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709759704; cv=none; b=HZO/8neojlFOG8WtWEi8ayVJpp6gMo/GGUU11mFG94sjkGhxjo6upP3OXPlJIdi/BudMdlyMEMqNffR/DZOdLeZI/Jp+StmbbRq6klpnasm9Rv/CAPfeUtlTW0ika7p4nRG061Xz8CvjuXQLGFODEPb37U3sEybk9E3k87dLn9Y= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709759704; c=relaxed/simple; bh=Im61rSYEkW0FepGWtFi39YlGVNasXF2NmzZls9nGJi0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ji5EEBXU9GTX6OP+k2cS3MObZURp6rE6JxQQN2mDjD4DIZCxAThG5PdbUuMPJP4MnjcgmiiP55Xw43iceymIM+Nl4DDlSioaQl3antuoutP35P0Z20XxSDbetwA2JkGwsmhZZY44F+XhIcUyldPdvBgL/7gBNsEQBs/FghSaeqc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=EWGPcl9/; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1709759700; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RvIF0BmF0Hlx3vvmjm5B0ksVs7bzrQ0T7kjSFQLHKuY=; b=EWGPcl9/F7mEimSEt1h4rqWghwfE6yH/mkEHpoqyGmx9mg/OMu7oSJV5q+J0aWWViBvo9J M1wzVqPQCrnseV5ZyJUnJazqmBLKUZcOwoS9L4wn2ZKTnYs3YGfpWezYh2EO7mybGAskzL ZwV0K2aMzbzoIWgQAn83S/i5NjsPWw4= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-546-eBBsEAi9N--yA91LR8-X4A-1; Wed, 06 Mar 2024 16:14:57 -0500 X-MC-Unique: eBBsEAi9N--yA91LR8-X4A-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5ADB5386A0A2; Wed, 6 Mar 2024 21:14:57 +0000 (UTC) Received: from omen.home.shazbot.org (unknown [10.22.33.99]) by smtp.corp.redhat.com (Postfix) with ESMTP id 60F0E37F6; Wed, 6 Mar 2024 21:14:56 +0000 (UTC) From: Alex Williamson To: alex.williamson@redhat.com Cc: kvm@vger.kernel.org, eric.auger@redhat.com, clg@redhat.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org, kevin.tian@intel.com Subject: [PATCH 2/7] vfio/pci: Lock external INTx masking ops Date: Wed, 6 Mar 2024 14:14:37 -0700 Message-ID: <20240306211445.1856768-3-alex.williamson@redhat.com> In-Reply-To: <20240306211445.1856768-1-alex.williamson@redhat.com> References: <20240306211445.1856768-1-alex.williamson@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Mask operations through config space changes to DisINTx may race INTx configuration changes via ioctl. Create wrappers that add locking for paths outside of the core interrupt code. In particular, irq_type is updated holding igate, therefore testing is_intx() requires holding igate. For example clearing DisINTx from config space can otherwise race changes of the interrupt configuration. This aligns interfaces which may trigger the INTx eventfd into two camps, one side serialized by igate and the other only enabled while INTx is configured. A subsequent patch introduces synchronization for the latter flows. Fixes: 89e1f7d4c66d ("vfio: Add PCI device driver") Reported-by: Reinette Chatre Signed-off-by: Alex Williamson --- drivers/vfio/pci/vfio_pci_intrs.c | 34 +++++++++++++++++++++++++------ 1 file changed, 28 insertions(+), 6 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 136101179fcb..75c85eec21b3 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -99,13 +99,15 @@ static void vfio_send_intx_eventfd(void *opaque, void *unused) } /* Returns true if the INTx vfio_pci_irq_ctx.masked value is changed. */ -bool vfio_pci_intx_mask(struct vfio_pci_core_device *vdev) +static bool __vfio_pci_intx_mask(struct vfio_pci_core_device *vdev) { struct pci_dev *pdev = vdev->pdev; struct vfio_pci_irq_ctx *ctx; unsigned long flags; bool masked_changed = false; + lockdep_assert_held(&vdev->igate); + spin_lock_irqsave(&vdev->irqlock, flags); /* @@ -143,6 +145,17 @@ bool vfio_pci_intx_mask(struct vfio_pci_core_device *vdev) return masked_changed; } +bool vfio_pci_intx_mask(struct vfio_pci_core_device *vdev) +{ + bool mask_changed; + + mutex_lock(&vdev->igate); + mask_changed = __vfio_pci_intx_mask(vdev); + mutex_unlock(&vdev->igate); + + return mask_changed; +} + /* * If this is triggered by an eventfd, we can't call eventfd_signal * or else we'll deadlock on the eventfd wait queue. Return >0 when @@ -194,12 +207,21 @@ static int vfio_pci_intx_unmask_handler(void *opaque, void *unused) return ret; } -void vfio_pci_intx_unmask(struct vfio_pci_core_device *vdev) +static void __vfio_pci_intx_unmask(struct vfio_pci_core_device *vdev) { + lockdep_assert_held(&vdev->igate); + if (vfio_pci_intx_unmask_handler(vdev, NULL) > 0) vfio_send_intx_eventfd(vdev, NULL); } +void vfio_pci_intx_unmask(struct vfio_pci_core_device *vdev) +{ + mutex_lock(&vdev->igate); + __vfio_pci_intx_unmask(vdev); + mutex_unlock(&vdev->igate); +} + static irqreturn_t vfio_intx_handler(int irq, void *dev_id) { struct vfio_pci_core_device *vdev = dev_id; @@ -563,11 +585,11 @@ static int vfio_pci_set_intx_unmask(struct vfio_pci_core_device *vdev, return -EINVAL; if (flags & VFIO_IRQ_SET_DATA_NONE) { - vfio_pci_intx_unmask(vdev); + __vfio_pci_intx_unmask(vdev); } else if (flags & VFIO_IRQ_SET_DATA_BOOL) { uint8_t unmask = *(uint8_t *)data; if (unmask) - vfio_pci_intx_unmask(vdev); + __vfio_pci_intx_unmask(vdev); } else if (flags & VFIO_IRQ_SET_DATA_EVENTFD) { struct vfio_pci_irq_ctx *ctx = vfio_irq_ctx_get(vdev, 0); int32_t fd = *(int32_t *)data; @@ -594,11 +616,11 @@ static int vfio_pci_set_intx_mask(struct vfio_pci_core_device *vdev, return -EINVAL; if (flags & VFIO_IRQ_SET_DATA_NONE) { - vfio_pci_intx_mask(vdev); + __vfio_pci_intx_mask(vdev); } else if (flags & VFIO_IRQ_SET_DATA_BOOL) { uint8_t mask = *(uint8_t *)data; if (mask) - vfio_pci_intx_mask(vdev); + __vfio_pci_intx_mask(vdev); } else if (flags & VFIO_IRQ_SET_DATA_EVENTFD) { return -ENOTTY; /* XXX implement me */ } -- 2.43.2