Received: by 2002:a05:6a10:206:0:0:0:0 with SMTP id 6csp102581pxj; Thu, 3 Jun 2021 01:51:48 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzii4FgD0GDqwZNCpA1ICGa2NFI208uZp2YsdFeo4EESzWam+U8GMyLzONJPcKtnBetb6YY X-Received: by 2002:aa7:d5c6:: with SMTP id d6mr6430598eds.290.1622710308766; Thu, 03 Jun 2021 01:51:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1622710308; cv=none; d=google.com; s=arc-20160816; b=CRtlCrZBqaz/MNuPbS6U0XH2Qj37DeDMS6+w04K8tmkK8R/egiVROIT/UL2QJy/Bnh vQBAdh/Md3iPv76LAyWeFdb/nZcc0odSskVubaeaCQE+n/Vp5BzoUmSNHLWyHdnaHG+2 cNSg45B11tdmOMjDcqFvXMDtd215nrKXLFRFSoDk6n72PW5bc888X1L5+LhPma0FhfVi bmL9PhEL+WILyILhyrtLER7njGviPUiDG98Zaj82c1f78b+AmHk5iIrRqBOP00GnO8zf +tVh/6Ij2FABH55mtHTe3avBhf1kFavSuIZ9fZ3gmL43YuQmZ5PiLLZaXac2veRXnX16 nbxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature:dkim-signature; bh=SeU5hLmakRpJp0wYHm9YpfQgd++fWAu9b5BVuriF3f4=; b=DNPQBp7myXF57yd6OdJrif4vvm6TPyE2Dyd+UVyWQNBzrBuDlH6Cj5nJGRAm5ujEKX t1qNsJc/rUTUMGyYoDImXISLoNVPjBm1YXmyUiVl81ubacRnHo2vQDUZ3r8B8aipz3vY BtF06SC0rfpYUYPVKirywnZN4Moc9Szu2wKD53I48E6/GSnWb9TNtOnKOBpE2Vqjkuic NDV1Sb19ulo5vUITVYlOnlUtYqYWbNHfZfpedl/ubPzVsDsLw+g/avrGPqBdwF81shBj DZ5HPl8WhiDKTzv2jEKDqmPiWtjKynFHlX3tOwSX/AiWowMwbmEdQJ7Mx1dHTaQLR6DV wCXw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@svenpeter.dev header.s=fm1 header.b=mJIhPobR; dkim=pass header.i=@messagingengine.com header.s=fm3 header.b=JtyiJHH5; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=svenpeter.dev Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id h8si1864424edz.253.2021.06.03.01.51.25; Thu, 03 Jun 2021 01:51:48 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@svenpeter.dev header.s=fm1 header.b=mJIhPobR; dkim=pass header.i=@messagingengine.com header.s=fm3 header.b=JtyiJHH5; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=svenpeter.dev Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229744AbhFCIwN (ORCPT + 99 others); Thu, 3 Jun 2021 04:52:13 -0400 Received: from new2-smtp.messagingengine.com ([66.111.4.224]:47361 "EHLO new2-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229486AbhFCIwM (ORCPT ); Thu, 3 Jun 2021 04:52:12 -0400 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailnew.nyi.internal (Postfix) with ESMTP id 7583E580F07; Thu, 3 Jun 2021 04:50:27 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Thu, 03 Jun 2021 04:50:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=svenpeter.dev; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; s=fm1; bh=SeU5hLmakRpJp0wYHm9YpfQgd+ +fWAu9b5BVuriF3f4=; b=mJIhPobRGuJNTiSMlnBfbnv2uDMGCqnD7gmJHZQMNK 7Z2AjI2AVt6uHOqCkE3j2XIC9i7qeUJiS3MjEnQfDygKdBRz5TPDccNA7GmnuFV/ /HEqH8B8JAfFB7VYfqUflkIauwTmTJCKeJ/hNrgBTVWiNxi+SuzFgOQ4ppnKtMWR bQmVZpWS+pyw3m/PUddbEfEJ7loVibUL4Si5W5dNV2Wi8nTEbC5OedahkQWxQQ3B 4KOdd+Nd+FnKosQ9UG3Edae6vTMn2A1hF4zBPemAsX8LXuzaLuH7aXL1Zt8CQXmj s/aZWqgPK6xEucyZeEGF9veFagkH49OTuD0ovhUKhjmw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :message-id:mime-version:subject:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=SeU5hLmakRpJp0wYH m9YpfQgd++fWAu9b5BVuriF3f4=; b=JtyiJHH5XvQhUlW67thTo5Q2G6H5TMbZ8 O5xzZF9Y6BW5OW/h0CNR0akpLpzDG7WZZRkqHER812z9MXDGX/X38iTtJ5g4HcCf Hxrlq596I1EiTDVZUMkPjcGE8nNET4fYjvNQFkFEosuxlZGwLJvGrMTJBxpg/SIn 4OS+LgGT9CmExRszynNz2ul/MrCIk9zCWdeMhWAYVaETafPw/9Pl2NN5SU+B4pNY k8zmc55XsJlAFMFzLZcdRX7F34ozb/qX87I0dyTTi5ItIZFqYw+dkCgsWhwJLVeY L2nVIiq9AzZEJ7xR6FNtAziKf+vccAAOrMzsWTWU9OMOGHfSArGvQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrvdelledgtdeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkffoggfgsedtkeertdertddtnecuhfhrohhmpefuvhgvnhcurfgv thgvrhcuoehsvhgvnhesshhvvghnphgvthgvrhdruggvvheqnecuggftrfgrthhtvghrnh epfeehudeftddvhfehvdduhedtjeejheeuudfftdfgvdekvdelleeuveelgfeflefgnecu ffhomhgrihhnpehkvghrnhgvlhdrohhrghenucevlhhushhtvghrufhiiigvpedtnecurf grrhgrmhepmhgrihhlfhhrohhmpehsvhgvnhesshhvvghnphgvthgvrhdruggvvh X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 3 Jun 2021 04:50:23 -0400 (EDT) From: Sven Peter To: Will Deacon , Robin Murphy , Joerg Roedel , Rob Herring Cc: Sven Peter , Arnd Bergmann , devicetree@vger.kernel.org, Hector Martin , linux-kernel@vger.kernel.org, Marc Zyngier , Mohamed Mediouni , Stan Skowronek , linux-arm-kernel@lists.infradead.org, Mark Kettenis , iommu@lists.linux-foundation.org, Alexander Graf Subject: [PATCH v3 0/3] Apple M1 DART IOMMU driver Date: Thu, 3 Jun 2021 10:50:00 +0200 Message-Id: <20210603085003.50465-1-sven@svenpeter.dev> X-Mailer: git-send-email 2.30.1 (Apple Git-130) MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi, This is v3 of my Apple M1 DART IOMMU driver series as a follow up to the original two versions [1][2]. Short summary: this series adds support for the iommu found in Apple's new M1 SoC which is required to use DMA on most peripherals. So far this code has been tested with dwc3 in host and device mode and PCIe on a M1 Mac Mini. This IOMMU comes with a hard-wired pagesize of 16K. This makes booting a kernel with 4K page challenging. For dwc3 this is no issue: As long as the iommu is set to bypass mode dwc3 works just fine. Translated mode isn't supported right now. The most controversial part on which I'd like to get feedback are the PCIe DARTs. These DARTs do not support hardware bypass mode and also limit the iova space to 32 bit. To still allow booting on kernels with a 4K pagesize I have introduced the following hack: I program a static pagetable that maps the entire 32bit iova space to the first 4GB of RAM starting at 0x8_0000_0000. Combined with an appropriate dma-ranges property in the pcie node this allows to create a fake bypass mode and successfully enables PCIe. Right now the RAM offset is hardcoded in the DART driver and this will likely have to change if this workaround is acceptable for now. I could either introduce a separate property for the iommu node or try to grab the offset from the dma-ranges once the first device is attached. Changes for v3: - fixed name of the iommu node in the device tree binding example pointed out by Arnd Bergmann - remove hardware specific checks from io-pgtable.c as pointed out by Will Deacon - introduced a fake bypass mode by programming static linear pagetables if the DART does not support regular bypass mode as proposed by Alex Graf - added checks to enforce bypass mode if there is a pagesize mismatch between the DART HW and the CPU. - fixed usage of GFP_KERNEL during a held spinlock found by Julia Lawall - rebased on v5.13-rc3 Changes for v2: - fixed devicetree binding linting issues pointed out by Rob Herring and reworked that file. - made DART-specific code in io-pgtable.c unconditional and removed flag from Kconfig as proposed by Robin Murphy. - allowed multiple DART nodes in the "iommus" property as proposed by Rob Herring and Robin Murphy. this resulted in significant changes to apple-iommu-dart.c. - the domain aperture is now forced to 32bit if translation is enabled after the original suggestion to limit the aperture by Mark Kettenis and the follow-up discussion and investigation with Mark Kettenis, Arnd Bergmann, Robin Murphy and Rob Herring. This change also simplified the code in io-pgtable.c and made some of the improvements suggested during review not apply anymore. - added support for bypassed and isolated domain modes. - reject IOMMU_MMIO and IOMMU_NOEXEC since it's unknown how to set these up for now or if the hardware even supports these flags. - renamed some registers to be less confusing (mainly s/DOMAIN/STREAM/ to prevent confusion with linux's iommu domain concept). [1] v1: https://lore.kernel.org/linux-iommu/20210320151903.60759-1-sven@svenpeter.dev/ [2] v2: https://lore.kernel.org/linux-iommu/20210328074009.95932-1-sven@svenpeter.dev/ Sven Peter (3): iommu: io-pgtable: add DART pagetable format dt-bindings: iommu: add DART iommu bindings iommu: dart: Add DART iommu driver .../devicetree/bindings/iommu/apple,dart.yaml | 81 ++ MAINTAINERS | 7 + drivers/iommu/Kconfig | 15 + drivers/iommu/Makefile | 1 + drivers/iommu/apple-dart-iommu.c | 966 ++++++++++++++++++ drivers/iommu/io-pgtable-arm.c | 62 ++ drivers/iommu/io-pgtable.c | 1 + include/linux/io-pgtable.h | 7 + 8 files changed, 1140 insertions(+) create mode 100644 Documentation/devicetree/bindings/iommu/apple,dart.yaml create mode 100644 drivers/iommu/apple-dart-iommu.c -- 2.25.1