Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34;
Message-ID: <f7ee378e-6fd4-620f-abf9-2a8aa3643f94@amd.com>
Date:   Wed, 4 Oct 2023 08:26:26 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.15.1
Subject: Re: dma_map_resource() has a bad performance in pcie peer to peer
 transactions when iommu enabled in Linux
Content-Language: en-US
To:     Robin Murphy <robin.murphy@arm.com>,
        Kelly Devilliv <kelly.devilliv@outlook.com>,
        "joro@8bytes.org" <joro@8bytes.org>,
        "will@kernel.org" <will@kernel.org>
Cc:     "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
References: <TY2PR0101MB3136A7480B842E009BCBDA8384C3A@TY2PR0101MB3136.apcprd01.prod.exchangelabs.com>
 <13d73200-2d53-7f95-fb93-5b5f422251cd@arm.com>
From:   =?UTF-8?Q?Christian_K=c3=b6nig?= <christian.koenig@amd.com>
In-Reply-To: <13d73200-2d53-7f95-fb93-5b5f422251cd@arm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MmcxUkQ1Y0pNNmtsWnM5V2pNck03SmtlUlYvbUVHY0RQeGRxODBNd1pEU2xq?=
 =?utf-8?B?Y24rRmZkenc5dmZGUGYzTDU2SHdRNjZEc2J3NXY3ODAzM1JDSWRINHZzcUxz?=
 =?utf-8?B?WS83Sll2UGxvbXpVV2I3eWYrWU9yTGV6NWdZR3VNdjRhdlJ0b2dQUXpHa0JY?=
 =?utf-8?B?ODU0YTFCaTZzTG1EUTh0ekRzSUs5V1BvWFgzT2ZyajIzcTdkOUdxUUttaURu?=
 =?utf-8?B?eFZHMUlLTHFkWVBEN0tHQWg0R0dERysxb3doWENTMmRzaWVLNXhkNWdLcVdK?=
 =?utf-8?B?bmpjeXRkZFJMUURGNWdjY3ZHb3drSWlPbndPOWl5RXVZTGI2UHpLdHpnV1Zw?=
 =?utf-8?B?cWhodGRNeEd5MEVuVndFMHVtK1c5QXlFYnBGckE1REFza2JEZnBwYU1jcUF2?=
 =?utf-8?B?RTBZU24zOFNlUUl6bHE3R0Nwcm9sQ3BoNzJINlY4OUZncUNRaUgzZmlSYnNx?=
 =?utf-8?B?RWVScVlhSXNpUWtzL1hJc1hISzh0RmkwUWpVYWpCQTgzYXFxZG1YOElYQ1hP?=
 =?utf-8?B?SlpKbVBDSm9TeUs3SGoydG1QYVFTWmFXeTBxb2xXL2hUcndKdWlLN1BYVS9U?=
 =?utf-8?B?cWp0dU9mSlR4a25yTHlCRnkwQ2VhdEI4R0s1bytnQjRlMU1XTGVIYkVHMHhR?=
 =?utf-8?B?NUsrK3gwTGxleGRKRis5ZnBzWGpValFSNHdaeURTNERxRkZVcXBVZkthaVNt?=
 =?utf-8?B?RE43U3RYZ2locXY0OUY3OWRTblN1WmQxYXJRT3dHR3V5Wndpc1BCU0VXUWJk?=
 =?utf-8?B?Nml2L0dSMUxEbm5ZT2JwcWhUcE5lWFNFOGgzZy9DanExQTJrWFNlYnl2aVIw?=
 =?utf-8?B?UlQyckxrUVFlS0VsWDM1bUIwK3BhMFJWOHdqZ1ZnUUJxWW0yS2VxaW40V0hQ?=
 =?utf-8?B?dlB5U0ozZFFjZkNHZUM2MnZlVUNqR3c1bXNzQkx0bTZkOW8vZWp6Qk5iVjlx?=
 =?utf-8?B?dXRvVWNna0hSK1ptUVNhd1pKYTQ1THAxZ1lHZzhVNWZrNWozNTBDbVJ4bVB2?=
 =?utf-8?B?QkVtd1ZONENUS1NMOHFRNDV6NlIxM1FMOWhaU0hFWVJtY3JTa2N6TVlLc0tB?=
 =?utf-8?B?OUxYdGtPRHNCckpWV3dWRHc3ay9PZHp1ZG1yVThSczBDcTRhSHowZXQ2TzVy?=
 =?utf-8?B?TGtOZXEvWnhpb3FVclZROTRtRnFEeHZvL2RHMW1IQkFYdnBBKzBQSXFlcjBS?=
 =?utf-8?B?ZzhWYnZBZENwTkp3TjF6aGZwUWQwL0w4cmIrV1MrT09uOTlzWWVMMjNNc0l5?=
 =?utf-8?B?a205TXlZMk1GNHJjWGhHcU9hVDFRZHArdWFBbDk3QXh2bjVXY2hZS0NlWEVi?=
 =?utf-8?B?WjZoYkU5SVhjRHliR2JRN1hoa2YydE5URis4Z2tCRnlCdHc0Z2lHTVI3bnl1?=
 =?utf-8?B?bEtKRDdYdlR1RVJRMHE5emMxMUJiN1BHYnA2aE95bHltMEhSb0hvcktuemhF?=
 =?utf-8?B?eHhDQUR6VVNpTCt2UUYyZmNwS3lNYjEvTkhHaG9OSUxoczY3OWJWa0h6K284?=
 =?utf-8?B?UkNaNU5oNEU4SThJYk1QOWZkc1UzbWVFMWgyU0xpclRtSXJheFU5SUdRZGlL?=
 =?utf-8?B?SkZkRHNBblh3OUhhVklYVWpBc1YxOFlsbG9MVWZKTThtUURYY1N1Y0FrUWl2?=
 =?utf-8?B?NlM1b0NTZ0tHK2hjdi95NWVhcFhGeHp5VThyMXpUZDk1SUxxYk5JVk5CU3hQ?=
 =?utf-8?B?UTd0MHN2dno0SVB3QXhQWkRYei9wbFJwaXArQmtjZGFWOEtWT3RtQUw1amdM?=
 =?utf-8?B?M2IzTHpTY0NHZ3hTdDhDNnBBUzU4d29TMEFhYlRXdjBLOXhadkpRbmFEQTVE?=
 =?utf-8?B?TlI5eVRuS1JDTE5uRWdlMDRwVkpPWWFxRllDdjRpMFVoRlVXbFM3b1ZKY2sx?=
 =?utf-8?B?QTBDRm53dlFFM1AvMkNZTzEzcTRNb0N3NG5ndlZkYnAvZW1WaHYwZEJvT2d1?=
 =?utf-8?B?NTBPc2hiUWJlc1BpWldrTGJGbFZrOVJxcWlmLzFEQXVJM2lRbnZ2ZU1BS3Rh?=
 =?utf-8?B?bWExQjNnelZxajd1ZnBWYklPK1E5cElERlJGZ1BMeW9paTRpT0V1MzV3T1ZB?=
 =?utf-8?B?aGR2YVhmaGlGV1BEcm42aU80ZkVFSlU3WGg1UzdKNERSNGFtM0c1ZHFzNTZI?=
 =?utf-8?Q?gO1c=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ecc8777c-778b-4292-fef1-08dbc4a2d87a
X-MS-Exchange-CrossTenant-AuthSource: BN8PR12MB3587.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Oct 2023 06:26:30.0922
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: VD673md6vsCpXpRcgwT24ZTGp7RU8hE7cQPzHb/dbgOSdak+fNa7Kb/CYBhmd/9k
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6331
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        NICE_REPLY_A,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,
        SPF_NONE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Tue, 03 Oct 2023 23:26:41 -0700 (PDT)

Am 02.10.23 um 22:08 schrieb Robin Murphy:
> On 2023-09-26 16:30, Kelly Devilliv wrote:
>> [SNIP]
>> Hi Robin,
>>
>> Is there any chance to extend the dma_map_resource() API as discussed 
>> above?
>
> As Christoph says, no. There are things one could do to make a 
> minimal-effort bodge in a downstream kernel, but upstream we already 
> have a dedicated PCI peer-to-peer API, so we have no reason and no 
> desire to also attempt to crowbar P2P support into a different API 
> which isn't designed for it. Sure there exist some drivers that went 
> ahead of the game and did their own thing before we realised that 
> dma_map_resource() just fundamentally wouldn't be a good fit for P2P 
> as initially suggested, but all that means is that if they're still 
> doing that today, they're now lagging behind and it's on them to 
> update to the newer solution if they want to benefit from all its 
> goodness. Note that this isn't just maintainer semantics or relatively 
> straightforward things like memory attributes; I believe the proper 
> API can also handle stuff like direct P2P when you do have an IOMMU 
> but don't have ACS upstream redirect, which dma_map_resource() could 
> never do.

The problem is that the new API requires to have struct pages for PCIe 
resources which the graphics drivers absolutely don't want to do.

Allocating struct pages for memory mapped I/O simply doesn't make to 
much sense when the underlying resource is not even memory.

Regards,
Christian.

>
> Thanks,
> Robin.