Received: by 2002:a25:8b91:0:0:0:0:0 with SMTP id j17csp3974497ybl; Mon, 9 Dec 2019 03:19:26 -0800 (PST) X-Google-Smtp-Source: APXvYqymePweu+iA+u7a3f5sNRXFeGxviZrjQW+zM9iedmMB2UqZq8K12/CS3gfSEVkKwq04GJYw X-Received: by 2002:a05:6830:1116:: with SMTP id w22mr21415505otq.216.1575890366182; Mon, 09 Dec 2019 03:19:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1575890366; cv=none; d=google.com; s=arc-20160816; b=qfD9TpmA8ELhBa+kjCJSDZy3WgrpESFAPrd9vs4XulNEG0zU2Tf/MAdr5vyR+vJ+YH 4ieIkTvpHtBasMrWf6f42X6OeSf3sh3TF6XNNlAyrUKAegLnoRMARdUYqaMsxQToXTpp bfnm4kFjk5AI1a+jJsohOSYApfdqHfOTmTodL910tai88VfNJ9rcuJ2pFIChtIFC64LJ e7GHcpR5OsnjhsBDbaxpj6uhRuyWFLuUUPS9l4XdfzoIUt+LVwdyvABEqdnpJUf/gwP/ iJv7QpRwQd8w5+Wiwusm3Po7WUEib9xTnpepo7kl7QW4ZBNJFc7mT0pzoOxjppNFLeiN y5kA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:cc:to:subject :message-id:date:from:in-reply-to:references:mime-version :dkim-signature; bh=gePB6fSq/rlJBPu1erNdnBIWumv6Nl4xKIQxuD48vdE=; b=zb5fUwE/mnERdOoDKv83MA60JktW6qQ03Hzq7OxQQBFDCoO5Abrrt5EPsxhhCFqlBX NY5bh0hKC4vr1Dmsqq2RQFVkmStvnh59XFd++DA+hq+cWvxik9sFBk5P+pyTaWdkB4Jb FYJBkkLH5HyiLUdOlCZReZeEPwB9oLrKWMkz1YMuDceuIshfRMsBnoV5twyBEJU4IRal 4t403vMMD8YKsjurdWYcbeQhW7uHrzuXgyPnYnVrPwu69q4gWPsyvjuj+8TvErS7k7zK ZUqtd3KrcRf3cLewaTT0Z/8Rg/EqNX+5nGQ1rfmuc9BnVyFaOWa6pkiqaP+nJUs4vYoC +trA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=OsFJjiQH; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id d17si11568552oij.136.2019.12.09.03.19.13; Mon, 09 Dec 2019 03:19:26 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=OsFJjiQH; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727566AbfLILR1 (ORCPT + 99 others); Mon, 9 Dec 2019 06:17:27 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:39995 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727074AbfLILR1 (ORCPT ); Mon, 9 Dec 2019 06:17:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1575890246; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gePB6fSq/rlJBPu1erNdnBIWumv6Nl4xKIQxuD48vdE=; b=OsFJjiQHnTxrupAUDyHY4Bhs+Yqlfh4ym4Gg3ObMxuSjxtkCT967bulI7xqhqy5SnQ49QB MNDOEJJkSnfLQYTyRcAoNnQ5+5q6CNv6YhWtOSxvwtQYLWtS+rMF6L2j/Uze8S/rzp2lqr Ziyfy9iUxSqP4O66qxvwYuNaLyozFhU= Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-335-F3FgXZhFNMSaPeiyv34R1A-1; Mon, 09 Dec 2019 06:17:22 -0500 Received: by mail-qt1-f200.google.com with SMTP id o24so10817538qtr.17 for ; Mon, 09 Dec 2019 03:17:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=CWTaEBFOGggC4ce4fgk8cEZpKpRsvl0bTsvJF/j06c4=; b=NOmgLfytOJA76ueRPq3Cy2YNJ69lEqlhH31vFNd7x0uok9k4CuK2QCzQADF6jF7k80 EMGRU3nbWGsSIckAKBmidxkCPhqMQ2jQhjmYBbRrOEer2IzxqN9KMF+DwA4QAF5o496a Etnpig54IRWANADj8pjK2B8qIYNesuO1UHf2jvWlI0QS/oSeXB29Fgn29vIokbQI5Zff xHJ9lmZwbFV4bQNMtFB5SL14C6O7ykugQ1pMKGyDZwqaTTpKi6dNeM7L62xaKw5Esf4w CkOotssfMMhcbJHq7vQKhz0FqnUnNA6dqSkhdKkwn2kB0HKBjextypeCWJpHzNQaP2CG WoAg== X-Gm-Message-State: APjAAAWHfDO/m3CQN09AFrC8ss2T+BQ2uQ8ZTDD02i9Jh26zxbSHPUZV a3QGezccBjRA9+biPxKIqlWXzjmuVB+rb3ajhCjeLlf040TANaDf8XJdtF1KeJjxPk2XtpxyON4 jSq/rIHhuJE7Ne+fPUZmvwX8YEwEdvrhL40QXda0M X-Received: by 2002:aed:2ae7:: with SMTP id t94mr24558830qtd.130.1575890242233; Mon, 09 Dec 2019 03:17:22 -0800 (PST) X-Received: by 2002:aed:2ae7:: with SMTP id t94mr24558808qtd.130.1575890241963; Mon, 09 Dec 2019 03:17:21 -0800 (PST) MIME-Version: 1.0 References: <20191121112821.GU11621@lahna.fi.intel.com> <20191121114610.GW11621@lahna.fi.intel.com> <20191127114856.GZ11621@lahna.fi.intel.com> In-Reply-To: From: Karol Herbst Date: Mon, 9 Dec 2019 12:17:10 +0100 Message-ID: Subject: Re: [PATCH v4] pci: prevent putting nvidia GPUs into lower device states on certain intel bridges To: Lyude Paul Cc: Mika Westerberg , "Rafael J. Wysocki" , Bjorn Helgaas , LKML , "Rafael J . Wysocki" , Linux PCI , Linux PM , dri-devel , nouveau , Dave Airlie , Mario Limonciello X-MC-Unique: F3FgXZhFNMSaPeiyv34R1A-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org anybody any other ideas? It seems that both patches don't really fix the issue and I have no idea left on my side to try out. The only thing left I could do to further investigate would be to reverse engineer the Nvidia driver as they support runpm on Turing+ GPUs now, but I've heard users having similar issues to the one Lyude told us about... and I couldn't verify that the patches help there either in a reliable way. On Wed, Nov 27, 2019 at 8:55 PM Lyude Paul wrote: > > On Wed, 2019-11-27 at 12:51 +0100, Karol Herbst wrote: > > On Wed, Nov 27, 2019 at 12:49 PM Mika Westerberg > > wrote: > > > On Tue, Nov 26, 2019 at 06:10:36PM -0500, Lyude Paul wrote: > > > > Hey-this is almost certainly not the right place in this thread to > > > > respond, > > > > but this thread has gotten so deep evolution can't push the subject > > > > further to > > > > the right, heh. So I'll just respond here. > > > > > > :) > > > > > > > I've been following this and helping out Karol with testing here an= d > > > > there. > > > > They had me test Bjorn's PCI branch on the X1 Extreme 2nd generatio= n, > > > > which > > > > has a turing GPU and 8086:1901 PCI bridge. > > > > > > > > I was about to say "the patch fixed things, hooray!" but it seems t= hat > > > > after > > > > trying runtime suspend/resume a couple times things fall apart agai= n: > > > > > > You mean $subject patch, no? > > > > > > > no, I told Lyude to test the pci/pm branch as the runpm errors we saw > > on that machine looked different. Some BAR error the GPU reported > > after it got resumed, so I was wondering if the delays were helping > > with that. But after some cycles it still caused the same issue, that > > the GPU disappeared. Later testing also showed that my patch also > > didn't seem to help with this error sadly :/ > > > > > > [ 686.883247] nouveau 0000:01:00.0: DRM: suspending object tree... > > > > [ 752.866484] ACPI Error: Aborting method \_SB.PCI0.PEG0.PEGP.NVPO= due > > > > to previous error (AE_AML_LOOP_TIMEOUT) (20190816/psparse-529) > > > > [ 752.866508] ACPI Error: Aborting method \_SB.PCI0.PGON due to > > > > previous error (AE_AML_LOOP_TIMEOUT) (20190816/psparse-529) > > > > [ 752.866521] ACPI Error: Aborting method \_SB.PCI0.PEG0.PG00._ON = due > > > > to previous error (AE_AML_LOOP_TIMEOUT) (20190816/psparse-529) > > > > > > This is probably the culprit. The same AML code fails to properly tur= n > > > on the device. > > > > > > Is acpidump from this system available somewhere? > > Attached it to this email > > > > > -- > Cheers, > Lyude Paul