Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32;
Date:   Mon, 25 Sep 2023 09:10:11 -0700
From:   Ira Weiny <ira.weiny@intel.com>
To:     Tomas Glozar <tglozar@redhat.com>, <nvdimm@lists.linux.dev>
CC:     <dan.j.williams@intel.com>, <vishal.l.verma@intel.com>,
        <dave.jiang@intel.com>, <ira.weiny@intel.com>,
        <linux-kernel@vger.kernel.org>, Tomas Glozar <tglozar@redhat.com>
Subject: Re: [PATCH v2] nd_btt: Make BTT lanes preemptible
Message-ID: <6511b0e3a4627_b00c32947a@iweiny-mobl.notmuch>
References: <20230920053712.499439-1-tglozar@redhat.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20230920053712.499439-1-tglozar@redhat.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?3t7M1PSi2oQZ9ZIjV2x/ujMuX66/PhgwbrKqxkGgBo7lUFole1gaYDeCtvMQ?=
 =?us-ascii?Q?J7YPwnMD6EaV/ufh39znuAVv08zQ2Q4AWW6QYAccGJH7nzsCkPP4IRS9zVQ3?=
 =?us-ascii?Q?xlJI6ln49+PSXTM4y8USXn3Xvo6q11y1+QmdOlnjZ0GpDLEWZrfylel2+GCH?=
 =?us-ascii?Q?326zwPFNv99A3SJ5MMjr9M44jViEMO8z7YaBuBsFatndvzB0xSEOiMkv9dXx?=
 =?us-ascii?Q?Rk9mTfWR7LK3jWGfpgyg7fFpH9lL5O4m7EBQdz1RQ4PMfidCkBhWr/NqwMRO?=
 =?us-ascii?Q?PoUxeD1LeyTpqTdjog+gNvuLy6B36sv388hqDqOxUWObdqTY512KIZ+yHI/u?=
 =?us-ascii?Q?92IOa/htvaJP371riRe0XPdkE35V6nD4R7mFISvOi/Ho3JY67/lhmOjqZbYU?=
 =?us-ascii?Q?T66EYuiZt07U9ktawOmSotOEZRdaFYZ2Y7VPakgw4VwqJw5+fIzSyep3btbc?=
 =?us-ascii?Q?lRzgSF1h19O0Csnr9xf+oz2UAP8ZidB9l0hYXf5a2QMwJiUj1ilEytSwEGPt?=
 =?us-ascii?Q?4vqLZ0GXcXwFy7+pnYurBtRjAueOTNhFguFWuDh2Dj0EgIDG7U0CSVMrIstN?=
 =?us-ascii?Q?KRfgiUwYVWCFEVDXRuz2FnyIdi4rDgwlHRRdpn3rCAqT0u6InIlGIPNxbCTU?=
 =?us-ascii?Q?Y4eua1LreEk3S4tcBL911qtdN2lF85OFoLrc4te6JmvVBFyxcC45rTscZba3?=
 =?us-ascii?Q?0HNbUNrbahZpVRfAcowNMDNiPM47bC5oSgqD2boCqoe1UIh5CwAKP+8ugySS?=
 =?us-ascii?Q?UhmMKvb2OQvbPeEfAzTULzVN8EhyLcPUloeD2j+gYbWobaqvuSOjQQ0WKLAx?=
 =?us-ascii?Q?aZOUGe46BdhTt1+sc6fX3aqiEXJbZ2ok7YF0QHTDzSGj2m3O1RIRZjLWnmCG?=
 =?us-ascii?Q?qfDAd/m/PuQZlwA96iekSjPzxmzmyl0+z8wH+3YppOOaM9EQ1R+Y/uFWBxAO?=
 =?us-ascii?Q?hlsOEQH6SluBNQndSlDYFM4Ry4Z8L9xfvHy6JHEdCHOM4Cg5o3Q1s0JdXQeW?=
 =?us-ascii?Q?DoXANtFZX37BGx8gZJQPfeb1K3EOB30+LkVye/q2S4pYSZZ6Ry5XBqbiLG+S?=
 =?us-ascii?Q?AwBF1eRwgW06zL4HEoAIMKIQWLopMmN33xzY2/tdvBdIukagHF6gLQPsvKZr?=
 =?us-ascii?Q?U1OlDBRwOxICZEioZk1msCMc9SLmAdWtOQmrPNtHOndSH/k+n9OKsSgYSC49?=
 =?us-ascii?Q?hLZY4SIq2YvaVmkyyHCgFGzmgI2TUzjgnrLgDnv80shL8EFr/kexiQf/X009?=
 =?us-ascii?Q?UdFb3druwBqUB7Ewbsp92jYZiuLVPrzCJA7goHplqlt6rFE0xToJPVDq/7eq?=
 =?us-ascii?Q?eZwtIT+XHcD6PjlOVhx+BOIQVaoK2tchToibU6YgkoUz7wcjrxNiXoUpjFLD?=
 =?us-ascii?Q?Z5F/LHsXBaSnHlstK0aV/kv4kBDcsgiFMPqlPqJ20fRvgp/0ke13fhuLkLZR?=
 =?us-ascii?Q?aVelbepBHujAsFIPbWoieBSDwtfn2kX5/trLeAmrhlh1NBquidxosqNQ4b8w?=
 =?us-ascii?Q?X0Ieii1vjiwkCE2b9WzbfstJC4w9FpausTv0SXoXpm+9kjsPNeByH8bfwje/?=
 =?us-ascii?Q?PLDtukp5E9vsurtXe6Pmsi17r3J+oyDHJasP7KyX?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 400e1fcd-2ce1-4d4d-86d5-08dbbde1e7e3
X-MS-Exchange-CrossTenant-AuthSource: SA1PR11MB6733.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Sep 2023 16:10:16.0923
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: RJWp580sd+rTg/oKeXhmXW7+0hvc95QIFJRub8xFSkyRhRjd2kMc2BHYZSaNGD0Cb4JevaFbc3DD+02PVPocOg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR11MB8279
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Mon, 25 Sep 2023 09:14:07 -0700 (PDT)

Tomas Glozar wrote:
> nd_region_acquire_lane uses get_cpu, which disables preemption. This is
> an issue on PREEMPT_RT kernels, since btt_write_pg and also
> nd_region_acquire_lane itself take a spin lock, resulting in BUG:
> sleeping function called from invalid context.
> 
> Fix the issue by replacing get_cpu with smp_process_id and
> migrate_disable when needed. This makes BTT operations preemptible, thus
> permitting the use of spin_lock.
> 
> BUG example occurring when running ndctl tests on PREEMPT_RT kernel:
> 
> BUG: sleeping function called from invalid context at
> kernel/locking/spinlock_rt.c:48
> in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 4903, name:
> libndctl
> preempt_count: 1, expected: 0
> RCU nest depth: 0, expected: 0
> Preemption disabled at:
> [<ffffffffc1313db5>] nd_region_acquire_lane+0x15/0x90 [libnvdimm]
> Call Trace:
>  <TASK>
>  dump_stack_lvl+0x8e/0xb0
>  __might_resched+0x19b/0x250
>  rt_spin_lock+0x4c/0x100
>  ? btt_write_pg+0x2d7/0x500 [nd_btt]
>  btt_write_pg+0x2d7/0x500 [nd_btt]
>  ? local_clock_noinstr+0x9/0xc0
>  btt_submit_bio+0x16d/0x270 [nd_btt]
>  __submit_bio+0x48/0x80
>  __submit_bio_noacct+0x7e/0x1e0
>  submit_bio_wait+0x58/0xb0
>  __blkdev_direct_IO_simple+0x107/0x240
>  ? inode_set_ctime_current+0x51/0x110
>  ? __pfx_submit_bio_wait_endio+0x10/0x10
>  blkdev_write_iter+0x1d8/0x290
>  vfs_write+0x237/0x330
>  ...
>  </TASK>
> 
> Fixes: 5212e11fde4d ("nd_btt: atomic sector updates")
> Signed-off-by: Tomas Glozar <tglozar@redhat.com>

Thanks for the clarification.

Reviewed-by: Ira Weiny <ira.weiny@intel.com>