Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 18 Nov 2022 10:43:17 -0800
From:   Ira Weiny <ira.weiny@intel.com>
To:     David Laight <David.Laight@aculab.com>
CC:     Bjorn Helgaas <bhelgaas@google.com>,
        Dan Williams <dan.j.williams@intel.com>,
        Bjorn Helgaas <helgaas@kernel.org>,
        Gregory Price <gregory.price@memverge.com>,
        Jonathan Cameron <Jonathan.Cameron@huawei.com>,
        Lukas Wunner <lukas@wunner.de>,
        Vishal Verma <vishal.l.verma@intel.com>,
        "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-cxl@vger.kernel.org" <linux-cxl@vger.kernel.org>
Subject: Re: [PATCH V2] PCI/DOE: Detect on stack work items automatically
Message-ID: <Y3fSRbpjeC1CTPA1@iweiny-desk3>
References: <20221118000524.1477383-1-ira.weiny@intel.com>
 <e59f83f3ca4149d098efe43b48fecd1b@AcuMS.aculab.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <e59f83f3ca4149d098efe43b48fecd1b@AcuMS.aculab.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?5j+Po0k5OF46Dp+UDAlfDSMGNqvb/UZtvDGBRjIYhQC05Rgr+UoiuRnYGoh/?=
 =?us-ascii?Q?l4AQuWpOTfz5HeOka32JJNSyTvA88ZddyU4jEQHvMHCSpgAzTx+0RzLGt3oE?=
 =?us-ascii?Q?sfybnuI7u3j+y8QW7X3c02UNoLG/fPTabIJj51c7zVvM3kguiipdBgp2yyKE?=
 =?us-ascii?Q?aPCtsUwbYMYkJgAJpXpG6vRrfGKxDcRRE6GQFn09iFGgrDm9h3gzCRQ+9XkV?=
 =?us-ascii?Q?d9ySkoq4l9jqqyvbs0cdRWD0oRVcf9spIvceODWip1GBOPoqIPFneO6PyKTn?=
 =?us-ascii?Q?Q/sYiiSIbe1EkOF3EB6eoWX34XjtBYya8z1CJ4/atF3nhkI773CK1hvcMC/T?=
 =?us-ascii?Q?uy2kPbgwxH1VdAbD73BV9zMmqbt15u0LcdgzDyszhZw0UKEngPbPr6WjaQtH?=
 =?us-ascii?Q?5x4V7YGIu2zCh1APR+6a103QUyZ0eGBspV2JyFVzaPQfQnT7J6iKe64bsQTi?=
 =?us-ascii?Q?P96IrQcGwHNrJC7Me+4Wz1C3jpI/fkaRP2Yjiw2HnuXHiBkgODphOz/QTsFG?=
 =?us-ascii?Q?RgHj0prqSi5h6Eds1JeNZq3sbj8FXBy3QmjYF+g8iB9I/5xzElnSMAKuHBvU?=
 =?us-ascii?Q?936xjL4oncqw7OXwom6yqPsAau+CU31qnRi1ObWRXvVkSBTamhSPepMcfJid?=
 =?us-ascii?Q?dwJ5X8UfinlNIV+O+YpiE0j+H147VlYxcyPsj1diJTC4jq9IwtHBznJaUnfv?=
 =?us-ascii?Q?1bQ8r7UyT08FAhI1KaI2rfByQZ1HXsoWTmj9mPxANmKQOq1O5PLFRhx6U3df?=
 =?us-ascii?Q?wQ4iIIPrSJkXpy8+3L2h08czrnUE0IFi8CAiy9/S1NO1mpy82dAhGvng3Ngr?=
 =?us-ascii?Q?w94a43bNWgfHeRMyttzzBtErqfF/+XR+GQLHR46fAqoOPiYgRMIoa307yoK9?=
 =?us-ascii?Q?ATVXF/8U4/kWEQV9Rx55iX8Mg7k8uhgy5Yjw5rqEPholWaSYd3J9+NIlHUNg?=
 =?us-ascii?Q?uitgLDE1CPCEE+9caHFePJuuo91/9FOH4paDnY4teG0uufCagKY+apX+4OFV?=
 =?us-ascii?Q?TUWqG5eRzJ0LtHGCxgZAo18dtR1TWutzAOpyFjIX845ALIbd4zL0QaRdXTKg?=
 =?us-ascii?Q?CUxoaV3ijbSw/BbMfUyZE+pdTERedAI38XfkLOWlqI8MPF3X0CHolVdx+je+?=
 =?us-ascii?Q?I70lIhpIFEoU1g0HZ/40yvIrqGcQG/m0wdYR8n7bCYQypmjj0dRKiGGBotlT?=
 =?us-ascii?Q?nJKMJ6ARA0hDn9ND/sJBsczQIoOGxWxVXkIyUvffdmIDWxAFnN4c+bUAciye?=
 =?us-ascii?Q?vVlQ7LQiFP8GrwPJ63j4AbSkg9MWaI2n00UydC77SAXFn3kvYJdlr/meVKiy?=
 =?us-ascii?Q?Pq+XACtAoKCgAY84BNgoQFzCUFkWlYyiZVXjCkzojzPQXyMaZax93CfSjh6Y?=
 =?us-ascii?Q?39ztGo0suNIQc8qrzZ0sm7H3Mr1DCg+dOjZ2R02zu8vun3PPdg1orZhJfMd3?=
 =?us-ascii?Q?lkHPNGxot4EGqSzzaBamn3qknSIdk64QMN1Hyx2TOGhvZq2cqjrZH7IlY+gZ?=
 =?us-ascii?Q?U0pS8s1m2zgtZYsCEi7DMbTPsDnKOflLvd/Lyv2SC70f7+/pZ+B1ZPY9egX/?=
 =?us-ascii?Q?rlDk5PGTLBeyxDZRZyB5aAvW3pX85r6hrX/TNdV9?=
X-MS-Exchange-CrossTenant-Network-Message-Id: d0989892-ecc5-4e85-e115-08dac994c51e
X-MS-Exchange-CrossTenant-AuthSource: SA1PR11MB6733.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Nov 2022 18:43:22.8011
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 84qS8s2OCS1s0MdQ7rT7KawXhP1LD3tx7cD8zl+n7g9XTE7MzYgusGPWNyuU4NtAHFG9LIxTKjX6OSPugaWNDg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR11MB5778
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,
        SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Nov 18, 2022 at 09:20:38AM +0000, David Laight wrote:
> From: ira.weiny@intel.com
> > Sent: 18 November 2022 00:05
> > 
> > Work item initialization needs to be done with either
> > INIT_WORK_ONSTACK() or INIT_WORK() depending on how the work item is
> > allocated.
> > 
> > The callers of pci_doe_submit_task() allocate struct pci_doe_task on the
> > stack and pci_doe_submit_task() incorrectly used INIT_WORK().
> > 
> > Jonathan suggested creating doe task allocation macros such as
> > DECLARE_CDAT_DOE_TASK_ONSTACK().[1]  The issue with this is the work
> > function is not known to the callers and must be initialized correctly.
> > 
> > A follow up suggestion was to have an internal 'pci_doe_work' item
> > allocated by pci_doe_submit_task().[2]  This requires an allocation which
> > could restrict the context where tasks are used.
> > 
> > Another idea was to have an intermediate step to initialize the task
> > struct with a new call.[3]  This added a lot of complexity.
> > 
> > Lukas pointed out that object_is_on_stack() is available to detect this
> > automatically.
> > 
> > Use object_is_on_stack() to determine the correct init work function to
> > call.
> 
> This is all a bit strange.
> The 'onstack' flag is needed for the diagnostic check:
> 	is_on_stack = object_is_on_stack(addr);
> 	if (is_on_stack == onstack)
> 		return;
> 	pr_warn(...);
> 	WARN_ON(1);
> 

:-(

> So setting the flag to the location of the buffer just subverts the check.
> It that is sane there ought to be a proper way to do it.

Ok this brings me back to my previous point and suggested patch.[*]  The
fundamental bug is that the work item is allocated in different code from
the code which uses it.  Separating the work item from the task.

[*] https://lore.kernel.org/linux-cxl/20221014151045.24781-1-Jonathan.Cameron@huawei.com/T/#m63c636c5135f304480370924f4d03c00357be667

Bjorn would this solution be acceptable and just use GFP_KERNEL and mark the
required context for pci_doe_submit_task()?

> OTOH using an on-stack structure for INIT_WORK seems rather strange.
> Since the kernel thread must sleep waiting for the 'work' to complete
> why not just perform the required code there.

It is not strange if some task submitters want to wait while others do not.  It
was suggested that all submit task operations be async and the callers who
wanted to be synchronous would wait like this.

As Dan said there is a difference between submit_bio() and submit_bio_wait().

We have simply left the wait part up to the users who all wait right now.

> 
> Also you really don't want to OOPS with anything from the stack
> linked into global kernel data structures.

I'm not following what you mean here.  I'm not seeing anything like this in the
current code nor any of the solutions suggested.

Ira

> While wait queues are pretty limited in scope and probably ok,
> this looks like a big accident waiting to happen.
> 
> 	David
> 
> -
> Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
> Registration No: 1397386 (Wales)
>