Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 24 Aug 2022 22:05:31 -0700
From:   Dan Williams <dan.j.williams@intel.com>
To:     Dan Williams <dan.j.williams@intel.com>,
        HORIGUCHI =?utf-8?B?TkFPWUEo5aCA5Y+j44CA55u05LmfKQ==?= 
        <naoya.horiguchi@nec.com>
CC:     Shiyang Ruan <ruansy.fnst@fujitsu.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-xfs@vger.kernel.org" <linux-xfs@vger.kernel.org>,
        "nvdimm@lists.linux.dev" <nvdimm@lists.linux.dev>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
        "djwong@kernel.org" <djwong@kernel.org>,
        "david@fromorbit.com" <david@fromorbit.com>,
        "hch@infradead.org" <hch@infradead.org>,
        "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
        "jane.chu@oracle.com" <jane.chu@oracle.com>,
        "rgoldwyn@suse.de" <rgoldwyn@suse.de>,
        "viro@zeniv.linux.org.uk" <viro@zeniv.linux.org.uk>,
        "willy@infradead.org" <willy@infradead.org>,
        "linmiaohe@huawei.com" <linmiaohe@huawei.com>,
        Christoph Hellwig <hch@lst.de>
Subject: Re: [PATCH v2 05/14] mm: Introduce mf_dax_kill_procs() for fsdax case
Message-ID: <6307031b763be_18ed729455@dwillia2-xfh.jf.intel.com.notmuch>
References: <20220603053738.1218681-1-ruansy.fnst@fujitsu.com>
 <20220603053738.1218681-6-ruansy.fnst@fujitsu.com>
 <63069db388d43_1b3229426@dwillia2-xfh.jf.intel.com.notmuch>
 <20220824234142.GA850225@hori.linux.bs1.fc.nec.co.jp>
 <6306fbabab4cd_18ed7294e2@dwillia2-xfh.jf.intel.com.notmuch>
Content-Type: text/plain; charset="utf-8"
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <6306fbabab4cd_18ed7294e2@dwillia2-xfh.jf.intel.com.notmuch>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NHQwdUJoUEtYRTg3S3pFMTF4RVhWSTBIMmFoNGVVTisxczhOZXpDcUxQcjcv?=
 =?utf-8?B?ZkMvd3Q0M0Z0Ulhsdy92aUNtcVdzT2xDMjVHVHVLZlJBRElWcFFjTGtIYWFu?=
 =?utf-8?B?NnlhSU5LTEhKN1RLcXdQOW1rQTJiS250WkZtRHRhWFpxRVIzRmpCbW1wMjlj?=
 =?utf-8?B?dk8wbFQ3bkRsV0U0Zk1HdEtzeDJnR2JQMTI2aXQxa1RuU05OcHdCUjhFU25C?=
 =?utf-8?B?QlBZMVdZTDRpVnlTakN1YlAxWllRQnVnd3JaNnBseTJlcVlZSlpIUVVtSWk0?=
 =?utf-8?B?OTY2bTdKSUk0QUR5bklyY3h5L0UwUTlTR1JmVEd6QlBuWE0vSi9LT0dxWXg2?=
 =?utf-8?B?eXBNZXFqSzRqL1ByY2FBVGw5cTduVDhwd0pqVUk1cndhSnlFcDJsVTlJU1Ev?=
 =?utf-8?B?K2F1Mloyb0M1b3J4WEgxbGtFWXR2V1g5ZEdQdFNxTm9pSXFscldGYWFVVlA2?=
 =?utf-8?B?L1J6WVhvUTZWOUtzVk1wb21CNnpLRmpVeHBia3BUaEtVcG4zWEY5UlAzajBX?=
 =?utf-8?B?QVRaaEtJSkFnUVMvZjZtMWVZckhSN3JhNlNXam5ReUUvaVoyLzZoYmc1S1ZB?=
 =?utf-8?B?RlUxRm94WDIyVEFEUzM0dmV0UlZDQytNMW9FZnFrTWFhWHRvMmcvSnJtUUl0?=
 =?utf-8?B?Ny82ZncrWXQrL0U4UTE0ZmNtOHU5U2U1UWRWc2RQMW5vZDJYK1hyUjRKNUsx?=
 =?utf-8?B?T05vMDBpbXdQeXFPSEhKcHlML0hJSDVidXhmcTdxRjM3bGhlcTFsT2U2a0U2?=
 =?utf-8?B?VjdoL3hLeXpJWTR1K0JNWnRsZjJjQWJLUVVjRHFSU3YwR1U0bWFOVlJGYVI2?=
 =?utf-8?B?ZWs2QVBtTzR0bitrOWRuMnNKUzFTV245ZW54a2RJU0hOMXJhMWNabWRYVm9N?=
 =?utf-8?B?NlYwbXEwc3RSa20rbFhtL0tEUkE3ekZTMEJSQkVNbGpuNnUzdSt6NkxiODRP?=
 =?utf-8?B?Y21uNVFKMmtJVDdvTC9tblBJNlR4Q1oxNGpJck05N0VZZzlzTkRnY1NPT2lM?=
 =?utf-8?B?WlZDU3pBSVJ1TWZIUUtCRDZLelZIQm1DTzZHd3NIKzk5elVIZmxrbGhTRTR5?=
 =?utf-8?B?VVp1MS9oR1djMWJjZEVzSkZ5bE1yS1NMN3NzRUI3OVJCaGtIZER6T3puS3c0?=
 =?utf-8?B?YThNKzI5VUhCRUFFNXVMd1FSRmFyMGxDZTdacDc0b2F6WWRwNHlNOHZ5SFpi?=
 =?utf-8?B?TWRIenZvUjFsT0Z6TzFzZnd1a0V4OVlHbml0KzJWQXNZR3ptYnBJeEQ1UUhy?=
 =?utf-8?B?cTJ0c3dKNElUNGpQWjFHOVJMSGQ1TVdwTTZWWWZ2SzZHd3VXL2t6TVNGLzBn?=
 =?utf-8?B?NUxnUXgzbkoxWmF1MUNPcFpTcFJvemJ1UXRYR1Z6TTF5Ukg5Y3pPWVhMbzk3?=
 =?utf-8?B?emNyYmhBU3RSRUE4Nk1tVytTSGxuallkMUU0L0FxTUV3cEhpK1ZEM2FWS2ox?=
 =?utf-8?B?N21HTlBmeE9SMC9GUllZUGh3WCtVVEJscnJLYlU0anhOc2hVYlMvTERpd25T?=
 =?utf-8?B?dU5KZHdvVTN0OFR5U29xcGZ5VkVlcUZtRE9PSHo1UjY0czB5Qk1NdVc4b2VZ?=
 =?utf-8?B?UytGV1h2T2c5bGgvYXlEREt4ZXpRUGd2T2NpNUhaMXYwQXpvV05sdVEwcm0z?=
 =?utf-8?B?a0lJdkcraGZORXVJU2ZwTElHSk9XaHFVbW5KQ1c4WUhKVk9BNXpVZnNyY0s4?=
 =?utf-8?B?ZlFUMG9XdUd6OWxwejN1S2pDVlQyMCtuOFY3NkY2cVFBdFc0SnpKZDlaajdG?=
 =?utf-8?B?eTMrMjBGMEhJTE1nbW0yY29BU094ZUNoZXRGbnBQQzR6RG1zazNMVzBWV2ZT?=
 =?utf-8?B?ci9xV29zQnZVUDl6bW92alZneHJGUVZLVEFpaWhqMENHWUc3d1d6bTc1eUJ3?=
 =?utf-8?B?bnAyWlZYUFNxWGJJbFh3NmF3Z3A5Uk1zdVJtTkRWamptUFMreng4aVJjaUs0?=
 =?utf-8?B?MnV3V2sxT2JLY0RoMVJFTUNJaEIwaHJ0cGs5SlQ5Z1ZlSGI2dkF3aDFmczVz?=
 =?utf-8?B?OUs1a0RpaHdHc01tMitWaGNYRCtyajF4VzNFZVd6NXF2UU1BZW9Ua0JHN0U4?=
 =?utf-8?B?T3JmNm13empIOUdPU21iVjB4NW5nRmhPWVRVVklzeWNIZEVEcVg5NkZZdTRv?=
 =?utf-8?B?Z3Yranl5TXcycnhSNU54NHh0SlY5N1I3TnY1MTYrODk3UkIrdU5HNTNOV3VE?=
 =?utf-8?B?aEE9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 099141f1-7ba8-458b-ba8e-08da8657715d
X-MS-Exchange-CrossTenant-AuthSource: MWHPR1101MB2126.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Aug 2022 05:05:35.0778
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: MrFjZWIKQczwoR/w986n7XQYYIcMlVlQiOg13VLVvAl+QohYkz0gp/3SA8rLzJfDUc2hz7sk2JnzbyVGwh8pVL++pA+aglnJPTaUVyDZNDA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR11MB4325
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,
        SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Dan Williams wrote:
> HORIGUCHI NAOYA(堀口　直也) wrote:
> > On Wed, Aug 24, 2022 at 02:52:51PM -0700, Dan Williams wrote:
> > > Shiyang Ruan wrote:
> > > > This new function is a variant of mf_generic_kill_procs that accepts a
> > > > file, offset pair instead of a struct to support multiple files sharing
> > > > a DAX mapping.  It is intended to be called by the file systems as part
> > > > of the memory_failure handler after the file system performed a reverse
> > > > mapping from the storage address to the file and file offset.
> > > > 
> > > > Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
> > > > Reviewed-by: Dan Williams <dan.j.williams@intel.com>
> > > > Reviewed-by: Christoph Hellwig <hch@lst.de>
> > > > Reviewed-by: Darrick J. Wong <djwong@kernel.org>
> > > > Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>
> > > > ---
> > > >  include/linux/mm.h  |  2 +
> > > >  mm/memory-failure.c | 96 ++++++++++++++++++++++++++++++++++++++++-----
> > > >  2 files changed, 88 insertions(+), 10 deletions(-)
> > > 
> > > Unfortunately my test suite was only running the "non-destructive" set
> > > of 'ndctl' tests which skipped some of the complex memory-failure cases.
> > > Upon fixing that, bisect flags this commit as the source of the following
> > > crash regression:
> > 
> > Thank you for testing/reporting.
> > 
> > > 
> > >  kernel BUG at mm/memory-failure.c:310!
> > >  invalid opcode: 0000 [#1] PREEMPT SMP PTI
> > >  CPU: 26 PID: 1252 Comm: dax-pmd Tainted: G           OE     5.19.0-rc4+ #58
> > >  Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015
> > >  RIP: 0010:add_to_kill+0x304/0x400
> > > [..]
> > >  Call Trace:
> > >   <TASK>
> > >   collect_procs.part.0+0x2c8/0x470
> > >   memory_failure+0x979/0xf30
> > >   do_madvise.part.0.cold+0x9c/0xd3
> > >   ? lock_is_held_type+0xe3/0x140
> > >   ? find_held_lock+0x2b/0x80
> > >   ? lock_release+0x145/0x2f0
> > >   ? lock_is_held_type+0xe3/0x140
> > >   ? syscall_enter_from_user_mode+0x20/0x70
> > >   __x64_sys_madvise+0x56/0x70
> > >   do_syscall_64+0x3a/0x80
> > >   entry_SYSCALL_64_after_hwframe+0x46/0xb0
> > 
> > This stacktrace shows that VM_BUG_ON_VMA() in dev_pagemap_mapping_shift()
> > was triggered.  I think that BUG_ON is too harsh here because address ==
> > -EFAULT means that there's no mapping for the address.  The subsequent
> > code considers "tk->size_shift == 0" as "no mapping" cases, so
> > dev_pagemap_mapping_shift() can return 0 in such a case?
> > 
> > Could the following diff work for the issue?
> 
> This passes the "dax-ext4.sh" and "dax-xfs.sh" tests from the ndctl
> suite.
> 
> It then fails on the "device-dax" test with this signature:
> 
>  BUG: kernel NULL pointer dereference, address: 0000000000000010
>  #PF: supervisor read access in kernel mode
>  #PF: error_code(0x0000) - not-present page
>  PGD 8000000205073067 P4D 8000000205073067 PUD 2062b3067 PMD 0 
>  Oops: 0000 [#1] PREEMPT SMP PTI
>  CPU: 22 PID: 4535 Comm: device-dax Tainted: G           OE    N 6.0.0-rc2+ #59
>  Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015
>  RIP: 0010:memory_failure+0x667/0xba0
> [..]
>  Call Trace:
>   <TASK>
>   ? _printk+0x58/0x73
>   do_madvise.part.0.cold+0xaf/0xc5
> 
> Which is:
> 
> (gdb) li *(memory_failure+0x667)
> 0xffffffff813b7f17 is in memory_failure (mm/memory-failure.c:1933).
> 1928
> 1929            /*
> 1930             * Call driver's implementation to handle the memory failure, otherwise
> 1931             * fall back to generic handler.
> 1932             */
> 1933            if (pgmap->ops->memory_failure) {
> 1934                    rc = pgmap->ops->memory_failure(pgmap, pfn, 1, flags);
> 
> 
> ...I think this is just a simple matter of:
> 
> @@ -1928,7 +1930,7 @@ static int memory_failure_dev_pagemap(unsigned long pfn, int flags,
>          * Call driver's implementation to handle the memory failure, otherwise
>          * fall back to generic handler.
>          */
> -       if (pgmap->ops->memory_failure) {
> +       if (pgmap->ops && pgmap->ops->memory_failure) {
>                 rc = pgmap->ops->memory_failure(pgmap, pfn, 1, flags);
>                 /*
>                  * Fall back to generic handler too if operation is not
> 
> 
> ...since device-dax does not implement pagemap ops.
> 
> I will see what else pops up and make sure that this regression always
> runs going forward.

Ok, that was last of the regression fallout that I could find.