Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <bd43b7c9-a710-7251-46e0-b2c31b27d812@oracle.com>
Date:   Mon, 12 Dec 2022 05:57:02 -0800
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0)
 Gecko/20100101 Thunderbird/102.5.1
Subject: Re: [PATCH 1/1] NFSD: fix use-after-free in __nfs42_ssc_open()
Content-Language: en-US
To:     Jeff Layton <jlayton@kernel.org>, chuck.lever@oracle.com
Cc:     kolga@netapp.com, hdthky0@gmail.com, linux-nfs@vger.kernel.org,
        security@kernel.org
References: <1670786549-27041-1-git-send-email-dai.ngo@oracle.com>
 <7f68cb23445820b4a1c12b74dce0954f537ae5e2.camel@kernel.org>
 <56b0cb4f-dfe9-6892-7fef-1a2965cf1d99@oracle.com>
 <0ab8efccae708faa092a56c6935c33564814bfed.camel@kernel.org>
From:   dai.ngo@oracle.com
In-Reply-To: <0ab8efccae708faa092a56c6935c33564814bfed.camel@kernel.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NDRrbUlRaCtNR1M2NFFZWjVqaHZKSTBkWEd4K1ZxODBjU1JlWjhSeFZOQ2Fv?=
 =?utf-8?B?eEQxZ1NxYUtLSFNqTFdjNURIZ2szWGN5NHloUlp1eWl2azJEL2xnYnpNbDBE?=
 =?utf-8?B?UHA4RTFqc3B1b0hoeDN5Nmp0SlhYNjhuUWgrcDNGN0VUUENRWjBWL3JDV0xm?=
 =?utf-8?B?NlZCbEc3Nms4K29ORDA5WFB0ODI2eXNLK0hva0JSekQweUNYdmkrUWVlWjh5?=
 =?utf-8?B?b251VHdoV1k4ajZuR1RrSGFHcVhNb1UzajYzT2did3FqV0dJaWd6UnBVL3ZY?=
 =?utf-8?B?M2thMk9sSnBYMGUzZnQ3S1A1WFVoUzIvRjhGanhDVUtZSW1VWHlGTUloQUxY?=
 =?utf-8?B?K3Boa25XUkRJVmYyYWVUZFBEK0dhRXJMTDlnQW9NenFBT1JwcHdNaFZmWkhq?=
 =?utf-8?B?eEp4NjdQKzVBR2R0ZHdYaXpmZk9GWE5TQ1ZVb1BWTnpFTmdTeFQ4RFhvQkx2?=
 =?utf-8?B?cUM3RHYraElYdmJmRHNoblJ2UjZEUzRjQ1F5djFPUERTejkxUFlBRW1HQk5P?=
 =?utf-8?B?L3h6UXgrejFKZ3VQOE1iUnNWOXFJQ3JXUnNreTdtM3ZnODFoMkJGS3FOekgv?=
 =?utf-8?B?VVJxQTZmMXZvRWtYdWVhVVN0dTBwSnk1VkVWVnNxbEtmYjgvR2lSeS9vVk00?=
 =?utf-8?B?VmFGRm1nc2ovRXFDRC9iOE54SDlCVUJZbVdzWUlrSVZudHFlMXZGWGt3c204?=
 =?utf-8?B?SFJqRXk4QnF3eEV1Ymg4ZGlxWUtpZ21Tbzd5UTRoaXZmVGZKM3JOZXhnQktQ?=
 =?utf-8?B?WjNlR3VqZnBMYW5IaFdOdXVFL094SmUvMSswdm9wWnltZGIwK3lPV3A0SFF1?=
 =?utf-8?B?SmErTkxTaW02R3pGZUNIWEk4cjNTNFQ3aUs0dnJzb3BqNXdSM2dBWjVHRmln?=
 =?utf-8?B?WWRkWkpUdVNQZCs4Yk1XcmFhb3BRR2gyRHg4Z1pWT3FXc1FjODVpdmFmQ1B5?=
 =?utf-8?B?TVE3T0t4WE0rV0VnVTRtSWI1Q2VWRTJkMndmNWwyTlBGRVVEbnBsNlk5K1cy?=
 =?utf-8?B?TEx4UlNqN3hpRGlTNzNQMzRyNWY4UEhvQ1cxZlRXSGpvdWY2dFNGOUR1bnpG?=
 =?utf-8?B?b0JFMnM1VVF4V1gyUWkvaEd2Y0dXdXhpekNTVHFmNWxwSnV4WFV0QXk5VXRR?=
 =?utf-8?B?NzlwVjZnS0RJRmpoclVHWVRaNW1RT0twY3N6MWJQMFYxeWd1TXdGdWpPQ2lx?=
 =?utf-8?B?TzJnNGdhV3ZZbzBhSlkzdndLZG9KTmM1OEF3cUxKS29lVGhFQk1JQ29nRlBC?=
 =?utf-8?B?UHNzODJmRUViUElpVmhJSEdlajR1OUxDb1MyM3pjOEl1Q1BRNWZzam9zdnh4?=
 =?utf-8?B?N3VRa3dBNzFtTStkQlFacnpYZW16VjZVTHpFeU90RGd0bWJyNFZGLzhVR1Rz?=
 =?utf-8?B?S09QU0VIYjB5VEtMdDVyVmdReTMva0hHTlVpU1FQQlFXVVNucUNFL3dubGVh?=
 =?utf-8?B?NE1uL1lTc2lieFdkSHRTT3hFTXFCSGl1OVlJZUJQZ0U3eHFuVlhQTjg3WjZZ?=
 =?utf-8?B?TTNJaXgvSnpzSmhuWjhlajhtSFc4aDdkZVdQOXF2Ky83WWdNQnQ2bXdOTTJX?=
 =?utf-8?B?cUpaWWFYbDJScVpkdjRieGpuVk1qUlZOTjJ5SzV4Yk5PeFFSb1ZqaSthSklq?=
 =?utf-8?B?STRnMGNCSEt4ZXFab1VEY2xLNlozdmpGWmJtT2FjQS9iYzRQTngrV1YyQWdo?=
 =?utf-8?B?NmM1b0FyS1hnd3RWY3BpSkpTSkI3WkhIdHZyMWZTWjhNVEd5TEhhbjRNR3Rj?=
 =?utf-8?B?cFJCOEZtU0p5c0svTlhJUVdMOUhzZU9mRmdqV29EMmphVjNpcmpvd29FdGll?=
 =?utf-8?B?bkg1V0pCNVc0cXdxaVQ5RTRqV0JINWo3dFNtNERhT2p6YVRMcWpnNE5lNWMw?=
 =?utf-8?B?N0RnUENFQlAyY1U2VHlvdEw3K29JNDlOSE1OOVZOQzFMaUl2UDBPVGFPQ3pR?=
 =?utf-8?B?L1Q2elk4QXpGajhuZmtORzlxNmprVW5heTNjU3dGN0lLUEpsU1BiMmp2OHI2?=
 =?utf-8?B?RXRETGR5NkJDcGErUEo0SldTa2xYTWdhY2picXVuQndrNkxrTVY5N1hFRUZJ?=
 =?utf-8?B?N1plaDJTTjNCaDVIaXlSbmJ3RjJVZ1BxUDFPenRuWVhRNmtvTklEVndlS0ZP?=
 =?utf-8?Q?J9FgI3VVJv9UcF4q6Jfz13BF9?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fc5c570d-8edc-4996-ea7c-08dadc48c09f
X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4257.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Dec 2022 13:57:05.5525
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: AwRKGzUUI6QWe8MBQEl9nBAFFaErhKpNFkdzLygvFx55K2NKNGAVy48zqFDGGeueMP9AR5F1zjuTt5+a1VtflA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR10MB7054
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1
 definitions=2022-12-12_02,2022-12-12_01,2022-06-22_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 phishscore=0
 adultscore=0 bulkscore=0 spamscore=0 suspectscore=0 mlxlogscore=999
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2210170000 definitions=main-2212120128
X-Proofpoint-ORIG-GUID: 57DlRLEgqjGjrYzo0JATbMgzdfoYwuwV
X-Proofpoint-GUID: 57DlRLEgqjGjrYzo0JATbMgzdfoYwuwV
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


On 12/12/22 5:40 AM, Jeff Layton wrote:
> On Mon, 2022-12-12 at 05:34 -0800, dai.ngo@oracle.com wrote:
>> On 12/12/22 4:22 AM, Jeff Layton wrote:
>>> On Sun, 2022-12-11 at 11:22 -0800, Dai Ngo wrote:
>>>> Problem caused by source's vfsmount being unmounted but remains
>>>> on the delayed unmount list. This happens when nfs42_ssc_open()
>>>> return errors.
>>>> Fixed by removing nfsd4_interssc_connect(), leave the vfsmount
>>>> for the laundromat to unmount when idle time expires.
>>>>
>>>> Reported-by: Xingyuan Mo <hdthky0@gmail.com>
>>>> Signed-off-by: Dai Ngo <dai.ngo@oracle.com>
>>>> ---
>>>>    fs/nfsd/nfs4proc.c | 23 +++++++----------------
>>>>    1 file changed, 7 insertions(+), 16 deletions(-)
>>>>
>>>> diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
>>>> index 8beb2bc4c328..756e42cf0d01 100644
>>>> --- a/fs/nfsd/nfs4proc.c
>>>> +++ b/fs/nfsd/nfs4proc.c
>>>> @@ -1463,13 +1463,6 @@ nfsd4_interssc_connect(struct nl4_server *nss, struct svc_rqst *rqstp,
>>>>    	return status;
>>>>    }
>>>>    
>>>> -static void
>>>> -nfsd4_interssc_disconnect(struct vfsmount *ss_mnt)
>>>> -{
>>>> -	nfs_do_sb_deactive(ss_mnt->mnt_sb);
>>>> -	mntput(ss_mnt);
>>>> -}
>>>> -
>>>>    /*
>>>>     * Verify COPY destination stateid.
>>>>     *
>>>> @@ -1572,11 +1565,6 @@ nfsd4_cleanup_inter_ssc(struct vfsmount *ss_mnt, struct file *filp,
>>>>    {
>>>>    }
>>>>    
>>>> -static void
>>>> -nfsd4_interssc_disconnect(struct vfsmount *ss_mnt)
>>>> -{
>>>> -}
>>>> -
>>>>    static struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
>>>>    				   struct nfs_fh *src_fh,
>>>>    				   nfs4_stateid *stateid)
>>>> @@ -1762,7 +1750,8 @@ static int nfsd4_do_async_copy(void *data)
>>>>    		struct file *filp;
>>>>    
>>>>    		filp = nfs42_ssc_open(copy->ss_mnt, &copy->c_fh,
>>>> -				      &copy->stateid);
>>>> +					&copy->stateid);
>>>> +
>>>>    		if (IS_ERR(filp)) {
>>>>    			switch (PTR_ERR(filp)) {
>>>>    			case -EBADF:
>>>> @@ -1771,7 +1760,7 @@ static int nfsd4_do_async_copy(void *data)
>>>>    			default:
>>>>    				nfserr = nfserr_offload_denied;
>>>>    			}
>>>> -			nfsd4_interssc_disconnect(copy->ss_mnt);
>>>> +			/* ss_mnt will be unmounted by the laundromat */
>>>>    			goto do_callback;
>>>>    		}
>>>>    		nfserr = nfsd4_do_copy(copy, filp, copy->nf_dst->nf_file,
>>>> @@ -1852,8 +1841,10 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>>>>    	if (async_copy)
>>>>    		cleanup_async_copy(async_copy);
>>>>    	status = nfserrno(-ENOMEM);
>>>> -	if (nfsd4_ssc_is_inter(copy))
>>>> -		nfsd4_interssc_disconnect(copy->ss_mnt);
>>>> +	/*
>>>> +	 * source's vfsmount of inter-copy will be unmounted
>>>> +	 * by the laundromat
>>>> +	 */
>>>>    	goto out;
>>>>    }
>>>>    
>>> This looks reasonable at first glance, but I have some concerns with the
>>> refcounting around ss_mnt elsewhere in this code. nfsd4_ssc_setup_dul
>>> looks for an existing connection and bumps the ni->nsui_refcnt if it
>>> finds one.
>>>
>>> But then later, nfsd4_cleanup_inter_ssc has a couple of cases where it
>>> just does a bare mntput:
>>>
>>>           if (!nn) {
>>>                   mntput(ss_mnt);
>>>                   return;
>>>           }
>>> ...
>>>           if (!found) {
>>>                   mntput(ss_mnt);
>>>                   return;
>>>           }
>>>
>>> The first one looks bogus. Can net_generic return NULL? If so how, and
>>> why is it not a problem elsewhere in the kernel?
>> it looks like net_generic can not fail, no where else check for NULL
>> so I will remove this check.
>>
>>> For the second case, if the ni is no longer on the list, where did the
>>> extra ss_mnt reference come from? Maybe that should be a WARN_ON or
>>> BUG_ON?
>> if ni is not found on the list then it's a bug somewhere so I will add
>> a BUG_ON on this.
>>
> Probably better to just WARN_ON and let any references leak in that
> case. A BUG_ON implies a panic in some environments, and it's best to
> avoid that unless there really is no choice.

ok, thanks Jeff!

-Dai