Received: by 2002:a05:6358:d09b:b0:dc:cd0c:909e with SMTP id jc27csp2944288rwb; Wed, 30 Nov 2022 13:06:16 -0800 (PST) X-Google-Smtp-Source: AA0mqf64Jcq1hk1D5RD3OiMpVDqKRnHTANPrmno6ObOSV8roy/dTw16S2aktGxJWwZWifp4pUhY4 X-Received: by 2002:a17:906:e0d2:b0:7ad:b45c:dbe1 with SMTP id gl18-20020a170906e0d200b007adb45cdbe1mr54256399ejb.726.1669842376247; Wed, 30 Nov 2022 13:06:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669842376; cv=none; d=google.com; s=arc-20160816; b=fao1p3fhpAsZ5t23U7dCdNxsztwcb4TX5je9q8TLGM+jXsrQlKaKLZ+qLIEEn0tGCF ytEHkF7oAp5EbbKPFHy5SYpGtUeN4oRHI/ZQ54vWZBfEbSz/MkhL4Sw4+U8nzmeT1sb9 o6KsoV8ka4hFD7JudxMkjdNkZj4cfdvVN/jU/DGUCCYNJceG80E69LzMZpcYv0Sf9bLP VzEzds+ZL/VomuCNdgULKs9t//P3SrSVBCtkdWVlbK01F86uuWd2ynlT1352+uzla6l4 STV1Dl5OZDTK0InRl9DJ52a0ZASAZ8ru//Hglpvp2hLdo0Q8kMEvbNRM9sJCWAiWh7O8 ZxYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=O68hHJNIJUydgqvoVYgEDwHFsib2OTku7a+BuTE6Dtw=; b=ip/OeXUlDmCihVQObDREQvxiR3oBBlV4wbtBuYm2++tcARxGJPsO9L7mxiGMQLZbNs 2brWOVUFivz61GAxkcQnuPIM5wGR9i/5uJZfN2TrWaeOdnUwHjVsT/Ud8D2Td+YH4jvG FyvIZxlc+idgjCQAHF9Lx7p6gnyC+X0k7MsU07m4QKC9xY19PGz2huVc0w8UsuLHYutJ 3T9ZS1TxWsxb6Mq/aDQNVdQ5zchJ6MtGadD+FZPyK9jfqPf/WXL1Mlx+JeGiH+I5c/l4 17TEbbrRvCVZ/ianPWUIYpcOz0Z31MlAuWp29jqG17FGUzqTEuH3Il4VbpTdk1CE7Ted h35A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=XUA2wtMq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a26-20020aa7d75a000000b00469d667117csi2133312eds.167.2022.11.30.13.05.56; Wed, 30 Nov 2022 13:06:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=XUA2wtMq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229728AbiK3UcV (ORCPT + 83 others); Wed, 30 Nov 2022 15:32:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42366 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230283AbiK3Ubx (ORCPT ); Wed, 30 Nov 2022 15:31:53 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 31FF1920AA; Wed, 30 Nov 2022 12:31:18 -0800 (PST) Received: from pps.filterd (m0279866.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2AU58B5O010377; Wed, 30 Nov 2022 20:31:00 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=O68hHJNIJUydgqvoVYgEDwHFsib2OTku7a+BuTE6Dtw=; b=XUA2wtMqDluih7T0/6JDQv+RJlzvnCLZHGuaEsYQqtbmQjjlNqLdrEULb8woqXqR6iEs Y68iINSxkcVphCWvTZBbO5d6y9I9KWfTj43DBiZ5W6wO1rAPmtmcC+Op2zMqqp1//Un/ jLBprv8lrjkAu6YuO1wNuGqk3fISeJGaIdwo9G7oI1vqLkWLCwS51hvZzkE1CY/SJX7X DxRIua+++1/UybMmHYw6g8sP7y4H4LqZKsd4H3j7sJlQf6hCqn3u0JeJntiI0t/QHxqM tOs1rj+tpOeyYNpA1c1ezNQmvk5wOv1/vAijMxAnu8y09W+LAuzYD35gjsZctefzZAKP bg== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3m5mhc56hq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 30 Nov 2022 20:31:00 +0000 Received: from nasanex01a.na.qualcomm.com ([10.52.223.231]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2AUKUxxw025571 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 30 Nov 2022 20:30:59 GMT Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 30 Nov 2022 12:30:59 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , Krzysztof Kozlowski , Arthur Simchaev , Jinyoung Choi , "open list" Subject: [PATCH v8 15/16] ufs: core: mcq: Add completion support in poll Date: Wed, 30 Nov 2022 12:27:56 -0800 Message-ID: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: 3C04WtXFQGpi03mf6ZccsuN7I3PtoRhW X-Proofpoint-ORIG-GUID: 3C04WtXFQGpi03mf6ZccsuN7I3PtoRhW X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-30_04,2022-11-30_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 mlxlogscore=999 mlxscore=0 priorityscore=1501 spamscore=0 bulkscore=0 malwarescore=0 phishscore=0 adultscore=0 impostorscore=0 lowpriorityscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211300144 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Complete cqe requests in poll. Assumption is that several poll completion may happen in different CPUs for the same completion queue. Hence a spin lock protection is added. Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Bart Van Assche Reviewed-by: Manivannan Sadhasivam --- drivers/ufs/core/ufs-mcq.c | 13 +++++++++++++ drivers/ufs/core/ufshcd-priv.h | 2 ++ drivers/ufs/core/ufshcd.c | 7 +++++++ include/ufs/ufshcd.h | 2 ++ 4 files changed, 24 insertions(+) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index 68c4097..f99c912 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -294,6 +294,18 @@ unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, return completed_reqs; } +unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, + struct ufs_hw_queue *hwq) +{ + unsigned long completed_reqs; + + spin_lock(&hwq->cq_lock); + completed_reqs = ufshcd_mcq_poll_cqe_nolock(hba, hwq); + spin_unlock(&hwq->cq_lock); + + return completed_reqs; +} + void ufshcd_mcq_make_queues_operational(struct ufs_hba *hba) { struct ufs_hw_queue *hwq; @@ -390,6 +402,7 @@ int ufshcd_mcq_init(struct ufs_hba *hba) hwq = &hba->uhq[i]; hwq->max_entries = hba->nutrs; spin_lock_init(&hwq->sq_lock); + spin_lock_init(&hwq->cq_lock); } /* The very first HW queue serves device commands */ diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index 70e3416..ff03aa5 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -75,6 +75,8 @@ unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, struct ufs_hw_queue *hwq); struct ufs_hw_queue *ufshcd_mcq_req_to_hwq(struct ufs_hba *hba, struct request *req); +unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, + struct ufs_hw_queue *hwq); #define UFSHCD_MCQ_IO_QUEUE_OFFSET 1 #define SD_ASCII_STD true diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 8d743c3..adf3597 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -5475,6 +5475,13 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num) struct ufs_hba *hba = shost_priv(shost); unsigned long completed_reqs, flags; u32 tr_doorbell; + struct ufs_hw_queue *hwq; + + if (is_mcq_enabled(hba)) { + hwq = &hba->uhq[queue_num + UFSHCD_MCQ_IO_QUEUE_OFFSET]; + + return ufshcd_mcq_poll_cqe_lock(hba, hwq); + } spin_lock_irqsave(&hba->outstanding_lock, flags); tr_doorbell = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL); diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index 8441c46..f20557b 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -1072,6 +1072,7 @@ struct ufs_hba { * @sq_lock: serialize submission queue access * @cq_tail_slot: current slot to which CQ tail pointer is pointing * @cq_head_slot: current slot to which CQ head pointer is pointing + * @cq_lock: Synchronize between multiple polling instances */ struct ufs_hw_queue { void __iomem *mcq_sq_head; @@ -1089,6 +1090,7 @@ struct ufs_hw_queue { spinlock_t sq_lock; u32 cq_tail_slot; u32 cq_head_slot; + spinlock_t cq_lock; }; static inline bool is_mcq_enabled(struct ufs_hba *hba) -- 2.7.4