Received: by 2002:a05:6a10:f347:0:0:0:0 with SMTP id d7csp969487pxu; Wed, 2 Dec 2020 07:52:54 -0800 (PST) X-Google-Smtp-Source: ABdhPJzE/bY46a3vR+STlq6OrV604BGWAYgXRg+xh5TJtwVoj7D1OkZyCAL9A2DDngOHkTwvx1Jz X-Received: by 2002:a17:906:bc9b:: with SMTP id lv27mr336991ejb.505.1606924374654; Wed, 02 Dec 2020 07:52:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1606924374; cv=none; d=google.com; s=arc-20160816; b=w3OCGt0z5YPOtceofIOi8ccg76F2QqoaN07Ih37i39faU3f3cKwPXjptnU02etaKcu raoh91GGyfZdC5yDejChAcU3LrzQU/ZHFtNAKxF2Pp5hnaOA9kGkcEt+HYZ/H/k2g2Aj plh5I0TxcixtDxZC+3v0dVxGIMCiEcZ3eUKERzoqK4Ca5ix1TY3/uXNa0Ap2wKbK4lD0 6+7Mjj2e9AWRo7zCzcH7lusBvOeA3z1U8MNjs0AwDfS0HJn1tylNXtx8t6TZxUAMP3/P hAP5LaR+Aj1hcXwXyJvjcDkDbFAo9P1OMAyl+MvttiYRS531qDBqsTX55ZqdBBsgqs/6 Qocw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:content-language :in-reply-to:mime-version:user-agent:date:message-id:from:references :cc:to:subject:dkim-signature; bh=fBPxYPgsvkf+CoxdCEVMZ5J+rf844SeLR5d894vr+Sg=; b=vTzjQdO5PElt5COxOVjZlEjgoNwvnbS1LFkAp6+/LfiNbTaJ+4vVC9YCHgVfbmHYrW XAX2taqruBh8mWViirnRTD7Dd1RPQ+c+AiF99p3oVvXe+URQ1RNvfyFUvpgc3YgiXfiQ HSLGryya2PvZPLO/UMrn8VBDqDWzvEWeKT0QTsBM7kRhVSkaOWSNhwYrv7Px2goea2n5 /KiMpfHvWzQqw00awJPsWpjQ2NRP3yILMK4MKxFkbWxp2v7IfFHiMZdCRxCT5U/GOJDK JiPyGZj/mWas9TDiKhTMwCEnLG5wO5UCufv7o0Tl/hPQ/P70UkwV4qAu9Tnq82RR6yTZ ZI3A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=Si+pIbKu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id f10si218600edj.366.2020.12.02.07.52.31; Wed, 02 Dec 2020 07:52:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=Si+pIbKu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387931AbgLBPtk (ORCPT + 99 others); Wed, 2 Dec 2020 10:49:40 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:1140 "EHLO mx0b-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728608AbgLBPtj (ORCPT ); Wed, 2 Dec 2020 10:49:39 -0500 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B2Fl2B5096554; Wed, 2 Dec 2020 10:48:17 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=subject : to : cc : references : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=pp1; bh=fBPxYPgsvkf+CoxdCEVMZ5J+rf844SeLR5d894vr+Sg=; b=Si+pIbKuCX8jk9NuW50eMeIB/D8bRlBHt+BdnHD9AwH3dFBqWYFew17XpCiqG3YWX/zd WjNsgUex326033rrEkN9+CAZm9ELj4YP2jSmocIgfDZUFPHKVtcYSkErNQxVtt0KZcRU tMhevjY23AVduUF39zKTyreIWZIRTnuuQwXBHGWmr30UfrKRMsQjynyRMtgOimGEas4O f824XllSrLpr+qlUqc+PqcJSyyxFcvt4oB9/XSwsjVWk5IjWZHEKaC1n9hWfjjMZ2vTj cHJtn6wzXjh4pgZIA6/N574lramrKoA1JP8PgTdoiDRLBXdnm2T5zw4Yuxo9W2Mz1E7Z LQ== Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0a-001b2d01.pphosted.com with ESMTP id 355y6d8skp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 10:48:16 -0500 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B2FYud1021218; Wed, 2 Dec 2020 15:48:16 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma01wdc.us.ibm.com with ESMTP id 355vrfqwyx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 15:48:16 +0000 Received: from b01ledav005.gho.pok.ibm.com (b01ledav005.gho.pok.ibm.com [9.57.199.110]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B2Fl0345505756 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 2 Dec 2020 15:47:00 GMT Received: from b01ledav005.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 607E9AE064; Wed, 2 Dec 2020 15:47:00 +0000 (GMT) Received: from b01ledav005.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 857B4AE05F; Wed, 2 Dec 2020 15:46:59 +0000 (GMT) Received: from oc6034535106.ibm.com (unknown [9.211.78.151]) by b01ledav005.gho.pok.ibm.com (Postfix) with ESMTP; Wed, 2 Dec 2020 15:46:59 +0000 (GMT) Subject: Re: [PATCH v2 06/17] ibmvfc: add handlers to drain and complete Sub-CRQ responses To: Tyrel Datwyler , james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com References: <20201202005329.4538-1-tyreld@linux.ibm.com> <20201202005329.4538-7-tyreld@linux.ibm.com> From: Brian King Message-ID: <32b08be7-4c1e-a572-c70c-1f182f1d0259@linux.vnet.ibm.com> Date: Wed, 2 Dec 2020 09:46:58 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.4.0 MIME-Version: 1.0 In-Reply-To: <20201202005329.4538-7-tyreld@linux.ibm.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312,18.0.737 definitions=2020-12-02_08:2020-11-30,2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 mlxlogscore=999 mlxscore=0 suspectscore=2 priorityscore=1501 clxscore=1015 adultscore=0 lowpriorityscore=0 malwarescore=0 impostorscore=0 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012020095 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 12/1/20 6:53 PM, Tyrel Datwyler wrote: > The logic for iterating over the Sub-CRQ responses is similiar to that > of the primary CRQ. Add the necessary handlers for processing those > responses. > > Signed-off-by: Tyrel Datwyler > --- > drivers/scsi/ibmvscsi/ibmvfc.c | 77 ++++++++++++++++++++++++++++++++++ > 1 file changed, 77 insertions(+) > > diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c > index 97f00fefa809..e9da3f60c793 100644 > --- a/drivers/scsi/ibmvscsi/ibmvfc.c > +++ b/drivers/scsi/ibmvscsi/ibmvfc.c > @@ -3381,6 +3381,83 @@ static int ibmvfc_toggle_scrq_irq(struct ibmvfc_sub_queue *scrq, int enable) > return rc; > } > > +static void ibmvfc_handle_scrq(struct ibmvfc_crq *crq, struct ibmvfc_host *vhost) > +{ > + struct ibmvfc_event *evt = (struct ibmvfc_event *)be64_to_cpu(crq->ioba); > + unsigned long flags; > + > + switch (crq->valid) { > + case IBMVFC_CRQ_CMD_RSP: > + break; > + case IBMVFC_CRQ_XPORT_EVENT: > + return; > + default: > + dev_err(vhost->dev, "Got and invalid message type 0x%02x\n", crq->valid); > + return; > + } > + > + /* The only kind of payload CRQs we should get are responses to > + * things we send. Make sure this response is to something we > + * actually sent > + */ > + if (unlikely(!ibmvfc_valid_event(&vhost->pool, evt))) { > + dev_err(vhost->dev, "Returned correlation_token 0x%08llx is invalid!\n", > + crq->ioba); > + return; > + } > + > + if (unlikely(atomic_read(&evt->free))) { > + dev_err(vhost->dev, "Received duplicate correlation_token 0x%08llx!\n", > + crq->ioba); > + return; > + } > + > + spin_lock_irqsave(vhost->host->host_lock, flags); > + del_timer(&evt->timer); > + list_del(&evt->queue); > + ibmvfc_trc_end(evt); > + spin_unlock_irqrestore(vhost->host->host_lock, flags); > + evt->done(evt); > +} > + > +static struct ibmvfc_crq *ibmvfc_next_scrq(struct ibmvfc_sub_queue *scrq) > +{ > + struct ibmvfc_crq *crq; > + > + crq = &scrq->msgs[scrq->cur].crq; > + if (crq->valid & 0x80) { > + if (++scrq->cur == scrq->size) You are incrementing the cur pointer without any locks held. Although unlikely, could you also be in ibmvfc_reset_crq in another thread? If so, you'd have a subtle race condition here where the cur pointer could be read, then ibmvfc_reset_crq writes it to zero, then this thread writes it to a non zero value, which would then cause you to be out of sync with the VIOS as to where the cur pointer is. > + scrq->cur = 0; > + rmb(); > + } else > + crq = NULL; > + > + return crq; > +} > + -- Brian King Power Linux I/O IBM Linux Technology Center