From: "Tran, Quinn" <Quinn.Tran@cavium.com>
To: "Nicholas A. Bellinger" <nab@linux-iscsi.org>,
        target-devel <target-devel@vger.kernel.org>
CC: linux-scsi <linux-scsi@vger.kernel.org>,
        lkml <linux-kernel@vger.kernel.org>,
        "Madhani, Himanshu" <Himanshu.Madhani@cavium.com>,
        Mike Christie <mchristi@redhat.com>, Hannes Reinecke <hare@suse.com>,
        Christoph Hellwig <hch@lst.de>
Subject: Re: [PATCH 1/3] target: Add support for TMR percpu reference counting
Thread-Topic: [PATCH 1/3] target: Add support for TMR percpu reference
 counting
Thread-Index: AQHS3LXA4nPXq287nEyePHL7wXCjiKIYJ8mA
Date: Wed, 7 Jun 2017 01:13:32 +0000
Message-ID: <386D71C4-353C-4237-8345-3043E8773EAF@cavium.com>
References: <1496527808-28789-1-git-send-email-nab@linux-iscsi.org>
 <1496527808-28789-2-git-send-email-nab@linux-iscsi.org>
In-Reply-To: <1496527808-28789-2-git-send-email-nab@linux-iscsi.org>
Accept-Language: en-US
Content-Language: en-US
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="utf-8"
Content-ID: <746EC59593ECE143AA45857A96AAED99@namprd07.prod.outlook.com>
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Jun 2017 01:13:32.2168
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 711e4ccf-2e9b-4bcf-a551-4094005b6194
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR07MB3445
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from base64 to 8bit by mail.home.local id v571Dggf025600
Content-Length: 4376
Lines: 111

Looks good.

Regards,
Quinn Tran

-----Original Message-----
From: Nicholas Bellinger <nab@linux-iscsi.org>
Date: Saturday, June 3, 2017 at 3:10 PM
To: target-devel <target-devel@vger.kernel.org>
Cc: linux-scsi <linux-scsi@vger.kernel.org>, lkml <linux-kernel@vger.kernel.org>, Nicholas Bellinger <nab@linux-iscsi.org>, "Madhani, Himanshu" <Himanshu.Madhani@cavium.com>, "Tran, Quinn" <Quinn.Tran@cavium.com>, Mike Christie <mchristi@redhat.com>, Hannes Reinecke <hare@suse.com>, Christoph Hellwig <hch@lst.de>
Subject: [PATCH 1/3] target: Add support for TMR percpu reference counting

    From: Nicholas Bellinger <nab@linux-iscsi.org>
    
    This patch introduces TMR percpu reference counting using
    se_lun->lun_ref in transport_lookup_tmr_lun(), following
    how existing non TMR per se_lun reference counting works
    within transport_lookup_cmd_lun().
    
    It also adds explicit transport_lun_remove_cmd() calls to
    drop the reference in the three tmr related locations that
    invoke transport_cmd_check_stop_to_fabric();
    
       - target_tmr_work() during normal ->queue_tm_rsp()
       - target_complete_tmr_failure() during error ->queue_tm_rsp()
       - transport_generic_handle_tmr() during early failure
    
    Also, note the exception paths in transport_generic_free_cmd()
    and transport_cmd_finish_abort() already check SCF_SE_LUN_CMD,
    and will invoke transport_lun_remove_cmd() when necessary.
    
    Cc: Himanshu Madhani <himanshu.madhani@cavium.com>
    Cc: Quinn Tran <quinn.tran@cavium.com>
    Cc: Mike Christie <mchristi@redhat.com>
    Cc: Hannes Reinecke <hare@suse.com>
    Cc: Christoph Hellwig <hch@lst.de>
    Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
    ---
     drivers/target/target_core_device.c    | 14 ++++++++++----
     drivers/target/target_core_transport.c |  3 +++
     2 files changed, 13 insertions(+), 4 deletions(-)
    
    diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
    index 8f0e0e3..11c80c4 100644
    --- a/drivers/target/target_core_device.c
    +++ b/drivers/target/target_core_device.c
    @@ -168,11 +168,20 @@ int transport_lookup_tmr_lun(struct se_cmd *se_cmd, u64 unpacked_lun)
     	rcu_read_lock();
     	deve = target_nacl_find_deve(nacl, unpacked_lun);
     	if (deve) {
    -		se_cmd->se_lun = rcu_dereference(deve->se_lun);
     		se_lun = rcu_dereference(deve->se_lun);
    +
    +		if (!percpu_ref_tryget_live(&se_lun->lun_ref)) {
    +			se_lun = NULL;
    +			goto out_unlock;
    +		}
    +
    +		se_cmd->se_lun = rcu_dereference(deve->se_lun);
     		se_cmd->pr_res_key = deve->pr_res_key;
     		se_cmd->orig_fe_lun = unpacked_lun;
    +		se_cmd->se_cmd_flags |= SCF_SE_LUN_CMD;
    +		se_cmd->lun_ref_active = true;
     	}
    +out_unlock:
     	rcu_read_unlock();
     
     	if (!se_lun) {
    @@ -182,9 +191,6 @@ int transport_lookup_tmr_lun(struct se_cmd *se_cmd, u64 unpacked_lun)
     			unpacked_lun);
     		return -ENODEV;
     	}
    -	/*
    -	 * XXX: Add percpu se_lun->lun_ref reference count for TMR
    -	 */
     	se_cmd->se_dev = rcu_dereference_raw(se_lun->lun_se_dev);
     	se_tmr->tmr_dev = rcu_dereference_raw(se_lun->lun_se_dev);
     
    diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c
    index f16a789..83bfc97 100644
    --- a/drivers/target/target_core_transport.c
    +++ b/drivers/target/target_core_transport.c
    @@ -1588,6 +1588,7 @@ static void target_complete_tmr_failure(struct work_struct *work)
     	se_cmd->se_tmr_req->response = TMR_LUN_DOES_NOT_EXIST;
     	se_cmd->se_tfo->queue_tm_rsp(se_cmd);
     
    +	transport_lun_remove_cmd(se_cmd);
     	transport_cmd_check_stop_to_fabric(se_cmd);
     }
     
    @@ -3199,6 +3200,7 @@ static void target_tmr_work(struct work_struct *work)
     	cmd->se_tfo->queue_tm_rsp(cmd);
     
     check_stop:
    +	transport_lun_remove_cmd(cmd);
     	transport_cmd_check_stop_to_fabric(cmd);
     }
     
    @@ -3221,6 +3223,7 @@ int transport_generic_handle_tmr(
     		pr_warn_ratelimited("handle_tmr caught CMD_T_ABORTED TMR %d"
     			"ref_tag: %llu tag: %llu\n", cmd->se_tmr_req->function,
     			cmd->se_tmr_req->ref_task_tag, cmd->tag);
    +		transport_lun_remove_cmd(cmd);
     		transport_cmd_check_stop_to_fabric(cmd);
     		return 0;
     	}
    -- 
    1.9.1