Received: by 2002:a05:6a10:206:0:0:0:0 with SMTP id 6csp379673pxj; Wed, 16 Jun 2021 04:44:38 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwqBdzBJG+CpNoliHZXc15qsFglAbWnarIxvcA8IA9hNuwtkDUoI4B1qEUNvQWuUtif2PZ5 X-Received: by 2002:a05:6e02:1d89:: with SMTP id h9mr3257899ila.235.1623843878033; Wed, 16 Jun 2021 04:44:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1623843878; cv=none; d=google.com; s=arc-20160816; b=lN1Ea1vBX4omXx6Gq54ZZDwnYOncnWY8dAYFEUa6w0XIOZ/WhCf0EWK9PBpYEV3P14 qRLSftC1+Icdgt0ufjSSnGLKPfUxdSWfTF0myz/F+YUhEs5cUgin2vieZYYaTQOkBqMq lsidm/6ww7iUGoOvr66ocwXg+b78Z0Mcr4ocpjTbS9dPWLo1xzUTRvS/Op4Q0xCpugAh hJzoe+T86qPjan0kiWRmBZd5iT9QFrj3cFZhFurN4NEmgZ3H+nRT7US5b4HMqHQxI2RR 3FCTthXj2b3WSGqxvFVxby1yEFrIth4AP30hZL6+x8SSTnwiZiBCIEXokNG1UydNTshO hKvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :wdcironportexception:ironport-sdr:ironport-sdr:ironport-sdr :dkim-signature; bh=Q7tglZ+1BcfxP45B4MYwu/VZkjysEfq2CeM8sl41XIU=; b=AtlGlK8jzOEYOg+ZviKOgqLUIfUTg8ALLxCEMwuIESvLI2lF+melTbNNBW2XVSkrGO EQDgFnxJ47xxxZJ9nweByhcwHtizsXkM3/Cqf67tv49bpAgQ/8Qg/ynujoPTLalS714C hHARIC/y48C69cxfOmEPoapKpBl8QFBsJFEh83XTPftdmuFEf2vgXHEjOMDHg6bybvus b4K8jd0xwYACLox1oZRTQ9RbTjY9mA2p1Hz6PuMyVLurGl+NyYmYeBbqjTfXeqkYql6O bz+nafZpm0Fl4Tfuvi/Xsa41yicpXTa80Wl0f0mBqnrw1bIl0TNk11q7vDd+ZYxTjhdg wWIQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@wdc.com header.s=dkim.wdc.com header.b=ir+tL1sU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=wdc.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id i15si1926671ils.139.2021.06.16.04.44.25; Wed, 16 Jun 2021 04:44:38 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=fail header.i=@wdc.com header.s=dkim.wdc.com header.b=ir+tL1sU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=wdc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232537AbhFPLbf (ORCPT + 99 others); Wed, 16 Jun 2021 07:31:35 -0400 Received: from esa5.hgst.iphmx.com ([216.71.153.144]:30224 "EHLO esa5.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232541AbhFPLbZ (ORCPT ); Wed, 16 Jun 2021 07:31:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1623842957; x=1655378957; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QmslEhEyWR2A2blac4IjPnOEJ4QqpiLz4G81vkdi4Cw=; b=ir+tL1sUB9lxyKFEvX7PuoqdeTkF0FxXNWCiZ1p0jzEjUgxn8RfvMtes gAc6fqxDP0/meqAHy8Hj9WWtk7hXyyq8dib6BcHWx9pn2KcnUaU5VF1eq CHqHXocTPwdVpIAqaMLBsGoAmWIlQtcNzL8vlcgPFwbH9iUSoJ6CfyqeG LNFOoRmJQPRbgmKKWGeVcG5IcOs089VUJWFI3Lidh//qSD3gCPU857Gr+ rWmdhIjl3/YUE4Z2dqYH2FZSyWPchrfqKzOYOwny5XS9mKNxJxDLUSAlT YZmNt2sC32XGtYAHtOx7y5/0DyVJVVx6BNZrsw/5RrjtumZM3N/5C97ry g==; IronPort-SDR: DZISGay1XOMLlbXb0wWmcdPtb2bg1MLqF18hlW7A0skV/dL1PbkpiaLwRi36rAjNRo+j3wbnte nmfXuBI5W+YT44Xr06hHeR1SZ/uAr1P4yP4cCeelRvuu3CBpG3jDkdhWbD2jdzBnCmML9hbFox fFyvNpJDUrUbP+ccmO9hD8R0uYWKB/2/K0o0NW7oBPHjGU9iC4+fzYq9MEYryGolvsxz4+ynKF sUMtV6f9c53uioCLcvf4bHbbw1UqcWEK4rgpw3ETiAUl52YZplwzGwq3srW502i/q/A7eokEOS mrQ= X-IronPort-AV: E=Sophos;i="5.83,277,1616428800"; d="scan'208";a="172091850" Received: from h199-255-45-15.hgst.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 16 Jun 2021 19:29:17 +0800 IronPort-SDR: nQm5RNgJveTV9r3KzDf+Qj+lHeIw7Tq7EAvthErX5xJECdqPvbpe2YT/TUCNCYQ5b7ZMAhb5pj AjqyVvKjakDlOicz6au0vujyMyK/NghFZCy50p5vwI+oUBQlMl8qNOPTFAyY1gQw/uwTs4i0qs vl4qopJ+DUOqYMrqFzsa+CpmF6XhmCgfUrAn5TBdotUZPY5vnENdUPPwrChSvnP0hiyRSnk1lQ DUQAde401HND7Nsi0J/GTGapF7otB923sDeBKEhv4cH6HzuuDJWjdGlSvlFn8+5kDM4qhZbLhb VKoVftTkTVAI9kvq5pSpEYDe Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jun 2021 04:06:46 -0700 IronPort-SDR: Edc71OKVf3Ijq14Nixfq1vrlemrrAdSY9RTm50x00pH2OM/dDWdQ38ulkhGKcSdKaDs2Fa013o QNIPKbHe+oLCC08wsOq1o2VHnfgvAuJprkPwln7KAhjnq4Vfg9a1mzT/i1W1PXLQKZEAq7DGfO T6c55sp6qUif6fUIMk8YcUBx/QScewfYfyeihFzOI2XaKlTyy+0ZR8V/Z+w17XHt1DHkh6KPNt OReeNuHKQIIpYT/NhUBa5OOUi9cWi2wpYNLjQOANZbxmFWWoPClVJNychIUrX6fgo0pnYJQLOv i+g= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip01.wdc.com with ESMTP; 16 Jun 2021 04:29:15 -0700 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v11 08/12] scsi: ufshpb: Add "Cold" regions timer Date: Wed, 16 Jun 2021 14:27:56 +0300 Message-Id: <20210616112800.52963-9-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210616112800.52963-1-avri.altman@wdc.com> References: <20210616112800.52963-1-avri.altman@wdc.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In order not to hang on to “cold” regions, we shall inactivate a region that has no READ access for a predefined amount of time - READ_TO_MS. For that purpose we shall monitor the active regions list, polling it on every POLLING_INTERVAL_MS. On timeout expiry we shall add the region to the "to-be-inactivated" list, unless it is clean and did not exhaust its READ_TO_EXPIRIES - another parameter. All this does not apply to pinned regions. Signed-off-by: Avri Altman Reviewed-by: Daejun Park --- drivers/scsi/ufs/ufshpb.c | 74 +++++++++++++++++++++++++++++++++++++-- drivers/scsi/ufs/ufshpb.h | 8 +++++ 2 files changed, 79 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 39b86e8b2eee..cf719831adb3 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -18,6 +18,9 @@ #define ACTIVATION_THRESHOLD 8 /* 8 IOs */ #define EVICTION_THRESHOLD (ACTIVATION_THRESHOLD << 5) /* 256 IOs */ +#define READ_TO_MS 1000 +#define READ_TO_EXPIRIES 100 +#define POLLING_INTERVAL_MS 200 /* memory management */ static struct kmem_cache *ufshpb_mctx_cache; @@ -1032,12 +1035,63 @@ static int ufshpb_check_srgns_issue_state(struct ufshpb_lu *hpb, return 0; } +static void ufshpb_read_to_handler(struct work_struct *work) +{ + struct ufshpb_lu *hpb = container_of(work, struct ufshpb_lu, + ufshpb_read_to_work.work); + struct victim_select_info *lru_info = &hpb->lru_info; + struct ufshpb_region *rgn, *next_rgn; + unsigned long flags; + LIST_HEAD(expired_list); + + if (test_and_set_bit(TIMEOUT_WORK_RUNNING, &hpb->work_data_bits)) + return; + + spin_lock_irqsave(&hpb->rgn_state_lock, flags); + + list_for_each_entry_safe(rgn, next_rgn, &lru_info->lh_lru_rgn, + list_lru_rgn) { + bool timedout = ktime_after(ktime_get(), rgn->read_timeout); + + if (timedout) { + rgn->read_timeout_expiries--; + if (is_rgn_dirty(rgn) || + rgn->read_timeout_expiries == 0) + list_add(&rgn->list_expired_rgn, &expired_list); + else + rgn->read_timeout = ktime_add_ms(ktime_get(), + READ_TO_MS); + } + } + + spin_unlock_irqrestore(&hpb->rgn_state_lock, flags); + + list_for_each_entry_safe(rgn, next_rgn, &expired_list, + list_expired_rgn) { + list_del_init(&rgn->list_expired_rgn); + spin_lock_irqsave(&hpb->rsp_list_lock, flags); + ufshpb_update_inactive_info(hpb, rgn->rgn_idx); + spin_unlock_irqrestore(&hpb->rsp_list_lock, flags); + } + + ufshpb_kick_map_work(hpb); + + clear_bit(TIMEOUT_WORK_RUNNING, &hpb->work_data_bits); + + schedule_delayed_work(&hpb->ufshpb_read_to_work, + msecs_to_jiffies(POLLING_INTERVAL_MS)); +} + static void ufshpb_add_lru_info(struct victim_select_info *lru_info, struct ufshpb_region *rgn) { rgn->rgn_state = HPB_RGN_ACTIVE; list_add_tail(&rgn->list_lru_rgn, &lru_info->lh_lru_rgn); atomic_inc(&lru_info->active_cnt); + if (rgn->hpb->is_hcm) { + rgn->read_timeout = ktime_add_ms(ktime_get(), READ_TO_MS); + rgn->read_timeout_expiries = READ_TO_EXPIRIES; + } } static void ufshpb_hit_lru_info(struct victim_select_info *lru_info, @@ -1819,6 +1873,7 @@ static int ufshpb_alloc_region_tbl(struct ufs_hba *hba, struct ufshpb_lu *hpb) INIT_LIST_HEAD(&rgn->list_inact_rgn); INIT_LIST_HEAD(&rgn->list_lru_rgn); + INIT_LIST_HEAD(&rgn->list_expired_rgn); if (rgn_idx == hpb->rgns_per_lu - 1) { srgn_cnt = ((hpb->srgns_per_lu - 1) % @@ -1840,6 +1895,7 @@ static int ufshpb_alloc_region_tbl(struct ufs_hba *hba, struct ufshpb_lu *hpb) } rgn->rgn_flags = 0; + rgn->hpb = hpb; } return 0; @@ -2063,9 +2119,12 @@ static int ufshpb_lu_hpb_init(struct ufs_hba *hba, struct ufshpb_lu *hpb) INIT_LIST_HEAD(&hpb->list_hpb_lu); INIT_WORK(&hpb->map_work, ufshpb_map_work_handler); - if (hpb->is_hcm) + if (hpb->is_hcm) { INIT_WORK(&hpb->ufshpb_normalization_work, ufshpb_normalization_work_handler); + INIT_DELAYED_WORK(&hpb->ufshpb_read_to_work, + ufshpb_read_to_handler); + } hpb->map_req_cache = kmem_cache_create("ufshpb_req_cache", sizeof(struct ufshpb_req), 0, 0, NULL); @@ -2099,6 +2158,10 @@ static int ufshpb_lu_hpb_init(struct ufs_hba *hba, struct ufshpb_lu *hpb) ufshpb_stat_init(hpb); ufshpb_param_init(hpb); + if (hpb->is_hcm) + schedule_delayed_work(&hpb->ufshpb_read_to_work, + msecs_to_jiffies(POLLING_INTERVAL_MS)); + return 0; release_pre_req_mempool: @@ -2165,9 +2228,10 @@ static void ufshpb_discard_rsp_lists(struct ufshpb_lu *hpb) static void ufshpb_cancel_jobs(struct ufshpb_lu *hpb) { - if (hpb->is_hcm) + if (hpb->is_hcm) { + cancel_delayed_work_sync(&hpb->ufshpb_read_to_work); cancel_work_sync(&hpb->ufshpb_normalization_work); - + } cancel_work_sync(&hpb->map_work); } @@ -2275,6 +2339,10 @@ void ufshpb_resume(struct ufs_hba *hba) continue; ufshpb_set_state(hpb, HPB_PRESENT); ufshpb_kick_map_work(hpb); + if (hpb->is_hcm) + schedule_delayed_work(&hpb->ufshpb_read_to_work, + msecs_to_jiffies(POLLING_INTERVAL_MS)); + } } diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufshpb.h index 43a95c670763..8309b59c7819 100644 --- a/drivers/scsi/ufs/ufshpb.h +++ b/drivers/scsi/ufs/ufshpb.h @@ -115,6 +115,7 @@ struct ufshpb_subregion { }; struct ufshpb_region { + struct ufshpb_lu *hpb; struct ufshpb_subregion *srgn_tbl; enum HPB_RGN_STATE rgn_state; int rgn_idx; @@ -132,6 +133,10 @@ struct ufshpb_region { /* region reads - for host mode */ spinlock_t rgn_lock; unsigned int reads; + /* region "cold" timer - for host mode */ + ktime_t read_timeout; + unsigned int read_timeout_expiries; + struct list_head list_expired_rgn; }; #define for_each_sub_region(rgn, i, srgn) \ @@ -223,6 +228,9 @@ struct ufshpb_lu { /* for selecting victim */ struct victim_select_info lru_info; struct work_struct ufshpb_normalization_work; + struct delayed_work ufshpb_read_to_work; + unsigned long work_data_bits; +#define TIMEOUT_WORK_RUNNING 0 /* pinned region information */ u32 lu_pinned_start; -- 2.25.1