Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Meng.Li@windriver.com
To:     Roy.Pledge@nxp.com, leoyang.li@nxp.com, youri.querry_1@nxp.com
Cc:     linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org,
        linux-arm-kernel@lists.infradead.org, meng.li@windriver.com
Subject: [PATCH] driver: soc: dpio: use the whole functions to protect critical zone
Date:   Tue, 19 Oct 2021 11:05:55 +0800
Message-Id: <20211019030555.29461-1-Meng.Li@windriver.com>
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?4DdH4d4qSYeYXitr2lhPMriqwmatpMoztSziDi0RcESMW+0FxlRRKhgLdUcf?=
 =?us-ascii?Q?3qD2S5idXDNiGU748SuS4cbtO7BCGee8wHqhAKFcKqlJLmQQavOksbPMkZWH?=
 =?us-ascii?Q?uvE8A4RtgR3ZcepYWnbi6dyy4JlY8lzu9xZqi8o5vnlLI1X+mHECUgRDQ3/i?=
 =?us-ascii?Q?lCvGPHyCiq5l2OSM9NKa8N6spV+96v2X2RMsVKMU/e6Iw7L+CyPJW+SfBKO/?=
 =?us-ascii?Q?n/IRkg7DBhkROP2eKQEkmF8MxfeWW3aEJN8xlLyEpgTD6+iiZnO0pOTgq1xG?=
 =?us-ascii?Q?UglPG0tF9TZtHJ2nkChLm/M1UbLYRXRIMnjD01BGAc9uB+v365B42A4DUr4g?=
 =?us-ascii?Q?QyPKG1/OJFDjQIf7o8ID7OhQnGXA2bCyoeSHmxexLhwWvKn5vvrh/nK8f/3+?=
 =?us-ascii?Q?0QUWAUjkMxzKV5Io81HYfJunSI73+vdk6c1JlVfAM1hyJsB4GhGvKFA9lc9z?=
 =?us-ascii?Q?QEAhNM11N1K8WSFOzE6Sc1oVFD5B8v2TpEo3d5Kbsm5aa2B6Q+qeSbr8IvgY?=
 =?us-ascii?Q?AEv22wggUGxjLa9N0zXrYrzqbP0Q2tZGRp7/ObuGh/e6TAgQ59JQvgoAaCza?=
 =?us-ascii?Q?mf0CN68/B8U9o36EUr96yt34Qua2T5tXlrStx4eecF3hOuuPhfyoEcM/EivZ?=
 =?us-ascii?Q?ovgvka8Vyd+LU+rNf7ODsAiqoX3gzu5qD28FYBzqgpIjL6zSIpRulhJbJ5SW?=
 =?us-ascii?Q?EAZrMXrSEIAwwdHes0bua+bUjYTJIIw/X6Y1h+aNyCNp2KBQAh38fPv1tXx5?=
 =?us-ascii?Q?4PFNQnihiOPszsGBp8gwET7ytFYeNrD3UEi3yuhVF5IehLPG5VcYd3Un2P8s?=
 =?us-ascii?Q?2GsvN26+ssJ0JaqKY73cq6WLpc+OzXaEbZ7xtmp+POi/CU9tT+mOAtzk5pf8?=
 =?us-ascii?Q?yWCbb6O7nJNvgTTvSVM1RhJENYC4mnBHRw6FJbEQh57WPiaJIjogPYJ+QRAN?=
 =?us-ascii?Q?SVtOjRdmcZINMfACOTH+YK2or00M9D5jbDTwO0B1Yt60m5MWZ0EVgEqR8miU?=
 =?us-ascii?Q?MG2brYqkfU2Cps9/LdQAodng1NvUKJHJqYYNRsnB6oR39pihRZfRObiCSPay?=
 =?us-ascii?Q?2+hYTupE+QPWt0W7/gtx1PoqvCg4BZR9tAFyRI3U/DUGOnOBwbsTotOuJzSl?=
 =?us-ascii?Q?qosL8trBEzjcqz0G7YVR2ZZ8ldIpQb+pCaF3LpyDQtRGio6OCtEoIBBAwNzz?=
 =?us-ascii?Q?BL6nh4i6KOsUwul0u28SKqa+VjOSXwqXvmXtJwLpIpLIrY53vKPmXihz7ezW?=
 =?us-ascii?Q?LoyYB76EY3tW9/zUDG97CYNp4xuqkf/1rspYspm32P85KD5QYV0ecrLBSQAA?=
 =?us-ascii?Q?eyjVKrkKQ1xqU5DX4Oud0BuJ?=
X-OriginatorOrg: windriver.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b8c57d1f-0c5c-47d9-2dec-08d992ad6b8f
X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB5191.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Oct 2021 03:06:18.2672
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 8ddb2873-a1ad-4a18-ae4e-4644631433be
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HrRQMHjeiuQU8wcDXQVDf85l45Onm6L7gyPeqTVkamA0cRooCrC95vD056dO2gQLjeVVhtgGmgRIq74TVoaeKA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB5927
X-Proofpoint-GUID: atanGH9l_5FrpMRtRI_FRs-tf4CTVbUM
X-Proofpoint-ORIG-GUID: atanGH9l_5FrpMRtRI_FRs-tf4CTVbUM
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475
 definitions=2021-10-18_07,2021-10-18_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 malwarescore=0
 phishscore=0 spamscore=0 mlxscore=0 priorityscore=1501 clxscore=1011
 impostorscore=0 bulkscore=0 mlxlogscore=259 suspectscore=0
 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2110190016
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

From: Meng Li <Meng.Li@windriver.com>

In orininal code, use 2 function spin_lock() and local_irq_save() to
protect the critical zone. But when enable the kernel debug config,
there are below inconsistent lock state detected.
================================
WARNING: inconsistent lock state
5.10.63-yocto-standard #1 Not tainted
--------------------------------
inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
lock_torture_wr/226 [HC0[0]:SC1[5]:HE1:SE0] takes:
ffff002005b2dd80 (&p->access_spinlock){+.?.}-{3:3}, at: qbman_swp_enqueue_multiple_mem_back+0x44/0x270
{SOFTIRQ-ON-W} state was registered at:
  lock_acquire.part.0+0xf8/0x250
  lock_acquire+0x68/0x84
  _raw_spin_lock+0x68/0x90
  qbman_swp_enqueue_multiple_mem_back+0x44/0x270
  ......
  cryptomgr_test+0x38/0x60
  kthread+0x158/0x164
  ret_from_fork+0x10/0x38
irq event stamp: 4498
hardirqs last  enabled at (4498): [<ffff800010fcf980>] _raw_spin_unlock_irqrestore+0x90/0xb0
hardirqs last disabled at (4497): [<ffff800010fcffc4>] _raw_spin_lock_irqsave+0xd4/0xe0
softirqs last  enabled at (4458): [<ffff8000100108c4>] __do_softirq+0x674/0x724
softirqs last disabled at (4465): [<ffff80001005b2a4>] __irq_exit_rcu+0x190/0x19c

other info that might help us debug this:
 Possible unsafe locking scenario:
       CPU0
       ----
  lock(&p->access_spinlock);
  <Interrupt>
    lock(&p->access_spinlock);
 *** DEADLOCK ***

So, in order to avoid deadlock, use the whole functinos
spin_lock_irqsave/spin_unlock_irqrestore() to protect critical zone.

Fixes: 3b2abda7d28c ("soc: fsl: dpio: Replace QMAN array mode with ring mode enqueue")
Cc: stable@vger.kernel.org
Signed-off-by: Meng Li <Meng.Li@windriver.com>
---
 drivers/soc/fsl/dpio/qbman-portal.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/drivers/soc/fsl/dpio/qbman-portal.c b/drivers/soc/fsl/dpio/qbman-portal.c
index 845e91416b58..a56dbe4de34f 100644
--- a/drivers/soc/fsl/dpio/qbman-portal.c
+++ b/drivers/soc/fsl/dpio/qbman-portal.c
@@ -785,8 +785,7 @@ int qbman_swp_enqueue_multiple_mem_back(struct qbman_swp *s,
 	int i, num_enqueued = 0;
 	unsigned long irq_flags;
 
-	spin_lock(&s->access_spinlock);
-	local_irq_save(irq_flags);
+	spin_lock_irqsave(&s->access_spinlock, irq_flags);
 
 	half_mask = (s->eqcr.pi_ci_mask>>1);
 	full_mask = s->eqcr.pi_ci_mask;
@@ -797,8 +796,7 @@ int qbman_swp_enqueue_multiple_mem_back(struct qbman_swp *s,
 		s->eqcr.available = qm_cyc_diff(s->eqcr.pi_ring_size,
 					eqcr_ci, s->eqcr.ci);
 		if (!s->eqcr.available) {
-			local_irq_restore(irq_flags);
-			spin_unlock(&s->access_spinlock);
+			spin_unlock_irqrestore(&s->access_spinlock, irq_flags);
 			return 0;
 		}
 	}
@@ -837,8 +835,7 @@ int qbman_swp_enqueue_multiple_mem_back(struct qbman_swp *s,
 	dma_wmb();
 	qbman_write_register(s, QBMAN_CINH_SWP_EQCR_PI,
 				(QB_RT_BIT)|(s->eqcr.pi)|s->eqcr.pi_vb);
-	local_irq_restore(irq_flags);
-	spin_unlock(&s->access_spinlock);
+	spin_unlock_irqrestore(&s->access_spinlock, irq_flags);
 
 	return num_enqueued;
 }
-- 
2.17.1