Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   Ioana Ciornei <ioana.ciornei@nxp.com>
To:     Roy Pledge <roy.pledge@nxp.com>,
        Youri Querry <youri.querry_1@nxp.com>,
        Leo Li <leoyang.li@nxp.com>
CC:     "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        Ioana Ciocoi Radulescu <ruxandra.radulescu@nxp.com>,
        Horia Geanta <horia.geanta@nxp.com>,
        Ioana Ciornei <ioana.ciornei@nxp.com>
Subject: [PATCH v2 2/2] soc: fsl: dpio: use a cpumask to identify which cpus
 are unused
Thread-Topic: [PATCH v2 2/2] soc: fsl: dpio: use a cpumask to identify which
 cpus are unused
Thread-Index: AQHUi8Nc+JCofmNIZU+pIcUraudpTg==
Date:   Tue, 4 Dec 2018 11:20:29 +0000
Message-ID: <1543922408-22950-3-git-send-email-ioana.ciornei@nxp.com>
References: <1543922408-22950-1-git-send-email-ioana.ciornei@nxp.com>
In-Reply-To: <1543922408-22950-1-git-send-email-ioana.ciornei@nxp.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-messagesentrepresentingtype: 1
x-originating-ip: [86.34.165.90]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1;AM5PR0402MB2721;6:xiA4lFkg/NqnqU5ztk18wkXcX0Xro+k7ka1Je/BpWGe+7yY6/eWwZrXxVm0Imxwdvooud6Dsdsr+sO33bkiQl+Yx6G2opqtLee0P0pxqxa9TGpY/sv5An4elnpjr1epJCKO76JVb3rFMMoeRW7wlAqVlxaK5EZEx6T1Zzzar6TEocsvAtAfoqqjwAC7sUIvSLvMJe7bobSSs18ApHjR3xQ0XYNt03EKAQxe2OH+6JL1hEmdb4oHN4mds6cyuPs76NfjiRaYdXslr2mWS3o5sI5HQQm8qpMJkVgrg3N4tCZro/KUB6Fklx60HeVnYCYfH8o0XT3cKDrAOIg/QfxDyNlmzRR8kiGgwRSlN6xojf0jeMIH2PfG4QfKVt4BZ3ZBybJKUXhPKuK29z7I3WdQTMX1c8MgGHuUmYpbwohgXiThjEkcgcCdh7P5XJLV4RBozC8beE6VMqwO0eBMKdc6tnQ==;5:NCRzxjDA+rEa3nGkZ3FjRlCy6SRQCnpWcDvSasXKsNZwmNGInsiWhfgAOIXg9zO08slNQ6zANpzSU6RH5aPrbSGLrj9ZEDOPoVBoGKZMpblzaxgjFbopSToAI4PW+Sy3qvAQOCGVA9nL3+ZV/ktpYLk0R4VW7skGk3hy0b2pBRQ=;7:/OESKqNc5PRd/x/3Ur0wA6ZZpO4TwQby0gj2ABur1gJkiwl66wzhZqYqKTcYCLNbZ8SzSP8t8s2PjNyV1DgVKufT/7Rzq45mHwU/ajdkkfOGf3UHEk5U+eCz4unOTQCptwMjF/eyREOahUkxjCxwuA==
x-ms-office365-filtering-correlation-id: 430eff27-e50b-4c34-baa5-08d659da7f41
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390098)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020);SRVR:AM5PR0402MB2721;
x-ms-traffictypediagnostic: AM5PR0402MB2721:
x-microsoft-antispam-prvs: <AM5PR0402MB2721C149B691FFAA0456B57BE0AF0@AM5PR0402MB2721.eurprd04.prod.outlook.com>
x-ms-exchange-senderadcheck: 1
x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(93006095)(93001095)(3231455)(999002)(944501493)(52105112)(3002001)(10201501046)(6055026)(148016)(149066)(150057)(6041310)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123560045)(20161123558120)(201708071742011)(7699051)(76991095);SRVR:AM5PR0402MB2721;BCL:0;PCL:0;RULEID:;SRVR:AM5PR0402MB2721;
x-forefront-prvs: 0876988AF0
x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(346002)(376002)(366004)(39860400002)(136003)(396003)(199004)(189003)(476003)(5660300001)(71190400001)(14444005)(102836004)(25786009)(6436002)(52116002)(71200400001)(66066001)(76176011)(386003)(26005)(7736002)(186003)(68736007)(256004)(6636002)(6486002)(305945005)(99286004)(14454004)(6506007)(110136005)(54906003)(106356001)(105586002)(97736004)(11346002)(486006)(53936002)(478600001)(44832011)(6512007)(6116002)(2616005)(36756003)(4326008)(8676002)(81156014)(3846002)(8936002)(316002)(81166006)(86362001)(446003)(2906002);DIR:OUT;SFP:1101;SCL:1;SRVR:AM5PR0402MB2721;H:AM5PR0402MB2785.eurprd04.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1;
received-spf: None (protection.outlook.com: nxp.com does not designate
 permitted sender hosts)
x-microsoft-antispam-message-info: s37hJVFfyxppVIaVix0bxzVNNM1adQiq5Zln0abjveeSOaXMHHtHOZgpNMe1an4n/dTAeknsRK+rKEhBX8G3lur/HIC/WfamHYWFbaNXgMBV+eVUaFnUUnQ3exNQojt/Jzh5z9ciuFBX47qrT0iR+Rj6PXGbeOs7PpqLglajZBtbkK9loO6g0gE4/GUMeZaH+E7O5rxNM0nzhafPZ3eUd806RnBmj87HZOfg07A7DN00zsfq2ScAl1unusu17Oi6DyVXzlAdSqek88XSAuO8Ff5egiYrnbsdWFgPH2KBhQihX+lP/I7CZANlHiGk02Zke9r0bz5v6FE9ca2wZj/vdZlmkjY/IS8fhz+K682vDTo=
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: nxp.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 430eff27-e50b-4c34-baa5-08d659da7f41
X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Dec 2018 11:20:29.5493
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0402MB2721
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

The current implementation of the dpio driver uses a static next_cpu
variable to keep track of the index of the next cpu available. This
approach does not handle well unbinding and binding dpio devices in a
random order. For example, unbinding a dpio and then binding it again
with the driver, will generate the below error:

$ echo dpio.5 > /sys/bus/fsl-mc/drivers/fsl_mc_dpio/unbind
$ echo dpio.5 > /sys/bus/fsl-mc/drivers/fsl_mc_dpio/bind
[  103.946380] fsl_mc_dpio dpio.5: probe failed. Number of DPIOs exceeds
NR_CPUS.
[  103.955157] fsl_mc_dpio dpio.5: fsl_mc_driver_probe failed: -34
-bash: echo: write error: No such device

Fix this error by keeping a global cpumask of unused cpus that will be
updated at every dpaa2_dpio_[probe,remove].

Signed-off-by: Ioana Ciornei <ioana.ciornei@nxp.com>
---
Changes in v2:
  - added kernel-doc comment to the dpaa2_io_get_cpu function

 drivers/soc/fsl/dpio/dpio-driver.c  | 25 ++++++++++++++++---------
 drivers/soc/fsl/dpio/dpio-service.c | 13 +++++++++++++
 include/soc/fsl/dpaa2-io.h          |  2 ++
 3 files changed, 31 insertions(+), 9 deletions(-)

diff --git a/drivers/soc/fsl/dpio/dpio-driver.c b/drivers/soc/fsl/dpio/dpio=
-driver.c
index e58fcc9..832175c 100644
--- a/drivers/soc/fsl/dpio/dpio-driver.c
+++ b/drivers/soc/fsl/dpio/dpio-driver.c
@@ -30,6 +30,8 @@ struct dpio_priv {
 	struct dpaa2_io *io;
 };
=20
+static cpumask_var_t cpus_unused_mask;
+
 static irqreturn_t dpio_irq_handler(int irq_num, void *arg)
 {
 	struct device *dev =3D (struct device *)arg;
@@ -86,7 +88,7 @@ static int dpaa2_dpio_probe(struct fsl_mc_device *dpio_de=
v)
 	struct dpio_priv *priv;
 	int err =3D -ENOMEM;
 	struct device *dev =3D &dpio_dev->dev;
-	static int next_cpu =3D -1;
+	int possible_next_cpu;
=20
 	priv =3D devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
 	if (!priv)
@@ -128,17 +130,14 @@ static int dpaa2_dpio_probe(struct fsl_mc_device *dpi=
o_dev)
 	desc.dpio_id =3D dpio_dev->obj_desc.id;
=20
 	/* get the cpu to use for the affinity hint */
-	if (next_cpu =3D=3D -1)
-		next_cpu =3D cpumask_first(cpu_online_mask);
-	else
-		next_cpu =3D cpumask_next(next_cpu, cpu_online_mask);
-
-	if (!cpu_possible(next_cpu)) {
+	possible_next_cpu =3D cpumask_first(cpus_unused_mask);
+	if (possible_next_cpu >=3D nr_cpu_ids) {
 		dev_err(dev, "probe failed. Number of DPIOs exceeds NR_CPUS.\n");
 		err =3D -ERANGE;
 		goto err_allocate_irqs;
 	}
-	desc.cpu =3D next_cpu;
+	desc.cpu =3D possible_next_cpu;
+	cpumask_clear_cpu(possible_next_cpu, cpus_unused_mask);
=20
 	/*
 	 * Set the CENA regs to be the cache inhibited area of the portal to
@@ -211,7 +210,7 @@ static int dpaa2_dpio_remove(struct fsl_mc_device *dpio=
_dev)
 {
 	struct device *dev;
 	struct dpio_priv *priv;
-	int err;
+	int err =3D 0, cpu;
=20
 	dev =3D &dpio_dev->dev;
 	priv =3D dev_get_drvdata(dev);
@@ -220,6 +219,9 @@ static int dpaa2_dpio_remove(struct fsl_mc_device *dpio=
_dev)
=20
 	dpio_teardown_irqs(dpio_dev);
=20
+	cpu =3D dpaa2_io_get_cpu(priv->io);
+	cpumask_set_cpu(cpu, cpus_unused_mask);
+
 	err =3D fsl_mc_portal_allocate(dpio_dev, 0, &dpio_dev->mc_io);
 	if (err) {
 		dev_err(dev, "MC portal allocation failed\n");
@@ -267,11 +269,16 @@ static int dpaa2_dpio_remove(struct fsl_mc_device *dp=
io_dev)
=20
 static int dpio_driver_init(void)
 {
+	if (!zalloc_cpumask_var(&cpus_unused_mask, GFP_KERNEL))
+		return -ENOMEM;
+	cpumask_copy(cpus_unused_mask, cpu_online_mask);
+
 	return fsl_mc_driver_register(&dpaa2_dpio_driver);
 }
=20
 static void dpio_driver_exit(void)
 {
+	free_cpumask_var(cpus_unused_mask);
 	fsl_mc_driver_unregister(&dpaa2_dpio_driver);
 }
 module_init(dpio_driver_init);
diff --git a/drivers/soc/fsl/dpio/dpio-service.c b/drivers/soc/fsl/dpio/dpi=
o-service.c
index 21c3e32..3b60258 100644
--- a/drivers/soc/fsl/dpio/dpio-service.c
+++ b/drivers/soc/fsl/dpio/dpio-service.c
@@ -215,6 +215,19 @@ irqreturn_t dpaa2_io_irq(struct dpaa2_io *obj)
 }
=20
 /**
+ * dpaa2_io_get_cpu() - get the cpu associated with a given DPIO object
+ *
+ * @d: the given DPIO object.
+ *
+ * Return the cpu associated with the DPIO object
+ */
+int dpaa2_io_get_cpu(struct dpaa2_io *d)
+{
+	return d->dpio_desc.cpu;
+}
+EXPORT_SYMBOL(dpaa2_io_get_cpu);
+
+/**
  * dpaa2_io_service_register() - Prepare for servicing of FQDAN or CDAN
  *                               notifications on the given DPIO service.
  * @d:   the given DPIO service.
diff --git a/include/soc/fsl/dpaa2-io.h b/include/soc/fsl/dpaa2-io.h
index ab51e40..1c1764f 100644
--- a/include/soc/fsl/dpaa2-io.h
+++ b/include/soc/fsl/dpaa2-io.h
@@ -90,6 +90,8 @@ struct dpaa2_io_notification_ctx {
 	void *dpio_private;
 };
=20
+int dpaa2_io_get_cpu(struct dpaa2_io *d);
+
 int dpaa2_io_service_register(struct dpaa2_io *service,
 			      struct dpaa2_io_notification_ctx *ctx);
 void dpaa2_io_service_deregister(struct dpaa2_io *service,
--=20
1.9.1