In a system with a single initiator node, and one or more memory-only
'target' nodes, the memory-only node(s) would fail to register their
initiator node correctly. i.e. in sysfs:
# ls /sys/devices/system/node/node0/access0/targets/
node0
Where as the correct behavior should be:
# ls /sys/devices/system/node/node0/access0/targets/
node0 node1
This happened because hmat_register_target_initiators() uses list_sort()
to sort the initiator list, but the sort comparision function
(initiator_cmp()) is overloaded to also set the node mask's bits.
In a system with a single initiator, the list is singular, and list_sort
elides the comparision helper call. Thus the node mask never gets set,
and the subsequent search for the best initiator comes up empty.
Add a new helper to sort the initiator list, and handle the singular
list corner case by setting the node mask for that explicitly.
Reported-by: Chris Piper <[email protected]>
Cc: <[email protected]>
Cc: Rafael J. Wysocki <[email protected]>
Cc: Liu Shixin <[email protected]>
Cc: Dan Williams <[email protected]>
Signed-off-by: Vishal Verma <[email protected]>
---
drivers/acpi/numa/hmat.c | 32 ++++++++++++++++++++++++++++++--
1 file changed, 30 insertions(+), 2 deletions(-)
diff --git a/drivers/acpi/numa/hmat.c b/drivers/acpi/numa/hmat.c
index 144a84f429ed..cd20b0e9cdfa 100644
--- a/drivers/acpi/numa/hmat.c
+++ b/drivers/acpi/numa/hmat.c
@@ -573,6 +573,30 @@ static int initiator_cmp(void *priv, const struct list_head *a,
return ia->processor_pxm - ib->processor_pxm;
}
+static int initiators_to_nodemask(unsigned long *p_nodes)
+{
+ /*
+ * list_sort doesn't call @cmp (initiator_cmp) for 0 or 1 sized lists.
+ * For a single-initiator system with other memory-only nodes, this
+ * means an empty p_nodes mask, since that is set by initiator_cmp().
+ * Special case the singular list, and make sure the node mask gets set
+ * appropriately.
+ */
+ if (list_empty(&initiators))
+ return -ENXIO;
+
+ if (list_is_singular(&initiators)) {
+ struct memory_initiator *initiator = list_first_entry(
+ &initiators, struct memory_initiator, node);
+
+ set_bit(initiator->processor_pxm, p_nodes);
+ return 0;
+ }
+
+ list_sort(p_nodes, &initiators, initiator_cmp);
+ return 0;
+}
+
static void hmat_register_target_initiators(struct memory_target *target)
{
static DECLARE_BITMAP(p_nodes, MAX_NUMNODES);
@@ -609,7 +633,9 @@ static void hmat_register_target_initiators(struct memory_target *target)
* initiators.
*/
bitmap_zero(p_nodes, MAX_NUMNODES);
- list_sort(p_nodes, &initiators, initiator_cmp);
+ if (initiators_to_nodemask(p_nodes) < 0)
+ return;
+
if (!access0done) {
for (i = WRITE_LATENCY; i <= READ_BANDWIDTH; i++) {
loc = localities_types[i];
@@ -643,7 +669,9 @@ static void hmat_register_target_initiators(struct memory_target *target)
/* Access 1 ignores Generic Initiators */
bitmap_zero(p_nodes, MAX_NUMNODES);
- list_sort(p_nodes, &initiators, initiator_cmp);
+ if (initiators_to_nodemask(p_nodes) < 0)
+ return;
+
for (i = WRITE_LATENCY; i <= READ_BANDWIDTH; i++) {
loc = localities_types[i];
if (!loc)
--
2.38.1
On Wed, Nov 16, 2022 at 12:57:36AM -0700, Vishal Verma wrote:
> In a system with a single initiator node, and one or more memory-only
> 'target' nodes, the memory-only node(s) would fail to register their
> initiator node correctly. i.e. in sysfs:
>
> # ls /sys/devices/system/node/node0/access0/targets/
> node0
>
> Where as the correct behavior should be:
>
> # ls /sys/devices/system/node/node0/access0/targets/
> node0 node1
>
> This happened because hmat_register_target_initiators() uses list_sort()
> to sort the initiator list, but the sort comparision function
> (initiator_cmp()) is overloaded to also set the node mask's bits.
>
> In a system with a single initiator, the list is singular, and list_sort
> elides the comparision helper call. Thus the node mask never gets set,
> and the subsequent search for the best initiator comes up empty.
>
> Add a new helper to sort the initiator list, and handle the singular
> list corner case by setting the node mask for that explicitly.
>
> Reported-by: Chris Piper <[email protected]>
> Cc: <[email protected]>
> Cc: Rafael J. Wysocki <[email protected]>
> Cc: Liu Shixin <[email protected]>
> Cc: Dan Williams <[email protected]>
> Signed-off-by: Vishal Verma <[email protected]>
> ---
> drivers/acpi/numa/hmat.c | 32 ++++++++++++++++++++++++++++++--
> 1 file changed, 30 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/acpi/numa/hmat.c b/drivers/acpi/numa/hmat.c
> index 144a84f429ed..cd20b0e9cdfa 100644
> --- a/drivers/acpi/numa/hmat.c
> +++ b/drivers/acpi/numa/hmat.c
> @@ -573,6 +573,30 @@ static int initiator_cmp(void *priv, const struct list_head *a,
> return ia->processor_pxm - ib->processor_pxm;
> }
>
> +static int initiators_to_nodemask(unsigned long *p_nodes)
> +{
> + /*
> + * list_sort doesn't call @cmp (initiator_cmp) for 0 or 1 sized lists.
> + * For a single-initiator system with other memory-only nodes, this
> + * means an empty p_nodes mask, since that is set by initiator_cmp().
> + * Special case the singular list, and make sure the node mask gets set
> + * appropriately.
> + */
> + if (list_empty(&initiators))
> + return -ENXIO;
> +
> + if (list_is_singular(&initiators)) {
> + struct memory_initiator *initiator = list_first_entry(
> + &initiators, struct memory_initiator, node);
> +
> + set_bit(initiator->processor_pxm, p_nodes);
> + return 0;
> + }
> +
> + list_sort(p_nodes, &initiators, initiator_cmp);
> + return 0;
> +}
> +
Hm. I think it indicates that these set_bit()s do not belong to
initiator_cmp().
Maybe remove both set_bit() from the compare helper and walk the list
separately to initialize the node mask? I think it will be easier to
follow.
--
Kiryl Shutsemau / Kirill A. Shutemov
On Wed, 2022-11-16 at 15:46 +0300, Kirill A. Shutemov wrote:
> On Wed, Nov 16, 2022 at 12:57:36AM -0700, Vishal Verma wrote:
> > In a system with a single initiator node, and one or more memory-only
> > 'target' nodes, the memory-only node(s) would fail to register their
> > initiator node correctly. i.e. in sysfs:
> >
> > # ls /sys/devices/system/node/node0/access0/targets/
> > node0
> >
> > Where as the correct behavior should be:
> >
> > # ls /sys/devices/system/node/node0/access0/targets/
> > node0 node1
> >
> > This happened because hmat_register_target_initiators() uses list_sort()
> > to sort the initiator list, but the sort comparision function
> > (initiator_cmp()) is overloaded to also set the node mask's bits.
> >
> > In a system with a single initiator, the list is singular, and list_sort
> > elides the comparision helper call. Thus the node mask never gets set,
> > and the subsequent search for the best initiator comes up empty.
> >
> > Add a new helper to sort the initiator list, and handle the singular
> > list corner case by setting the node mask for that explicitly.
> >
> > Reported-by: Chris Piper <[email protected]>
> > Cc: <[email protected]>
> > Cc: Rafael J. Wysocki <[email protected]>
> > Cc: Liu Shixin <[email protected]>
> > Cc: Dan Williams <[email protected]>
> > Signed-off-by: Vishal Verma <[email protected]>
> > ---
> > drivers/acpi/numa/hmat.c | 32 ++++++++++++++++++++++++++++++--
> > 1 file changed, 30 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/acpi/numa/hmat.c b/drivers/acpi/numa/hmat.c
> > index 144a84f429ed..cd20b0e9cdfa 100644
> > --- a/drivers/acpi/numa/hmat.c
> > +++ b/drivers/acpi/numa/hmat.c
> > @@ -573,6 +573,30 @@ static int initiator_cmp(void *priv, const struct list_head *a,
> > return ia->processor_pxm - ib->processor_pxm;
> > }
> >
> > +static int initiators_to_nodemask(unsigned long *p_nodes)
> > +{
> > + /*
> > + * list_sort doesn't call @cmp (initiator_cmp) for 0 or 1 sized lists.
> > + * For a single-initiator system with other memory-only nodes, this
> > + * means an empty p_nodes mask, since that is set by initiator_cmp().
> > + * Special case the singular list, and make sure the node mask gets set
> > + * appropriately.
> > + */
> > + if (list_empty(&initiators))
> > + return -ENXIO;
> > +
> > + if (list_is_singular(&initiators)) {
> > + struct memory_initiator *initiator = list_first_entry(
> > + &initiators, struct memory_initiator, node);
> > +
> > + set_bit(initiator->processor_pxm, p_nodes);
> > + return 0;
> > + }
> > +
> > + list_sort(p_nodes, &initiators, initiator_cmp);
> > + return 0;
> > +}
> > +
>
> Hm. I think it indicates that these set_bit()s do not belong to
> initiator_cmp().
>
> Maybe remove both set_bit() from the compare helper and walk the list
> separately to initialize the node mask? I think it will be easier to
> follow.
Yes - I thuoght about this, but went with the seemingly less intrusive
change. I can send a v2 which separates out the set_bit()s. I agree
that's cleaner and easier to follow than overloading initiator_cmp().
On Wed, Nov 16, 2022 at 06:02:32PM +0000, Verma, Vishal L wrote:
> On Wed, 2022-11-16 at 15:46 +0300, Kirill A. Shutemov wrote:
> > On Wed, Nov 16, 2022 at 12:57:36AM -0700, Vishal Verma wrote:
> > > In a system with a single initiator node, and one or more memory-only
> > > 'target' nodes, the memory-only node(s) would fail to register their
> > > initiator node correctly. i.e. in sysfs:
> > >
> > > ? # ls /sys/devices/system/node/node0/access0/targets/
> > > ? node0
> > >
> > > Where as the correct behavior should be:
> > >
> > > ? # ls /sys/devices/system/node/node0/access0/targets/
> > > ? node0 node1
> > >
> > > This happened because hmat_register_target_initiators() uses list_sort()
> > > to sort the initiator list, but the sort comparision function
> > > (initiator_cmp()) is overloaded to also set the node mask's bits.
> > >
> > > In a system with a single initiator, the list is singular, and list_sort
> > > elides the comparision helper call. Thus the node mask never gets set,
> > > and the subsequent search for the best initiator comes up empty.
> > >
> > > Add a new helper to sort the initiator list, and handle the singular
> > > list corner case by setting the node mask for that explicitly.
> > >
> > > Reported-by: Chris Piper <[email protected]>
> > > Cc: <[email protected]>
> > > Cc: Rafael J. Wysocki <[email protected]>
> > > Cc: Liu Shixin <[email protected]>
> > > Cc: Dan Williams <[email protected]>
> > > Signed-off-by: Vishal Verma <[email protected]>
> > > ---
> > > ?drivers/acpi/numa/hmat.c | 32 ++++++++++++++++++++++++++++++--
> > > ?1 file changed, 30 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/drivers/acpi/numa/hmat.c b/drivers/acpi/numa/hmat.c
> > > index 144a84f429ed..cd20b0e9cdfa 100644
> > > --- a/drivers/acpi/numa/hmat.c
> > > +++ b/drivers/acpi/numa/hmat.c
> > > @@ -573,6 +573,30 @@ static int initiator_cmp(void *priv, const struct list_head *a,
> > > ????????return ia->processor_pxm - ib->processor_pxm;
> > > ?}
> > > ?
> > > +static int initiators_to_nodemask(unsigned long *p_nodes)
> > > +{
> > > +???????/*
> > > +??????? * list_sort doesn't call @cmp (initiator_cmp) for 0 or 1 sized lists.
> > > +??????? * For a single-initiator system with other memory-only nodes, this
> > > +??????? * means an empty p_nodes mask, since that is set by initiator_cmp().
> > > +??????? * Special case the singular list, and make sure the node mask gets set
> > > +??????? * appropriately.
> > > +??????? */
> > > +???????if (list_empty(&initiators))
> > > +???????????????return -ENXIO;
> > > +
> > > +???????if (list_is_singular(&initiators)) {
> > > +???????????????struct memory_initiator *initiator = list_first_entry(
> > > +???????????????????????&initiators, struct memory_initiator, node);
> > > +
> > > +???????????????set_bit(initiator->processor_pxm, p_nodes);
> > > +???????????????return 0;
> > > +???????}
> > > +
> > > +???????list_sort(p_nodes, &initiators, initiator_cmp);
> > > +???????return 0;
> > > +}
> > > +
> >
> > Hm. I think it indicates that these set_bit()s do not belong to
> > initiator_cmp().
> >
> > Maybe remove both set_bit() from the compare helper and walk the list
> > separately to initialize the node mask? I think it will be easier to
> > follow.
>
>
> Yes - I thuoght about this, but went with the seemingly less intrusive
> change. I can send a v2 which separates out the set_bit()s. I agree
> that's cleaner and easier to follow than overloading initiator_cmp().
Yes, please make v2.
With current implementation set_bit() can be called multiple times on the
same initiator, depending on placement of the initiator in the list.
It is totally wrong place.
--
Kiryl Shutsemau / Kirill A. Shutemov