2014-12-10 16:33:11

by Rasmus Villemoes

Subject: [PATCH] fs: hfs: Fix comparison bug in hfs_cat_keycmp

Relying on the sign (after casting to int) of the difference of two
quantities for comparison is usually wrong. For example, should a-b
turn out to be 2^31, the return value of cmp(a,b) is -2^31; but that
would also be the return value from cmp(b, a). So a compares less than
b and b compares less than a. One can also easily find three values
a,b,c such that a compares less than b, b compares less than c, but a
does not compare less than c.

Signed-off-by: Rasmus Villemoes <[email protected]>
---
fs/hfs/catalog.c | 14 ++++++++------
1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/fs/hfs/catalog.c b/fs/hfs/catalog.c
index ff0316b925a5..db458ee3a546 100644
--- a/fs/hfs/catalog.c
+++ b/fs/hfs/catalog.c
@@ -162,14 +162,16 @@ err2:
*/
int hfs_cat_keycmp(const btree_key *key1, const btree_key *key2)
{
- int retval;
+ __be32 k1p, k2p;

- retval = be32_to_cpu(key1->cat.ParID) - be32_to_cpu(key2->cat.ParID);
- if (!retval)
- retval = hfs_strcmp(key1->cat.CName.name, key1->cat.CName.len,
- key2->cat.CName.name, key2->cat.CName.len);
+ k1p = key1->cat.ParID;
+ k2p = key2->cat.ParID;

- return retval;
+ if (k1p != k2p)
+ return be32_to_cpu(k1p) < be32_to_cpu(k2p) ? -1 : 1;
+
+ return hfs_strcmp(key1->cat.CName.name, key1->cat.CName.len,
+ key2->cat.CName.name, key2->cat.CName.len);
}

/* Try to get a catalog entry for given catalog id */
--
2.1.3

2014-12-10 19:24:23

by Viacheslav Dubeyko

Subject: Re: [PATCH] fs: hfs: Fix comparison bug in hfs_cat_keycmp

On Wed, 2014-12-10 at 17:32 +0100, Rasmus Villemoes wrote:
> Relying on the sign (after casting to int) of the difference of two
> quantities for comparison is usually wrong. For example, should a-b
> turn out to be 2^31, the return value of cmp(a,b) is -2^31; but that
> would also be the return value from cmp(b, a). So a compares less than
> b and b compares less than a. One can also easily find three values
> a,b,c such that a compares less than b, b compares less than c, but a
> does not compare less than c.
>

Looks good for me. Thank you for the fix.

Reviewed-by: Vyacheslav Dubeyko <[email protected]>

Thanks,
Vyacheslav Dubeyko.

> Signed-off-by: Rasmus Villemoes <[email protected]>
> ---
> fs/hfs/catalog.c | 14 ++++++++------
> 1 file changed, 8 insertions(+), 6 deletions(-)
>
> diff --git a/fs/hfs/catalog.c b/fs/hfs/catalog.c
> index ff0316b925a5..db458ee3a546 100644
> --- a/fs/hfs/catalog.c
> +++ b/fs/hfs/catalog.c
> @@ -162,14 +162,16 @@ err2:
> */
> int hfs_cat_keycmp(const btree_key *key1, const btree_key *key2)
> {
> - int retval;
> + __be32 k1p, k2p;
>
> - retval = be32_to_cpu(key1->cat.ParID) - be32_to_cpu(key2->cat.ParID);
> - if (!retval)
> - retval = hfs_strcmp(key1->cat.CName.name, key1->cat.CName.len,
> - key2->cat.CName.name, key2->cat.CName.len);
> + k1p = key1->cat.ParID;
> + k2p = key2->cat.ParID;
>
> - return retval;
> + if (k1p != k2p)
> + return be32_to_cpu(k1p) < be32_to_cpu(k2p) ? -1 : 1;
> +
> + return hfs_strcmp(key1->cat.CName.name, key1->cat.CName.len,
> + key2->cat.CName.name, key2->cat.CName.len);
> }
>
> /* Try to get a catalog entry for given catalog id */