MIME-Version: 1.0
References: <20220108205319.2046348-1-liambeguin@gmail.com> <20220108205319.2046348-10-liambeguin@gmail.com>
In-Reply-To: <20220108205319.2046348-10-liambeguin@gmail.com>
From:   Andy Shevchenko <andy.shevchenko@gmail.com>
Date:   Sun, 9 Jan 2022 15:02:01 +0200
Message-ID: <CAHp75Vd-FOV7BL0VjhGLyC5fhYXbW3x-hC5J1VvMT9W3Kfc_0Q@mail.gmail.com>
Subject: Re: [PATCH v12 09/16] iio: afe: rescale: fix accuracy for small
 fractional scales
To:     Liam Beguin <liambeguin@gmail.com>
Cc:     Peter Rosin <peda@axentia.se>, Jonathan Cameron <jic23@kernel.org>,
        Lars-Peter Clausen <lars@metafoo.de>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        linux-iio <linux-iio@vger.kernel.org>,
        devicetree <devicetree@vger.kernel.org>,
        Rob Herring <robh+dt@kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Sat, Jan 8, 2022 at 10:53 PM Liam Beguin <liambeguin@gmail.com> wrote:
>
> The approximation caused by integer divisions can be costly on smaller
> scale values since the decimal part is significant compared to the
> integer part. Switch to an IIO_VAL_INT_PLUS_NANO scale type in such
> cases to maintain accuracy.

...

> +               tmp = 1 << *val2;

Unfortunately, potential UB is still here. I think you missed a subtle
detail in the implementation of BIT()/BIT_ULL(). Let's put it here:

  #define BIT(nr) (UL(1) << (nr))

where

  #define UL(x) (_UL(x))
  #define _UL(x) (_AC(x, UL))

For non-assembler case

  #define __AC(X,Y) (X##Y)
  #define _AC(X,Y) __AC(X,Y)

Now you may easily see the difference.

...

> +               rem2 = *val % (int)tmp;
> +               *val = *val / (int)tmp;
> +
> +               *val2 = rem / (int)tmp;

Hmm... You divide s64 by 10^9, which means that the maximum value can
be ~10^10 / 2 (because 2^64-1 ~= 10^19), but this _might_ be bigger
than 'int' can hold. Can you confirm that tmp can't be so big?

-- 
With Best Regards,
Andy Shevchenko