From: Liam Beguin <[email protected]>
Some ADCs use IIO_VAL_INT_PLUS_{NANO,MICRO} scale types.
Add support for these to allow using the iio-rescaler with them.
Signed-off-by: Liam Beguin <[email protected]>
---
drivers/iio/afe/iio-rescale.c | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/drivers/iio/afe/iio-rescale.c b/drivers/iio/afe/iio-rescale.c
index 4c3cfd4d5181..a2b220b5ba86 100644
--- a/drivers/iio/afe/iio-rescale.c
+++ b/drivers/iio/afe/iio-rescale.c
@@ -92,7 +92,22 @@ static int rescale_read_raw(struct iio_dev *indio_dev,
do_div(tmp, 1000000000LL);
*val = tmp;
return ret;
+ case IIO_VAL_INT_PLUS_NANO:
+ tmp = ((s64)*val * 1000000000LL + *val2) * rescale->numerator;
+ do_div(tmp, rescale->denominator);
+
+ *val = div_s64(tmp, 1000000000LL);
+ *val2 = tmp - *val * 1000000000LL;
+ return ret;
+ case IIO_VAL_INT_PLUS_MICRO:
+ tmp = ((s64)*val * 1000000LL + *val2) * rescale->numerator;
+ do_div(tmp, rescale->denominator);
+
+ *val = div_s64(tmp, 1000000LL);
+ *val2 = tmp - *val * 1000000LL;
+ return ret;
default:
+ dev_err(&indio_dev->dev, "unsupported type %d\n", ret);
return -EOPNOTSUPP;
}
default:
--
2.30.1.489.g328c10930387
On 2021-07-15 05:12, Liam Beguin wrote:
> From: Liam Beguin <[email protected]>
>
> Some ADCs use IIO_VAL_INT_PLUS_{NANO,MICRO} scale types.
> Add support for these to allow using the iio-rescaler with them.
>
> Signed-off-by: Liam Beguin <[email protected]>
> ---
> drivers/iio/afe/iio-rescale.c | 15 +++++++++++++++
> 1 file changed, 15 insertions(+)
>
> diff --git a/drivers/iio/afe/iio-rescale.c b/drivers/iio/afe/iio-rescale.c
> index 4c3cfd4d5181..a2b220b5ba86 100644
> --- a/drivers/iio/afe/iio-rescale.c
> +++ b/drivers/iio/afe/iio-rescale.c
> @@ -92,7 +92,22 @@ static int rescale_read_raw(struct iio_dev *indio_dev,
> do_div(tmp, 1000000000LL);
> *val = tmp;
> return ret;
> + case IIO_VAL_INT_PLUS_NANO:
> + tmp = ((s64)*val * 1000000000LL + *val2) * rescale->numerator;
> + do_div(tmp, rescale->denominator);
> +
> + *val = div_s64(tmp, 1000000000LL);
> + *val2 = tmp - *val * 1000000000LL;
> + return ret;
This is too simplistic and prone to overflow. We need something like this
(untested)
tmp = (s64)*val * rescale->numerator;
rem = do_div(tmp, rescale->denominator);
*val = tmp;
tmp = ((s64)rem * 1000000000LL + (s64)*val2) * rescale->numerator;
do_div(tmp, rescale->denominator);
*val2 = tmp;
Still not very safe with numerator and denominator both "large", but much
better. And then we need normalizing the fraction part after the above, of
course.
And, of course, I'm not sure what *val == -1 and *val2 == 500000000 really
means. Is that -1.5 or -0.5? The above may very well need adjusting for
negative values...
Cheers,
Peter
> + case IIO_VAL_INT_PLUS_MICRO:
> + tmp = ((s64)*val * 1000000LL + *val2) * rescale->numerator;
> + do_div(tmp, rescale->denominator);
> +
> + *val = div_s64(tmp, 1000000LL);
> + *val2 = tmp - *val * 1000000LL;
> + return ret;
> default:
> + dev_err(&indio_dev->dev, "unsupported type %d\n", ret);
> return -EOPNOTSUPP;
> }
> default:
>
On Thu Jul 15, 2021 at 5:48 AM EDT, Peter Rosin wrote:
>
> On 2021-07-15 05:12, Liam Beguin wrote:
> > From: Liam Beguin <[email protected]>
> >
> > Some ADCs use IIO_VAL_INT_PLUS_{NANO,MICRO} scale types.
> > Add support for these to allow using the iio-rescaler with them.
> >
> > Signed-off-by: Liam Beguin <[email protected]>
> > ---
> > drivers/iio/afe/iio-rescale.c | 15 +++++++++++++++
> > 1 file changed, 15 insertions(+)
> >
> > diff --git a/drivers/iio/afe/iio-rescale.c b/drivers/iio/afe/iio-rescale.c
> > index 4c3cfd4d5181..a2b220b5ba86 100644
> > --- a/drivers/iio/afe/iio-rescale.c
> > +++ b/drivers/iio/afe/iio-rescale.c
> > @@ -92,7 +92,22 @@ static int rescale_read_raw(struct iio_dev *indio_dev,
> > do_div(tmp, 1000000000LL);
> > *val = tmp;
> > return ret;
> > + case IIO_VAL_INT_PLUS_NANO:
> > + tmp = ((s64)*val * 1000000000LL + *val2) * rescale->numerator;
> > + do_div(tmp, rescale->denominator);
> > +
> > + *val = div_s64(tmp, 1000000000LL);
> > + *val2 = tmp - *val * 1000000000LL;
> > + return ret;
>
> This is too simplistic and prone to overflow. We need something like
> this
> (untested)
>
> tmp = (s64)*val * rescale->numerator;
> rem = do_div(tmp, rescale->denominator);
> *val = tmp;
> tmp = ((s64)rem * 1000000000LL + (s64)*val2) * rescale->numerator;
> do_div(tmp, rescale->denominator);
> *val2 = tmp;
>
> Still not very safe with numerator and denominator both "large", but
> much
> better. And then we need normalizing the fraction part after the above,
> of
> course.
>
Understood, I'll test that.
> And, of course, I'm not sure what *val == -1 and *val2 == 500000000
> really
> means. Is that -1.5 or -0.5? The above may very well need adjusting for
> negative values...
>
I would've assumed the correct answer is -1 + 500000000e-9 = -0.5
but adding a test case to iio-test-format.c seems to return -1.5...
I believe that's a bug but we can work around if for now by moving the
integer part of *val2 to *val.
Liam
> Cheers,
> Peter
>
> > + case IIO_VAL_INT_PLUS_MICRO:
> > + tmp = ((s64)*val * 1000000LL + *val2) * rescale->numerator;
> > + do_div(tmp, rescale->denominator);
> > +
> > + *val = div_s64(tmp, 1000000LL);
> > + *val2 = tmp - *val * 1000000LL;
> > + return ret;
> > default:
> > + dev_err(&indio_dev->dev, "unsupported type %d\n", ret);
> > return -EOPNOTSUPP;
> > }
> > default:
> >
On 2021-07-16 21:18, Liam Beguin wrote:
> On Thu Jul 15, 2021 at 5:48 AM EDT, Peter Rosin wrote:
>>
>> On 2021-07-15 05:12, Liam Beguin wrote:
>>> From: Liam Beguin <[email protected]>
>>>
>>> Some ADCs use IIO_VAL_INT_PLUS_{NANO,MICRO} scale types.
>>> Add support for these to allow using the iio-rescaler with them.
>>>
>>> Signed-off-by: Liam Beguin <[email protected]>
>>> ---
>>> drivers/iio/afe/iio-rescale.c | 15 +++++++++++++++
>>> 1 file changed, 15 insertions(+)
>>>
>>> diff --git a/drivers/iio/afe/iio-rescale.c b/drivers/iio/afe/iio-rescale.c
>>> index 4c3cfd4d5181..a2b220b5ba86 100644
>>> --- a/drivers/iio/afe/iio-rescale.c
>>> +++ b/drivers/iio/afe/iio-rescale.c
>>> @@ -92,7 +92,22 @@ static int rescale_read_raw(struct iio_dev *indio_dev,
>>> do_div(tmp, 1000000000LL);
>>> *val = tmp;
>>> return ret;
>>> + case IIO_VAL_INT_PLUS_NANO:
>>> + tmp = ((s64)*val * 1000000000LL + *val2) * rescale->numerator;
>>> + do_div(tmp, rescale->denominator);
>>> +
>>> + *val = div_s64(tmp, 1000000000LL);
>>> + *val2 = tmp - *val * 1000000000LL;
>>> + return ret;
>>
>> This is too simplistic and prone to overflow. We need something like
>> this
>> (untested)
>>
>> tmp = (s64)*val * rescale->numerator;
>> rem = do_div(tmp, rescale->denominator);
>> *val = tmp;
>> tmp = ((s64)rem * 1000000000LL + (s64)*val2) * rescale->numerator;
>> do_div(tmp, rescale->denominator);
>> *val2 = tmp;
>>
>> Still not very safe with numerator and denominator both "large", but
>> much
>> better. And then we need normalizing the fraction part after the above,
>> of
>> course.
>>
>
> Understood, I'll test that.
I made a thinko. The remainder should not be re-multiplied with the
numerator...
tmp = (s64)*val * rescale->numerator;
rem = do_div(tmp, rescale->denominator);
*val = tmp;
tmp = (s64)rem * 1000000000LL + (s64)*val2 * rescale->numerator;
do_div(tmp, rescale->denominator);
*val2 = tmp;
And that actually reduces the risk of overflow too, which is nice!
Cheers,
Peter
>> And, of course, I'm not sure what *val == -1 and *val2 == 500000000
>> really
>> means. Is that -1.5 or -0.5? The above may very well need adjusting for
>> negative values...
>>
>
> I would've assumed the correct answer is -1 + 500000000e-9 = -0.5
> but adding a test case to iio-test-format.c seems to return -1.5...
>
> I believe that's a bug but we can work around if for now by moving the
> integer part of *val2 to *val.
>
> Liam
>
>> Cheers,
>> Peter
>>
>>> + case IIO_VAL_INT_PLUS_MICRO:
>>> + tmp = ((s64)*val * 1000000LL + *val2) * rescale->numerator;
>>> + do_div(tmp, rescale->denominator);
>>> +
>>> + *val = div_s64(tmp, 1000000LL);
>>> + *val2 = tmp - *val * 1000000LL;
>>> + return ret;
>>> default:
>>> + dev_err(&indio_dev->dev, "unsupported type %d\n", ret);
>>> return -EOPNOTSUPP;
>>> }
>>> default:
>>>
>
On Fri, 16 Jul 2021 15:18:33 -0400
"Liam Beguin" <[email protected]> wrote:
> On Thu Jul 15, 2021 at 5:48 AM EDT, Peter Rosin wrote:
> >
> > On 2021-07-15 05:12, Liam Beguin wrote:
> > > From: Liam Beguin <[email protected]>
> > >
> > > Some ADCs use IIO_VAL_INT_PLUS_{NANO,MICRO} scale types.
> > > Add support for these to allow using the iio-rescaler with them.
> > >
> > > Signed-off-by: Liam Beguin <[email protected]>
> > > ---
> > > drivers/iio/afe/iio-rescale.c | 15 +++++++++++++++
> > > 1 file changed, 15 insertions(+)
> > >
> > > diff --git a/drivers/iio/afe/iio-rescale.c b/drivers/iio/afe/iio-rescale.c
> > > index 4c3cfd4d5181..a2b220b5ba86 100644
> > > --- a/drivers/iio/afe/iio-rescale.c
> > > +++ b/drivers/iio/afe/iio-rescale.c
> > > @@ -92,7 +92,22 @@ static int rescale_read_raw(struct iio_dev *indio_dev,
> > > do_div(tmp, 1000000000LL);
> > > *val = tmp;
> > > return ret;
> > > + case IIO_VAL_INT_PLUS_NANO:
> > > + tmp = ((s64)*val * 1000000000LL + *val2) * rescale->numerator;
> > > + do_div(tmp, rescale->denominator);
> > > +
> > > + *val = div_s64(tmp, 1000000000LL);
> > > + *val2 = tmp - *val * 1000000000LL;
> > > + return ret;
> >
> > This is too simplistic and prone to overflow. We need something like
> > this
> > (untested)
> >
> > tmp = (s64)*val * rescale->numerator;
> > rem = do_div(tmp, rescale->denominator);
> > *val = tmp;
> > tmp = ((s64)rem * 1000000000LL + (s64)*val2) * rescale->numerator;
> > do_div(tmp, rescale->denominator);
> > *val2 = tmp;
> >
> > Still not very safe with numerator and denominator both "large", but
> > much
> > better. And then we need normalizing the fraction part after the above,
> > of
> > course.
> >
>
> Understood, I'll test that.
>
> > And, of course, I'm not sure what *val == -1 and *val2 == 500000000
> > really
> > means. Is that -1.5 or -0.5? The above may very well need adjusting for
> > negative values...
> >
>
> I would've assumed the correct answer is -1 + 500000000e-9 = -0.5
> but adding a test case to iio-test-format.c seems to return -1.5...
No. -1.5 is as intended, though the IIO_VAL_PLUS_MICRO is rather confusing
naming :( We should perhaps add more documentation for that. Signs were
always a bit of a pain with this two integer scheme for fixed point.
The intent is to have moderately readable look up tables with the problem that
we don't have a signed 0 available. Meh, maybe this decision a long time
back wasn't a the right one, but it may be a pain to change now as too many
drivers to check!
1, 0000000 == 1
0, 5000000 == 0.5
0, 0000000 == 0
0, -5000000 == -0.5
-1, 5000000 == -1.5
>
> I believe that's a bug but we can work around if for now by moving the
> integer part of *val2 to *val.
Yup. Fiddly corner cases..
Jonathan
>
> Liam
>
> > Cheers,
> > Peter
> >
> > > + case IIO_VAL_INT_PLUS_MICRO:
> > > + tmp = ((s64)*val * 1000000LL + *val2) * rescale->numerator;
> > > + do_div(tmp, rescale->denominator);
> > > +
> > > + *val = div_s64(tmp, 1000000LL);
> > > + *val2 = tmp - *val * 1000000LL;
> > > + return ret;
> > > default:
> > > + dev_err(&indio_dev->dev, "unsupported type %d\n", ret);
> > > return -EOPNOTSUPP;
> > > }
> > > default:
> > >
>
On Sat Jul 17, 2021 at 12:55 PM EDT, Jonathan Cameron wrote:
> On Fri, 16 Jul 2021 15:18:33 -0400
> "Liam Beguin" <[email protected]> wrote:
>
> > On Thu Jul 15, 2021 at 5:48 AM EDT, Peter Rosin wrote:
> > >
> > > On 2021-07-15 05:12, Liam Beguin wrote:
> > > > From: Liam Beguin <[email protected]>
> > > >
> > > > Some ADCs use IIO_VAL_INT_PLUS_{NANO,MICRO} scale types.
> > > > Add support for these to allow using the iio-rescaler with them.
> > > >
> > > > Signed-off-by: Liam Beguin <[email protected]>
> > > > ---
> > > > drivers/iio/afe/iio-rescale.c | 15 +++++++++++++++
> > > > 1 file changed, 15 insertions(+)
> > > >
> > > > diff --git a/drivers/iio/afe/iio-rescale.c b/drivers/iio/afe/iio-rescale.c
> > > > index 4c3cfd4d5181..a2b220b5ba86 100644
> > > > --- a/drivers/iio/afe/iio-rescale.c
> > > > +++ b/drivers/iio/afe/iio-rescale.c
> > > > @@ -92,7 +92,22 @@ static int rescale_read_raw(struct iio_dev *indio_dev,
> > > > do_div(tmp, 1000000000LL);
> > > > *val = tmp;
> > > > return ret;
> > > > + case IIO_VAL_INT_PLUS_NANO:
> > > > + tmp = ((s64)*val * 1000000000LL + *val2) * rescale->numerator;
> > > > + do_div(tmp, rescale->denominator);
> > > > +
> > > > + *val = div_s64(tmp, 1000000000LL);
> > > > + *val2 = tmp - *val * 1000000000LL;
> > > > + return ret;
> > >
> > > This is too simplistic and prone to overflow. We need something like
> > > this
> > > (untested)
> > >
> > > tmp = (s64)*val * rescale->numerator;
> > > rem = do_div(tmp, rescale->denominator);
> > > *val = tmp;
> > > tmp = ((s64)rem * 1000000000LL + (s64)*val2) * rescale->numerator;
> > > do_div(tmp, rescale->denominator);
> > > *val2 = tmp;
> > >
> > > Still not very safe with numerator and denominator both "large", but
> > > much
> > > better. And then we need normalizing the fraction part after the above,
> > > of
> > > course.
> > >
> >
> > Understood, I'll test that.
> >
> > > And, of course, I'm not sure what *val == -1 and *val2 == 500000000
> > > really
> > > means. Is that -1.5 or -0.5? The above may very well need adjusting for
> > > negative values...
> > >
> >
> > I would've assumed the correct answer is -1 + 500000000e-9 = -0.5
> > but adding a test case to iio-test-format.c seems to return -1.5...
>
Hi Jonathan,
> No. -1.5 is as intended, though the IIO_VAL_PLUS_MICRO is rather
> confusing
> naming :( We should perhaps add more documentation for that. Signs were
> always a bit of a pain with this two integer scheme for fixed point.
>
> The intent is to have moderately readable look up tables with the
> problem that
> we don't have a signed 0 available. Meh, maybe this decision a long time
> back wasn't a the right one, but it may be a pain to change now as too
> many
> drivers to check!
>
> 1, 0000000 == 1
> 0, 5000000 == 0.5
> 0, 0000000 == 0
> 0, -5000000 == -0.5
> -1, 5000000 == -1.5
>
Understood, thanks for clearing that out.
Liam
>
> >
> > I believe that's a bug but we can work around if for now by moving the
> > integer part of *val2 to *val.
>
> Yup. Fiddly corner cases..
>
> Jonathan
>
> >
> > Liam
> >
> > > Cheers,
> > > Peter
> > >
> > > > + case IIO_VAL_INT_PLUS_MICRO:
> > > > + tmp = ((s64)*val * 1000000LL + *val2) * rescale->numerator;
> > > > + do_div(tmp, rescale->denominator);
> > > > +
> > > > + *val = div_s64(tmp, 1000000LL);
> > > > + *val2 = tmp - *val * 1000000LL;
> > > > + return ret;
> > > > default:
> > > > + dev_err(&indio_dev->dev, "unsupported type %d\n", ret);
> > > > return -EOPNOTSUPP;
> > > > }
> > > > default:
> > > >
> >
On 2021-07-19 01:44, Liam Beguin wrote:
> On Sat Jul 17, 2021 at 12:55 PM EDT, Jonathan Cameron wrote:
>> On Fri, 16 Jul 2021 15:18:33 -0400
>> "Liam Beguin" <[email protected]> wrote:
>>
>>> On Thu Jul 15, 2021 at 5:48 AM EDT, Peter Rosin wrote:
>>>>
>>>> On 2021-07-15 05:12, Liam Beguin wrote:
>>>>> From: Liam Beguin <[email protected]>
>>>>>
>>>>> Some ADCs use IIO_VAL_INT_PLUS_{NANO,MICRO} scale types.
>>>>> Add support for these to allow using the iio-rescaler with them.
>>>>>
>>>>> Signed-off-by: Liam Beguin <[email protected]>
>>>>> ---
>>>>> drivers/iio/afe/iio-rescale.c | 15 +++++++++++++++
>>>>> 1 file changed, 15 insertions(+)
>>>>>
>>>>> diff --git a/drivers/iio/afe/iio-rescale.c b/drivers/iio/afe/iio-rescale.c
>>>>> index 4c3cfd4d5181..a2b220b5ba86 100644
>>>>> --- a/drivers/iio/afe/iio-rescale.c
>>>>> +++ b/drivers/iio/afe/iio-rescale.c
>>>>> @@ -92,7 +92,22 @@ static int rescale_read_raw(struct iio_dev *indio_dev,
>>>>> do_div(tmp, 1000000000LL);
>>>>> *val = tmp;
>>>>> return ret;
>>>>> + case IIO_VAL_INT_PLUS_NANO:
>>>>> + tmp = ((s64)*val * 1000000000LL + *val2) * rescale->numerator;
>>>>> + do_div(tmp, rescale->denominator);
>>>>> +
>>>>> + *val = div_s64(tmp, 1000000000LL);
>>>>> + *val2 = tmp - *val * 1000000000LL;
>>>>> + return ret;
>>>>
>>>> This is too simplistic and prone to overflow. We need something like
>>>> this
>>>> (untested)
>>>>
>>>> tmp = (s64)*val * rescale->numerator;
>>>> rem = do_div(tmp, rescale->denominator);
>>>> *val = tmp;
>>>> tmp = ((s64)rem * 1000000000LL + (s64)*val2) * rescale->numerator;
>>>> do_div(tmp, rescale->denominator);
>>>> *val2 = tmp;
>>>>
>>>> Still not very safe with numerator and denominator both "large", but
>>>> much
>>>> better. And then we need normalizing the fraction part after the above,
>>>> of
>>>> course.
>>>>
>>>
>>> Understood, I'll test that.
>>>
>>>> And, of course, I'm not sure what *val == -1 and *val2 == 500000000
>>>> really
>>>> means. Is that -1.5 or -0.5? The above may very well need adjusting for
>>>> negative values...
>>>>
>>>
>>> I would've assumed the correct answer is -1 + 500000000e-9 = -0.5
>>> but adding a test case to iio-test-format.c seems to return -1.5...
>>
>
> Hi Jonathan,
>
>> No. -1.5 is as intended, though the IIO_VAL_PLUS_MICRO is rather
>> confusing
>> naming :( We should perhaps add more documentation for that. Signs were
>> always a bit of a pain with this two integer scheme for fixed point.
>>
>> The intent is to have moderately readable look up tables with the
>> problem that
>> we don't have a signed 0 available. Meh, maybe this decision a long time
>> back wasn't a the right one, but it may be a pain to change now as too
>> many
>> drivers to check!
>>
>> 1, 0000000 == 1
>> 0, 5000000 == 0.5
>> 0, 0000000 == 0
>> 0, -5000000 == -0.5
>> -1, 5000000 == -1.5
>>
>
> Understood, thanks for clearing that out.
I just realized that do_div assumes unsigned operands...
:-(
Cheers,
Peter
On Mon Jul 19, 2021 at 4:31 AM EDT, Peter Rosin wrote:
>
>
> On 2021-07-19 01:44, Liam Beguin wrote:
> > On Sat Jul 17, 2021 at 12:55 PM EDT, Jonathan Cameron wrote:
> >> On Fri, 16 Jul 2021 15:18:33 -0400
> >> "Liam Beguin" <[email protected]> wrote:
> >>
> >>> On Thu Jul 15, 2021 at 5:48 AM EDT, Peter Rosin wrote:
> >>>>
> >>>> On 2021-07-15 05:12, Liam Beguin wrote:
> >>>>> From: Liam Beguin <[email protected]>
> >>>>>
> >>>>> Some ADCs use IIO_VAL_INT_PLUS_{NANO,MICRO} scale types.
> >>>>> Add support for these to allow using the iio-rescaler with them.
> >>>>>
> >>>>> Signed-off-by: Liam Beguin <[email protected]>
> >>>>> ---
> >>>>> drivers/iio/afe/iio-rescale.c | 15 +++++++++++++++
> >>>>> 1 file changed, 15 insertions(+)
> >>>>>
> >>>>> diff --git a/drivers/iio/afe/iio-rescale.c b/drivers/iio/afe/iio-rescale.c
> >>>>> index 4c3cfd4d5181..a2b220b5ba86 100644
> >>>>> --- a/drivers/iio/afe/iio-rescale.c
> >>>>> +++ b/drivers/iio/afe/iio-rescale.c
> >>>>> @@ -92,7 +92,22 @@ static int rescale_read_raw(struct iio_dev *indio_dev,
> >>>>> do_div(tmp, 1000000000LL);
> >>>>> *val = tmp;
> >>>>> return ret;
> >>>>> + case IIO_VAL_INT_PLUS_NANO:
> >>>>> + tmp = ((s64)*val * 1000000000LL + *val2) * rescale->numerator;
> >>>>> + do_div(tmp, rescale->denominator);
> >>>>> +
> >>>>> + *val = div_s64(tmp, 1000000000LL);
> >>>>> + *val2 = tmp - *val * 1000000000LL;
> >>>>> + return ret;
> >>>>
> >>>> This is too simplistic and prone to overflow. We need something like
> >>>> this
> >>>> (untested)
> >>>>
> >>>> tmp = (s64)*val * rescale->numerator;
> >>>> rem = do_div(tmp, rescale->denominator);
> >>>> *val = tmp;
> >>>> tmp = ((s64)rem * 1000000000LL + (s64)*val2) * rescale->numerator;
> >>>> do_div(tmp, rescale->denominator);
> >>>> *val2 = tmp;
> >>>>
> >>>> Still not very safe with numerator and denominator both "large", but
> >>>> much
> >>>> better. And then we need normalizing the fraction part after the above,
> >>>> of
> >>>> course.
> >>>>
> >>>
> >>> Understood, I'll test that.
> >>>
> >>>> And, of course, I'm not sure what *val == -1 and *val2 == 500000000
> >>>> really
> >>>> means. Is that -1.5 or -0.5? The above may very well need adjusting for
> >>>> negative values...
> >>>>
> >>>
> >>> I would've assumed the correct answer is -1 + 500000000e-9 = -0.5
> >>> but adding a test case to iio-test-format.c seems to return -1.5...
> >>
> >
> > Hi Jonathan,
> >
> >> No. -1.5 is as intended, though the IIO_VAL_PLUS_MICRO is rather
> >> confusing
> >> naming :( We should perhaps add more documentation for that. Signs were
> >> always a bit of a pain with this two integer scheme for fixed point.
> >>
> >> The intent is to have moderately readable look up tables with the
> >> problem that
> >> we don't have a signed 0 available. Meh, maybe this decision a long time
> >> back wasn't a the right one, but it may be a pain to change now as too
> >> many
> >> drivers to check!
> >>
> >> 1, 0000000 == 1
> >> 0, 5000000 == 0.5
> >> 0, 0000000 == 0
> >> 0, -5000000 == -0.5
> >> -1, 5000000 == -1.5
> >>
> >
> > Understood, thanks for clearing that out.
Hi Peter,
>
> I just realized that do_div assumes unsigned operands...
>
> :-(
I noticed the same thing after adding the kunit tests.
I added patches for that.
For IIO_VAL_PLUS_{MICRO,NANO} specifically, I have something working but
I like your approach better so I'll work on it a little more.
Thanks,
Liam
>
> Cheers,
> Peter