Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753030AbdDNONW (ORCPT ); Fri, 14 Apr 2017 10:13:22 -0400 Received: from saturn.retrosnub.co.uk ([178.18.118.26]:56268 "EHLO saturn.retrosnub.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751544AbdDNONT (ORCPT ); Fri, 14 Apr 2017 10:13:19 -0400 Subject: Re: [PATCH v2] IIO: bmp280-core.c: fix error in humidity calculation To: Matt Ranostay , Andreas Klinger References: <20170410170001.GA2524@project> Cc: Linus Walleij , Hartmut Knaack , Lars-Peter Clausen , Peter Meerwald , Vlad Dogaru , Akinobu Mita , Wei Yongjun , Aniroop Mathur , "linux-iio@vger.kernel.org" , "linux-kernel@vger.kernel.org" From: Jonathan Cameron Message-ID: <0955126a-e408-cf59-3f38-cd6daf4998ff@kernel.org> Date: Fri, 14 Apr 2017 15:13:16 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2075 Lines: 54 On 12/04/17 00:59, Matt Ranostay wrote: > On Mon, Apr 10, 2017 at 10:00 AM, Andreas Klinger wrote: >> While calculating the compensation of the humidity there are negative values >> interpreted as unsigned because of unsigned variables used. These values as >> well as the constants need to be casted to signed as indicated by the >> documentation of the sensor. >> >> Changes in v2: >> - cast also constants to signed; otherwise calculation is not correct >> >> I removed the Acked-By of Linus Walleij because of another change in the >> calculation. Don't know how it should be treated. >> >> Signed-off-by: Andreas Klinger > > Reviewed-by: Matt Ranostay Applied to the fixes-togreg branch of iio.git and marked for stable. Thanks, Jonathan > >> --- >> drivers/iio/pressure/bmp280-core.c | 11 ++++++----- >> 1 file changed, 6 insertions(+), 5 deletions(-) >> >> diff --git a/drivers/iio/pressure/bmp280-core.c b/drivers/iio/pressure/bmp280-core.c >> index 4d18826ac63c..d82b788374b6 100644 >> --- a/drivers/iio/pressure/bmp280-core.c >> +++ b/drivers/iio/pressure/bmp280-core.c >> @@ -175,11 +175,12 @@ static u32 bmp280_compensate_humidity(struct bmp280_data *data, >> } >> H6 = sign_extend32(tmp, 7); >> >> - var = ((s32)data->t_fine) - 76800; >> - var = ((((adc_humidity << 14) - (H4 << 20) - (H5 * var)) + 16384) >> 15) >> - * (((((((var * H6) >> 10) * (((var * H3) >> 11) + 32768)) >> 10) >> - + 2097152) * H2 + 8192) >> 14); >> - var -= ((((var >> 15) * (var >> 15)) >> 7) * H1) >> 4; >> + var = ((s32)data->t_fine) - (s32)76800; >> + var = ((((adc_humidity << 14) - (H4 << 20) - (H5 * var)) >> + + (s32)16384) >> 15) * (((((((var * H6) >> 10) >> + * (((var * (s32)H3) >> 11) + (s32)32768)) >> 10) >> + + (s32)2097152) * H2 + 8192) >> 14); >> + var -= ((((var >> 15) * (var >> 15)) >> 7) * (s32)H1) >> 4; >> >> return var >> 12; >> }; >> -- >> 2.1.4 >> >> >> --