Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756095Ab3JHXMc (ORCPT ); Tue, 8 Oct 2013 19:12:32 -0400 Received: from relay1.mentorg.com ([192.94.38.131]:45828 "EHLO relay1.mentorg.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755555Ab3JHXMb (ORCPT ); Tue, 8 Oct 2013 19:12:31 -0400 Date: Tue, 8 Oct 2013 23:12:27 +0000 From: "Joseph S. Myers" X-X-Sender: jsm28@digraph.polyomino.org.uk To: CC: Subject: [PATCH] math-emu: fix floating-point to integer unsigned saturation Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" X-OriginalArrivalTime: 08 Oct 2013 23:12:30.0426 (UTC) FILETIME=[DCC543A0:01CEC47B] Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1957 Lines: 54 From: Joseph Myers The math-emu macros _FP_TO_INT and _FP_TO_INT_ROUND are supposed to saturate their results for out-of-range arguments, except in the case rsigned == 2 (when instead the low bits of the result are taken). However, in the case rsigned == 0 (converting to unsigned integers), they mistakenly produce 0 for positive results and the maximum unsigned integer for negative results, the opposite of correct unsigned saturation. This patch fixes the logic. Signed-off-by: Joseph Myers --- I intend to make the corresponding changes to the glibc/libgcc copy of this code, given that it would be desirable to resync the Linux and glibc/libgcc copies (the latter has had many enhancements and bug fixes since it was copied into Linux), although strictly this incorrect saturation is only a bug when trying to emulate particular instruction semantics, not when used in userspace to implement C operations where the results of out-of-range conversions are unspecified or undefined. diff --git a/include/math-emu/op-common.h b/include/math-emu/op-common.h index 9696a5e..70fe5e9 100644 --- a/include/math-emu/op-common.h +++ b/include/math-emu/op-common.h @@ -685,7 +685,7 @@ do { \ else \ { \ r = 0; \ - if (X##_s) \ + if (!X##_s) \ r = ~r; \ } \ FP_SET_EXCEPTION(FP_EX_INVALID); \ @@ -762,7 +762,7 @@ do { \ if (!rsigned) \ { \ r = 0; \ - if (X##_s) \ + if (!X##_s) \ r = ~r; \ } \ else if (rsigned != 2) \ -- Joseph S. Myers joseph@codesourcery.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/