Received: by 2002:ad5:474a:0:0:0:0:0 with SMTP id i10csp2880474imu; Sun, 11 Nov 2018 03:03:49 -0800 (PST) X-Google-Smtp-Source: AJdET5e7SI5BxsiFetAu5LFYCRYD99Bj2xcyGkdhy3MUkwowlGfHZjaShbwTMkKpe5H/TVodsVDf X-Received: by 2002:a62:5cc4:: with SMTP id q187-v6mr15749225pfb.47.1541934229891; Sun, 11 Nov 2018 03:03:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1541934229; cv=none; d=google.com; s=arc-20160816; b=qRseozR4ak+3Cu+yHiMNT0sfoJRKjm7OxjOY9sGez6O4ZqvLKXw5qL9xMoh/necDb4 ylCeTBrS2az4f6eFdayxxrNbwfSOty4pWVdeUhpgPuSgrBekrkxXyAwIPxGJBkjntEDK jCj3190+Ma2KNxRm8pQwU+6GQCZrXBzIENxH/uP0OgPB8/PTJg5lHbX9kmOJemPZ+Fqx cQ4bchuhifVT70E7tQFGBygslXT6Tb0uKsWU37wyxyGuQI59nuB/iIZ72olG8Izwm42M DoYQN+LBDyo3Yz5uYKejCbCYZYhQE6Ifg/S7kHJ1YCJ36LwRt0YlsV6YqUm/lrE3YaVG AR7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-disposition:mime-version:references:message-id:subject:cc :to:from:date; bh=tT85OQLGkNbKjRZdtH9gVhHt8DpLC8oT3KP8zt0l7Xs=; b=TLpZ7aK5I2/t80KXZAfkTjYxFRBa7aK98Yc8NmlPdRSPrVHsahz1r4u1YDOpS8v4RY +Jsif8l7s4BQO1oxxgLMGOMu10ukvleW3+duyR3mqKA63wVWA8hKQO6JHUiDx2cpE6sP 5E9PY1C6/p2CyzIorcaQrE2gEO6iEcQeuF44dARQ04MlngYHeMo1uaekwOB1GLlZQl6O wrVSb1qnLVRJL3+WIoEh0ov7QgRZdf21/BdshY/5Kl35iI6NNl7UcZhI1x+yQNRwbnAW zCyAkvFzcBgMSMflI9Yzbwc+YDzNHQ+OmRy79ux0namheB4sEwjtbmHxFE95EBuPTNCJ Et6w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id w8si15771pgm.467.2018.11.11.03.03.34; Sun, 11 Nov 2018 03:03:49 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727747AbeKKUvH (ORCPT + 99 others); Sun, 11 Nov 2018 15:51:07 -0500 Received: from wtarreau.pck.nerim.net ([62.212.114.60]:41860 "EHLO 1wt.eu" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727344AbeKKUvH (ORCPT ); Sun, 11 Nov 2018 15:51:07 -0500 Received: (from willy@localhost) by pcw.home.local (8.15.2/8.15.2/Submit) id wABB2aqv004374; Sun, 11 Nov 2018 12:02:36 +0100 Date: Sun, 11 Nov 2018 12:02:36 +0100 From: Willy Tarreau To: Florian Weimer Cc: "Michael Kerrisk (man-pages)" , Daniel Colascione , linux-kernel , Joel Fernandes , Linux API , Vlastimil Babka , "Carlos O'Donell" , "libc-alpha@sourceware.org" Subject: Re: Official Linux system wrapper library? Message-ID: <20181111110236.GA4189@1wt.eu> References: <20181111081725.GA30248@1wt.eu> <87y39zx5xa.fsf@oldenburg.str.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87y39zx5xa.fsf@oldenburg.str.redhat.com> User-Agent: Mutt/1.6.1 (2016-04-27) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sun, Nov 11, 2018 at 11:30:25AM +0100, Florian Weimer wrote: > * Willy Tarreau: > > > On Sun, Nov 11, 2018 at 07:55:30AM +0100, Michael Kerrisk (man-pages) wrote: > >> [1] https://sourceware.org/bugzilla/show_bug.cgi?id=6399 is a > >> longstanding example. > > > > This one was a sad read and shows that applications will continue to > > suffer from glibc's prehistorical view on operating systems and will > > continue to have to define their own syscall wrappers to exploit the > > full potential of the modern operating systems they execute on. > > What's modern about a 15-bit thread identifier? It's 15-bit on 32-bit systems, and 22 on 64-bit, hence you can have 4 million threads and/or processes on a single system image provided you have the resources for that of course. > I understand that using this interface is required in some cases (which > includes some system calls for which glibc does provide wrappers), but I > assumed that it was at least understood that these reusable IDs for > tasks were an extremely poor interface. Aren't the resulting bugs > common knowledge? Sure, just as are the bugs created by people trying to implement their own syscall wrappers. It's not by denying access to some native system interfaces that you will prevent users from accessing them, you'll just force them to work around the restriction and make things even worse. > > This reminds me when one had to write their own spinlocks and atomics > > many years ago. Seeing comments suggesting an application should open > > /proc/$PID makes me really wonder if people actually want to use slow > > and insecure applications designed this way. > > I don't understand. If you want a non-reusable identifier, you have to > go through the /proc interface anyway. I think the recommendation is to > use the PID/start time combination to get a unique process identifier or > something like that. It depends what you want to achieve. If you just need the tid, the one you'll pass to sched_setaffinity(), gettid() is fine. There are two issues with abusing /proc to emulate syscalls : - it's sometimes much slower than the equivalent syscall and can encourage users to cache the resulting values when they should not - either it's done upon process startup and it may not get valid value or may not work if /proc is not mounted yet (think init, mount etc), or it's done upon first use and can break daemons which chroot() themselves. Syscalls don't have such limitations and are much safer to use. For other things it's quite possible that you cannot rely on this syscall at all, it's not a solution to everything, but it's a nice solution to all cases where you need to access the system-wide identifier to pin a thread to a given CPU set or renice it. > I wanted to add gettid to glibc this cycle, but your comments suggest to > me that if we did this, we'd likely never get a proper non-reusable > thread identifier from the kernel. So I'm not sure what do anymore. "Look people, I was about to do what we all refused to do for 10 years now and Willy's comment made me change my mind, I'm sorry". The *real* argument that most users could understand is "guys, we're sorry, but we are running out of time and we won't work on this low priority stuff, so someone else will have to take care of it". In my opinion what matters is not whether or not people will use it appropriately, but that its validity, side effects and wrong assumptions are properly documented so that users don't shoot themselves in the foot. But I guess that most of those defining it by themselves already figured this out and are happy to use this available syscall when their application wants to make use of certain feature that are offered by their operating system. Thanks, Willy