Received: by 2002:ab2:7903:0:b0:1fb:b500:807b with SMTP id a3csp675765lqj; Sun, 2 Jun 2024 17:33:36 -0700 (PDT) X-Forwarded-Encrypted: i=3; AJvYcCW/aHtH7F3C6DHJq/faD3GzlY6apCpRQJgmLjl2dgoaMuLEs0ANz6NQJNzyuZfU12VE+q/9ZQLdP0FX9j0ZWOyTH0eev/GMDc/0dp8/0A== X-Google-Smtp-Source: AGHT+IHqKzHj/+TZTmPBWTuD77JWlPjYubBHgQC8r7wbBOoOlW2H/noLC0YukpdJQYOp0GEjzmdZ X-Received: by 2002:a05:6512:3b0:b0:52b:9530:f0b9 with SMTP id 2adb3069b0e04-52b9530f164mr2364664e87.67.1717374816767; Sun, 02 Jun 2024 17:33:36 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1717374816; cv=pass; d=google.com; s=arc-20160816; b=gJKG1PI9nlzjd/BeJcVA6OXnjoCzsPG1WqL82CyzsmwxTU62d9LVny2Fnqw30nlxnP 650YWZHS85pRf3Seyg2PjVoV+FOf3sv5VUy2tCtZkVl/vVNSdEyvCoDkoiWacjQtV5Lb vFZtT9SS74QpyA49PUhYJFXvdwZjgoGAXXE6xAAIbkh6pMF59qmy0Jg6XoCZNsSewUP3 Q5pdgeqJh9lwrtGnaaZP30AyUfZJ1h4CVtFUEga80PrI6pSqqDVxoruqFtQID0qzm3T0 7umO/UgEENLGnNfg5SDgPPnWNAdRlfwwl33TT+SwIOpn3xytmKR6opIH8M81LtPv8QLO YvzA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:message-id:date:subject:cc:to :from:dkim-signature; bh=IeBJ/5oPlb9g+ArFLQs0RP3mSJ+NwqtBpi4ZC0JDGNM=; fh=zHmvfkjdNfBuJ8M3ivq9Vd8L3gqi7cNjIbsvZkBSDEk=; b=AyFK2qsZU0syTOC0tXlEJk5Rpc1ty7dk5ex4jW/JRIhlc8NsHxL7R1di7POu53ve3o 07cb+cC/u43oTnRz+UCnqf+UaWQ99HgBr3OfTk2co51rPg8qZOtxW24TccK9t0sBfKZ+ AqxJkuMvrA6QNsCygd5qq7Zz1qYPuXbek3K/aTgYbNzrcOL1UjgkpjdtdsGLaHAoke7c Bcqv9ahIBJA3iJIAaIXDURg4R7tLR3oP4rhDwFwDzucUuTOw1Wx3QMfl+fQ3cElTupPx swE0xNvUYTOdh2m/fgXtKcDH4qqZ3kx95snH6OxrLBHVQATiUMH7RTO5edVrdAu69lbo +aYQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=OYnoPBg6; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-198500-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-198500-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Return-Path: Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id a640c23a62f3a-a690a9b7074si55119866b.68.2024.06.02.17.33.36 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jun 2024 17:33:36 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel+bounces-198500-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=OYnoPBg6; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-198500-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-198500-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 77E551F213F6 for ; Mon, 3 Jun 2024 00:33:36 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8B3C023CE; Mon, 3 Jun 2024 00:33:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="OYnoPBg6" Received: from out-176.mta0.migadu.com (out-176.mta0.migadu.com [91.218.175.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 31E1917C9 for ; Mon, 3 Jun 2024 00:33:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.176 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717374800; cv=none; b=UW/cykH+ww5TwmkmVCQ/7u9hI6yLVX8v+MxhQbSic+GivLGCJGJontV40inVsD50Iyo5FuWzeah+Ux7xT4RzCfsd34t75ghW35zxrZ9iaLg/c8VkbIzhIidrzeDjj4pwNWE0AoWivRB3ArPZ37cF9iGr+945LmQc9jhmWAf+yNU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717374800; c=relaxed/simple; bh=b6eKfCltzw8fypFAkdr1Zz9lFnS5//CV8ehAWpoGhn4=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=TZ7vQXndyyO7fQSauq/t8EPFCODKhWWrBEvmo5gSN4Fc2oiZWw7EnO+xtkMMShnMrGuBQdiVlQOAdSOBFXcmD/UtbF8C/ctpXAWR0HKPTCNS3ZzxdQbtzOYl4IsZQHxSXVktY9yFgQQkrvD1bkB+SezLsElQk2AJzn6zx3o4vW4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=OYnoPBg6; arc=none smtp.client-ip=91.218.175.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Envelope-To: linux-fsdevel@vger.kernel.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1717374795; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=IeBJ/5oPlb9g+ArFLQs0RP3mSJ+NwqtBpi4ZC0JDGNM=; b=OYnoPBg6pxl4poMMON/6TpyfKQJV7Y0DVjK8agLVjT7YDZCqb4rCj8ep3GjQBWBiuiRtg4 Ns95sq0QeN0vY9KOWQC64MQt557Z7lPLZph/zJHTbjdEc6ecOLmWxzbN+kSfjyQVZKj94x wgefjmPwxUuC2053ufGXu/pdKgHimPM= X-Envelope-To: linux-kernel@vger.kernel.org X-Envelope-To: axboe@kernel.dk X-Envelope-To: kent.overstreet@linux.dev X-Envelope-To: brauner@kernel.org X-Envelope-To: viro@zeniv.linux.org.uk X-Envelope-To: bernd.schubert@fastmail.fm X-Envelope-To: linux-mm@kvack.org X-Envelope-To: josef@toxicpanda.com X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Kent Overstreet To: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Jens Axboe , Kent Overstreet , brauner@kernel.org, viro@zeniv.linux.org.uk, Bernd Schubert , linux-mm@kvack.org, Josef Bacik Subject: [PATCH 0/5] sys_ringbuffer Date: Sun, 2 Jun 2024 20:32:57 -0400 Message-ID: <20240603003306.2030491-1-kent.overstreet@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT New syscall for mapping generic ringbuffers for arbitary (supported) file descriptors. Ringbuffers can be created either when requested or at file open time, and can be mapped into multiple address spaces (naturally, since files can be shared as well). Initial motivation is for fuse, but I plan on adding support to pipes and possibly sockets as well - pipes are a particularly interesting use case, because if both the sender and receiver of a pipe opt in to the new ringbuffer interface, we can make them the _same_ ringbuffer for true zero copy IO, while being backwards compatible with existing pipes. the ringbuffer_wait and ringbuffer_wakeup syscalls are probably going away in a future iteration, in favor of just using futexes. In my testing, reading/writing from the ringbuffer 16 bytes at a time is ~7x faster than using read/write syscalls - and I was testing with mitigations off, real world benefit will be even higher. Kent Overstreet (5): darray: lift from bcachefs darray: Fix darray_for_each_reverse() when darray is empty fs: sys_ringbuffer ringbuffer: Test device ringbuffer: Userspace test helper MAINTAINERS | 7 + arch/x86/entry/syscalls/syscall_32.tbl | 3 + arch/x86/entry/syscalls/syscall_64.tbl | 3 + fs/Makefile | 2 + fs/bcachefs/Makefile | 1 - fs/bcachefs/btree_types.h | 2 +- fs/bcachefs/btree_update.c | 2 + fs/bcachefs/btree_write_buffer_types.h | 2 +- fs/bcachefs/fsck.c | 2 +- fs/bcachefs/journal_io.h | 2 +- fs/bcachefs/journal_sb.c | 2 +- fs/bcachefs/sb-downgrade.c | 3 +- fs/bcachefs/sb-errors_types.h | 2 +- fs/bcachefs/sb-members.h | 3 +- fs/bcachefs/subvolume.h | 1 - fs/bcachefs/subvolume_types.h | 2 +- fs/bcachefs/thread_with_file_types.h | 2 +- fs/bcachefs/util.h | 28 +- fs/ringbuffer.c | 474 ++++++++++++++++++++++++ fs/ringbuffer_test.c | 209 +++++++++++ {fs/bcachefs => include/linux}/darray.h | 61 +-- include/linux/darray_types.h | 22 ++ include/linux/fs.h | 2 + include/linux/mm_types.h | 4 + include/linux/ringbuffer_sys.h | 18 + include/uapi/linux/futex.h | 1 + include/uapi/linux/ringbuffer_sys.h | 40 ++ init/Kconfig | 9 + kernel/fork.c | 2 + lib/Kconfig.debug | 5 + lib/Makefile | 2 +- {fs/bcachefs => lib}/darray.c | 12 +- tools/ringbuffer/Makefile | 3 + tools/ringbuffer/ringbuffer-test.c | 254 +++++++++++++ 34 files changed, 1125 insertions(+), 62 deletions(-) create mode 100644 fs/ringbuffer.c create mode 100644 fs/ringbuffer_test.c rename {fs/bcachefs => include/linux}/darray.h (63%) create mode 100644 include/linux/darray_types.h create mode 100644 include/linux/ringbuffer_sys.h create mode 100644 include/uapi/linux/ringbuffer_sys.h rename {fs/bcachefs => lib}/darray.c (56%) create mode 100644 tools/ringbuffer/Makefile create mode 100644 tools/ringbuffer/ringbuffer-test.c -- 2.45.1