Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Song Liu <songliubraving@fb.com>
To:     Pavel Begunkov <asml.silence@gmail.com>
CC:     "io-uring@vger.kernel.org" <io-uring@vger.kernel.org>,
        Networking <netdev@vger.kernel.org>,
        "bpf@vger.kernel.org" <bpf@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Jens Axboe <axboe@kernel.dk>,
        Alexei Starovoitov <ast@kernel.org>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Andrii Nakryiko <andrii@kernel.org>, Martin Lau <kafai@fb.com>,
        Yonghong Song <yhs@fb.com>,
        John Fastabend <john.fastabend@gmail.com>,
        KP Singh <kpsingh@kernel.org>,
        Horst Schirmeier <horst.schirmeier@tu-dortmund.de>,
        "Franz-B . Tuneke" <franz-bernhard.tuneke@tu-dortmund.de>,
        Christian Dietrich <stettberger@dokucode.de>
Subject: Re: [PATCH 01/23] io_uring: shuffle rarely used ctx fields
Thread-Topic: [PATCH 01/23] io_uring: shuffle rarely used ctx fields
Thread-Index: AQHXTLk17dWezJtB9kirPhycVh7qbKrs6dAA
Date:   Thu, 20 May 2021 21:46:47 +0000
Message-ID: <0339DB35-27AC-4A50-B807-968C72ABB698@fb.com>
References: <cover.1621424513.git.asml.silence@gmail.com>
 <485abb65cf032f4ddf13dcc0bd60e5475638efc2.1621424513.git.asml.silence@gmail.com>
In-Reply-To: <485abb65cf032f4ddf13dcc0bd60e5475638efc2.1621424513.git.asml.silence@gmail.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-ID: <25B8C0346FCD994E878FBEE605F3887C@namprd15.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: ff54c24f-fba8-4010-018b-08d91bd8c4ae
X-MS-Exchange-CrossTenant-originalarrivaltime: 20 May 2021 21:46:47.4480
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: N19MRrij0jZlLjTltfObV+N1kMIoxVI8zymyqQuuzwvbb4ZyEKcOIJ5xmt8h6fuArzgz6Eag1lcLsf+uCYO5ig==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR15MB3256
X-OriginatorOrg: fb.com
X-Proofpoint-ORIG-GUID: IV8N-7d_jP2-O8BIySZAfN3M2q_xDKRm
X-Proofpoint-GUID: IV8N-7d_jP2-O8BIySZAfN3M2q_xDKRm
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.761
 definitions=2021-05-20_06:2021-05-20,2021-05-20 signatures=0
X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 spamscore=0
 lowpriorityscore=0 bulkscore=0 phishscore=0 clxscore=1011 mlxscore=0
 mlxlogscore=999 priorityscore=1501 impostorscore=0 suspectscore=0
 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx
 scancount=1 engine=8.12.0-2104190000 definitions=main-2105200136
X-FB-Internal: deliver
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> On May 19, 2021, at 7:13 AM, Pavel Begunkov <asml.silence@gmail.com> wrot=
e:
>=20
> There is a bunch of scattered around ctx fields that are almost never
> used, e.g. only on ring exit, plunge them to the end, better locality,
> better aesthetically.
>=20
> Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
> ---
> fs/io_uring.c | 36 +++++++++++++++++-------------------
> 1 file changed, 17 insertions(+), 19 deletions(-)
>=20
> diff --git a/fs/io_uring.c b/fs/io_uring.c
> index 9ac5e278a91e..7e3410ce100a 100644
> --- a/fs/io_uring.c
> +++ b/fs/io_uring.c
> @@ -367,9 +367,6 @@ struct io_ring_ctx {
> 		unsigned		cached_cq_overflow;
> 		unsigned long		sq_check_overflow;
>=20
> -		/* hashed buffered write serialization */
> -		struct io_wq_hash	*hash_map;
> -
> 		struct list_head	defer_list;
> 		struct list_head	timeout_list;
> 		struct list_head	cq_overflow_list;
> @@ -386,9 +383,6 @@ struct io_ring_ctx {
>=20
> 	struct io_rings	*rings;
>=20
> -	/* Only used for accounting purposes */
> -	struct mm_struct	*mm_account;
> -
> 	const struct cred	*sq_creds;	/* cred used for __io_sq_thread() */
> 	struct io_sq_data	*sq_data;	/* if using sq thread polling */
>=20
> @@ -409,14 +403,6 @@ struct io_ring_ctx {
> 	unsigned		nr_user_bufs;
> 	struct io_mapped_ubuf	**user_bufs;
>=20
> -	struct user_struct	*user;
> -
> -	struct completion	ref_comp;
> -
> -#if defined(CONFIG_UNIX)
> -	struct socket		*ring_sock;
> -#endif
> -
> 	struct xarray		io_buffers;
>=20
> 	struct xarray		personalities;
> @@ -460,12 +446,24 @@ struct io_ring_ctx {
>=20
> 	struct io_restriction		restrictions;
>=20
> -	/* exit task_work */
> -	struct callback_head		*exit_task_work;
> -
> 	/* Keep this last, we don't need it for the fast path */
> -	struct work_struct		exit_work;
> -	struct list_head		tctx_list;
> +	struct {

Why do we need an anonymous struct here? For cache line alignment?
Do we need ____cacheline_aligned_in_smp?

> +		#if defined(CONFIG_UNIX)
> +			struct socket		*ring_sock;
> +		#endif
> +		/* hashed buffered write serialization */
> +		struct io_wq_hash		*hash_map;
> +
> +		/* Only used for accounting purposes */
> +		struct user_struct		*user;
> +		struct mm_struct		*mm_account;
> +
> +		/* ctx exit and cancelation */
> +		struct callback_head		*exit_task_work;
> +		struct work_struct		exit_work;
> +		struct list_head		tctx_list;
> +		struct completion		ref_comp;
> +	};
> };
>=20
> struct io_uring_task {
> --=20
> 2.31.1
>=20