Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758643AbbGHTSE (ORCPT ); Wed, 8 Jul 2015 15:18:04 -0400 Received: from mx1.redhat.com ([209.132.183.28]:59075 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755439AbbGHTRy (ORCPT ); Wed, 8 Jul 2015 15:17:54 -0400 Message-ID: <559D7760.1020909@redhat.com> Date: Wed, 08 Jul 2015 15:17:52 -0400 From: Doug Ledford Organization: Red Hat, Inc. User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: mtk.manpages@gmail.com, Davidlohr Bueso CC: Marcus Gelderie , lkml , David Howells , Alexander Viro , John Duffy , Arto Bendiken , Linux API , Andrew Morton Subject: Re: [PATCH v3] ipc: Modify message queue accounting to not take kernel data structures into account References: <20150622222546.GA32432@ramsey.localdomain> <1435211229.11852.23.camel@stgolabs.net> <1435256484.11852.30.camel@stgolabs.net> <20150706154928.GA19828@ramsey.localdomain> <1436246210.12255.71.camel@stgolabs.net> In-Reply-To: OpenPGP: id=AE6B1BDA122B23B4265B1274B826A3330E572FDD; url=pgp.mit.edu Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="xXsc2F1spLJgj0U9qhXjv5F5pUVibQt2P" Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 6407 Lines: 162 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --xXsc2F1spLJgj0U9qhXjv5F5pUVibQt2P Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 07/07/2015 09:01 AM, Michael Kerrisk (man-pages) wrote: > Hi David, >=20 > On 7 July 2015 at 07:16, Davidlohr Bueso wrote: >> On Mon, 2015-07-06 at 17:49 +0200, Marcus Gelderie wrote: >>> A while back, the message queue implementation in the kernel was >>> improved to use btrees to speed up retrieval of messages (commit >>> d6629859b36). The patch introducing the improved kernel handling of >>> message queues (using btrees) has, as a by-product, changed the >>> meaning of the QSIZE field in the pseudo-file created for the queue. >>> Before, this field reflected the size of the user-data in the queue. >>> Since, it also takes kernel data structures into account. For >>> example, if 13 bytes of user data are in the queue, on my machine the= >>> file reports a size of 61 bytes. >>> >>> There was some discussion on this topic before (for example >>> https://lkml.org/lkml/2014/10/1/115). Commenting on a th lkml, Michae= l >>> Kerrisk gave the following background (https://lkml.org/lkml/2015/6/1= 6/74): >>> >>> The pseudofiles in the mqueue filesystem (usually mounted at >>> /dev/mqueue) expose fields with metadata describing a message >>> queue. One of these fields, QSIZE, as originally implemented, >>> showed the total number of bytes of user data in all messages in >>> the message queue, and this feature was documented from the >>> beginning in the mq_overview(7) page. In 3.5, some other (useful)= >>> work happened to break the user-space API in a couple of places, >>> including the value exposed via QSIZE, which now includes a measu= re >>> of kernel overhead bytes for the queue, a figure that renders QSI= ZE >>> useless for its original purpose, since there's no way to deduce >>> the number of overhead bytes consumed by the implementation. >>> (The other user-space breakage was subsequently fixed.) >> >> Michael, this breakage was never finally documented in the manpage, >> right? >=20 > Right. >=20 >> I took a look and there is no mention, but it was a quick look. >> It's just that if this patch goes in, I'd hate ending up with somethin= g >> like this in the manpage: >> >> as of 3.5 >> >> >> as of 4.3 >> >> >> If there are changes to be made to the manpage, it should probably be >> posted with this patch, methinks. >=20 > I think something like the above will have to end up in the man page. > The only thing I'd add is that the fix also has gone to -stable > kernels. At least: I think this patch should also be marked for > -stable. I'll take care of the man page updates as the patch goes > through. >=20 >>> This patch removes the accounting of kernel data structures in the >>> queue. Reporting the size of these data-structures in the QSIZE field= >>> was a breaking change (see Michael's comment above). Without the QSIZ= E >>> field reporting the total size of user-data in the queue, there is no= >>> way to deduce this number. >>> >>> It should be noted that the resource limit RLIMIT_MSGQUEUE is counted= >>> against the worst-case size of the queue (in both the old and the new= >>> implementation). Therefore, the kernel overhead accounting in QSIZE i= s >>> not necessary to help the user understand the limitations RLIMIT impo= ses >>> on the processes. >> >> Also, I would suggest adding some comment in struct mqueue_inode_info >> for future reference, ie: >> >> - unsigned long qsize; /* size of queue in memory (sum of all ms= gs) */ >> + /* >> + * Size of queue in memory (sum of all msgs). Accounts for >> + * only userspace overhead; ignoring any in-kernel rbtree node= s. >> + */ >> + unsigned long qsize; >> >> But no big deal in any case. >> >> I think this is the right approach, >=20 > Me too. >=20 >> but would still like to know if Doug >> has any concerns about it. >=20 > Looking on gmane, Doug does not appear to have been active on any > lists since late May! Not sure why. I responded yesterday in the v2 patch thread I believe. In any case, I think this patch is fine, and can go to stable. This patch doesn't actually change the math related to the rlimit checks (which is the main thing I wanted to correct in my original patches), instead it corrects a mistake I made. At the time, I mistakenly thought that the qsize included the current message data total + the struct msg_msg size total. It didn't, it was just the current user data total. I added the rbtree nodes in order to keep the total accurate but I shouldn't have added the rbtree nodes to this total because none of the other kernel usage was previously included. Acked-by: Doug Ledford --=20 Doug Ledford GPG KeyID: 0E572FDD --xXsc2F1spLJgj0U9qhXjv5F5pUVibQt2P Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJVnXdgAAoJELgmozMOVy/dCfQP/3YC8/pVEHka8+Vi51sutou9 M6gRc9s/kC11mwhpD39jHTf+nErHx9NYq+l4e1h8rmj6Gv+hSUWe26plhe1vtL/d 8MquiawxxS1PuGKh6Pckv4zkAhVnFyY9c0VbNnt2UcYGRk8fp5oLCPwc99acMz/T plphc58aTya9UO5Dx5KFwKDGuSM/5lyTFPUoINqgway69WEfbvChxrWWAhHj8l2s Q+tXrgz4D42RAvkxOJ5xevH/vEp5EvYHfREIN0RMriymlVwRSMEwSJDKwh5TBrcB HU/KouxBbc4cbhuU9HjkShg9Z0Besab0BY0g0IILydXMT05oj6uTCY/X0qdefE0Y b9FTf95U71EErChI9wgDpizSJ2EmR//P/xmA+hwLGHiFI03xql1racdNZstrBjMV oAniFFGZ2aHeTAosdmkjTcVQi9vAOJbRFl2+N0js603+XIQv6OmBkatedVjdpGIA B17imG1KoxH+FEfWphe74XenC9267lJq0jqyFuOVqY4wIiuGnyBmBqgljkzfuH73 NGLC5Rkx/cKBRzPZmCWWs5v1foUPPUJTldZRO3Hu16z59wgmq64rJ2jP1NeRQ/qh Xq+oiEIKwSt7xQAMFHZ2oKKxm7eXFfB4zQ2GJ/41400KEH+px+/GdTfoTaaT5CiW lrkKTIjzy2rFy1ymtBx4 =qXYz -----END PGP SIGNATURE----- --xXsc2F1spLJgj0U9qhXjv5F5pUVibQt2P-- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/