Received-SPF: pass (google.com: domain of linux-kernel+bounces-59488-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Message-ID: <2d3359b3-f124-4ddc-97e0-cd56d0e7b966@oracle.com>
Date: Fri, 9 Feb 2024 15:47:16 +0100
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3] Documentation: Document each netlink family
Content-Language: en-US
From: Vegard Nossum <vegard.nossum@oracle.com>
To: Breno Leitao <leitao@debian.org>, Jonathan Corbet <corbet@lwn.net>
Cc: Jani Nikula <jani.nikula@linux.intel.com>, kuba@kernel.org,
        "David S. Miller" <davem@davemloft.net>, linux-doc@vger.kernel.org,
        netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
        pabeni@redhat.com, edumazet@google.com
References: <20231121114831.3033560-1-leitao@debian.org>
 <874jevjgvo.fsf@intel.com> <87jznqewa7.fsf@meer.lwn.net>
 <ZbkebMW+xLqNhsoB@gmail.com>
 <63304f6a-d26f-414a-8c92-14d740774379@oracle.com>
Autocrypt: addr=vegard.nossum@oracle.com; keydata=
 xsFNBE4DTU8BEADTtNncvO6rZdvTSILZHHhUnJr9Vd7N/MSx8U9z0UkAtrcgP6HPsVdsvHeU
 C6IW7L629z7CSffCXNeF8xBYnGFhCh9L9fyX/nZ2gVw/0cVDCVMwVgeXo3m8AR1iSFYvO9vC
 Rcd1fN2y+vGsJaD4JoxhKBygUtPWqUKks88NYvqyIMKgIVNQ964Qh7M+qDGY+e/BaId1OK2Z
 92jfTNE7EaIhJfHX8hW1yJKXWS54qBMqBstgLHPx8rv8AmRunsehso5nKxjtlYa/Zw5J1Uyw
 tSl+e3g/8bmCj+9+7Gj2swFlmZQwBVpVVrAR38jjEnjbKe9dQZ7c8mHHSFDflcAJlqRB2RT1
 2JA3iX/XZ0AmcOvrk62S7B4I00+kOiY6fAERPptrA19n452Non7PD5VTe2iKsOIARIkf7LvD
 q2bjzB3r41A8twtB7DUEH8Db5tbiztwy2TGLD9ga+aJJwGdy9kR5kRORNLWvqMM6Bfe9+qbw
 cJ1NXTM1RFsgCgq7U6BMEXZNcsSg9Hbs6fqDPbbZXXxn7iA4TmOhyAqgY5KCa0wm68GxMhyG
 5Q5dWfwX42/U/Zx5foyiORvEFxDBWNWc6iP1h+w8wDiiEO/UM7eH06bxRaxoMEYmcYNeEjk6
 U6qnvjUiK8A35zDOoK67t9QD35aWlNBNQ2becGk9i8fuNJKqNQARAQABzShWZWdhcmQgTm9z
 c3VtIDx2ZWdhcmQubm9zc3VtQG9yYWNsZS5jb20+wsF4BBMBAgAiBQJX+8E+AhsDBgsJCAcD
 AgYVCAIJCgsEFgIDAQIeAQIXgAAKCRALzvTY/pi6WOTDD/46kJZT/yJsYVT44e+MWvWXnzi9
 G7Tcqo1yNS5guN0d49B8ei9VvRzYpRsziaj1nAQJ8bgGJeXjNsMLMOZgx4b5OTsn8t2zIm2h
 midgIE8b3nS73uNs+9E1ktJPnHClGtTECEIIwQibpdCPYCS3lpmoAagezfcnkOqtTdgSvBg9
 FxrxKpAclgoQFTKpUoI121tvYBHmaW9K5mBM3Ty16t7IPghnndgxab+liUUZQY0TZqDG8PPW
 SuRpiVJ9buszWQvm1MUJB/MNtj1rWHivsc1Xu559PYShvJiqJF1+NCNVUx3hfXEm3evTZ9Fm
 TQJBNaeROqCToGJHjdbOdtxeSdMhaiExuSnxghqcWN+76JNXAQLlVvYhHjQwzr4me4Efo1AN
 jinz1STmmeeAMYBfHPmBNjbyNMmYBH4ETbK9XKmtkLlEPuwTXu++7zKECgsgJJJ+kvAM1OOP
 VSOKCFouq1NiuJTDwIXQf/zc1ZB8ILoY/WljE+TO/ZNmRCZl8uj03FTUzLYhR7iWdyfG5gJ/
 UfNDs/LBk596rEAtlwn0qlFUmj01B1MVeevV8JJ711S1jiRrPCXg90P3wmUUQzO0apfk1Np6
 jZVlvsnbdK/1QZaYo1kdDPEVG+TQKOgdj4wbLMBV0rh82SYM1nc6YinoXWS3EuEfRLYTf8ad
 hbkmGzrwcc7BTQROA01PARAA5+ySdsvX2RzUF6aBwtohoGYV6m2P77wn4u9uNDMD9vfcqZxj
 y9QBMKGVADLY/zoL3TJx8CYS71YNz2AsFysTdfJjNgruZW7+j2ODTrHVTNWNSpMt5yRVW426
 vN12gYjqK95c5uKNWGreP9W99T7Tj8yJe2CcoXYb6kO8hGvAHFlSYpJe+Plph5oD9llnYWpO
 XOzzuICFi4jfm0I0lvneQGd2aPK47JGHWewHn1Xk9/IwZW2InPYZat0kLlSDdiQmy/1Kv1UL
 PfzSjc9lkZqUJEXunpE0Mdp8LqowlL3rmgdoi1u4MNXurqWwPTXf1MSH537exgjqMp6tddfw
 cLAIcReIrKnN9g1+rdHfAUiHJYhEVbJACQSy9a4Z+CzUgb4RcwOQznGuzDXxnuTSuwMRxvyz
 XpDvuZazsAqB4e4p/m+42hAjE5lKBfE/p/WWewNzRRxRKvscoLcWCLg1qZ6N1pNJAh7BQdDK
 pvLaUv6zQkrlsvK2bicGXqzPVhjwX+rTghSuG3Sbsn2XdzABROgHd7ImsqzV6QQGw7eIlTD2
 MT2b9gf0f76TaTgi0kZlLpQiAGVgjNhU2Aq3xIqOFTuiGnIQN0LV9/g6KqklzOGMBYf80Pgs
 kiObHTTzSvPIT+JcdIjPcKj2+HCbgbhmrYLtGJW8Bqp/I8w2aj2nVBa7l7UAEQEAAcLBXwQY
 AQIACQUCTgNNTwIbDAAKCRALzvTY/pi6WEWzD/4rWDeWc3P0DfOv23vWgx1qboMuFLxetair
 Utae7i60PQFIVj44xG997aMjohdxxzO9oBCTxUekn31aXzTBpUbRhStq78d1hQA5Rk7nJRS6
 Nl6UtIcuLTE6Zznrq3QdQHtqwQCm1OM2F5w0ezOxbhHgt9WTrjJHact4AsN/8Aa2jmxJYrup
 aKmHqPxCVwxrrSTnx8ljisPaZWdzLQF5qmgmAqIRvX57xAuCu8O15XyZ054u73dIEYb2MBBl
 aUYwDv/4So2e2MEUymx7BF8rKDJ1LvwxKYT+X1gSdeiSambCzuEZ3SQWsVv3gn5TTCn3fHDt
 KTUL3zejji3s2V/gBXoHX7NnTNx6ZDP7It259tvWXKlUDd+spxUCF4i5fbkoQ9A0PNCwe01i
 N71y5pRS0WlFS06cvPs9lZbkAj4lDFgnOVQwmg6Smqi8gjD8rjP0GWKY24tDqd6sptX5cTDH
 pcH+LjiY61m43d8Rx+tqiUGJNUfXE/sEB+nkpL1PFWzdI1XZp4tlG6R7T9VLLf01SfeA2wgo
 9BLDRko6MK5UxPwoYDHpYiyzzAdO24dlfTphNxNcDfspLCgOW1IQ3kGoTghU7CwDtV44x4rA
 jtz7znL1XTlXp6YJQ/FWWIJfsyFvr01kTmv+/QpnAG5/iLJ+0upU1blkWmVwaEo82BU6MrS2 8A==
In-Reply-To: <63304f6a-d26f-414a-8c92-14d740774379@oracle.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?UGFham1HZE5WVmU5Y1JIUWQ2VTRJM043ZW8zSEhwQkdSLytpN2JSRERvZEVZ?=
 =?utf-8?B?NHdqRU5JWUJFS1M1NEh5VE1BcW1KUWNVVlBnRk9sR0hrcmZrdE5CNnNMZFU5?=
 =?utf-8?B?VzBXU1hwM0UwQWFaZVp5Q2dPQk4yRmhJVXNuSTc1QVhtZ1ZPQkdwL1d1bXk3?=
 =?utf-8?B?anQzWFRFVWtoZWpBZHFJMk9iZll0eGk3bnV6K3JwV2VlemFFeXZIaEF3OVQy?=
 =?utf-8?B?ZVZ3UGdERTYvNHowUFM4MytHQzNUNlB5Vit4eDViaEI0OFJEOHpaZks3MFVN?=
 =?utf-8?B?c3RvcTRrdHUzZ2RJOWQ4dFJISUlSdFkzeVdIRzBucS9kaG5nbXZFNmo2MWo3?=
 =?utf-8?B?Z2lUb0VwQXVSZ1dkSjZlMmFqTXhwdERDRXpSMStIZlg1cWFtcUJrNlNOSmV6?=
 =?utf-8?B?MTM2SmkwL1VOTHR1aWxqdmFKejBYSjBRYW56UnVoVzd5ZUExYzB0alg3Rk1V?=
 =?utf-8?B?S3lFVHpXWG9heTZRaTkxSEtwQytTVzBvdUhaTjQ0Mk43MHhpN2ZDc1NDY2Y2?=
 =?utf-8?B?VHdpTEhTYURyR1ZDd1kvci92WlJSWnh5UWN1NlJRNHdBOFlKMFhvRzVrRFBy?=
 =?utf-8?B?R3F6bFByTXBrbVFvR0xBZnpqNS8rc3IrSHVDU1BqbGJUY1FBM1RvN1RueU9B?=
 =?utf-8?B?eDRiajNSc1J4TEtJNmJ3elhWeXhBb3hhU290ZnNNWjVBQ2xqQkFCTlFiUTFX?=
 =?utf-8?B?UXExUEJxUEdYaElCWVpLcitBZjgxeVdGSEpFWnh2S0pXcnpOV2ljRmFBMDR0?=
 =?utf-8?B?RGRvNDUxSmVhR2s0SHdiLzdKWVRQVUJJV0NMdUd2MmJlR2tjRHhab2pGUEN3?=
 =?utf-8?B?ZVJhU2F5SVB2Y2x2L1NDRHRDMjl4SFVRb1R3Uk1aNytKL1lHZnhxOHYvRGhp?=
 =?utf-8?B?ZWU3cUo5UWsvMXExUUFmZTBnbWQycEV0bURmOVFOT1IvWG9ucm1kVGdNdG5L?=
 =?utf-8?B?VHhjcXhmcGRmaUd6aTNGMXpnYy94QWVhMFRBdEd2OHZ3NS9FM0xnRDc2NUpi?=
 =?utf-8?B?STVvZksveGRPb1VFamFKRG9DNkFYNFVlSlAwUjBFcEhGaTNSY2w3RDBoN2s1?=
 =?utf-8?B?WDJpa3lZTWpZbWdFL3BGRWVWUlYybG9EU0V1STNIUVN0UDI0R2F1eE5WQmx6?=
 =?utf-8?B?MmVra0h3Z2EwSG9KajlaZVRsVStmaHZZNXFhQWVIT2pQcVZ6dHJEUFhoc09I?=
 =?utf-8?B?cm1zbjk4OGNmMWZ2YmN5NG8vYUxtRUN3dmtoUzlFUFh3T0NWQWhjeEZZdDF6?=
 =?utf-8?B?QnVaRGtCVm1tM0lwVzAwUDVOM2hkdFA3UXZ6ZHFCMWU2c1hMVVp2UzNvWmNS?=
 =?utf-8?B?Mnpqd0hNK2NVb0pFelFTd1VnNjV3OEJNQmJrbDA0VHdQc21NMXJlNHpHZ1Zv?=
 =?utf-8?B?emcrM1BnSGEvZUo0WjMrcjZsWEdnZzZzN1UrS09lT0xPNlEwYlUycEV6cDg4?=
 =?utf-8?B?SWx5ckxYY3RMakttRk9lN0UrZTg1dmxSdTJrMC9UNlprY1paZ2dZU3ZGTVM3?=
 =?utf-8?B?YmVBUUo5MDhUbFJYSzFRRGtOa2xwNXZrUlJHdjAybW9rbnpJYVZBdFlEYWZk?=
 =?utf-8?B?SEZ6NUhGZVIzdEVFbjA1bnRWNkQzYlVMMGVadSs5ZWFlNGVMVVNLQ0hybWVX?=
 =?utf-8?B?blFZTjQvL0FDRUdKOERNM0RWbmR4VmhSV2trODkxS2VoS1dhSVRzcWJEMUNl?=
 =?utf-8?B?QldxbnlTLzB0VmVucWgrNEZQVXUwWGdicU56dUFBaTYwbjRVL3VxL1kxTHla?=
 =?utf-8?B?Uld5SVRqOXhuSFlRR3RJVGNGM0JzRjVCR0prNTI1RlJoVlNZQmZZeVlNV1M2?=
 =?utf-8?B?djRyRHhnd2I2RlJFNmo3azhWTHVhU0hseGdxWU96VVBxWmNqN0M2OUgzU1pL?=
 =?utf-8?B?UDFYeFkzMGZiWHA3cElLSWV1T3I1M2swN0NST0xTbFptMVk5WjE0SWsxa1dC?=
 =?utf-8?B?Q3NXU3Z4VldSV1EyRm1qcVFHZ3l5bDlqbXk1R0ZzS095YWxrcnBWM2ZiazZm?=
 =?utf-8?B?aHNmTjByb3dDbWdidzJvYVQ0am1VbUJmTGVOQXUwNllFMGFlNExzSXQ4NHJq?=
 =?utf-8?B?Nm5memx3U1VYZTJGczFORWF1bnpGcHhJWlluc2lDTmUrNEZnNlAzRTRSZHJ3?=
 =?utf-8?B?VDVmbEF0dUtmT093ZXI3M3k2VVdqdy9FRFRBMUx1VnZpdmtOTEZuRDFBWVFu?=
 =?utf-8?B?VGc9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	4cNLusjAeYzWFAO+f2DTeqvdrw56XmsSm6/LA/CPp8WhT/dt2PTf89EOUtL1Hf9oSnxeyIqcfg/aZGlkysQAVwAKnjRlLmbeTTdNCIdVUL3xqFR432wGvudX/WUCijBOLkoHe64UcQyJuYPZEWrbX3+0ScVjeCPt6hQ1FUA93/CYMS0X7td5CKkA4Nz3J/9vRwwCIJq52TPn7IZYDApkTjXBwqeOIdrZwBDrX+NlNG7+OI+1mDjPocCedOyfYU5EsCvhn/ggOkxOBssMWu3Tnj25md3wKjCxfwuPwq/kkY9ZQMqpvISUu6rScOLd3dt2T3x1S1G0VQeVoGul0/Y22/+WxrJGPw5HLaggHVKE847UYULjk9vMImYcgIhiv4j+S2OGac43nvm5luqb54P6DmoEnmsJf9mVA1os9KHNEKgo2p6MUeFv5UAVTXji10YwSAjlU4wwkAYxYM2sV5SVp0LEL//CMKoogKKekalxRsX4qMb0pOnbdW6a8u2Iwk5LsrpXVDGp09VCezD6mHrXwTwtXRo5hiseQtkHUgWTIG0gP5an/199Hp92O181KUL8MABZjvBQJcNIp1lxJp5BRQepmI5/SLTL2i6hHTUtgMw=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 66c23df6-75f9-4f95-7d08-08dc297e05bf
X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5433.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Feb 2024 14:47:21.9994
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HfZ/yhczovHhDOCqv1tEBijlzIfWEzHiEnLRO8nL7CLMSRwogL8UD5XPuXyOxGBHQWl5C0RVuHpkedCppTTmaDzcheLq75ppeJ8Md2OJDy8=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR10MB7283
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-02-09_12,2024-02-08_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0
 mlxlogscore=739 bulkscore=0 mlxscore=0 phishscore=0 adultscore=0
 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2311290000 definitions=main-2402090108
X-Proofpoint-GUID: mwArZ7_-Hf5wTi77IoDoSJJm2YwiMa8Z
X-Proofpoint-ORIG-GUID: mwArZ7_-Hf5wTi77IoDoSJJm2YwiMa8Z


On 30/01/2024 17:23, Vegard Nossum wrote:
> We should still fix the writing of .rst to $(srctree), though -- our
> use of parse-headers.pl seems to sidestep this by writing the 
> intermediate .rst output to Documentation/output/, I'll have to look
> a bit more closely.

I have now spent quite a bit of time investigating this.

The main result is that Sphinx really does NOT like it when documents
are located outside of the source root.

What makes the parse-headers.pl approach work is the 'kernel-include'
custom directive (from kernel_include.py) -- this allows you to include
files from outside the source root into a file that _is_ in the source
root. So the .rst files generated by parse-headers.pl are included using
this directive, rather than being treated as real documents that you can
refer to by their name.

If we did the same, one option would be to create one .rst file for each
yaml file currently in Documentation/netlink/specs/ and just have them
'kernel-include' the corresponding generated file. I don't think this is
a great idea because we'd always have to remember to create new files
when a corresponding .yaml file is added -- and it feels like an ugly
workaround since each file would just be a single line including the
generated file.

Sphinx 7.2.0 introduces a mapping of document names (like
"networking/index") to paths, and it looks doable to write an extension
that would discover additional .rst files from outside the source path
to process. But before this there is essentially no way to have files
outside the source root appear in the toctree.

But I wouldn't like to bump the minimum required version from 2.4 to 7.2
either. Also, the extension wouldn't be using a public/standard API so
it would essentially be unsupported and could break again at any time.
It might not even be possible at all even today.

Another idea would be to have another layer of directories in the output
directory, so e.g.:

   $(BUILDDIR)/Documentation -> $(srctree)/Documentation (symlink)
   $(BUILDDIR)/generated/...

Of course this would break ALL the web links like

   https://docs.kernel.org/process/stable-kernel-rules.html

since it would now be

   https://docs.kernel.org/Documentation/process/stable-kernel-rules.html

so I consider this a no-go as well. (We could consider creating N
symlinks instead; one for each directory under Documentation/ -- but
this again goes well into hacky territory IMHO.)

As far as I can tell, the most viable remaining option is to use a true
Sphinx extension that defines a new directive (that points directly to a
YAML file) and never write out the .rst to a file.

I think this was Jani's preferred solution as well.

Jon, Breno -- what do you think?


Vegard