Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <005a944c-ed2e-6010-a534-26d5947402da@virtuozzo.com>
Date:   Thu, 11 May 2023 12:12:32 +0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.9.0
Subject: Re: [RFD] posix-timers: CRIU woes
Content-Language: en-US
To:     Thomas Gleixner <tglx@linutronix.de>,
        Frederic Weisbecker <frederic@kernel.org>
Cc:     LKML <linux-kernel@vger.kernel.org>,
        Anna-Maria Behnsen <anna-maria@linutronix.de>,
        Peter Zijlstra <peterz@infradead.org>,
        syzbot+5c54bd3eb218bb595aa9@syzkaller.appspotmail.com,
        Dmitry Vyukov <dvyukov@google.com>,
        Sebastian Siewior <bigeasy@linutronix.de>,
        Michael Kerrisk <mtk.manpages@gmail.com>,
        Andrei Vagin <avagin@openvz.org>,
        Christian Brauner <brauner@kernel.org>,
        Alexander Mikhalitsyn <aleksandr.mikhalitsyn@canonical.com>,
        Pavel Emelyanov <xemul@openvz.org>
References: <20230425181827.219128101@linutronix.de>
 <20230425183312.932345089@linutronix.de> <ZFUXrCZtWyNG3Esi@lothringen>
 <87zg6i2xn3.ffs@tglx> <87v8h62vwp.ffs@tglx> <878rdy32ri.ffs@tglx>
 <87v8h126p2.ffs@tglx> <875y911xeg.ffs@tglx> <87ednpyyeo.ffs@tglx>
 <009e7658-1377-cc79-7a42-4dda8fec5af0@virtuozzo.com> <87wn1gy4e6.ffs@tglx>
From:   Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
In-Reply-To: <87wn1gy4e6.ffs@tglx>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?OUFZT2diY1k3UVUvSWdrdHM1ZDI3RVJZanAxbFFtVEJESC9jVXdtUE03M1k1?=
 =?utf-8?B?cFExbUFCZDhsU2lHUWs0UW9zUktEdjMvK0VaazduSEpETHJOUnBjZUVqdHo4?=
 =?utf-8?B?ejR6NUpkd0crTE45ckNrU3F1UERzNy9JWVZyNDJ5VjFqdXlyZjV5Rnp0cnBm?=
 =?utf-8?B?cVR6RHRZMXQ2UU1YVlFHMHRuQXhCVVR5OWlzbExUTytkMUxZcVFOSGltYjZD?=
 =?utf-8?B?Rm1tQzFHTG1UYmo2VHo1MENjUUdRTGJjMklnR2EybVRlVTh4UXY4eWVOdHJU?=
 =?utf-8?B?dEdTZDFDSEg3dFpueVg1MjB2TTBUZzM0U2llQms5L283dnFOUkQ1dmd1YXRG?=
 =?utf-8?B?THRLWkVJL3BWemNCS1RRUFRpcnd3NjcxY1kwK0tQYy9WbHdBcXNEell5bzBQ?=
 =?utf-8?B?MzhwdjlmbWlxNllFNXpJUEtSNkpVNURELzZzREdpOXdxb2RKS0VmbGpjL0Rt?=
 =?utf-8?B?RDR1YkVEc21ZQmEvb2tIa1FXWkY3QkFGeEVmWnMweEMxelBxT1RzMEhCY0RS?=
 =?utf-8?B?QW1TbDA2c3lxd2lLRkc1VjBCNFloTjQrMFBVNDhrZ3Q1VlFscnBCeFdsVW5X?=
 =?utf-8?B?MXVtU1ZWZlhXbWl5Ri9Sa0dXR0EwTXlTeHhHOU5BNWJvalk2WUszYXhTc3VJ?=
 =?utf-8?B?d0owWm1yL1dSdWNYd0Riai9ZOFVZT21DdDV0NnR3OFoyL2xzWHE0bHZkMStl?=
 =?utf-8?B?Y0E2cE1VVWFVNFlWMlBJcStmMFloeExRUWxoUStDVGY0L0hpTVgzeFUvLzcr?=
 =?utf-8?B?NGJoZGxYb3NEMlV6U0lxMGFiMi9OSXVzM1dLU2dnWFo1cEpVTlNhU21DOWNH?=
 =?utf-8?B?NmR1T3pjUUVlRmNIMVM2YzNaTkc4NWRyWEE5VS9ERWVsYXlqZWZic3NrbnNY?=
 =?utf-8?B?eHlvWU4zQVgxQ05ISFhGNDdjaUwzMGdGWExwWkxkMENNQm9UUytUTm55L3ll?=
 =?utf-8?B?TFhXcmJmQTNEeUFEQjdJZlJ4UnBtNmVTNTI5OFRPaG9DU2NSLzY5RzliNFcx?=
 =?utf-8?B?QWUwSVBwb0FJZUE2MzludjgwU1A1S2p1Q2t1djV2Q05jNEZRcG1Pam9FVkVy?=
 =?utf-8?B?bUtJQWlZNlorS1cxVnFUUE9HcjhKU1NKOEhuL2cvR3RXaHhWMDR6N3EwWFhq?=
 =?utf-8?B?Ym5KcFFBVHVIb0xQMjBDZUdIWTYvK0JFd2tROStWWFFpQ1V2TzFsMXFKZE1B?=
 =?utf-8?B?eHBjdTlNVHFpY0JqN3VLeG5VYkJ0ZnhuSlRFdmtTV0ZKSC9kRXlNalZHRVZu?=
 =?utf-8?B?M0ZZN1d2ZnRuRmErMU5SZHBBdE9zbDV2QVRvT0lVZDlMSFFLUmJYOE5zOXlS?=
 =?utf-8?B?ZUNlMVVSN2ZXRE9rODhDd0xHKzVLU2NubTRjdHk4QVY1eWZmQm53V2FTdE8x?=
 =?utf-8?B?a29MNEpYRUk4b0VSV3pWaExzWkFHOGcyeEZ2cFYrMXplelozTG9SbzdVcjB6?=
 =?utf-8?B?Wm1lRnBUV0ZtSnNoOXFpRmZZU1MwUWtnUm1ia1BoanBOOVNrMFgvanFtdDZL?=
 =?utf-8?B?d2pkSXJqN29aUVByb0tqL041b1RLMEp4VUF6YXFGNWFaNTFoVlFOYi80YmNr?=
 =?utf-8?B?U2Y5cENwR0tRUU4vQ1NoRkZSTkw1UWhjQUFTcFovQmhHdHdBY3AzRG1RZjdm?=
 =?utf-8?B?bEMvOWxpZG1KcHV2TCtZVUx2QXBPVVVKMzB5ZTR6NHJRclFTM0x0OWFVcmd3?=
 =?utf-8?B?VWJqV3N4WHRtSVh4bVVXTzFRa2hvZ3B2QXU2ZHBTYzBDMnZ6cURDUlI0TE9S?=
 =?utf-8?B?aGpnYkpKVi9PRDN3WDhSSVlJYU81UURlekRvaDJtbDRaM0h6cU5Vd09kcXY0?=
 =?utf-8?B?L1dCaytqSzdqN0Z1b0tRQkI3Vno1L2lhUVE3V3RCUm9GV3lUTjdLcU0xVEwy?=
 =?utf-8?B?b2pWQk1vRmk1SnhQSGtKeEZtNm8zbUdVdGdENStDNlR3cE5ZYm8rWHpzRWV4?=
 =?utf-8?B?MnJESzc4UXJ3ZE1NM1BrNlB0TTFyaVlCbmZSUFJaUmVQWFpKU0xVK1IydDV4?=
 =?utf-8?B?cjFLRnVTaWhUYkRjQWNuVHhuTERkTzBHRU5aQTZQWTViRzJIWVd5L1lGNUdM?=
 =?utf-8?B?WHowZGJYTHYveWFyUmFKRjNOMkxuTG8vY0xiNW1GQ1hnUlNrcUhTR1ducis4?=
 =?utf-8?B?Tk5rQlNIc1RTTFMzRWV2ZnR6SjNtVUErR3VkdnNRaXJBdVNMVWY0TDArVTds?=
 =?utf-8?B?VngwM29MMStMc081RjI2RW9kcm9CUmFrUWZoT0hoRGx2aFZEQzJsS0NnQVlI?=
 =?utf-8?B?TnNYbVhXK3huamRCQ1NZcXFtVXBnPT0=?=
X-OriginatorOrg: virtuozzo.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a4454c33-0721-4dbf-9919-08db51d5f786
X-MS-Exchange-CrossTenant-AuthSource: VE1PR08MB4989.eurprd08.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2023 04:12:42.9089
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: so4w6W4Ddw82daQZyfScUqVNMWKSR378oNdYsqNMWNJDPpE3Ni5w+oh5z3o4WUoBTBTLvOi+MPBvAagg2MsY63m/WNSQiEpbrA0olAVNwXs=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6696
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,NICE_REPLY_A,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 10.05.2023 16:30, Thomas Gleixner wrote:
> Pavel!
> 
> On Wed, May 10 2023 at 12:36, Pavel Tikhomirov wrote:
>> On 10.05.2023 05:42, Thomas Gleixner wrote:
>>> So because of that half thought out user space ABI we are now up the
>>> regression creek without a paddle, unless CRIU can accomodate to a
>>> different restore mechanism to lift this restriction from the kernel.
>>>
>>> Thoughts?
>>
>> Maybe we can do something similar to /proc/sys/kernel/ns_last_pid?
>> Switch to per-(process->signal) idr based approach with idr_set_cursor
>> to set next id for next posix timer from new sysctl?
> 
> I'm not a fan of such sysctls. We have already too many of them and that
> particular one does not buy much.

Sorry, it was a bad idea, what you suggest below is much better.

> 
> We can simply let timer_create() or a new syscall create a timer at a
> given ID.

Yes this would work for CRIU. (note: in neighbor thread Andrei writes 
about adding a bit to sigevent.sigev_notify to request a timer with a 
specified id, new syscall is also a good option)

> 
> That allows CRIU to restore any checkpointed process no matter which
> kernel version it came from without doing this insane create/delete
> dance.

Yes, for CRIU this kind of API change is a big improvement.

> 
> The downside is that this allows to create stupidly sparse timer IDs
> even for the non CRIU case, which increases per process kernel memory
> consumption and creates slightly more overhead in the signal delivery
> path. The latter is a burden on the process owning the timer and not
> affecting expiry, which is a context stealing operation. The memory part
> needs eventually some thoughts vs. accounting.
> 
> If the 'explicit at ID' option is not used then the ID mechanism is
> optimzied for dense IDs by using the first available ID in a bottom up
> search, which recovers holes created by a timer_delete() operation.

Not sure how kernel memory consumption increases with sparse timer IDs, 
global hashtable (posix_timers_hashtable) is the same size anyway, 
entries in hlists can be distributed differently as hash depends on id 
directly but we have same number of entries. Probably I miss something, 
why do we need dense IDs?

> 
> Thanks,
> 
>          tglx

-- 
Best regards, Tikhomirov Pavel
Senior Software Developer, Virtuozzo.