Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <865e4da3-94fe-95dc-cbe3-161fa8c2146d@virtuozzo.com>
Date:   Mon, 25 Jul 2022 14:27:47 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.9.1
Subject: Re: [PATCH v5 1/1] Create debugfs file with virtio balloon usage
 information
Content-Language: en-US
To:     David Hildenbrand <david@redhat.com>,
        "Michael S. Tsirkin" <mst@redhat.com>,
        Jason Wang <jasowang@redhat.com>
Cc:     kernel@openvz.org, virtualization@lists.linux-foundation.org,
        linux-kernel@vger.kernel.org
References: <7fd5e645-3892-6e0d-de80-08728e29b150@redhat.com>
 <20220714132053.56323-1-alexander.atanasov@virtuozzo.com>
 <0b660226-28dc-1f1d-e22e-34805dc5b25d@redhat.com>
From:   Alexander Atanasov <alexander.atanasov@virtuozzo.com>
In-Reply-To: <0b660226-28dc-1f1d-e22e-34805dc5b25d@redhat.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?K3M5eWxIaG5CeGVIeUYrNTdVSGZqWkFkK1kvRnpNdTZZMFFNOHNNSEtTMjlr?=
 =?utf-8?B?SHpKY0E4V3U1SmtjMlVvOE5nNmgwdVJLRjFURG1vYWllcXhObjZkY0tLWjYy?=
 =?utf-8?B?UHk2TVhwZ3JZVURPUjFlWWxac3ZCRS82T2tEbzJUWVlEbVZFNVFYYjh4WWt6?=
 =?utf-8?B?eDdQYzAyakJOY2ZycTN4cFY4eTFWNVRmMUlBeWNEWm80Z2tzTjg1cytzQW5I?=
 =?utf-8?B?K084SW5FSG9KMVhERUpucEtDKzcyQW1iY0tlSUk2TWd2Mkk0V1NvMHZQQlZn?=
 =?utf-8?B?ekZ1dzlhd2grazFOWmRCVkI4R0Q1S2dwdXJYaW9pVjF5d0hqMUpJcDFaNmcr?=
 =?utf-8?B?d1dsN3JSRW1JaUlYZ3NiOGw5VmYzY29wNUFkSzYyVGdSOXFQMnpMNnNvZk13?=
 =?utf-8?B?Ri9TblRMekZDQkxtTGZjQVh2TENrdjdRZkNuWnNGTFBpTXVwREJYby9vYUVj?=
 =?utf-8?B?cDA4UzBFYW02cXU1MncxRjdtcFoyaTBVelA5NUhIK2xray9ET1BiRjNmbmdv?=
 =?utf-8?B?OXdmbTR0Vm41Y3dDcFZlNEdyV2krcC9IUms4QUoxNnB2NDVVSjFBcjBpa1h5?=
 =?utf-8?B?dTRLb0pueEZQeFdOellNMnFKNEhxTCtabFE2WGhjUGJvbmZLUmZIMytVQWVX?=
 =?utf-8?B?a2lBUjJid0gwelNiSE1JM1JKcDdpOGxwbmU4SnlZbWlhN3YzOTczUWFRZWxJ?=
 =?utf-8?B?ZHFHeDhYR0VrZEpVOERTVjhQUFNtRjdIMUR3RXJ4Vkxqc1d2bEhuWG96Ukpm?=
 =?utf-8?B?dVZJMnhSdlRvV2tyQlg5SzAwQmMzcDJOVVoxRU5VYlB4eitSMnRNTGJlYnA5?=
 =?utf-8?B?NzlENVdOdXZzcGpGUzBtYnJxU3Jwa3M5azVmM0sxS3JVbXBnNElIZmxORkJz?=
 =?utf-8?B?YmdhUWhmNmRFelowalFZclN0V2FnVnR4SHhOOTJwQ1JZOEthT21EZ2VhQzRM?=
 =?utf-8?B?ejFRcWs3dmRUZGE1a0RMcE9YYjF5QkF3MU1RN1BXT044b0k3YktzOUFnTGFE?=
 =?utf-8?B?UE1FQVh5a1pHODlyd0NqUW1oeUdFNkwxNFp5bkwvR1VLcnpIQ3hIa2dYQUJS?=
 =?utf-8?B?SXI3bFBmbHBXVnptYUszdUw1Vy9NUmZwbXZIQzQ2V1I0ZzVsZTVRTVJUYXJM?=
 =?utf-8?B?ZGNpZlp6TDlhOWFnU1ZpWUpQTUpSQ2Ntckl5bytCaGVQdHhvT2xCbElqRTgw?=
 =?utf-8?B?NGttaHZoeWNGQ25lL25iMnNqUnEweTRFbmxhRFBqbVRrY1gra3RWS011UStK?=
 =?utf-8?B?VE1nSHJPdWlCZ29BTk1FK3NwMEV5amlmVmpVNW1oeWkzSzZrNm05am13c2pI?=
 =?utf-8?B?aXNvaVhVa09EbXJwZGhqSUdQVDJFVGZvR3p5UWhsUXVkUWRwNnlGbDdBSXEr?=
 =?utf-8?B?MWw2MmxrSnVXeDQyK2Z5NHJZUmdTbVNxQ1AvMXhiWWpYV0k4WmdiTEJMVjhU?=
 =?utf-8?B?NzdGSHRYODdiL2tiSmNSd1lFR0lWVjByZzVpS0tCMkVKbnQ3WlNVSmE3RUlo?=
 =?utf-8?B?R2hPZDVGK09TbkVNSDE3NmlMd0h2Nmd6S3JoeG1qM3k3N0d3MGw3eXRLUEtk?=
 =?utf-8?B?K1JYTG1vT0ErZDl5aFV6enZIc3huay8yOFZQaUt1bzJtckZlU0craEdteDg1?=
 =?utf-8?B?WjdyQzZBSGNpS0g1eTkwSzlEYUVZOXVVT1dmUFhYcmZ6anRYVGFzbWdFN3NC?=
 =?utf-8?B?QzlCa2dhOTI1aWtpcnB4Z0c0NHMwQnF6citFbnFmTThCL1hvcE1oN0FYWnl1?=
 =?utf-8?B?Qk1QYWJNVDRmQTEwdGJ5TDdrbzA0Mm9UcmcyZG91Qkc5SjIrTDF0a0grQ3Bw?=
 =?utf-8?B?c0kvTmdITFdSQUlsOGVtdVpQOEhTVlBQQ1UzemNrTFdsN1VEL2ZxQzlINW5p?=
 =?utf-8?B?NTFTYjM1cjU0M3hIMThvS25YRzNmY3B6R0gyUkNpQ2FpKzBhbzd2MnFvVG15?=
 =?utf-8?B?UkJ6MWU1MnBuRjRuTVczazd0cTFlbndUSmoxUTNpT2VHYWlzbENrendOQXV3?=
 =?utf-8?B?R0oxZi8ySEUxRGJ4cngwWURZVmkzTkZjUTQ4NzIyZjJWSm1NdlNHdWY3c1h5?=
 =?utf-8?B?WDJyOTdqL1BPRy9lOEV3dVg5S1J5b04zSERCc29pSktsTWNTcXNPc3IzbGhM?=
 =?utf-8?B?SVhQeERVSHE4cWo2Y0ZNT0pLZnhhSS8rM3MvVDFoWkVFMnJTdktENHc3dndZ?=
 =?utf-8?B?Mmc9PQ==?=
X-OriginatorOrg: virtuozzo.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 92f9edd9-3c07-46eb-a397-08da6e30b4ed
X-MS-Exchange-CrossTenant-AuthSource: VE1PR08MB4765.eurprd08.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jul 2022 11:27:50.0242
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +jAqu47hUZhVXoa5IgSSIOM3iCLld6tUIpJMB+9Pa1OfVkLL1vy8WgNQCBzL6UxMcDIqpan6K8n7V23hpxkNMbHTyJiXFejp3HEbVf2Lo6R+l34t2kMPYH+v8ZtQw8fC
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4175
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,NICE_REPLY_A,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE
        autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi,

On 18/07/2022 14:35, David Hildenbrand wrote:
> On 14.07.22 15:20, Alexander Atanasov wrote:
>> Allow the guest to know how much it is ballooned by the host.
>> It is useful when debugging out of memory conditions.
>>
>> When host gets back memory from the guest it is accounted
>> as used memory in the guest but the guest have no way to know
>> how much it is actually ballooned.
>>
>> Signed-off-by: Alexander Atanasov <alexander.atanasov@virtuozzo.com>
>> ---
>>   drivers/virtio/virtio_balloon.c     | 79 +++++++++++++++++++++++++++++
>>   include/uapi/linux/virtio_balloon.h |  1 +
>>   2 files changed, 80 insertions(+)
>>
>> V2:
>>   - fixed coding style
>>   - removed pretty print
>> V3:
>>   - removed dublicate of features
>>   - comment about balooned_pages more clear
>>   - convert host pages to balloon pages
>> V4:
>>   - added a define for BALLOON_PAGE_SIZE to make things clear
>> V5:
>>   - Made the calculatons work properly for both ways of memory accounting
>>     with or without deflate_on_oom
>>   - dropped comment
>>
[....]
>> +	u32 num_pages, total_pages, current_pages;
>> +	struct sysinfo i;
>> +
>> +	si_meminfo(&i);
>> +
>> +	seq_printf(f, "%-22s: %d\n", "page_size", VIRTIO_BALLOON_PAGE_SIZE);
> Why? Either export all in ordinary page size or in kB. No need to
> over-complicate the interface with a different page size that users
> don't actually care about.
>
> I'd just stick to /proc/meminfo and use kB.

The point is to expose some internal balloon data. Balloon works with 
pages not with kB  and users can easily calculate kB.

>> +
>> +	seq_printf(f, "%-22s: %u\n", "ballooned_pages", num_pages);
>> +
>> +	if (virtio_has_feature(b->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM)) {
>> +		total_pages = guest_to_balloon_pages(i.totalram);
>> +		current_pages = guest_to_balloon_pages(i.totalram) - num_pages;
>> +	} else {
>> +		total_pages = guest_to_balloon_pages(i.totalram) +  num_pages;
>> +		current_pages = guest_to_balloon_pages(i.totalram);
>> +	}
>> +
>> +	/* Total Memory for the guest from host */
>> +	seq_printf(f, "%-22s: %u\n", "total_pages", total_pages);
>> +
>> +	/* Current memory for the guest */
>> +	seq_printf(f, "%-22s: %u\n", "current_pages", current_pages);
> The think I detest about that interface (total/current) is that it's
> simply not correct -- because i.totalram for example doesn't include
> things like (similar to MemTotal in /proc/meminfo)
>
> a) crashkernel
> b) early boot allocations (e.g., memmap)
> c) any kind of possible memory (un)hotplug in the future
>
> I'd really suggest to just KIS and not mess with i.totalram.
>
> Exposing how much memory is inflated and some way to identify how that
> memory is accounted in /proc/meminfo should be good enough.
>
> E.g., the output here could simply be
>
> "Inflated: 1024 kB"
> "MemTotalReduced: 1024 kB"
>
> That would even allow in the future for flexibility when it comes to how
> much/what was actually subtracted from MemTotal.


The point of the debug interface is to expose some of the balloon driver 
internals to the guest.

The users of this are user space processes that try to work according to 
the memory pressure and nested virtualization.

I haven't seen one userspace software that expects total ram to change, 
it should be constant with one exception hotplug. But if you do  hotplug 
RAM you know that and you can restart what you need to restart.

So instead of messing with totalram with adding or removing pages /it 
would even be optimization since now it is done page by page/ i suggest 
to just account the inflated memory as used as in the deflate_on_oom 
case now. It is confusing and inconsistent as it is now. How to  explain 
to a vps user why his RAM is constantly changing?

And the file can go just to

inflated: <pages>

ballon_page_size:  4096

or

inflated: kB

I prefer pages because on theory guest and host can different size and 
if at some point guest starts to make requests to the host for pages it 
must somehow know what the host/balloon page is. Since you have all the 
information at one place it is easy to calculate kB. But you can not 
calculate pages from only kB.

Later on when hotplug comes it can add it's own data to the file so it 
can be used to see the amount that is added to the total ram.

It can add

hotplugged: <pages>


What do you think?


>> diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virtio_balloon.h
>> index ddaa45e723c4..f3ff7c4e5884 100644
>> --- a/include/uapi/linux/virtio_balloon.h
>> +++ b/include/uapi/linux/virtio_balloon.h
>> @@ -40,6 +40,7 @@
>>   
>>   /* Size of a PFN in the balloon interface. */
>>   #define VIRTIO_BALLOON_PFN_SHIFT 12
>> +#define VIRTIO_BALLOON_PAGE_SIZE (1<<VIRTIO_BALLOON_PFN_SHIFT)
> We prefer extra spacing
>
>   (1 << VIRTIO_BALLOON_PFN_SHIFT)


Ok, i am working on my  bad habits - thanks! I'll address the style 
issues and variable naming in the next version along with the future 
feedback.

-- 
Regards,
Alexander Atanasov