Received: by 2002:a25:c593:0:0:0:0:0 with SMTP id v141csp1084404ybe; Wed, 4 Sep 2019 12:18:46 -0700 (PDT) X-Google-Smtp-Source: APXvYqxHZeOjy01hONnnscNzNshipt28hXu50D70fqORA47dXZOTaGfcPo5gwBXTbzgY+/SrdTml X-Received: by 2002:a63:9245:: with SMTP id s5mr37238494pgn.123.1567624726087; Wed, 04 Sep 2019 12:18:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1567624726; cv=none; d=google.com; s=arc-20160816; b=dm1S2wg7z7ceixYPAESwM1wrw3LRP1W0o/dRFnoUkeUEiPVtivEBs9hKM6AgW1UJ6f qTHiYjEb0S8qXsrYi4GR704+kj1/Vixc0uZo0d6lezOTVhf/dw++8g1G4dTSGtGnpwtQ VxN801icRR11HJI7hNfgUWqwBGeiF2XmPxTrUCuBpoMcU7zu0kM9+0a8L6OApj/aLslJ LA6fIXflfmEVcSokNgBKnpJy6S0o5xCwEIC00Jl8gsQNqnzIpeIwEcLtCI1V3m7Fk3hF aJAutIvF6N8hTl2TUjPotZ2E/hpDIqTNSC7100SqI92LF2BQyMePTlwn0xu6qoLT7es4 /Oew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:in-reply-to:content-disposition :mime-version:references:message-id:subject:cc:to:from:date; bh=NLpKeAwvZauzvcOfJSaBzm71bWfF1nQX4VNdTFWyNbM=; b=osTSI9F5rM7e/tudu/BY1uU7OfJ0howXDcd/e2F1YNbY9lUK4fp4tLBfYvK5FB6HXp kYVXXg1nfvko+V6HEOfHUD5wxedQwp/olNycGez9Z0OUOtmd9lGPlnw/v6Rl515wrjN6 HBsDKrfbTDXfgKA2UMm9I4oNLGrMOMemS2vC3HRoHE83+/LjZJJqTDqvgrzxuOD5eLjl sPR8Ua75dlBKsyaRm37XIoQtIGNDIcvmRm4qnwvJliwDEvwfizp5lZmi9J53SjZKJyrE VdeKFFfFWWq03OFGVFC7RSPdbwt8O2lsY2LdmitaHuZF/T3hsuxdclbBQKKU/DTYNuMW 2itQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id p187si21889633pfp.5.2019.09.04.12.18.29; Wed, 04 Sep 2019 12:18:46 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732969AbfIDTRl (ORCPT + 99 others); Wed, 4 Sep 2019 15:17:41 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53088 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732132AbfIDTRk (ORCPT ); Wed, 4 Sep 2019 15:17:40 -0400 Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B59E212E5 for ; Wed, 4 Sep 2019 19:17:39 +0000 (UTC) Received: by mail-qt1-f198.google.com with SMTP id u7so13223215qtg.7 for ; Wed, 04 Sep 2019 12:17:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=NLpKeAwvZauzvcOfJSaBzm71bWfF1nQX4VNdTFWyNbM=; b=CjG1lAo57OZauL1w67b5zH5r57W7KoxzuIe6WwyHDilfTINhyoOX1oZNHLoFOOhOUa 2ANnulTN7vA3ibfgeu3RevDFDiXiakkuI/8ZbJjIQsawvUL6bmted/R5We2nraDf/HK7 tzdLbJKoyjFJvgh9vEgd2XL4+4/qbrQdKpFvB3+4iyyn5JcD8JLcXhUdVn5CGULEFCoV mUKaWMICA1XnQEOBOpMq8XJfxTXFeGio4KoF9f11JbjuJBXObNSfLwUaPy0M5Sgx+Xdc ENXNJYXiki0UwQbsZAM2QTCdwm+eH1SEidL/7WeUdADYq3otWhdhIUlChi9/fTQaQOxb MqPw== X-Gm-Message-State: APjAAAU+sLm4qOBAHAtrqqncZVPWcNGyD/QoPkg1qKOoKluorwVp8qOa X8awoYWW0eVroIMLJpp9jDNVzNSbGnBS4LLMtlixvS4k3Nz7+bNHMaHvvETj8KVDl8z9tmVcV12 SXY6/9RRGrbyW7Tiesr5WB6Lw X-Received: by 2002:ae9:f00b:: with SMTP id l11mr3225423qkg.322.1567624659001; Wed, 04 Sep 2019 12:17:39 -0700 (PDT) X-Received: by 2002:ae9:f00b:: with SMTP id l11mr3225401qkg.322.1567624658823; Wed, 04 Sep 2019 12:17:38 -0700 (PDT) Received: from redhat.com (bzq-79-176-40-226.red.bezeqint.net. [79.176.40.226]) by smtp.gmail.com with ESMTPSA id d13sm5728359qkj.18.2019.09.04.12.17.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2019 12:17:37 -0700 (PDT) Date: Wed, 4 Sep 2019 15:17:30 -0400 From: "Michael S. Tsirkin" To: Alexander Duyck Cc: nitesh@redhat.com, kvm@vger.kernel.org, david@redhat.com, dave.hansen@intel.com, linux-kernel@vger.kernel.org, willy@infradead.org, mhocko@kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, virtio-dev@lists.oasis-open.org, osalvador@suse.de, yang.zhang.wz@gmail.com, pagupta@redhat.com, riel@surriel.com, konrad.wilk@oracle.com, lcapitulino@redhat.com, wei.w.wang@intel.com, aarcange@redhat.com, pbonzini@redhat.com, dan.j.williams@intel.com, alexander.h.duyck@linux.intel.com Subject: Re: [PATCH v7 6/6] virtio-balloon: Add support for providing unused page reports to host Message-ID: <20190904151506-mutt-send-email-mst@kernel.org> References: <20190904150920.13848.32271.stgit@localhost.localdomain> <20190904151102.13848.65770.stgit@localhost.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190904151102.13848.65770.stgit@localhost.localdomain> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Sep 04, 2019 at 08:11:02AM -0700, Alexander Duyck wrote: > From: Alexander Duyck > > Add support for the page reporting feature provided by virtio-balloon. > Reporting differs from the regular balloon functionality in that is is > much less durable than a standard memory balloon. Instead of creating a > list of pages that cannot be accessed the pages are only inaccessible > while they are being indicated to the virtio interface. Once the > interface has acknowledged them they are placed back into their respective > free lists and are once again accessible by the guest system. > > Signed-off-by: Alexander Duyck > --- > drivers/virtio/Kconfig | 1 + > drivers/virtio/virtio_balloon.c | 65 +++++++++++++++++++++++++++++++++++ > include/uapi/linux/virtio_balloon.h | 1 + > 3 files changed, 67 insertions(+) > > diff --git a/drivers/virtio/Kconfig b/drivers/virtio/Kconfig > index 078615cf2afc..4b2dd8259ff5 100644 > --- a/drivers/virtio/Kconfig > +++ b/drivers/virtio/Kconfig > @@ -58,6 +58,7 @@ config VIRTIO_BALLOON > tristate "Virtio balloon driver" > depends on VIRTIO > select MEMORY_BALLOON > + select PAGE_REPORTING > ---help--- > This driver supports increasing and decreasing the amount > of memory within a KVM guest. > diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c > index 2c19457ab573..0b400bb382c0 100644 > --- a/drivers/virtio/virtio_balloon.c > +++ b/drivers/virtio/virtio_balloon.c > @@ -19,6 +19,7 @@ > #include > #include > #include > +#include > > /* > * Balloon device works in 4K page units. So each page is pointed to by > @@ -37,6 +38,9 @@ > #define VIRTIO_BALLOON_FREE_PAGE_SIZE \ > (1 << (VIRTIO_BALLOON_FREE_PAGE_ORDER + PAGE_SHIFT)) > > +/* limit on the number of pages that can be on the reporting vq */ > +#define VIRTIO_BALLOON_VRING_HINTS_MAX 16 > + > #ifdef CONFIG_BALLOON_COMPACTION > static struct vfsmount *balloon_mnt; > #endif > @@ -46,6 +50,7 @@ enum virtio_balloon_vq { > VIRTIO_BALLOON_VQ_DEFLATE, > VIRTIO_BALLOON_VQ_STATS, > VIRTIO_BALLOON_VQ_FREE_PAGE, > + VIRTIO_BALLOON_VQ_REPORTING, > VIRTIO_BALLOON_VQ_MAX > }; > > @@ -113,6 +118,10 @@ struct virtio_balloon { > > /* To register a shrinker to shrink memory upon memory pressure */ > struct shrinker shrinker; > + > + /* Unused page reporting device */ > + struct virtqueue *reporting_vq; > + struct page_reporting_dev_info ph_dev_info; > }; > > static struct virtio_device_id id_table[] = { > @@ -152,6 +161,32 @@ static void tell_host(struct virtio_balloon *vb, struct virtqueue *vq) > > } > > +void virtballoon_unused_page_report(struct page_reporting_dev_info *ph_dev_info, > + unsigned int nents) > +{ > + struct virtio_balloon *vb = > + container_of(ph_dev_info, struct virtio_balloon, ph_dev_info); > + struct virtqueue *vq = vb->reporting_vq; > + unsigned int unused, err; > + > + /* We should always be able to add these buffers to an empty queue. */ > + err = virtqueue_add_inbuf(vq, ph_dev_info->sg, nents, vb, > + GFP_NOWAIT | __GFP_NOWARN); > + > + /* > + * In the extremely unlikely case that something has changed and we > + * are able to trigger an error we will simply display a warning > + * and exit without actually processing the pages. > + */ > + if (WARN_ON(err)) > + return; > + > + virtqueue_kick(vq); > + > + /* When host has read buffer, this completes via balloon_ack */ > + wait_event(vb->acked, virtqueue_get_buf(vq, &unused)); > +} > + So just to make sure I understand, this always passes a single buf to the vq and then waits until that completes, correct? Thus there are never outstanding bufs on the vq and this is why we don't need e.g. any cleanup. > static void set_page_pfns(struct virtio_balloon *vb, > __virtio32 pfns[], struct page *page) > { > @@ -476,6 +511,7 @@ static int init_vqs(struct virtio_balloon *vb) > names[VIRTIO_BALLOON_VQ_DEFLATE] = "deflate"; > names[VIRTIO_BALLOON_VQ_STATS] = NULL; > names[VIRTIO_BALLOON_VQ_FREE_PAGE] = NULL; > + names[VIRTIO_BALLOON_VQ_REPORTING] = NULL; > > if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_STATS_VQ)) { > names[VIRTIO_BALLOON_VQ_STATS] = "stats"; > @@ -487,11 +523,19 @@ static int init_vqs(struct virtio_balloon *vb) > callbacks[VIRTIO_BALLOON_VQ_FREE_PAGE] = NULL; > } > > + if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING)) { > + names[VIRTIO_BALLOON_VQ_REPORTING] = "reporting_vq"; > + callbacks[VIRTIO_BALLOON_VQ_REPORTING] = balloon_ack; > + } > + > err = vb->vdev->config->find_vqs(vb->vdev, VIRTIO_BALLOON_VQ_MAX, > vqs, callbacks, names, NULL, NULL); > if (err) > return err; > > + if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING)) > + vb->reporting_vq = vqs[VIRTIO_BALLOON_VQ_REPORTING]; > + > vb->inflate_vq = vqs[VIRTIO_BALLOON_VQ_INFLATE]; > vb->deflate_vq = vqs[VIRTIO_BALLOON_VQ_DEFLATE]; > if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_STATS_VQ)) { > @@ -931,12 +975,30 @@ static int virtballoon_probe(struct virtio_device *vdev) > if (err) > goto out_del_balloon_wq; > } > + > + vb->ph_dev_info.report = virtballoon_unused_page_report; > + if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING)) { > + unsigned int capacity; > + > + capacity = min_t(unsigned int, > + virtqueue_get_vring_size(vb->reporting_vq), > + VIRTIO_BALLOON_VRING_HINTS_MAX); > + vb->ph_dev_info.capacity = capacity; > + > + err = page_reporting_startup(&vb->ph_dev_info); > + if (err) > + goto out_unregister_shrinker; > + } > + > virtio_device_ready(vdev); > > if (towards_target(vb)) > virtballoon_changed(vdev); > return 0; > > +out_unregister_shrinker: > + if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM)) > + virtio_balloon_unregister_shrinker(vb); > out_del_balloon_wq: > if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_FREE_PAGE_HINT)) > destroy_workqueue(vb->balloon_wq); > @@ -965,6 +1027,8 @@ static void virtballoon_remove(struct virtio_device *vdev) > { > struct virtio_balloon *vb = vdev->priv; > > + if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING)) > + page_reporting_shutdown(&vb->ph_dev_info); > if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM)) > virtio_balloon_unregister_shrinker(vb); > spin_lock_irq(&vb->stop_update_lock); > @@ -1034,6 +1098,7 @@ static int virtballoon_validate(struct virtio_device *vdev) > VIRTIO_BALLOON_F_DEFLATE_ON_OOM, > VIRTIO_BALLOON_F_FREE_PAGE_HINT, > VIRTIO_BALLOON_F_PAGE_POISON, > + VIRTIO_BALLOON_F_REPORTING, > }; > > static struct virtio_driver virtio_balloon_driver = { > diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virtio_balloon.h > index a1966cd7b677..19974392d324 100644 > --- a/include/uapi/linux/virtio_balloon.h > +++ b/include/uapi/linux/virtio_balloon.h > @@ -36,6 +36,7 @@ > #define VIRTIO_BALLOON_F_DEFLATE_ON_OOM 2 /* Deflate balloon on OOM */ > #define VIRTIO_BALLOON_F_FREE_PAGE_HINT 3 /* VQ to report free pages */ > #define VIRTIO_BALLOON_F_PAGE_POISON 4 /* Guest is using page poisoning */ > +#define VIRTIO_BALLOON_F_REPORTING 5 /* Page reporting virtqueue */ > > /* Size of a PFN in the balloon interface. */ > #define VIRTIO_BALLOON_PFN_SHIFT 12