Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753137AbbKYQYm (ORCPT ); Wed, 25 Nov 2015 11:24:42 -0500 Received: from mail-ig0-f193.google.com ([209.85.213.193]:34723 "EHLO mail-ig0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752752AbbKYQYj (ORCPT ); Wed, 25 Nov 2015 11:24:39 -0500 MIME-Version: 1.0 In-Reply-To: <5655DB99.3040007@intel.com> References: <1448372298-28386-1-git-send-email-tianyu.lan@intel.com> <1448372298-28386-4-git-send-email-tianyu.lan@intel.com> <20151124230551-mutt-send-email-mst@redhat.com> <56554994.1090305@intel.com> <20151125142437-mutt-send-email-mst@redhat.com> <5655DB99.3040007@intel.com> Date: Wed, 25 Nov 2015 08:24:38 -0800 Message-ID: Subject: Re: [RFC PATCH V2 3/3] Ixgbevf: Add migration support for ixgbevf driver From: Alexander Duyck To: "Lan, Tianyu" Cc: "Michael S. Tsirkin" , a.motakis@virtualopensystems.com, Alex Williamson , b.reynal@virtualopensystems.com, Bjorn Helgaas , Carolyn Wyborny , "Skidmore, Donald C" , eddie.dong@intel.com, nrupal.jani@intel.com, Alexander Graf , kvm@vger.kernel.org, Paolo Bonzini , qemu-devel@nongnu.org, "Tantilov, Emil S" , Or Gerlitz , "Rustad, Mark D" , Eric Auger , intel-wired-lan , Jeff Kirsher , "Brandeburg, Jesse" , "Ronciak, John" , linux-api@vger.kernel.org, "linux-kernel@vger.kernel.org" , "Vick, Matthew" , Mitch Williams , Netdev , "Nelson, Shannon" , Wei Yang , zajec5@gmail.com Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2766 Lines: 52 On Wed, Nov 25, 2015 at 8:02 AM, Lan, Tianyu wrote: > On 11/25/2015 8:28 PM, Michael S. Tsirkin wrote: >> >> Frankly, I don't really see what this short term hack buys us, >> and if it goes in, we'll have to maintain it forever. >> > > The framework of how to notify VF about migration status won't be > changed regardless of stopping VF or not before doing migration. > We hope to reach agreement on this first. Tracking dirty memory still > need to more discussions and we will continue working on it. Stop VF may > help to work around the issue and make tracking easier. The problem is you still have to stop the device at some point for the same reason why you have to halt the VM. You seem to think you can get by without doing that but you can't. All you do is open the system up to multiple races if you leave the device running. The goal should be to avoid stopping the device until the last possible moment, however it will still have to be stopped eventually. It isn't as if you can migrate memory and leave the device doing DMA and expect to get a clean state. I agree with Michael. The focus needs to be on first addressing dirty page tracking. Once you have that you could use a variation on the bonding solution where you postpone the hot-plug event until near the end of the migration just before you halt the guest instead of having to do it before you start the migration. Then after that we could look at optimizing things further by introducing a variation that you could further improve on things by introducing a variation of hot-plug that would pause the device as I suggested instead of removing it. At that point you should be able to have almost all of the key issues addresses so that you could drop the bond interface entirely. >> Also, assuming you just want to do ifdown/ifup for some reason, it's >> easy enough to do using a guest agent, in a completely generic way. >> > > Just ifdown/ifup is not enough for migration. It needs to restore some PCI > settings before doing ifup on the target machine That is why I have been suggesting making use of suspend/resume logic that is already in place for PCI power management. In the case of a suspend/resume we already have to deal with the fact that the device will go through a D0->D3->D0 reset so we have to restore all of the existing state. It would take a significant load off of Qemu since the guest would be restoring its own state instead of making Qemu have to do all of the device migration work. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/