Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758177AbcDEMky (ORCPT ); Tue, 5 Apr 2016 08:40:54 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45540 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757696AbcDEMkw (ORCPT ); Tue, 5 Apr 2016 08:40:52 -0400 Date: Tue, 5 Apr 2016 08:40:46 -0400 From: Luiz Capitulino To: Yang Zhang Cc: Rik van Riel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, rkrcmar@redhat.com, mtosatti@redhat.com, bsd@redhat.com Subject: Re: [PATCH] kvm: x86: make lapic hrtimer pinned Message-ID: <20160405084046.21f68608@redhat.com> In-Reply-To: <57035899.6080609@gmail.com> References: <20160404164607.09e306fa@redhat.com> <1459803623.6219.28.camel@redhat.com> <57035899.6080609@gmail.com> Organization: Red Hat MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2309 Lines: 52 On Tue, 5 Apr 2016 14:18:01 +0800 Yang Zhang wrote: > On 2016/4/5 5:00, Rik van Riel wrote: > > On Mon, 2016-04-04 at 16:46 -0400, Luiz Capitulino wrote: > >> When a vCPU runs on a nohz_full core, the hrtimer used by > >> the lapic emulation code can be migrated to another core. > >> When this happens, it's possible to observe milisecond > >> latency when delivering timer IRQs to KVM guests. > >> > >> The huge latency is mainly due to the fact that > >> apic_timer_fn() expects to run during a kvm exit. It > >> sets KVM_REQ_PENDING_TIMER and let it be handled on kvm > >> entry. However, if the timer fires on a different core, > >> we have to wait until the next kvm exit for the guest > >> to see KVM_REQ_PENDING_TIMER set. > >> > >> This problem became visible after commit 9642d18ee. This > >> commit changed the timer migration code to always attempt > >> to migrate timers away from nohz_full cores. While it's > >> discussable if this is correct/desirable (I don't think > >> it is), it's clear that the lapic emulation code has > >> a requirement on firing the hrtimer in the same core > >> where it was started. This is achieved by making the > >> hrtimer pinned. > > > > Given that delivering a timer to a guest seems to > > involve trapping from the guest to the host, anyway, > > I don't see a downside to your patch. > > > > If that is ever changed (eg. allowing delivery of > > a timer interrupt to a VCPU without trapping to the > > host), we may want to revisit this. > > > Posted interrupt helps in this case. Currently, KVM doesn't use PI for > lapic timer is due to same affinity for lapic timer and VCPU. Now, we > can change to use PI for lapic timer. The only concern is what's > frequency of timer migration in upstream Linux? If it is frequently, > will it bring additional cost? I can't answer this questions. > BTW, in what case the migration of timers during VCPU scheduling will fail? For hrtimers (which is the lapic emulation case), it only succeeds if the destination core has a hrtimer expiring before the hrtimer being migrated. Also, if the hrtimer callback function is already running (that is, the timer fired already) it's not migrated either. But I _guess_ this case doesn't affect KVM (and there's no much do about it anyways).