CPU may spend a lot of time in a guest mode while other CPUs wait for
rcu grace period to elapse. This patch series makes guest mode into
quiescent state to shorten wait time.
Gleb Natapov (2):
rcu: provide rcu_virt_note_context_switch() function.
KVM: make guest mode entry to be rcu quiescent state
include/linux/kvm_host.h | 9 +++++++++
include/linux/rcutiny.h | 8 ++++++++
include/linux/rcutree.h | 10 ++++++++++
kernel/rcutree.c | 1 +
4 files changed, 28 insertions(+), 0 deletions(-)
--
1.7.4.4
Provide rcu_virt_note_context_switch() for vitalization use to note
quiescent state during guest entry.
Signed-off-by: Gleb Natapov <[email protected]>
---
include/linux/rcutiny.h | 8 ++++++++
include/linux/rcutree.h | 10 ++++++++++
kernel/rcutree.c | 1 +
3 files changed, 19 insertions(+), 0 deletions(-)
diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h
index 30ebd7c..52b3e02 100644
--- a/include/linux/rcutiny.h
+++ b/include/linux/rcutiny.h
@@ -100,6 +100,14 @@ static inline void rcu_note_context_switch(int cpu)
}
/*
+ * Take advantage of the fact that there is only one CPU, which
+ * allows us to ignore virtualization-based context switches.
+ */
+static inline void rcu_virt_note_context_switch(int cpu)
+{
+}
+
+/*
* Return the number of grace periods.
*/
static inline long rcu_batches_completed(void)
diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h
index 3a93348..b2a906e 100644
--- a/include/linux/rcutree.h
+++ b/include/linux/rcutree.h
@@ -35,6 +35,16 @@ extern void rcu_note_context_switch(int cpu);
extern int rcu_needs_cpu(int cpu);
extern void rcu_cpu_stall_reset(void);
+/*
+ * Note a virtualization-based context switch. This is simply a
+ * wrapper around rcu_note_context_switch(), which allows TINY_RCU
+ * to save a few bytes.
+ */
+static inline void rcu_virt_note_context_switch(int cpu)
+{
+ rcu_note_context_switch(cpu);
+}
+
#ifdef CONFIG_TREE_PREEMPT_RCU
extern void exit_rcu(void);
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index dd4aea8..0837d63 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -124,6 +124,7 @@ void rcu_note_context_switch(int cpu)
rcu_sched_qs(cpu);
rcu_preempt_note_context_switch(cpu);
}
+EXPORT_SYMBOL_GPL(rcu_note_context_switch);
#ifdef CONFIG_NO_HZ
DEFINE_PER_CPU(struct rcu_dynticks, rcu_dynticks) = {
--
1.7.4.4
KVM does not hold any references to rcu protected data when it switches
CPU into a guest mode. In fact switching to a guest mode is very similar
to exiting to userspase from rcu point of view. In addition CPU may stay
in a guest mode for quite a long time (up to one time slice). Lets treat
guest mode as quiescent state, just like we do with user-mode execution.
Signed-off-by: Gleb Natapov <[email protected]>
---
include/linux/kvm_host.h | 9 +++++++++
1 files changed, 9 insertions(+), 0 deletions(-)
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 0bc3d37..244413f 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -591,8 +591,17 @@ static inline int kvm_deassign_device(struct kvm *kvm,
static inline void kvm_guest_enter(void)
{
+ BUG_ON(preemptible());
account_system_vtime(current);
current->flags |= PF_VCPU;
+ /* KVM does not hold any references to rcu protected data when it
+ * switches CPU into a guest mode. In fact switching to a guest mode
+ * is very similar to exiting to userspase from rcu point of view. In
+ * addition CPU may stay in a guest mode for quite a long time (up to
+ * one time slice). Lets treat guest mode as quiescent state, just like
+ * we do with user-mode execution.
+ */
+ rcu_virt_note_context_switch(smp_processor_id());
}
static inline void kvm_guest_exit(void)
--
1.7.4.4
On Wed, May 04, 2011 at 04:31:03PM +0300, Gleb Natapov wrote:
> Provide rcu_virt_note_context_switch() for vitalization use to note
> quiescent state during guest entry.
Very good, queued on -rcu.
Unless you tell me otherwise, I will assume that you want to carry the
patch modifying KVM to use this.
Thanx, Paul
> Signed-off-by: Gleb Natapov <[email protected]>
> ---
> include/linux/rcutiny.h | 8 ++++++++
> include/linux/rcutree.h | 10 ++++++++++
> kernel/rcutree.c | 1 +
> 3 files changed, 19 insertions(+), 0 deletions(-)
>
> diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h
> index 30ebd7c..52b3e02 100644
> --- a/include/linux/rcutiny.h
> +++ b/include/linux/rcutiny.h
> @@ -100,6 +100,14 @@ static inline void rcu_note_context_switch(int cpu)
> }
>
> /*
> + * Take advantage of the fact that there is only one CPU, which
> + * allows us to ignore virtualization-based context switches.
> + */
> +static inline void rcu_virt_note_context_switch(int cpu)
> +{
> +}
> +
> +/*
> * Return the number of grace periods.
> */
> static inline long rcu_batches_completed(void)
> diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h
> index 3a93348..b2a906e 100644
> --- a/include/linux/rcutree.h
> +++ b/include/linux/rcutree.h
> @@ -35,6 +35,16 @@ extern void rcu_note_context_switch(int cpu);
> extern int rcu_needs_cpu(int cpu);
> extern void rcu_cpu_stall_reset(void);
>
> +/*
> + * Note a virtualization-based context switch. This is simply a
> + * wrapper around rcu_note_context_switch(), which allows TINY_RCU
> + * to save a few bytes.
> + */
> +static inline void rcu_virt_note_context_switch(int cpu)
> +{
> + rcu_note_context_switch(cpu);
> +}
> +
> #ifdef CONFIG_TREE_PREEMPT_RCU
>
> extern void exit_rcu(void);
> diff --git a/kernel/rcutree.c b/kernel/rcutree.c
> index dd4aea8..0837d63 100644
> --- a/kernel/rcutree.c
> +++ b/kernel/rcutree.c
> @@ -124,6 +124,7 @@ void rcu_note_context_switch(int cpu)
> rcu_sched_qs(cpu);
> rcu_preempt_note_context_switch(cpu);
> }
> +EXPORT_SYMBOL_GPL(rcu_note_context_switch);
>
> #ifdef CONFIG_NO_HZ
> DEFINE_PER_CPU(struct rcu_dynticks, rcu_dynticks) = {
> --
> 1.7.4.4
>
On 05/04/2011 07:35 PM, Paul E. McKenney wrote:
> On Wed, May 04, 2011 at 04:31:03PM +0300, Gleb Natapov wrote:
> > Provide rcu_virt_note_context_switch() for vitalization use to note
> > quiescent state during guest entry.
>
> Very good, queued on -rcu.
>
> Unless you tell me otherwise, I will assume that you want to carry the
> patch modifying KVM to use this.
Is -rcu a fast-forward-only tree (like tip)? If so I'll merge it and
apply patch 2.
--
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
On Mon, May 09, 2011 at 11:51:34AM +0300, Avi Kivity wrote:
> On 05/04/2011 07:35 PM, Paul E. McKenney wrote:
> >On Wed, May 04, 2011 at 04:31:03PM +0300, Gleb Natapov wrote:
> >> Provide rcu_virt_note_context_switch() for vitalization use to note
> >> quiescent state during guest entry.
> >
> >Very good, queued on -rcu.
> >
> >Unless you tell me otherwise, I will assume that you want to carry the
> >patch modifying KVM to use this.
>
> Is -rcu a fast-forward-only tree (like tip)? If so I'll merge it
> and apply patch 2.
Yep, -rcu is subject to rebase and feeds into -tip. The patch is
SHA 29ce831000081dd757d3116bf774aafffc4b6b20 in
git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git
Branch is rcu/next. My guess is that this commit will show up in
-tip soon.
Thanx, Paul
On 05/04/2011 04:31 PM, Gleb Natapov wrote:
> KVM does not hold any references to rcu protected data when it switches
> CPU into a guest mode. In fact switching to a guest mode is very similar
> to exiting to userspase from rcu point of view. In addition CPU may stay
> in a guest mode for quite a long time (up to one time slice). Lets treat
> guest mode as quiescent state, just like we do with user-mode execution.
Applied, thanks.
--
error compiling committee.c: too many arguments to function