The following patch implements set_cpus_allowed() for stock 2.4 without
the O(1) scheduler.
The calling semantics and behavior remain the same as 2.5's method.
This is to provide a backward-compatible interface, specifically for
those interested in back-porting the new workqueue code to 2.4 -
set_cpus_allowed() seems to be the only nit preventing a straight
drop-in.
Patch is against 2.4.20-pre8 and untested but does compile.
Robert Love
diff -urN linux-2.4.20-pre8/include/linux/sched.h linux/include/linux/sched.h
--- linux-2.4.20-pre8/include/linux/sched.h Mon Sep 30 17:41:22 2002
+++ linux/include/linux/sched.h Tue Oct 1 18:35:28 2002
@@ -163,6 +164,12 @@
extern int start_context_thread(void);
extern int current_is_keventd(void);
+#if CONFIG_SMP
+extern void set_cpus_allowed(struct task_struct *p, unsigned long new_mask);
+#else
+# define set_cpus_allowed(p, new_mask) do { } while (0)
+#endif
+
/*
* The default fd array needs to be at least BITS_PER_LONG,
* as this is the granularity returned by copy_fdset().
diff -urN linux-2.4.20-pre8/kernel/ksyms.c linux/kernel/ksyms.c
--- linux-2.4.20-pre8/kernel/ksyms.c Mon Sep 30 17:41:22 2002
+++ linux/kernel/ksyms.c Tue Oct 1 18:34:41 2002
@@ -451,6 +451,9 @@
EXPORT_SYMBOL(interruptible_sleep_on_timeout);
EXPORT_SYMBOL(schedule);
EXPORT_SYMBOL(schedule_timeout);
+#if CONFIG_SMP
+EXPORT_SYMBOL(set_cpus_allowed);
+#endif
EXPORT_SYMBOL(yield);
EXPORT_SYMBOL(__cond_resched);
EXPORT_SYMBOL(jiffies);
diff -urN linux-2.4.20-pre8/kernel/sched.c linux/kernel/sched.c
--- linux-2.4.20-pre8/kernel/sched.c Mon Sep 30 17:41:22 2002
+++ linux/kernel/sched.c Tue Oct 1 18:54:49 2002
@@ -850,6 +850,46 @@
void scheduling_functions_end_here(void) { }
+#if CONFIG_SMP
+
+/**
+ * set_cpus_allowed() - change a given task's processor affinity
+ * @p: task to bind
+ * @new_mask: bitmask of allowed processors
+ *
+ * Upon return, the task is running on a legal processor. Note the caller
+ * must have a valid reference to the task: it must not exit() prematurely.
+ * This call can sleep; do not hold locks on call.
+ */
+void set_cpus_allowed(struct task_struct *p, unsigned long new_mask)
+{
+ new_mask &= cpu_online_map;
+ BUG_ON(!new_mask);
+
+ p->cpus_allowed = new_mask;
+
+ /*
+ * If the task is on a no-longer-allowed processor, we need to move
+ * it. If the task is not current, then set need_resched and send
+ * its processor an IPI to reschedule.
+ */
+ if (!(p->cpus_runnable & p->cpus_allowed)) {
+ if (p != current) {
+ p->need_resched = 1;
+ smp_send_reschedule(p->processor);
+ }
+ /*
+ * Wait until we are on a legal processor. If the task is
+ * current, then we should be on a legal processor the next
+ * time we reschedule. Otherwise, we need to wait for the IPI.
+ */
+ while (!(p->cpus_runnable & p->cpus_allowed))
+ schedule();
+ }
+}
+
+#endif /* CONFIG_SMP */
+
#ifndef __alpha__
/*
diff -urN linux-2.4.20-pre8/kernel/softirq.c linux/kernel/softirq.c
--- linux-2.4.20-pre8/kernel/softirq.c Mon Sep 30 17:41:22 2002
+++ linux/kernel/softirq.c Tue Oct 1 18:53:01 2002
@@ -368,9 +368,8 @@
sigfillset(¤t->blocked);
/* Migrate to the right CPU */
- current->cpus_allowed = 1UL << cpu;
- while (smp_processor_id() != cpu)
- schedule();
+ set_cpus_allowed(current, 1UL << cpu);
+ BUG_ON(smp_processor_id() != cpu);
sprintf(current->comm, "ksoftirqd_CPU%d", bind_cpu);
On Tue, Oct 01, 2002 at 07:03:28PM -0400, Robert Love wrote:
> The following patch implements set_cpus_allowed() for stock 2.4 without
> the O(1) scheduler.
>
> The calling semantics and behavior remain the same as 2.5's method.
>
> This is to provide a backward-compatible interface, specifically for
> those interested in back-porting the new workqueue code to 2.4 -
> set_cpus_allowed() seems to be the only nit preventing a straight
> drop-in.
>
> Patch is against 2.4.20-pre8 and untested but does compile.
Patch looks good to me, and I'd really like to have it in XFS :)
BTW, now that you have the core functionality I wonder why you don't
also add the cpu affinity syscalls..
On Wed, 2002-10-02 at 09:01, Christoph Hellwig wrote:
> Patch looks good to me, and I'd really like to have it in XFS :)
Good :)
> BTW, now that you have the core functionality I wonder why you don't
> also add the cpu affinity syscalls..
I already wrote them for 2.4, although I guess I should redo them for
the new set_cpus_allowed()... I have been waiting to send them to
Marcelo to ensure the 2.5 interfaces were solid and did not change. As
we approach the feature freeze, I guess we are getting there.
Robert Love
Hi Marcelo,
now that all vendors ship a backport of Ingo's O(1) scheduler external projects
like XFS have to track those projects in addition to the mainline kernel.
Having the common new APIs available in mainline would be a very good think for
thos projects. We already have a proper yield() in 2.4.20, but the
set_cpus_allowed() API used e.g. for kernelthreads bound to CPUs is still missing.
Any chance you could apply Robert Love's patch to add it for 2.4.20-rc2? Note
that it does not change any existing code but just adds that interface.
diff -urN linux-2.4.20-pre8/include/linux/sched.h linux/include/linux/sched.h
--- linux-2.4.20-pre8/include/linux/sched.h Mon Sep 30 17:41:22 2002
+++ linux/include/linux/sched.h Tue Oct 1 18:35:28 2002
@@ -163,6 +164,12 @@
extern int start_context_thread(void);
extern int current_is_keventd(void);
+#if CONFIG_SMP
+extern void set_cpus_allowed(struct task_struct *p, unsigned long new_mask);
+#else
+# define set_cpus_allowed(p, new_mask) do { } while (0)
+#endif
+
/*
* The default fd array needs to be at least BITS_PER_LONG,
* as this is the granularity returned by copy_fdset().
diff -urN linux-2.4.20-pre8/kernel/ksyms.c linux/kernel/ksyms.c
--- linux-2.4.20-pre8/kernel/ksyms.c Mon Sep 30 17:41:22 2002
+++ linux/kernel/ksyms.c Tue Oct 1 18:34:41 2002
@@ -451,6 +451,9 @@
EXPORT_SYMBOL(interruptible_sleep_on_timeout);
EXPORT_SYMBOL(schedule);
EXPORT_SYMBOL(schedule_timeout);
+#if CONFIG_SMP
+EXPORT_SYMBOL(set_cpus_allowed);
+#endif
EXPORT_SYMBOL(yield);
EXPORT_SYMBOL(__cond_resched);
EXPORT_SYMBOL(jiffies);
diff -urN linux-2.4.20-pre8/kernel/sched.c linux/kernel/sched.c
--- linux-2.4.20-pre8/kernel/sched.c Mon Sep 30 17:41:22 2002
+++ linux/kernel/sched.c Tue Oct 1 18:54:49 2002
@@ -850,6 +850,46 @@
void scheduling_functions_end_here(void) { }
+#if CONFIG_SMP
+
+/**
+ * set_cpus_allowed() - change a given task's processor affinity
+ * @p: task to bind
+ * @new_mask: bitmask of allowed processors
+ *
+ * Upon return, the task is running on a legal processor. Note the caller
+ * must have a valid reference to the task: it must not exit() prematurely.
+ * This call can sleep; do not hold locks on call.
+ */
+void set_cpus_allowed(struct task_struct *p, unsigned long new_mask)
+{
+ new_mask &= cpu_online_map;
+ BUG_ON(!new_mask);
+
+ p->cpus_allowed = new_mask;
+
+ /*
+ * If the task is on a no-longer-allowed processor, we need to move
+ * it. If the task is not current, then set need_resched and send
+ * its processor an IPI to reschedule.
+ */
+ if (!(p->cpus_runnable & p->cpus_allowed)) {
+ if (p != current) {
+ p->need_resched = 1;
+ smp_send_reschedule(p->processor);
+ }
+ /*
+ * Wait until we are on a legal processor. If the task is
+ * current, then we should be on a legal processor the next
+ * time we reschedule. Otherwise, we need to wait for the IPI.
+ */
+ while (!(p->cpus_runnable & p->cpus_allowed))
+ schedule();
+ }
+}
+
+#endif /* CONFIG_SMP */
+
#ifndef __alpha__
/*
On Mon, Nov 04, 2002 at 10:37:25PM -0500, Christoph Hellwig wrote:
>...
> now that all vendors ship a backport of Ingo's O(1) scheduler external projects
>...
Your "all vendors" doesn't include Debian?
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
On Wed, Nov 06, 2002 at 04:32:17PM +0100, Adrian Bunk wrote:
> On Mon, Nov 04, 2002 at 10:37:25PM -0500, Christoph Hellwig wrote:
> >...
> > now that all vendors ship a backport of Ingo's O(1) scheduler external projects
> >...
>
> Your "all vendors" doesn't include Debian?
No. Replace all vendors with all commercial vendors or all recently
releaseased distribution :)
On November 4, Christoph Hellwig wrote:
> +void set_cpus_allowed(struct task_struct *p, unsigned long new_mask)
> +{
> + new_mask &= cpu_online_map;
> + BUG_ON(!new_mask);
> +
> + p->cpus_allowed = new_mask;
> +
> + /*
> + * If the task is on a no-longer-allowed processor, we need to move
> + * it. If the task is not current, then set need_resched and send
> + * its processor an IPI to reschedule.
> + */
> + if (!(p->cpus_runnable & p->cpus_allowed)) {
> + if (p != current) {
> + p->need_resched = 1;
> + smp_send_reschedule(p->processor);
> + }
> + /*
> + * Wait until we are on a legal processor. If the task is
> + * current, then we should be on a legal processor the next
> + * time we reschedule. Otherwise, we need to wait for the IPI.
> + */
> + while (!(p->cpus_runnable & p->cpus_allowed))
> + schedule();
> + }
> +}
Is this implementation of set_cpus_allowed() Ok for all 2.4 kernels,
even if they (like RH8.0's) use a non-vanilla scheduler?
I'm asking because I need to put a set_cpus_allowed() implementation
in my performance counters driver's compat layer. If it makes any
difference, I'll only use set_cpus_allowed(p, new_mask) when p == current
or p is stopped and under ptrace() control by current.
/Mikael
On Mon, Dec 02, 2002 at 06:12:05PM +0100, Mikael Pettersson wrote:
> Is this implementation of set_cpus_allowed() Ok for all 2.4 kernels,
> even if they (like RH8.0's) use a non-vanilla scheduler?
No, it's for the stock scheduler. But RH8.0 already has set_cpus_allowed().
Christoph Hellwig writes:
> On Mon, Dec 02, 2002 at 06:12:05PM +0100, Mikael Pettersson wrote:
> > Is this implementation of set_cpus_allowed() Ok for all 2.4 kernels,
> > even if they (like RH8.0's) use a non-vanilla scheduler?
>
> No, it's for the stock scheduler. But RH8.0 already has set_cpus_allowed().
I knew RH8.0 has set_cpus_allowed(), but I wanted to avoid having to check
for being compiled in a RH-hacked kernel. LINUX_VERSION_CODE doesn't
distinguish between standard and "with tons of vendor-specific changes" :-(
I'll use your code then on stock 2.4 kernels, and work out some kludge
for the RH case.
Thanks,
/Mikael
On Mon, 2002-12-02 at 12:47, Mikael Pettersson wrote:
> I knew RH8.0 has set_cpus_allowed(), but I wanted to avoid having to check
> for being compiled in a RH-hacked kernel. LINUX_VERSION_CODE doesn't
> distinguish between standard and "with tons of vendor-specific changes" :-(
>
> I'll use your code then on stock 2.4 kernels, and work out some kludge
> for the RH case.
The code only works on the stock scheduler. It is the same interface
and has the same behavior as the O(1) scheduler version, but the code is
very very different.
If this patch is merged, you can safely use set_cpus_allowed() in either
kernel (which is the intention). But you cannot use this routine's code
on either scheduler.
Robert Love