2007-12-19 20:02:09

by Remy Bohmer

[permalink] [raw]
Subject: [patch 3/3] Enable setting of IRQ-thread priorities from kernel cmdline. (repost:CC to LKML)

The RT-patch originally creates all its softirq-threads at
priority 50. Of course, this is not a usable default for many
realtime systems and therefor these priorities has to be tuneable
for each RT-system. But, currently there is no way within the
kernel to adjust this to the needs of the user. Some scripts
are floating around to do this from userspace, but for several
applications the priorities should already be set properly by
the kernel itself before userland is started.

This patch changes this by adding a kernel cmd-line option that
can handle a map of priorities.

Remarks:
* Priorities are only set at creation time of the softirq.
* Priorities has to be set per-cpu.
* If userland overrules, it is NOT restored by this code.
* if no new kernel cmdline options are given, the kernel works
as before, and all softirqs start at 50.
* No wildcards are supported on the cmdline.

See kernel/Documentation/kernel-parameters.txt for usage info.

Signed-off-by: Remy Bohmer <[email protected]>

---
Documentation/kernel-parameters.txt | 11 +++
kernel/softirq.c | 105 ++++++++++++++++++++++++++++++------
2 files changed, 99 insertions(+), 17 deletions(-)

Index: linux-2.6.24-rc5-rt1/Documentation/kernel-parameters.txt
===================================================================
--- linux-2.6.24-rc5-rt1.orig/Documentation/kernel-parameters.txt 2007-12-18 22:06:11.000000000 +0100
+++ linux-2.6.24-rc5-rt1/Documentation/kernel-parameters.txt 2007-12-18 22:07:19.000000000 +0100
@@ -806,6 +806,17 @@ and is between 256 and 4096 characters.
If this cmdline argument is ommitted, every thread
runs at prio 50.

+ sirq_pmap= [IRQ-threading] List of priorities each softirq
+ thread must have.
+ Format: sirq_pmap=block/0:90,sched/0:75,50
+ The priorities have to be specified per-cpu.
+ The first field without ':', is the default prio.
+ The names have to match the softirq_names[] table in
+ kernel/softirq.c, (thus without 'softirq-' prefix) to
+ keep the cmd-line short.
+ If this cmdline argument is ommitted, every softirq
+ runs at prio 50.
+
ports= [IP_VS_FTP] IPVS ftp helper module
Default is 21.
Up to 8 (IP_VS_APP_MAX_PORTS) ports
Index: linux-2.6.24-rc5-rt1/kernel/softirq.c
===================================================================
--- linux-2.6.24-rc5-rt1.orig/kernel/softirq.c 2007-12-18 22:06:11.000000000 +0100
+++ linux-2.6.24-rc5-rt1/kernel/softirq.c 2007-12-18 22:08:54.000000000 +0100
@@ -66,6 +66,10 @@ struct softirqdata {

static DEFINE_PER_CPU(struct softirqdata [MAX_SOFTIRQ], ksoftirqd);

+static char *cmdline;
+static int default_prio = MAX_USER_RT_PRIO/2;
+
+
#ifdef CONFIG_PREEMPT_SOFTIRQS
/*
* Preempting the softirq causes cases that would not be a
@@ -770,10 +774,28 @@ EXPORT_SYMBOL(tasklet_unlock_wait);

#endif

+static const char *softirq_names [] =
+{
+ [HI_SOFTIRQ] = "high",
+ [SCHED_SOFTIRQ] = "sched",
+ [TIMER_SOFTIRQ] = "timer",
+ [NET_TX_SOFTIRQ] = "net-tx",
+ [NET_RX_SOFTIRQ] = "net-rx",
+ [BLOCK_SOFTIRQ] = "block",
+ [TASKLET_SOFTIRQ] = "tasklet",
+#ifdef CONFIG_HIGH_RES_TIMERS
+ [HRTIMER_SOFTIRQ] = "hrtimer",
+#endif
+ [RCU_SOFTIRQ] = "rcu",
+};
+
+static int get_softirq_prio(const char *name);
+
static int ksoftirqd(void * __data)
{
- struct sched_param param = { .sched_priority = MAX_USER_RT_PRIO/2 };
+ struct sched_param param = { 0, };
struct softirqdata *data = __data;
+ char buf[50];
u32 softirq_mask = (1 << data->nr);
struct softirq_action *h;
int cpu = data->cpu;
@@ -781,8 +803,12 @@ static int ksoftirqd(void * __data)
#ifdef CONFIG_PREEMPT_SOFTIRQS
init_waitqueue_head(&data->wait);
#endif
-
+ /* Lookup the priority of this softirq, and set the prio accordingly */
+ snprintf(buf, sizeof(buf), "%s/%lu",
+ softirq_names[data->nr], data->cpu);
+ param.sched_priority = get_softirq_prio(buf);
sys_sched_setscheduler(current->pid, SCHED_FIFO, &param);
+
current->flags |= PF_SOFTIRQ;
set_current_state(TASK_INTERRUPTIBLE);

@@ -911,21 +937,6 @@ void takeover_tasklets(unsigned int cpu)
}
#endif /* CONFIG_HOTPLUG_CPU */

-static const char *softirq_names [] =
-{
- [HI_SOFTIRQ] = "high",
- [SCHED_SOFTIRQ] = "sched",
- [TIMER_SOFTIRQ] = "timer",
- [NET_TX_SOFTIRQ] = "net-tx",
- [NET_RX_SOFTIRQ] = "net-rx",
- [BLOCK_SOFTIRQ] = "block",
- [TASKLET_SOFTIRQ] = "tasklet",
-#ifdef CONFIG_HIGH_RES_TIMERS
- [HRTIMER_SOFTIRQ] = "hrtimer",
-#endif
- [RCU_SOFTIRQ] = "rcu",
-};
-
static int __cpuinit cpu_callback(struct notifier_block *nfb,
unsigned long action,
void *hcpu)
@@ -1051,3 +1062,63 @@ int on_each_cpu(void (*func) (void *info
}
EXPORT_SYMBOL(on_each_cpu);
#endif
+
+/*
+ * Lookup the softirq thread priority.
+ * A map for the priorities can be given on the kernel commandline.
+ * if name is NULL, or no kernel commandline options is given, the default
+ * prio is returned, which is either MAX_PRIO/2, or the default given on the
+ * kernel commandline.
+ */
+static int get_softirq_prio(const char *name)
+{
+ int prio = default_prio; /* Set the default for all threads */
+
+ /* If no commandline options, use thread prio defaults the old style.*/
+ if (!cmdline)
+ return prio;
+
+ if (!get_map_option(cmdline, name, &prio)) {
+ prio = default_prio;
+ if (name != NULL)
+ printk(KERN_INFO
+ "No priority specified for 'softirq-%s', " \
+ "using default(=%d)\n", name, prio);
+ } else {
+ if ((prio <= 0) || (prio >= MAX_USER_RT_PRIO))
+ prio = default_prio;
+
+ if (name != NULL)
+ printk(KERN_INFO "Using priority %d for 'softirq-%s'\n",
+ prio, name);
+ }
+ return prio;
+}
+
+/*
+ * Store the pointer to the arguments in a global var, and store the
+ * default prio globally
+ */
+static int __init softirq_prio_map_setup(char *str)
+{
+ if (!str) /* sanity check */
+ return 1;
+
+ cmdline = str; /* store it for later use */
+ default_prio = get_softirq_prio(NULL);
+
+ printk(KERN_INFO "Default softirq priority: %d\n", default_prio);
+
+ return 1;
+}
+
+/*
+ * The commandline looks like this:
+ * sirq_pmap=block/0:90,sched/0:75,50
+ * The first field without the ':' is used as the default for all
+ * soft-irq-threads. The priorities are only set on creation of the
+ * softirq threads. Unknown or redundant fields are ignored.
+ * The names have to match the softirq_names[] table without 'softirq-' prefix
+ * to keep the cmd-line short.
+ */
+__setup("sirq_pmap=", softirq_prio_map_setup);

--


2007-12-19 21:04:29

by Randy Dunlap

[permalink] [raw]
Subject: Re: [patch 3/3] Enable setting of IRQ-thread priorities from kernel cmdline. (repost:CC to LKML)

On Wed, 19 Dec 2007 20:45:54 +0100 Remy Bohmer wrote:

> Documentation/kernel-parameters.txt | 11 +++
> kernel/softirq.c | 105 ++++++++++++++++++++++++++++++------
> 2 files changed, 99 insertions(+), 17 deletions(-)
>
> Index: linux-2.6.24-rc5-rt1/Documentation/kernel-parameters.txt
> ===================================================================
> --- linux-2.6.24-rc5-rt1.orig/Documentation/kernel-parameters.txt 2007-12-18 22:06:11.000000000 +0100
> +++ linux-2.6.24-rc5-rt1/Documentation/kernel-parameters.txt 2007-12-18 22:07:19.000000000 +0100
> @@ -806,6 +806,17 @@ and is between 256 and 4096 characters.
> If this cmdline argument is ommitted, every thread
> runs at prio 50.
>
> + sirq_pmap= [IRQ-threading] List of priorities each softirq
> + thread must have.
> + Format: sirq_pmap=block/0:90,sched/0:75,50
> + The priorities have to be specified per-cpu.
> + The first field without ':', is the default prio.

Drop comma.

> + The names have to match the softirq_names[] table in
> + kernel/softirq.c, (thus without 'softirq-' prefix) to
> + keep the cmd-line short.
> + If this cmdline argument is ommitted, every softirq

omitted,

> + runs at prio 50.
> +
> ports= [IP_VS_FTP] IPVS ftp helper module
> Default is 21.
> Up to 8 (IP_VS_APP_MAX_PORTS) ports
> Index: linux-2.6.24-rc5-rt1/kernel/softirq.c
> ===================================================================
> --- linux-2.6.24-rc5-rt1.orig/kernel/softirq.c 2007-12-18 22:06:11.000000000 +0100
> +++ linux-2.6.24-rc5-rt1/kernel/softirq.c 2007-12-18 22:08:54.000000000 +0100
> @@ -1051,3 +1062,63 @@ int on_each_cpu(void (*func) (void *info
> }
> EXPORT_SYMBOL(on_each_cpu);
> #endif
> +
> +/*
> + * Lookup the softirq thread priority.
> + * A map for the priorities can be given on the kernel commandline.
> + * if name is NULL, or no kernel commandline options is given, the default

If ...................................... option is given,

> + * prio is returned, which is either MAX_PRIO/2, or the default given on the
> + * kernel commandline.
> + */
> +static int get_softirq_prio(const char *name)
> +{
...
> +}

---
~Randy

2007-12-20 11:59:38

by Jaswinder Singh

[permalink] [raw]
Subject: Re: [patch 3/3] Enable setting of IRQ-thread priorities from kernel cmdline. (repost:CC to LKML)

On 12/20/07, Remy Bohmer <[email protected]> wrote:
> The RT-patch originally creates all its softirq-threads at
> priority 50.

Is it possible to run softirq-threads or IRQs as a non-realtime tasks
in RT kernel.

If yes, then how.

Thank you,

Jaswinder Singh.

2007-12-20 12:26:00

by Remy Bohmer

[permalink] [raw]
Subject: Re: [patch 3/3] Enable setting of IRQ-thread priorities from kernel cmdline. (repost:CC to LKML)

Hello Jaswinder,

> Is it possible to run softirq-threads or IRQs as a non-realtime tasks
> in RT kernel.

Currently the answer is no. (Unless overruled from userspace) I did
not take this requirement into account.
So, Is this a serious requirement? Should this be possible?
I cannot imagine a purpose in which that is wanted, it would cause
unpredictable interrupt handling, and I can imagine that many
drivers/devices will not like that.

The purpose is usually to spread the threads across several
priorities, to create room for some RT-process to become more
important than an IRQ that generates large latencies, and each
RT-system designer shall divide the priorities different, due to
different system requirements.

Kind Regards,

Remy

2007/12/20, Jaswinder Singh <[email protected]>:
> On 12/20/07, Remy Bohmer <[email protected]> wrote:
> > The RT-patch originally creates all its softirq-threads at
> > priority 50.
>
> Is it possible to run softirq-threads or IRQs as a non-realtime tasks
> in RT kernel.
>
> If yes, then how.
>
> Thank you,
>
> Jaswinder Singh.
>

2007-12-20 12:45:54

by Jaswinder Singh

[permalink] [raw]
Subject: Re: [patch 3/3] Enable setting of IRQ-thread priorities from kernel cmdline. (repost:CC to LKML)

Hello Remy,

On 12/20/07, Remy Bohmer <[email protected]> wrote:
> So, Is this a serious requirement? Should this be possible?

I have noticed this problem:
[email protected]:~# cat /proc/loadavgrt
1.00 1.00 1.00 0/52 1158
[email protected]:~# cat /proc/loadavg
0.00 0.00 0.02 1/52 1159
[email protected]:~#

So I am curious, if possible, user can switch softirq-threads or IRQs
RT tasks to non-RT tasks for slow hardware or least important hardware
for NON-RT tasks. So this will improve RT behaviour.

By default, all softirq-threads or IRQs will be RT task but if there
is some option user can switch it to NON-RT task then it will be good.

Thank you,

Jaswinder Singh.

2007-12-20 12:51:00

by Remy Bohmer

[permalink] [raw]
Subject: Re: [patch 3/3] Enable setting of IRQ-thread priorities from kernel cmdline. (repost:CC to LKML)

> By default, all softirq-threads or IRQs will be RT task but if there
> is some option user can switch it to NON-RT task then it will be good.

Okay, understood.
I will think about how to add that too. But, probably we can add it
seperately from this patchset.

Kind Regards,

Remy


2007/12/20, Jaswinder Singh <[email protected]>:
> Hello Remy,
>
> On 12/20/07, Remy Bohmer <[email protected]> wrote:
> > So, Is this a serious requirement? Should this be possible?
>
> I have noticed this problem:
> [email protected]:~# cat /proc/loadavgrt
> 1.00 1.00 1.00 0/52 1158
> [email protected]:~# cat /proc/loadavg
> 0.00 0.00 0.02 1/52 1159
> [email protected]:~#
>
> So I am curious, if possible, user can switch softirq-threads or IRQs
> RT tasks to non-RT tasks for slow hardware or least important hardware
> for NON-RT tasks. So this will improve RT behaviour.
>
> By default, all softirq-threads or IRQs will be RT task but if there
> is some option user can switch it to NON-RT task then it will be good.
>
> Thank you,
>
> Jaswinder Singh.
>

2007-12-20 13:19:50

by Juergen Borleis

[permalink] [raw]
Subject: Re: [patch 3/3] Enable setting of IRQ-thread priorities from kernel cmdline. (repost:CC to LKML)

On Thursday 20 December 2007 13:45, Jaswinder Singh wrote:
> On 12/20/07, Remy Bohmer <[email protected]> wrote:
> > So, Is this a serious requirement? Should this be possible?
>
> I have noticed this problem:
> [email protected]:~# cat /proc/loadavgrt
> 1.00 1.00 1.00 0/52 1158
> [email protected]:~# cat /proc/loadavg
> 0.00 0.00 0.02 1/52 1159
> [email protected]:~#
>
> So I am curious, if possible, user can switch softirq-threads or IRQs
> RT tasks to non-RT tasks for slow hardware or least important hardware
> for NON-RT tasks. So this will improve RT behaviour.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Why?

IMHO: Simply decrease the RT priority of less important IRQs or increase all
other more important IRQs. IRQs are always more important than other
processes in a system, also in non RT systems.

Juergen
--
Dipl.-Ing. Juergen Beisert | http://www.pengutronix.de
?Pengutronix - Linux Solutions for Science and Industry
? Handelsregister: Amtsgericht Hildesheim, HRA 2686
? ? ? Vertretung Sued/Muenchen, Germany
Phone: +49-8766-939 228 | Fax: +49-5121-206917-9

2007-12-20 13:41:56

by Jaswinder Singh

[permalink] [raw]
Subject: Re: [patch 3/3] Enable setting of IRQ-thread priorities from kernel cmdline. (repost:CC to LKML)

hello Juergen,

On 12/20/07, Juergen Beisert <[email protected]> wrote:
> On Thursday 20 December 2007 13:45, Jaswinder Singh wrote:
> > So I am curious, if possible, user can switch softirq-threads or IRQs
> > RT tasks to non-RT tasks for slow hardware or least important hardware
> > for NON-RT tasks. So this will improve RT behaviour.
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> Why?
>
> IMHO: Simply decrease the RT priority of less important IRQs or increase all
> other more important IRQs. IRQs are always more important than other
> processes in a system, also in non RT systems.
>

In RT Kernel we have :-
RT Tasks and Non-RT tasks.

So we can also support :-
RT softirq-threads/IRQs and non-RT softirq-threads/IRQs

So RT softirq-threads/IRQs for RT Tasks
and non-RT softirq-threads/IRQs for non-RT Tasks.

OR also remove Non-RT tasks from RT Kernel and simply decrease the RT
priority of less important task as per your suggestions.

Thank you,

Jaswinder Singh.

> Juergen
> --
> Dipl.-Ing. Juergen Beisert | http://www.pengutronix.de
> Pengutronix - Linux Solutions for Science and Industry
> Handelsregister: Amtsgericht Hildesheim, HRA 2686
> Vertretung Sued/Muenchen, Germany
> Phone: +49-8766-939 228 | Fax: +49-5121-206917-9
>