The problem we observed was that the performance was lower with a large
number of threads (># of CPUs, such as 2x) with SPECjbb. With fewer
threads, the sched-domain scheduler performed slightly better. What we
found was that the sched-domain changes balance_interval (between
min_interaval and max_interval) reflecting success/failure of load
balancing, whereas the base scheduler does not. That value determines
how often we do inter and intra node baloancing, and we see the same
performance if we use the same hard code value as the base scheduler
does.
Nick,
That algorithm sounds reasonable to me, but how did you pick up
min_interval and max_interval, especially for NUMA?
Jun
>-----Original Message-----
>From: [email protected] [mailto:lse-tech-
>[email protected]] On Behalf Of Nakajima, Jun
>Sent: Thursday, March 25, 2004 7:15 AM
>To: Andi Kleen; Ingo Molnar
>Cc: [email protected]; [email protected];
[email protected];
>[email protected]; [email protected]; [email protected];
>[email protected]; [email protected]; [email protected]
>Subject: RE: [Lse-tech] [patch] sched-domain cleanups,
sched-2.6.5-rc2-mm2-
>A3
>
>We have found some performance regressions (e.g. SPECjbb) with the
>scheduler on a large IA-64 NUMA machine, and we are debugging it. On
SMP
>machines, we haven't seen performance regressions.
>
>Jun
>
>>-----Original Message-----
>>From: Andi Kleen [mailto:[email protected]]
>>Sent: Wednesday, March 24, 2004 8:56 PM
>>To: Ingo Molnar
>>Cc: [email protected]; [email protected];
>[email protected];
>>[email protected]; [email protected]; Nakajima, Jun;
>>[email protected]; [email protected]; [email protected];
>>[email protected]
>>Subject: Re: [Lse-tech] [patch] sched-domain cleanups,
>sched-2.6.5-rc2-mm2-
>>A3
>>
>>On Thu, 25 Mar 2004 09:28:09 +0100
>>Ingo Molnar <[email protected]> wrote:
>>
>>> i've reviewed the sched-domains balancing patches for upstream
>inclusion
>>> and they look mostly fine.
>>
>>The main problem it has is that it performs quite badly on Opteron
NUMA
>>e.g. in the OpenMP STREAM test (much worse than the normal scheduler)
>>
>>-Andi
>
>
>-------------------------------------------------------
>This SF.Net email is sponsored by: IBM Linux Tutorials
>Free Linux tutorial presented by Daniel Robbins, President and CEO of
>GenToo technologies. Learn everything from fundamentals to system
>administration.http://ads.osdn.com/?ad_id70&alloc_id638&op=ick
>_______________________________________________
>Lse-tech mailing list
>[email protected]
>https://lists.sourceforge.net/lists/listinfo/lse-tech
Any idea what the context switch rate was on the large wh runs? Any
sched_yield type locking can put a lot of demand on the scheduler, and with a
lower balance frequency it can make the problem worse, one side affect being
idle time. Do you have increased idle time with the lower throughput?
Also, what JVM are you using?
On Tuesday 30 March 2004 15:40, Nakajima, Jun wrote:
> The problem we observed was that the performance was lower with a large
> number of threads (># of CPUs, such as 2x) with SPECjbb. With fewer
> threads, the sched-domain scheduler performed slightly better. What we
> found was that the sched-domain changes balance_interval (between
> min_interaval and max_interval) reflecting success/failure of load
> balancing, whereas the base scheduler does not. That value determines
> how often we do inter and intra node baloancing, and we see the same
> performance if we use the same hard code value as the base scheduler
> does.
>
> Nick,
> That algorithm sounds reasonable to me, but how did you pick up
> min_interval and max_interval, especially for NUMA?
>
> Jun
>
> >-----Original Message-----
>
> From: [email protected] [mailto:lse-tech-
>
> >[email protected]] On Behalf Of Nakajima, Jun
> >Sent: Thursday, March 25, 2004 7:15 AM
> >To: Andi Kleen; Ingo Molnar
> >Cc: [email protected]; [email protected];
>
> [email protected];
>
> >[email protected]; [email protected]; [email protected];
> >[email protected]; [email protected]; [email protected]
> >Subject: RE: [Lse-tech] [patch] sched-domain cleanups,
>
> sched-2.6.5-rc2-mm2-
>
> >A3
> >
> >We have found some performance regressions (e.g. SPECjbb) with the
> >scheduler on a large IA-64 NUMA machine, and we are debugging it. On
>
> SMP
>
> >machines, we haven't seen performance regressions.
> >
> >Jun
> >
> >>-----Original Message-----
> >>From: Andi Kleen [mailto:[email protected]]
> >>Sent: Wednesday, March 24, 2004 8:56 PM
> >>To: Ingo Molnar
> >>Cc: [email protected]; [email protected];
> >
> >[email protected];
> >
> >>[email protected]; [email protected]; Nakajima, Jun;
> >>[email protected]; [email protected]; [email protected];
> >>[email protected]
> >>Subject: Re: [Lse-tech] [patch] sched-domain cleanups,
> >
> >sched-2.6.5-rc2-mm2-
> >
> >>A3
> >>
> >>On Thu, 25 Mar 2004 09:28:09 +0100
> >>
> >>Ingo Molnar <[email protected]> wrote:
> >>> i've reviewed the sched-domains balancing patches for upstream
> >
> >inclusion
> >
> >>> and they look mostly fine.
> >>
> >>The main problem it has is that it performs quite badly on Opteron
>
> NUMA
>
> >>e.g. in the OpenMP STREAM test (much worse than the normal scheduler)
> >>
> >>-Andi
> >
> >-------------------------------------------------------
> >This SF.Net email is sponsored by: IBM Linux Tutorials
> >Free Linux tutorial presented by Daniel Robbins, President and CEO of
> >GenToo technologies. Learn everything from fundamentals to system
> >administration.http://ads.osdn.com/?ad_id70&alloc_id638&op=ick
> >_______________________________________________
> >Lse-tech mailing list
> >[email protected]
> >https://lists.sourceforge.net/lists/listinfo/lse-tech
>
> -------------------------------------------------------
> This SF.Net email is sponsored by: IBM Linux Tutorials
> Free Linux tutorial presented by Daniel Robbins, President and CEO of
> GenToo technologies. Learn everything from fundamentals to system
> administration.http://ads.osdn.com/?ad_id70&alloc_id638&op=Click
> _______________________________________________
> Lse-tech mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/lse-tech