Objective
~~~~~~~~~
The objective of the io-throttle controller is to improve IO performance
predictability of different cgroups that share the same block devices.
State of the art (quick overview)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A recent work made by Vivek propose a weighted BW solution introducing
fair queuing support in the elevator layer and modifying the existent IO
schedulers to use that functionality
(https://lists.linux-foundation.org/pipermail/containers/2009-March/016129.html).
For the fair queuing part Vivek's IO controller makes use of the BFQ
code as posted by Paolo and Fabio (http://lkml.org/lkml/2008/11/11/148).
The dm-ioband controller by the valinux guys is also proposing a
proportional ticket-based solution fully implemented at the device
mapper level (http://people.valinux.co.jp/~ryov/dm-ioband/).
The bio-cgroup patch (http://people.valinux.co.jp/~ryov/bio-cgroup/) is
a BIO tracking mechanism for cgroups, implemented in the cgroup memory
subsystem. It is maintained by Ryo and it allows dm-ioband to track
writeback requests issued by kernel threads (pdflush).
Another work by Satoshi implements the cgroup awareness in CFQ, mapping
per-cgroup priority to CFQ IO priorities and this also provide only the
proportional BW support (http://lwn.net/Articles/306772/).
Please correct me or integrate if I missed someone or something. :)
Proposed solution
~~~~~~~~~~~~~~~~~
Respect to other priority/weight-based solutions the approach used by
this controller is to explicitly choke applications' requests that
directly or indirectly generate IO activity in the system (this
controller addresses both synchronous IO and writeback/buffered IO).
The bandwidth and iops limiting method has the advantage of improving
the performance predictability at the cost of reducing, in general, the
overall performance of the system in terms of throughput.
IO throttling and accounting is performed during the submission of IO
requests and it is independent of the particular IO scheduler.
Detailed informations about design, goal and usage are described in the
documentation (see [PATCH 1/7]).
Implementation
~~~~~~~~~~~~~~
Patchset against latest Linus' git:
[PATCH 0/7] cgroup: block device IO controller (v14)
[PATCH 1/7] io-throttle documentation
[PATCH 2/7] res_counter: introduce ratelimiting attributes
[PATCH 3/7] page_cgroup: provide a generic page tracking infrastructure
[PATCH 4/7] io-throttle controller infrastructure
[PATCH 5/7] kiothrottled: throttle buffered (writeback) IO
[PATCH 6/7] io-throttle instrumentation
[PATCH 7/7] export per-task io-throttle statistics to userspace
The v14 all-in-one patch, along with the previous versions, can be found at:
http://download.systemimager.org/~arighi/linux/patches/io-throttle/
What's new
~~~~~~~~~~
In this new version I've embedded the bio-cgroup code inside
io-throttle, providing the page_cgroup page tracking infrastructure.
This completely removes the complexity and the overhead of associating
multiple IO controllers (bio-cgroup groups and io-throttle groups) from
userspace, preserving the same tracking and throttling functionalities
for writeback IO. And it is also possibel to bind other cgroup
subsystems with io-throttle.
I've removed the tracking of IO generated by anonymous pages (swap), to
reduce the overhead of the page tracking functionality (and probably is
not a good idea to delay IO requests that come from swap-in/swap-out
operations).
I've also removed the ext3 specific patch to tag journal IO with
BIO_RW_META to never throttle such IO requests.
As suggested by Ted and Jens we need a more specific solution, where
filesystems inform the IO subsystem which IO requests come from tasks
that are holding filesystem exclusive resources (journal IO, metadata,
etc.). Then, the IO subsystem (both the IO scheduler and the IO
controller) will be able to dispatched those "special" requests at the
highest priority to avoid the classic priority inversion problems.
Changelog (v13 -> v14)
~~~~~~~~~~~~~~~~~~~~~~
* implemented the bio-cgroup functionality as pure infrastructure for page
tracking capability
* removed the tracking and throttling of IO generated by anonymous pages (swap)
* updated documentation
Overall diffstat
~~~~~~~~~~~~~~~~
Documentation/cgroups/io-throttle.txt | 417 +++++++++++++++++
block/Makefile | 1 +
block/blk-core.c | 8 +
block/blk-io-throttle.c | 822 +++++++++++++++++++++++++++++++++
block/kiothrottled.c | 341 ++++++++++++++
fs/aio.c | 12 +
fs/buffer.c | 2 +
fs/proc/base.c | 18 +
include/linux/blk-io-throttle.h | 144 ++++++
include/linux/cgroup_subsys.h | 6 +
include/linux/memcontrol.h | 6 +
include/linux/mmzone.h | 4 +-
include/linux/page_cgroup.h | 33 ++-
include/linux/res_counter.h | 69 ++-
include/linux/sched.h | 7 +
init/Kconfig | 16 +
kernel/fork.c | 7 +
kernel/res_counter.c | 72 +++
mm/Makefile | 3 +-
mm/bounce.c | 2 +
mm/filemap.c | 2 +
mm/memcontrol.c | 6 +
mm/page-writeback.c | 2 +
mm/page_cgroup.c | 95 ++++-
mm/readahead.c | 3 +
25 files changed, 2065 insertions(+), 33 deletions(-)
-Andrea
Andrea Righi wrote:
...
>
> Implementation
> ~~~~~~~~~~~~~~
> Patchset against latest Linus' git:
>
> [PATCH 0/7] cgroup: block device IO controller (v14)
> [PATCH 1/7] io-throttle documentation
> [PATCH 2/7] res_counter: introduce ratelimiting attributes
> [PATCH 3/7] page_cgroup: provide a generic page tracking infrastructure
> [PATCH 4/7] io-throttle controller infrastructure
> [PATCH 5/7] kiothrottled: throttle buffered (writeback) IO
> [PATCH 6/7] io-throttle instrumentation
> [PATCH 7/7] export per-task io-throttle statistics to userspace
Hi Andrea,
I'd like to have a try this patchset, would you tell what's the kernel version
and bio-cgroup version based on?
--
Regards
Gui Jianfeng
On Mon, Apr 20, 2009 at 04:39:29PM +0800, Gui Jianfeng wrote:
> Andrea Righi wrote:
> ...
> >
> > Implementation
> > ~~~~~~~~~~~~~~
> > Patchset against latest Linus' git:
> >
> > [PATCH 0/7] cgroup: block device IO controller (v14)
> > [PATCH 1/7] io-throttle documentation
> > [PATCH 2/7] res_counter: introduce ratelimiting attributes
> > [PATCH 3/7] page_cgroup: provide a generic page tracking infrastructure
> > [PATCH 4/7] io-throttle controller infrastructure
> > [PATCH 5/7] kiothrottled: throttle buffered (writeback) IO
> > [PATCH 6/7] io-throttle instrumentation
> > [PATCH 7/7] export per-task io-throttle statistics to userspace
>
> Hi Andrea,
>
> I'd like to have a try this patchset, would you tell what's the kernel version
> and bio-cgroup version based on?
>
Latest Linus' git is the kernel and bio-cgroup v7 (from
http://people.valinux.co.jp/~ryov/bio-cgroup/).
Thanks!
-Andrea
Andrea Righi wrote:
> On Mon, Apr 20, 2009 at 04:39:29PM +0800, Gui Jianfeng wrote:
>> Andrea Righi wrote:
>> ...
>>> Implementation
>>> ~~~~~~~~~~~~~~
>>> Patchset against latest Linus' git:
>>>
>>> [PATCH 0/7] cgroup: block device IO controller (v14)
>>> [PATCH 1/7] io-throttle documentation
>>> [PATCH 2/7] res_counter: introduce ratelimiting attributes
>>> [PATCH 3/7] page_cgroup: provide a generic page tracking infrastructure
>>> [PATCH 4/7] io-throttle controller infrastructure
>>> [PATCH 5/7] kiothrottled: throttle buffered (writeback) IO
>>> [PATCH 6/7] io-throttle instrumentation
>>> [PATCH 7/7] export per-task io-throttle statistics to userspace
>> Hi Andrea,
>>
>> I'd like to have a try this patchset, would you tell what's the kernel version
>> and bio-cgroup version based on?
>>
>
> Latest Linus' git is the kernel and bio-cgroup v7 (from
> http://people.valinux.co.jp/~ryov/bio-cgroup/).
Sorry, I still can't apply it...
[root@localhost linux-2.6.30-rc1]# patch -p1 --dry-run < ../cgroup-io-throttle-v14.patch patching file Documentation/cgroups/io-throttle.txt
patching file block/Makefile
patching file block/blk-core.c
patching file block/blk-io-throttle.c
patching file block/kiothrottled.c
patching file fs/aio.c
patching file fs/buffer.c
Hunk #1 FAILED at 36.
Hunk #2 FAILED at 669.
2 out of 2 hunks FAILED -- saving rejects to file fs/buffer.c.rej
patching file fs/proc/base.c
patching file include/linux/blk-io-throttle.h
patching file include/linux/cgroup_subsys.h
Hunk #1 succeeded at 49 (offset 6 lines).
patching file include/linux/memcontrol.h
Reversed (or previously applied) patch detected! Assume -R? [n] n
Apply anyway? [n] n
Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file include/linux/memcontrol.h.rej
patching file include/linux/mmzone.h
Hunk #1 FAILED at 607.
Hunk #2 FAILED at 958.
...
>
> Thanks!
> -Andrea
>
>
>
--
Regards
Gui Jianfeng
On Tue, Apr 21, 2009 at 09:16:22AM +0800, Gui Jianfeng wrote:
> Andrea Righi wrote:
> > On Mon, Apr 20, 2009 at 04:39:29PM +0800, Gui Jianfeng wrote:
> >> Andrea Righi wrote:
> >> ...
> >>> Implementation
> >>> ~~~~~~~~~~~~~~
> >>> Patchset against latest Linus' git:
> >>>
> >>> [PATCH 0/7] cgroup: block device IO controller (v14)
> >>> [PATCH 1/7] io-throttle documentation
> >>> [PATCH 2/7] res_counter: introduce ratelimiting attributes
> >>> [PATCH 3/7] page_cgroup: provide a generic page tracking infrastructure
> >>> [PATCH 4/7] io-throttle controller infrastructure
> >>> [PATCH 5/7] kiothrottled: throttle buffered (writeback) IO
> >>> [PATCH 6/7] io-throttle instrumentation
> >>> [PATCH 7/7] export per-task io-throttle statistics to userspace
> >> Hi Andrea,
> >>
> >> I'd like to have a try this patchset, would you tell what's the kernel version
> >> and bio-cgroup version based on?
> >>
> >
> > Latest Linus' git is the kernel and bio-cgroup v7 (from
> > http://people.valinux.co.jp/~ryov/bio-cgroup/).
>
> Sorry, I still can't apply it...
>
> [root@localhost linux-2.6.30-rc1]# patch -p1 --dry-run < ../cgroup-io-throttle-v14.patch patching file Documentation/cgroups/io-throttle.txt
> patching file block/Makefile
> patching file block/blk-core.c
> patching file block/blk-io-throttle.c
> patching file block/kiothrottled.c
> patching file fs/aio.c
> patching file fs/buffer.c
> Hunk #1 FAILED at 36.
> Hunk #2 FAILED at 669.
> 2 out of 2 hunks FAILED -- saving rejects to file fs/buffer.c.rej
> patching file fs/proc/base.c
> patching file include/linux/blk-io-throttle.h
> patching file include/linux/cgroup_subsys.h
> Hunk #1 succeeded at 49 (offset 6 lines).
> patching file include/linux/memcontrol.h
> Reversed (or previously applied) patch detected! Assume -R? [n] n
> Apply anyway? [n] n
> Skipping patch.
> 2 out of 2 hunks ignored -- saving rejects to file include/linux/memcontrol.h.rej
> patching file include/linux/mmzone.h
> Hunk #1 FAILED at 607.
> Hunk #2 FAILED at 958.
> ...
Could you checkout the latest Linus' git?
$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git linux-2.6
Or at least try with 2.6.30-rc2. It should apply cleanly also to this
version.
-Andrea
Andrea Righi wrote:
> On Tue, Apr 21, 2009 at 09:16:22AM +0800, Gui Jianfeng wrote:
>> Andrea Righi wrote:
>>> On Mon, Apr 20, 2009 at 04:39:29PM +0800, Gui Jianfeng wrote:
>>>> Andrea Righi wrote:
>>>> ...
>>>>> Implementation
>>>>> ~~~~~~~~~~~~~~
>>>>> Patchset against latest Linus' git:
>>>>>
>>>>> [PATCH 0/7] cgroup: block device IO controller (v14)
>>>>> [PATCH 1/7] io-throttle documentation
>>>>> [PATCH 2/7] res_counter: introduce ratelimiting attributes
>>>>> [PATCH 3/7] page_cgroup: provide a generic page tracking infrastructure
>>>>> [PATCH 4/7] io-throttle controller infrastructure
>>>>> [PATCH 5/7] kiothrottled: throttle buffered (writeback) IO
>>>>> [PATCH 6/7] io-throttle instrumentation
>>>>> [PATCH 7/7] export per-task io-throttle statistics to userspace
>>>> Hi Andrea,
>>>>
>>>> I'd like to have a try this patchset, would you tell what's the kernel version
>>>> and bio-cgroup version based on?
>>>>
>>> Latest Linus' git is the kernel and bio-cgroup v7 (from
>>> http://people.valinux.co.jp/~ryov/bio-cgroup/).
>> Sorry, I still can't apply it...
>>
>> [root@localhost linux-2.6.30-rc1]# patch -p1 --dry-run < ../cgroup-io-throttle-v14.patch patching file Documentation/cgroups/io-throttle.txt
>> patching file block/Makefile
>> patching file block/blk-core.c
>> patching file block/blk-io-throttle.c
>> patching file block/kiothrottled.c
>> patching file fs/aio.c
>> patching file fs/buffer.c
>> Hunk #1 FAILED at 36.
>> Hunk #2 FAILED at 669.
>> 2 out of 2 hunks FAILED -- saving rejects to file fs/buffer.c.rej
>> patching file fs/proc/base.c
>> patching file include/linux/blk-io-throttle.h
>> patching file include/linux/cgroup_subsys.h
>> Hunk #1 succeeded at 49 (offset 6 lines).
>> patching file include/linux/memcontrol.h
>> Reversed (or previously applied) patch detected! Assume -R? [n] n
>> Apply anyway? [n] n
>> Skipping patch.
>> 2 out of 2 hunks ignored -- saving rejects to file include/linux/memcontrol.h.rej
>> patching file include/linux/mmzone.h
>> Hunk #1 FAILED at 607.
>> Hunk #2 FAILED at 958.
>> ...
>
> Could you checkout the latest Linus' git?
>
> $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git linux-2.6
>
> Or at least try with 2.6.30-rc2. It should apply cleanly also to this
> version.
Yes, I tried them all, but didn't work. :(
Would you try "cgroup-io-throttle-v14.patch" too?
>
> -Andrea
>
>
>
--
Regards
Gui Jianfeng
On Wed, Apr 22, 2009 at 09:09:16AM +0800, Gui Jianfeng wrote:
> Andrea Righi wrote:
> > On Tue, Apr 21, 2009 at 09:16:22AM +0800, Gui Jianfeng wrote:
> >> Andrea Righi wrote:
> >>> On Mon, Apr 20, 2009 at 04:39:29PM +0800, Gui Jianfeng wrote:
> >>>> Andrea Righi wrote:
> >>>> ...
> >>>>> Implementation
> >>>>> ~~~~~~~~~~~~~~
> >>>>> Patchset against latest Linus' git:
> >>>>>
> >>>>> [PATCH 0/7] cgroup: block device IO controller (v14)
> >>>>> [PATCH 1/7] io-throttle documentation
> >>>>> [PATCH 2/7] res_counter: introduce ratelimiting attributes
> >>>>> [PATCH 3/7] page_cgroup: provide a generic page tracking infrastructure
> >>>>> [PATCH 4/7] io-throttle controller infrastructure
> >>>>> [PATCH 5/7] kiothrottled: throttle buffered (writeback) IO
> >>>>> [PATCH 6/7] io-throttle instrumentation
> >>>>> [PATCH 7/7] export per-task io-throttle statistics to userspace
> >>>> Hi Andrea,
> >>>>
> >>>> I'd like to have a try this patchset, would you tell what's the kernel version
> >>>> and bio-cgroup version based on?
> >>>>
> >>> Latest Linus' git is the kernel and bio-cgroup v7 (from
> >>> http://people.valinux.co.jp/~ryov/bio-cgroup/).
> >> Sorry, I still can't apply it...
> >>
> >> [root@localhost linux-2.6.30-rc1]# patch -p1 --dry-run < ../cgroup-io-throttle-v14.patch patching file Documentation/cgroups/io-throttle.txt
> >> patching file block/Makefile
> >> patching file block/blk-core.c
> >> patching file block/blk-io-throttle.c
> >> patching file block/kiothrottled.c
> >> patching file fs/aio.c
> >> patching file fs/buffer.c
> >> Hunk #1 FAILED at 36.
> >> Hunk #2 FAILED at 669.
> >> 2 out of 2 hunks FAILED -- saving rejects to file fs/buffer.c.rej
> >> patching file fs/proc/base.c
> >> patching file include/linux/blk-io-throttle.h
> >> patching file include/linux/cgroup_subsys.h
> >> Hunk #1 succeeded at 49 (offset 6 lines).
> >> patching file include/linux/memcontrol.h
> >> Reversed (or previously applied) patch detected! Assume -R? [n] n
> >> Apply anyway? [n] n
> >> Skipping patch.
> >> 2 out of 2 hunks ignored -- saving rejects to file include/linux/memcontrol.h.rej
> >> patching file include/linux/mmzone.h
> >> Hunk #1 FAILED at 607.
> >> Hunk #2 FAILED at 958.
> >> ...
> >
> > Could you checkout the latest Linus' git?
> >
> > $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git linux-2.6
> >
> > Or at least try with 2.6.30-rc2. It should apply cleanly also to this
> > version.
>
> Yes, I tried them all, but didn't work. :(
> Would you try "cgroup-io-throttle-v14.patch" too?
It works for me:
$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git linux-2.6
$ cd linux-2.6
$ wget http://download.systemimager.org/~arighi/linux/patches/io-throttle/cgroup-io-throttle-v14.patch
$ git apply cgroup-io-throttle-v14.patch
(no error reported)
The same for v13.
-Andrea
Andrea Righi wrote:
> On Wed, Apr 22, 2009 at 09:09:16AM +0800, Gui Jianfeng wrote:
>> Andrea Righi wrote:
>>> On Tue, Apr 21, 2009 at 09:16:22AM +0800, Gui Jianfeng wrote:
>>>> Andrea Righi wrote:
>>>>> On Mon, Apr 20, 2009 at 04:39:29PM +0800, Gui Jianfeng wrote:
>>>>>> Andrea Righi wrote:
>>>>>> ...
>>>>>>> Implementation
>>>>>>> ~~~~~~~~~~~~~~
>>>>>>> Patchset against latest Linus' git:
>>>>>>>
>>>>>>> [PATCH 0/7] cgroup: block device IO controller (v14)
>>>>>>> [PATCH 1/7] io-throttle documentation
>>>>>>> [PATCH 2/7] res_counter: introduce ratelimiting attributes
>>>>>>> [PATCH 3/7] page_cgroup: provide a generic page tracking infrastructure
>>>>>>> [PATCH 4/7] io-throttle controller infrastructure
>>>>>>> [PATCH 5/7] kiothrottled: throttle buffered (writeback) IO
>>>>>>> [PATCH 6/7] io-throttle instrumentation
>>>>>>> [PATCH 7/7] export per-task io-throttle statistics to userspace
>>>>>> Hi Andrea,
>>>>>>
>>>>>> I'd like to have a try this patchset, would you tell what's the kernel version
>>>>>> and bio-cgroup version based on?
>>>>>>
>>>>> Latest Linus' git is the kernel and bio-cgroup v7 (from
>>>>> http://people.valinux.co.jp/~ryov/bio-cgroup/).
>>>> Sorry, I still can't apply it...
>>>>
>>>> [root@localhost linux-2.6.30-rc1]# patch -p1 --dry-run < ../cgroup-io-throttle-v14.patch patching file Documentation/cgroups/io-throttle.txt
>>>> patching file block/Makefile
>>>> patching file block/blk-core.c
>>>> patching file block/blk-io-throttle.c
>>>> patching file block/kiothrottled.c
>>>> patching file fs/aio.c
>>>> patching file fs/buffer.c
>>>> Hunk #1 FAILED at 36.
>>>> Hunk #2 FAILED at 669.
>>>> 2 out of 2 hunks FAILED -- saving rejects to file fs/buffer.c.rej
>>>> patching file fs/proc/base.c
>>>> patching file include/linux/blk-io-throttle.h
>>>> patching file include/linux/cgroup_subsys.h
>>>> Hunk #1 succeeded at 49 (offset 6 lines).
>>>> patching file include/linux/memcontrol.h
>>>> Reversed (or previously applied) patch detected! Assume -R? [n] n
>>>> Apply anyway? [n] n
>>>> Skipping patch.
>>>> 2 out of 2 hunks ignored -- saving rejects to file include/linux/memcontrol.h.rej
>>>> patching file include/linux/mmzone.h
>>>> Hunk #1 FAILED at 607.
>>>> Hunk #2 FAILED at 958.
>>>> ...
>>> Could you checkout the latest Linus' git?
>>>
>>> $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git linux-2.6
>>>
>>> Or at least try with 2.6.30-rc2. It should apply cleanly also to this
>>> version.
>> Yes, I tried them all, but didn't work. :(
>> Would you try "cgroup-io-throttle-v14.patch" too?
>
> It works for me:
>
> $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git linux-2.6
> $ cd linux-2.6
> $ wget http://download.systemimager.org/~arighi/linux/patches/io-throttle/cgroup-io-throttle-v14.patch
> $ git apply cgroup-io-throttle-v14.patch
> (no error reported)
>
> The same for v13.
Ok, so we don't need to apply bio-cgroup.... I made a mistake, i applied bio-cgroup firstly :(
It's Ok now, thanks for the explanation.
I tried to build, but got the following error.
LD .tmp_vmlinux1
block/built-in.o: In function `get_iothrottle_from_page':
/kernel/linux-2.6.30-rc2/block/blk-io-throttle.c:670: undefined reference to `page_cgroup_get_owner'
block/built-in.o: In function `iothrottle_set_page_owner':
/kernel/linux-2.6.30-rc2/block/blk-io-throttle.c:704: undefined reference to `page_cgroup_set_owner'
block/built-in.o: In function `iothrottle_copy_page_owner':
/kernel/linux-2.6.30-rc2/block/blk-io-throttle.c:720: undefined reference to `page_cgroup_copy_owner'
make: *** [.tmp_vmlinux1] Error 1
Signed-off-by: Gui Jianfeng <[email protected]>
---
mm/page_cgroup.c | 65 ++++++++++++++++++++++++++---------------------------
1 files changed, 32 insertions(+), 33 deletions(-)
diff --git a/mm/page_cgroup.c b/mm/page_cgroup.c
index b3b394c..9c392f1 100644
--- a/mm/page_cgroup.c
+++ b/mm/page_cgroup.c
@@ -244,6 +244,38 @@ static int __meminit page_cgroup_callback(struct notifier_block *self,
#endif
+void __init page_cgroup_init(void)
+{
+ unsigned long pfn;
+ int fail = 0;
+
+ if (mem_cgroup_disabled() && iothrottle_disabled())
+ return;
+
+ for (pfn = 0; !fail && pfn < max_pfn; pfn += PAGES_PER_SECTION) {
+ if (!pfn_present(pfn))
+ continue;
+ fail = init_section_page_cgroup(pfn);
+ }
+ if (fail) {
+ printk(KERN_CRIT
+ "try cgroup_disable=memory,blockio boot option\n");
+ panic("Out of memory");
+ } else {
+ hotplug_memory_notifier(page_cgroup_callback, 0);
+ }
+ printk(KERN_INFO "allocated %ld bytes of page_cgroup\n", total_usage);
+ printk(KERN_INFO
+ "try cgroup_disable=memory,blockio option if you don't want\n");
+}
+
+void __meminit pgdat_page_cgroup_init(struct pglist_data *pgdat)
+{
+ return;
+}
+
+#endif
+
/**
* page_cgroup_get_owner() - get the owner ID of a page
* @page: the page we want to find the owner
@@ -317,39 +349,6 @@ int page_cgroup_copy_owner(struct page *npage, struct page *opage)
return 0;
}
-void __init page_cgroup_init(void)
-{
- unsigned long pfn;
- int fail = 0;
-
- if (mem_cgroup_disabled() && iothrottle_disabled())
- return;
-
- for (pfn = 0; !fail && pfn < max_pfn; pfn += PAGES_PER_SECTION) {
- if (!pfn_present(pfn))
- continue;
- fail = init_section_page_cgroup(pfn);
- }
- if (fail) {
- printk(KERN_CRIT
- "try cgroup_disable=memory,blockio boot option\n");
- panic("Out of memory");
- } else {
- hotplug_memory_notifier(page_cgroup_callback, 0);
- }
- printk(KERN_INFO "allocated %ld bytes of page_cgroup\n", total_usage);
- printk(KERN_INFO
- "try cgroup_disable=memory,blockio option if you don't want\n");
-}
-
-void __meminit pgdat_page_cgroup_init(struct pglist_data *pgdat)
-{
- return;
-}
-
-#endif
-
-
#ifdef CONFIG_CGROUP_MEM_RES_CTLR_SWAP
static DEFINE_MUTEX(swap_cgroup_mutex);
--
1.5.4.rc3
On Thu, Apr 23, 2009 at 10:58:16AM +0800, Gui Jianfeng wrote:
> Andrea Righi wrote:
> > On Wed, Apr 22, 2009 at 09:09:16AM +0800, Gui Jianfeng wrote:
> >> Andrea Righi wrote:
> >>> On Tue, Apr 21, 2009 at 09:16:22AM +0800, Gui Jianfeng wrote:
> >>>> Andrea Righi wrote:
> >>>>> On Mon, Apr 20, 2009 at 04:39:29PM +0800, Gui Jianfeng wrote:
> >>>>>> Andrea Righi wrote:
> >>>>>> ...
> >>>>>>> Implementation
> >>>>>>> ~~~~~~~~~~~~~~
> >>>>>>> Patchset against latest Linus' git:
> >>>>>>>
> >>>>>>> [PATCH 0/7] cgroup: block device IO controller (v14)
> >>>>>>> [PATCH 1/7] io-throttle documentation
> >>>>>>> [PATCH 2/7] res_counter: introduce ratelimiting attributes
> >>>>>>> [PATCH 3/7] page_cgroup: provide a generic page tracking infrastructure
> >>>>>>> [PATCH 4/7] io-throttle controller infrastructure
> >>>>>>> [PATCH 5/7] kiothrottled: throttle buffered (writeback) IO
> >>>>>>> [PATCH 6/7] io-throttle instrumentation
> >>>>>>> [PATCH 7/7] export per-task io-throttle statistics to userspace
> >>>>>> Hi Andrea,
> >>>>>>
> >>>>>> I'd like to have a try this patchset, would you tell what's the kernel version
> >>>>>> and bio-cgroup version based on?
> >>>>>>
> >>>>> Latest Linus' git is the kernel and bio-cgroup v7 (from
> >>>>> http://people.valinux.co.jp/~ryov/bio-cgroup/).
> >>>> Sorry, I still can't apply it...
> >>>>
> >>>> [root@localhost linux-2.6.30-rc1]# patch -p1 --dry-run < ../cgroup-io-throttle-v14.patch patching file Documentation/cgroups/io-throttle.txt
> >>>> patching file block/Makefile
> >>>> patching file block/blk-core.c
> >>>> patching file block/blk-io-throttle.c
> >>>> patching file block/kiothrottled.c
> >>>> patching file fs/aio.c
> >>>> patching file fs/buffer.c
> >>>> Hunk #1 FAILED at 36.
> >>>> Hunk #2 FAILED at 669.
> >>>> 2 out of 2 hunks FAILED -- saving rejects to file fs/buffer.c.rej
> >>>> patching file fs/proc/base.c
> >>>> patching file include/linux/blk-io-throttle.h
> >>>> patching file include/linux/cgroup_subsys.h
> >>>> Hunk #1 succeeded at 49 (offset 6 lines).
> >>>> patching file include/linux/memcontrol.h
> >>>> Reversed (or previously applied) patch detected! Assume -R? [n] n
> >>>> Apply anyway? [n] n
> >>>> Skipping patch.
> >>>> 2 out of 2 hunks ignored -- saving rejects to file include/linux/memcontrol.h.rej
> >>>> patching file include/linux/mmzone.h
> >>>> Hunk #1 FAILED at 607.
> >>>> Hunk #2 FAILED at 958.
> >>>> ...
> >>> Could you checkout the latest Linus' git?
> >>>
> >>> $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git linux-2.6
> >>>
> >>> Or at least try with 2.6.30-rc2. It should apply cleanly also to this
> >>> version.
> >> Yes, I tried them all, but didn't work. :(
> >> Would you try "cgroup-io-throttle-v14.patch" too?
> >
> > It works for me:
> >
> > $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git linux-2.6
> > $ cd linux-2.6
> > $ wget http://download.systemimager.org/~arighi/linux/patches/io-throttle/cgroup-io-throttle-v14.patch
> > $ git apply cgroup-io-throttle-v14.patch
> > (no error reported)
> >
> > The same for v13.
>
> Ok, so we don't need to apply bio-cgroup.... I made a mistake, i applied bio-cgroup firstly :(
> It's Ok now, thanks for the explanation.
>
> I tried to build, but got the following error.
>
> LD .tmp_vmlinux1
> block/built-in.o: In function `get_iothrottle_from_page':
> /kernel/linux-2.6.30-rc2/block/blk-io-throttle.c:670: undefined reference to `page_cgroup_get_owner'
> block/built-in.o: In function `iothrottle_set_page_owner':
> /kernel/linux-2.6.30-rc2/block/blk-io-throttle.c:704: undefined reference to `page_cgroup_set_owner'
> block/built-in.o: In function `iothrottle_copy_page_owner':
> /kernel/linux-2.6.30-rc2/block/blk-io-throttle.c:720: undefined reference to `page_cgroup_copy_owner'
> make: *** [.tmp_vmlinux1] Error 1
page_cgroup_*_onwer are defined in mm/page_cgroup.c, that is built if
CONFIG_PAGE_TRACKING=y. But from init/Kconfig:
config CGROUP_IO_THROTTLE
bool "Enable cgroup I/O throttling"
depends on CGROUPS && RESOURCE_COUNTERS && EXPERIMENTAL
select MM_OWNER
select PAGE_TRACKING
^^^^^^^^^^^^^
mmmh??? it should be correct.
And I don't understand how the following patch can fix this problem...
Could you post your .config?
Thanks,
-Andrea
>
> Signed-off-by: Gui Jianfeng <[email protected]>
> ---
> mm/page_cgroup.c | 65 ++++++++++++++++++++++++++---------------------------
> 1 files changed, 32 insertions(+), 33 deletions(-)
>
> diff --git a/mm/page_cgroup.c b/mm/page_cgroup.c
> index b3b394c..9c392f1 100644
> --- a/mm/page_cgroup.c
> +++ b/mm/page_cgroup.c
> @@ -244,6 +244,38 @@ static int __meminit page_cgroup_callback(struct notifier_block *self,
>
> #endif
>
> +void __init page_cgroup_init(void)
> +{
> + unsigned long pfn;
> + int fail = 0;
> +
> + if (mem_cgroup_disabled() && iothrottle_disabled())
> + return;
> +
> + for (pfn = 0; !fail && pfn < max_pfn; pfn += PAGES_PER_SECTION) {
> + if (!pfn_present(pfn))
> + continue;
> + fail = init_section_page_cgroup(pfn);
> + }
> + if (fail) {
> + printk(KERN_CRIT
> + "try cgroup_disable=memory,blockio boot option\n");
> + panic("Out of memory");
> + } else {
> + hotplug_memory_notifier(page_cgroup_callback, 0);
> + }
> + printk(KERN_INFO "allocated %ld bytes of page_cgroup\n", total_usage);
> + printk(KERN_INFO
> + "try cgroup_disable=memory,blockio option if you don't want\n");
> +}
> +
> +void __meminit pgdat_page_cgroup_init(struct pglist_data *pgdat)
> +{
> + return;
> +}
> +
> +#endif
> +
> /**
> * page_cgroup_get_owner() - get the owner ID of a page
> * @page: the page we want to find the owner
> @@ -317,39 +349,6 @@ int page_cgroup_copy_owner(struct page *npage, struct page *opage)
> return 0;
> }
>
> -void __init page_cgroup_init(void)
> -{
> - unsigned long pfn;
> - int fail = 0;
> -
> - if (mem_cgroup_disabled() && iothrottle_disabled())
> - return;
> -
> - for (pfn = 0; !fail && pfn < max_pfn; pfn += PAGES_PER_SECTION) {
> - if (!pfn_present(pfn))
> - continue;
> - fail = init_section_page_cgroup(pfn);
> - }
> - if (fail) {
> - printk(KERN_CRIT
> - "try cgroup_disable=memory,blockio boot option\n");
> - panic("Out of memory");
> - } else {
> - hotplug_memory_notifier(page_cgroup_callback, 0);
> - }
> - printk(KERN_INFO "allocated %ld bytes of page_cgroup\n", total_usage);
> - printk(KERN_INFO
> - "try cgroup_disable=memory,blockio option if you don't want\n");
> -}
> -
> -void __meminit pgdat_page_cgroup_init(struct pglist_data *pgdat)
> -{
> - return;
> -}
> -
> -#endif
> -
> -
> #ifdef CONFIG_CGROUP_MEM_RES_CTLR_SWAP
>
> static DEFINE_MUTEX(swap_cgroup_mutex);
> --
> 1.5.4.rc3
Andrea Righi wrote:
> page_cgroup_*_onwer are defined in mm/page_cgroup.c, that is built if
> CONFIG_PAGE_TRACKING=y. But from init/Kconfig:
page_cgroup_*_onwer also depend on CONFIG_SPARSEMEM, if !defined(CONFIG_SPARSEMEM),
they won't be built. So just move them out of CONFIG_SPARSEMEM block.
>
> config CGROUP_IO_THROTTLE
> bool "Enable cgroup I/O throttling"
> depends on CGROUPS && RESOURCE_COUNTERS && EXPERIMENTAL
> select MM_OWNER
> select PAGE_TRACKING
> ^^^^^^^^^^^^^
> mmmh??? it should be correct.
>
> And I don't understand how the following patch can fix this problem...
>
> Could you post your .config?
>
> Thanks,
> -Andrea
>
>> Signed-off-by: Gui Jianfeng <[email protected]>
>> ---
>> mm/page_cgroup.c | 65 ++++++++++++++++++++++++++---------------------------
>> 1 files changed, 32 insertions(+), 33 deletions(-)
>>
>> diff --git a/mm/page_cgroup.c b/mm/page_cgroup.c
>> index b3b394c..9c392f1 100644
>> --- a/mm/page_cgroup.c
>> +++ b/mm/page_cgroup.c
>> @@ -244,6 +244,38 @@ static int __meminit page_cgroup_callback(struct notifier_block *self,
>>
>> #endif
>>
>> +void __init page_cgroup_init(void)
>> +{
>> + unsigned long pfn;
>> + int fail = 0;
>> +
>> + if (mem_cgroup_disabled() && iothrottle_disabled())
>> + return;
>> +
>> + for (pfn = 0; !fail && pfn < max_pfn; pfn += PAGES_PER_SECTION) {
>> + if (!pfn_present(pfn))
>> + continue;
>> + fail = init_section_page_cgroup(pfn);
>> + }
>> + if (fail) {
>> + printk(KERN_CRIT
>> + "try cgroup_disable=memory,blockio boot option\n");
>> + panic("Out of memory");
>> + } else {
>> + hotplug_memory_notifier(page_cgroup_callback, 0);
>> + }
>> + printk(KERN_INFO "allocated %ld bytes of page_cgroup\n", total_usage);
>> + printk(KERN_INFO
>> + "try cgroup_disable=memory,blockio option if you don't want\n");
>> +}
>> +
>> +void __meminit pgdat_page_cgroup_init(struct pglist_data *pgdat)
>> +{
>> + return;
>> +}
>> +
>> +#endif
>> +
>> /**
>> * page_cgroup_get_owner() - get the owner ID of a page
>> * @page: the page we want to find the owner
>> @@ -317,39 +349,6 @@ int page_cgroup_copy_owner(struct page *npage, struct page *opage)
>> return 0;
>> }
>>
>> -void __init page_cgroup_init(void)
>> -{
>> - unsigned long pfn;
>> - int fail = 0;
>> -
>> - if (mem_cgroup_disabled() && iothrottle_disabled())
>> - return;
>> -
>> - for (pfn = 0; !fail && pfn < max_pfn; pfn += PAGES_PER_SECTION) {
>> - if (!pfn_present(pfn))
>> - continue;
>> - fail = init_section_page_cgroup(pfn);
>> - }
>> - if (fail) {
>> - printk(KERN_CRIT
>> - "try cgroup_disable=memory,blockio boot option\n");
>> - panic("Out of memory");
>> - } else {
>> - hotplug_memory_notifier(page_cgroup_callback, 0);
>> - }
>> - printk(KERN_INFO "allocated %ld bytes of page_cgroup\n", total_usage);
>> - printk(KERN_INFO
>> - "try cgroup_disable=memory,blockio option if you don't want\n");
>> -}
>> -
>> -void __meminit pgdat_page_cgroup_init(struct pglist_data *pgdat)
>> -{
>> - return;
>> -}
>> -
>> -#endif
>> -
>> -
>> #ifdef CONFIG_CGROUP_MEM_RES_CTLR_SWAP
>>
>> static DEFINE_MUTEX(swap_cgroup_mutex);
>> --
>> 1.5.4.rc3
>
>
>
--
Regards
Gui Jianfeng
On Fri, Apr 24, 2009 at 09:10:15AM +0800, Gui Jianfeng wrote:
> Andrea Righi wrote:
> > page_cgroup_*_onwer are defined in mm/page_cgroup.c, that is built if
> > CONFIG_PAGE_TRACKING=y. But from init/Kconfig:
>
> page_cgroup_*_onwer also depend on CONFIG_SPARSEMEM, if !defined(CONFIG_SPARSEMEM),
> they won't be built. So just move them out of CONFIG_SPARSEMEM block.
You're right! I moved all these functions out of the CONFIG_SPARSEMEM
block. I'll include this fix in the next io-throttle version.
Thanks!!
-Andrea