Hi Konrad,
On Tue, Jul 31, 2012 at 01:51:42PM -0400, Konrad Rzeszutek Wilk wrote:
> On Tue, Jul 31, 2012 at 09:19:16AM -0700, Greg Kroah-Hartman wrote:
> > On Tue, Jul 31, 2012 at 11:58:43AM -0400, Konrad Rzeszutek Wilk wrote:
> > > So in my head I feel that it is Ok to:
> > > 1) address the concerns that zcache has before it is unstaged
> > > 2) rip out the two-engine system with a one-engine system
> > > (and see how well it behaves)
> > > 3) sysfs->debugfs as needed
> > > 4) other things as needed
> > >
> > > I think we are getting hung-up what Greg said about adding features
> > > and the two-engine->one engine could be understood as that.
> > > While I think that is part of a staging effort to clean up the
> > > existing issues. Lets see what Greg thinks.
> >
> > Greg has no idea, except I want to see the needed fixups happen before
> > new features get added. Add the new features _after_ it is out of
> > staging.
>
> I think we (that is me, Seth, Minchan, Dan) need to talk to have a good
> understanding of what each of us thinks are fixups.
>
> Would Monday Aug 6th at 1pm EST on irc.freenode.net channel #zcache work
> for people?
1pm EST is 2am KST(Korea Standard Time) so it's not good for me. :)
I know it's hard to adjust my time for yours so let you talk without
me. Instead, I will write it down my requirement. It's very simple and
trivial.
1) Please don't add any new feature like replace zsmalloc with zbud.
It's totally untested so it needs more time for stable POV bug,
or performance/fragementation.
2) Factor out common code between zcache and ramster. It should be just
clean up code and should not change current behavior.
3) Add lots of comment to public functions
4) make function/varabiel names more clearly.
They are necessary for promotion and after promotion,
let's talk about new great features.
>
> >
> > greg k-h
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to [email protected]. For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
--
Kind regards,
Minchan Kim
> > I think we (that is me, Seth, Minchan, Dan) need to talk to have a good
> > understanding of what each of us thinks are fixups.
> >
> > Would Monday Aug 6th at 1pm EST on irc.freenode.net channel #zcache work
> > for people?
>
> 1pm EST is 2am KST(Korea Standard Time) so it's not good for me. :)
> I know it's hard to adjust my time for yours so let you talk without
> me. Instead, I will write it down my requirement. It's very simple and
> trivial.
>
> 1) Please don't add any new feature like replace zsmalloc with zbud.
> It's totally untested so it needs more time for stable POV bug,
> or performance/fragementation.
>
> 2) Factor out common code between zcache and ramster. It should be just
> clean up code and should not change current behavior.
>
> 3) Add lots of comment to public functions
>
> 4) make function/varabiel names more clearly.
>
> They are necessary for promotion and after promotion,
> let's talk about new great features.
Hi Minchan --
I hope you had a great vacation!
Since we won't be able to discuss this by phone/irc, I guess I
need to reply here.
Let me first restate my opinion as author of zcache.
The zcache in staging is really a "demo" version. It was written 21
months ago (and went into staging 16 months ago) primarily to show,
at Andrew Morton's suggestion, that frontswap and cleancache had value
in a normal standalone kernel (i.e. virtualization not required). When
posted in early 2011 zcache was known to have some fundamental flaws in the design...
that's why it went into "staging". The "demo" version in staging still has
those flaws and the change from xvmalloc to zsmalloc makes one of those flaws
worse. These design flaws are now fixed in the new code base I posted last
week AND the new code base has improved factoring, comments and the code is
properly re-merged with the zcache "fork" in ramster.
We are not talking about new "features"... I have tried to back out the
new features from the new code base already posted and will post them separately.
So I think we have four choices:
A) Try to promote zcache as is. (Seth's proposal)
B) Clean up zcache with no new functionality. (Minchan's proposal above)
C) New code base (in mm/tmem/) after review. (Dan's proposal)
D) New code base but retrofit as a series of patches (Konrad's suggestion)
Minchan, if we go with your proposal (B) are you volunteering
to do the work? And if you do, doesn't it have the same issue
that it is also totally untested? And, since (B) doesn't solve the
fundamental design issues, are you volunteering to fix those next?
And, in the meantime, doesn't this mean we have THREE versions
of zcache?
IMHO, the fastest way to get the best zcache into the kernel and
to distros and users is to throw away the "demo" version, move forward
to a new solid well-designed zcache code base, and work together to
build on it. There's still a lot to do so I hope we can work together.
Thanks,
Dan
On Mon, Aug 6, 2012 at 6:24 PM, Dan Magenheimer
<[email protected]> wrote:
> IMHO, the fastest way to get the best zcache into the kernel and
> to distros and users is to throw away the "demo" version, move forward
> to a new solid well-designed zcache code base, and work together to
> build on it. There's still a lot to do so I hope we can work together.
I'm not convinced it's the _fastest way_. You're effectively
invalidating all the work done under drivers/staging so you might end up
in review limbo with your shiny new code...
AFAICT, your best bet is to first clean up zcache under driver/staging
and get that promoted under mm/zcache.c. You can then move on to the
more controversial ramster and figure out where to put the clustering
code, etc.
Pekka
> From: Pekka Enberg [mailto:[email protected]]
> Subject: Re: [PATCH 0/4] promote zcache from staging
>
> On Mon, Aug 6, 2012 at 6:24 PM, Dan Magenheimer
> <[email protected]> wrote:
> > IMHO, the fastest way to get the best zcache into the kernel and
> > to distros and users is to throw away the "demo" version, move forward
> > to a new solid well-designed zcache code base, and work together to
> > build on it. There's still a lot to do so I hope we can work together.
>
> I'm not convinced it's the _fastest way_.
<grin> I guess I meant "optimal", combining "fast" and "best".
> You're effectively
> invalidating all the work done under drivers/staging so you might end up
> in review limbo with your shiny new code...
Fixing the fundamental design flaws will sooner or later invalidate
most (or all) of the previous testing/work anyway, won't it? Since
any kernel built with staging is "tainted" already, I feel like now
is a better time to make a major design transition.
I suppose:
(E) replace "demo" zcache with new code base and keep it
in staging for another cycle
is another alternative, but I think gregkh has said no to that.
Dan
On Mon, Aug 06, 2012 at 09:21:22AM -0700, Dan Magenheimer wrote:
> I suppose:
>
> (E) replace "demo" zcache with new code base and keep it
> in staging for another cycle
>
> is another alternative, but I think gregkh has said no to that.
No I have not. If you all feel that the existing code needs to be
dropped and replaced with a totally new version, that's fine with me.
It's forward progress, which is all that I ask for.
Hope this helps,
greg k-h
> From: Greg Kroah-Hartman [mailto:[email protected]]
> Subject: Re: [PATCH 0/4] promote zcache from staging
>
> On Mon, Aug 06, 2012 at 09:21:22AM -0700, Dan Magenheimer wrote:
> > I suppose:
> >
> > (E) replace "demo" zcache with new code base and keep it
> > in staging for another cycle
> >
> > is another alternative, but I think gregkh has said no to that.
>
> No I have not. If you all feel that the existing code needs to be
> dropped and replaced with a totally new version, that's fine with me.
> It's forward progress, which is all that I ask for.
>
> Hope this helps,
> greg k-h
Hi Greg --
Cool! I guess I mistakenly assumed that your "no new features"
requirement also implied "no fixes of fundamental design flaws". :-)
Having option (E) should make it easier to decide the best
technical solution, separate from the promotion timing and "where
does it land" question.
We'll get back to you soon...
Thanks!
Dan
Hi Dan,
On Mon, Aug 06, 2012 at 09:21:22AM -0700, Dan Magenheimer wrote:
> > From: Pekka Enberg [mailto:[email protected]]
> > Subject: Re: [PATCH 0/4] promote zcache from staging
> >
> > On Mon, Aug 6, 2012 at 6:24 PM, Dan Magenheimer
> > <[email protected]> wrote:
> > > IMHO, the fastest way to get the best zcache into the kernel and
> > > to distros and users is to throw away the "demo" version, move forward
> > > to a new solid well-designed zcache code base, and work together to
> > > build on it. There's still a lot to do so I hope we can work together.
> >
> > I'm not convinced it's the _fastest way_.
>
> <grin> I guess I meant "optimal", combining "fast" and "best".
>
> > You're effectively
> > invalidating all the work done under drivers/staging so you might end up
> > in review limbo with your shiny new code...
>
> Fixing the fundamental design flaws will sooner or later invalidate
> most (or all) of the previous testing/work anyway, won't it? Since
> any kernel built with staging is "tainted" already, I feel like now
> is a better time to make a major design transition.
>
> I suppose:
>
> (E) replace "demo" zcache with new code base and keep it
> in staging for another cycle
I go for (E). Please send your refactoring code as formal patch.
Thanks.
--
Kind regards,
Minchan Kim
On Mon, Aug 06, 2012 at 09:38:16AM +0900, Minchan Kim wrote:
> Hi Konrad,
>
> On Tue, Jul 31, 2012 at 01:51:42PM -0400, Konrad Rzeszutek Wilk wrote:
> > On Tue, Jul 31, 2012 at 09:19:16AM -0700, Greg Kroah-Hartman wrote:
> > > On Tue, Jul 31, 2012 at 11:58:43AM -0400, Konrad Rzeszutek Wilk wrote:
> > > > So in my head I feel that it is Ok to:
> > > > 1) address the concerns that zcache has before it is unstaged
> > > > 2) rip out the two-engine system with a one-engine system
> > > > (and see how well it behaves)
> > > > 3) sysfs->debugfs as needed
> > > > 4) other things as needed
> > > >
> > > > I think we are getting hung-up what Greg said about adding features
> > > > and the two-engine->one engine could be understood as that.
> > > > While I think that is part of a staging effort to clean up the
> > > > existing issues. Lets see what Greg thinks.
> > >
> > > Greg has no idea, except I want to see the needed fixups happen before
> > > new features get added. Add the new features _after_ it is out of
> > > staging.
> >
> > I think we (that is me, Seth, Minchan, Dan) need to talk to have a good
> > understanding of what each of us thinks are fixups.
> >
> > Would Monday Aug 6th at 1pm EST on irc.freenode.net channel #zcache work
> > for people?
>
> 1pm EST is 2am KST(Korea Standard Time) so it's not good for me. :)
> I know it's hard to adjust my time for yours so let you talk without
> me. Instead, I will write it down my requirement. It's very simple and
> trivial.
OK, Thank you.
We had a lengthy chat (full chat log attached). The summary was that
we all want to promote zcache (for various reasons), but we are hang up
whether we are OK unstaging it wherein it lowers the I/Os but potentially
not giving large performance increase (when doing 'make -jN') or that we
want both of those characteristics in. Little history: v3.3 had
swap readahead patches that made the amount of pages going in swap dramatically
decrease - as such the performance of zcache is not anymore amazing, but ok.
Seth and Robert (and I surmise Minchan too) are very interested in zcache
as its lowers the amount of I/Os but performance is secondary. Dan is interested
in having less I/Os and providing a performance boost with the such workloads as
'make -jN' - in short less I/Os and better performance. Dan would like both
before unstaging.
The action items that came out are:
- Seth posted some benchmarks - he is going to rerun them with v3.5
to see how it behaves in terms of performance (make -jN benchmark).
- Robert is going to take a swing at Minchan refactor and adding comments, etc
(But after we get over the hump of agreeing on the next step).
- Konrad to rummage in his mbox to find any other technical objections
that were raised on zcache earlier to make sure to address them.
- Once Seth is finished Konrad is going to take another swing
at driving this discussion - either via email, IRC or conference call.