Hey Guys ,
I am new to reading and writing kernel code.I got interested in
writing code for btrfs as it seems to
need more work then other file systems and this seems other then
drivers, a good use of time on my part.
I interested in helping improving the compression of btrfs by using a
set of threads using work queues like XFS
or reads and keeping the page cache after reading compressed blocks as
these seem to be a great way to improve
on compression performance mostly with large partitions of compressed
data. I am not asking you to write the code
for me but as I am new a little guidance and help would be greatly
appreciated as this seems like too much work for just a newbie.
Thanks A lot,
Nick
On Wed, Jul 30, 2014 at 12:37 AM, Gareth Pye <[email protected]> wrote:
> You've been replied to politely, now listen and do or shut up.
>
>
> On Wed, Jul 30, 2014 at 1:54 PM, Nick Krause <[email protected]> wrote:
>>
>> Hey Guys ,
>> I am new to reading and writing kernel code.I got interested in
>> writing code for btrfs as it seems to
>> need more work then other file systems and this seems other then
>> drivers, a good use of time on my part.
>> I interested in helping improving the compression of btrfs by using a
>> set of threads using work queues like XFS
>> or reads and keeping the page cache after reading compressed blocks as
>> these seem to be a great way to improve
>> on compression performance mostly with large partitions of compressed
>> data. I am not asking you to write the code
>> for me but as I am new a little guidance and help would be greatly
>> appreciated as this seems like too much work for just a newbie.
>> Thanks A lot,
>> Nick
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
>
>
> --
> Gareth Pye
> Level 2 Judge, Melbourne, Australia
> Australian MTG Forum: mtgau.com
> [email protected] - http://www.rockpaperdynamite.wordpress.com
> "Dear God, I would like to file a bug report"
Gareth
I am asking for advice on not writing the code but can someone please
test this for me as I don't have many hard drives lying around. In addition
I am new to btrfs , so I would like to known if this is a good idea or should
I just drop it?
Nick
On Tue, Jul 29, 2014 at 11:54:20PM -0400, Nick Krause wrote:
> Hey Guys ,
> I am new to reading and writing kernel code.I got interested in
> writing code for btrfs as it seems to
> need more work then other file systems and this seems other then
> drivers, a good use of time on my part.
> I interested in helping improving the compression of btrfs by using a
> set of threads using work queues like XFS
> or reads and keeping the page cache after reading compressed blocks as
> these seem to be a great way to improve
> on compression performance mostly with large partitions of compressed
> data. I am not asking you to write the code
> for me but as I am new a little guidance and help would be greatly
> appreciated as this seems like too much work for just a newbie.
* Documentation/workqueue.txt (in general, grep in Documentation
usually throws up something useful)
* grep -r alloc_workqueue fs/ shows a lot of uses (including in
btrfs), so it should be fairly easy to see how to create and manage
a workqueue.
I suspect that this may be a medium-sized project, rather than a
small one. My gut feeling (based on limited experience) is that the
fallocate extensions project would be considerably simpler.
I also noticed from the public reply to the private mail (don't do
this without getting permission from the other person) that you posted
to LKML in this thread (don't switch mailing lists mid-thread) that
you anticipated having problems testing with limited disks -- what you
will find is that testing new kernel code is something that you don't
do on your main development OS installation. Instead, you will need
either a scratch machine that you can easily update, or one or more
virtual machines. qemu/kvm is good for this, because it has a mode
that bypasses the BIOS and bootloader emulation, and just directly
runs a kernel from a file on the host machine. This is fast. You can
pass large sparse files to the VM to act as scratch disks, plus keep
another smaller file for the guest OS (and a copy of it so that you
can throw one away and make another one quickly and easily).
Hugo.
--
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
--- You've read the project plan. Forget that. We're going to Do ---
Stuff and Have Fun doing it.
On Wed, Jul 30, 2014 at 10:38:21AM +0100, Hugo Mills wrote:
> qemu/kvm is good for this, because it has a mode
> that bypasses the BIOS and bootloader emulation, and just directly
> runs a kernel from a file on the host machine. This is fast. You can
> pass large sparse files to the VM to act as scratch disks, plus keep
> another smaller file for the guest OS (and a copy of it so that you
> can throw one away and make another one quickly and easily).
Nick,
The xfstests-bld/kvm-xfstests git tree I pointed out to you has an
example of a test infrastructure which does this for ext4. It's been
on my todo list to support other file systems, and I have some plans
for how to do thi, but it's been low on my priority list. If someone
in the btrfs development community is interested in working with me on
this, they should contact me.
Or the btrfs developers can create their own automated systems which
then gets promulgated, but either way, it's something that I strongly
recommend. The set of test configs that ext4 developers would run
significantly increased after I cleaned up my test scripts and made it
something that other people could use, and it means that I can ask
people who are implementing new features that they run a complete
regression test run, with many different file systems features enabled
and disabled, so they find the problems before they propose a patch
for merging.
Cheers,
- Ted
On 07/30/2014 10:13 AM, Theodore Ts'o wrote:
> On Wed, Jul 30, 2014 at 10:38:21AM +0100, Hugo Mills wrote:
>> qemu/kvm is good for this, because it has a mode
>> that bypasses the BIOS and bootloader emulation, and just directly
>> runs a kernel from a file on the host machine. This is fast. You can
>> pass large sparse files to the VM to act as scratch disks, plus keep
>> another smaller file for the guest OS (and a copy of it so that you
>> can throw one away and make another one quickly and easily).
>
> Nick,
>
> The xfstests-bld/kvm-xfstests git tree I pointed out to you has an
> example of a test infrastructure which does this for ext4. It's been
> on my todo list to support other file systems, and I have some plans
> for how to do thi, but it's been low on my priority list. If someone
> in the btrfs development community is interested in working with me on
> this, they should contact me.
Ted,
Where is that git tree? I've been planning to set up a unit test and
regression suite for tty/serial, and wouldn't mind cribbing the
infrastructure from someone's existing work.
Regards,
Peter Hurley
On 07/30/2014 12:54 AM, Nick Krause wrote:
> On Wed, Jul 30, 2014 at 12:37 AM, Gareth Pye <[email protected]> wrote:
>> You've been replied to politely, now listen and do or shut up.
>>
>>
>> On Wed, Jul 30, 2014 at 1:54 PM, Nick Krause <[email protected]> wrote:
>>>
>>> Hey Guys ,
>>> I am new to reading and writing kernel code.I got interested in
>>> writing code for btrfs as it seems to
>>> need more work then other file systems and this seems other then
>>> drivers, a good use of time on my part.
>>> I interested in helping improving the compression of btrfs by using a
>>> set of threads using work queues like XFS
>>> or reads and keeping the page cache after reading compressed blocks as
>>> these seem to be a great way to improve
>>> on compression performance mostly with large partitions of compressed
>>> data. I am not asking you to write the code
>>> for me but as I am new a little guidance and help would be greatly
>>> appreciated as this seems like too much work for just a newbie.
>>> Thanks A lot,
>>> Nick
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>> the body of a message to [email protected]
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>>
>>
>>
>> --
>> Gareth Pye
>> Level 2 Judge, Melbourne, Australia
>> Australian MTG Forum: mtgau.com
>> [email protected] - http://www.rockpaperdynamite.wordpress.com
>> "Dear God, I would like to file a bug report"
>
> Gareth
> I am asking for advice on not writing the code but can someone please
> test this for me as I don't have many hard drives lying around. In addition
> I am new to btrfs , so I would like to known if this is a good idea or should
> I just drop it?
> Nick
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
I would look into using usermode linux (linux built with ARCH=um) for at
least the initial debugging, it is marginally more efficient than
virtualization, doesn't need root privileges, and allows you to run the
kernel under GDB without needing separate hardware with a serial console
or PS/2 keyboard.
On Wed, Jul 30, 2014 at 11:14 PM, Austin S Hemmelgarn
<[email protected]> wrote:
> On 07/30/2014 12:54 AM, Nick Krause wrote:
>> On Wed, Jul 30, 2014 at 12:37 AM, Gareth Pye <[email protected]> wrote:
>>> You've been replied to politely, now listen and do or shut up.
>>>
>>>
>>> On Wed, Jul 30, 2014 at 1:54 PM, Nick Krause <[email protected]> wrote:
>>>>
>>>> Hey Guys ,
>>>> I am new to reading and writing kernel code.I got interested in
>>>> writing code for btrfs as it seems to
>>>> need more work then other file systems and this seems other then
>>>> drivers, a good use of time on my part.
>>>> I interested in helping improving the compression of btrfs by using a
>>>> set of threads using work queues like XFS
>>>> or reads and keeping the page cache after reading compressed blocks as
>>>> these seem to be a great way to improve
>>>> on compression performance mostly with large partitions of compressed
>>>> data. I am not asking you to write the code
>>>> for me but as I am new a little guidance and help would be greatly
>>>> appreciated as this seems like too much work for just a newbie.
>>>> Thanks A lot,
>>>> Nick
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>>> the body of a message to [email protected]
>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>
>>>
>>>
>>>
>>> --
>>> Gareth Pye
>>> Level 2 Judge, Melbourne, Australia
>>> Australian MTG Forum: mtgau.com
>>> [email protected] - http://www.rockpaperdynamite.wordpress.com
>>> "Dear God, I would like to file a bug report"
>>
>> Gareth
>> I am asking for advice on not writing the code but can someone please
>> test this for me as I don't have many hard drives lying around. In addition
>> I am new to btrfs , so I would like to known if this is a good idea or should
>> I just drop it?
>> Nick
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at http://www.tux.org/lkml/
>>
> I would look into using usermode linux (linux built with ARCH=um) for at
> least the initial debugging, it is marginally more efficient than
> virtualization, doesn't need root privileges, and allows you to run the
> kernel under GDB without needing separate hardware with a serial console
> or PS/2 keyboard.
>
That's what I am thinking the only issue is I am going to start my
reading on work queues first.
Regards Nick
On Wed, Jul 30, 2014 at 10:36:57AM -0400, Peter Hurley wrote:
>
> Where is that git tree? I've been planning to set up a unit test and
> regression suite for tty/serial, and wouldn't mind cribbing the
> infrastructure from someone's existing work.
https://git.kernel.org/cgit/fs/ext2/xfstests-bld.git/