Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Wed, 23 Oct 2019 01:03:47 +0800
From:   Gao Xiang <hsiangkao@aol.com>
To:     Pratik Shinde <pratikshinde320@gmail.com>
Cc:     linux-erofs@lists.ozlabs.org, Gao Xiang <gaoxiang25@huawei.com>,
        yuchao0@huawei.com, linux-kernel@vger.kernel.org
Subject: Re: [PATCH-v2] erofs: code for verifying superblock checksum of an
 erofs image.
Message-ID: <20191022170340.GA4641@hsiangkao-HP-ZHAN-66-Pro-G1>
References: <20191022153956.16867-1-pratikshinde320@gmail.com>
 <20191022163403.GA3201@hsiangkao-HP-ZHAN-66-Pro-G1>
 <CAGu0czRBfVa+1WPCBYP2JAYRhRBXkn20-YmVw9J3XUbCxw0cNg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAGu0czRBfVa+1WPCBYP2JAYRhRBXkn20-YmVw9J3XUbCxw0cNg@mail.gmail.com>
User-Agent: Mutt/1.9.4 (2018-02-28)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Tue, Oct 22, 2019 at 10:18:23PM +0530, Pratik Shinde wrote:
> Hi Gao,
> 
> Understood your concern.
> 
> Can we do something like :
> 
> 1) Allocate one buf of size EROFS_BLKSIZ
> 2) Read one page at a time into buf(memcpy) .call crc32c for it.
> 
> In this way we won't be writing directly into page data and will not do
> large allocation.
> What do you think?

Yes, that is the right way, but the buf is only be used to calc
the 0th block (super_block), for the rest blocks, no need to do this.

buf = kmalloc(EROFS_BLKSIZ);
memcpy(buf, data);
dsb->checksum = 0;
crc = crc32c(0, buf);
kfree(buf);

for (i = 1; i < nblocks; ++i) {
   page = get_meta_page(sb, i);

   data = kmap_atomic(page);
   crc = crc32c(crc, data);
   kunmap_atomic(data);
   unlock_page(page);
   put_page(page);
}
...

Or some better approach, but it's not ok to modify page cache directly.

Thanks,
Gao Xiang

> 
> --Pratik
> 
> On Tue, 22 Oct, 2019, 10:04 PM Gao Xiang, <hsiangkao@aol.com> wrote:
> 
> > Hi Pratik,
> >
> > Some comments as below...
> >
> > On Tue, Oct 22, 2019 at 09:09:56PM +0530, Pratik Shinde wrote:
> > > Patch for kernel side changes of checksum feature.Used kernel's
> > > crc32c library for calculating the checksum.
> > >
> > > Signed-off-by: Pratik Shinde <pratikshinde320@gmail.com>
> > > ---
> > >  fs/erofs/erofs_fs.h |  5 +++--
> > >  fs/erofs/internal.h |  3 ++-
> > >  fs/erofs/super.c    | 33 +++++++++++++++++++++++++++++++++
> > >  3 files changed, 38 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/fs/erofs/erofs_fs.h b/fs/erofs/erofs_fs.h
> > > index b1ee565..4d8097a 100644
> > > --- a/fs/erofs/erofs_fs.h
> > > +++ b/fs/erofs/erofs_fs.h
> > > @@ -17,6 +17,7 @@
> > >   */
> > >  #define EROFS_FEATURE_INCOMPAT_LZ4_0PADDING  0x00000001
> > >  #define EROFS_ALL_FEATURE_INCOMPAT
> >  EROFS_FEATURE_INCOMPAT_LZ4_0PADDING
> > > +#define EROFS_FEATURE_COMPAT_SB_CHKSUM               0x00000001
> > >
> > >  /* 128-byte erofs on-disk super block */
> > >  struct erofs_super_block {
> > > @@ -37,8 +38,8 @@ struct erofs_super_block {
> > >       __u8 uuid[16];          /* 128-bit uuid for volume */
> > >       __u8 volume_name[16];   /* volume name */
> > >       __le32 feature_incompat;
> > > -
> > > -     __u8 reserved2[44];
> > > +     __le32 chksum_blocks;   /* number of blocks used for checksum */
> > > +     __u8 reserved2[40];
> > >  };
> > >
> > >  /*
> > > diff --git a/fs/erofs/internal.h b/fs/erofs/internal.h
> > > index 544a453..cec27ca 100644
> > > --- a/fs/erofs/internal.h
> > > +++ b/fs/erofs/internal.h
> > > @@ -86,7 +86,7 @@ struct erofs_sb_info {
> > >       u8 uuid[16];                    /* 128-bit uuid for volume */
> > >       u8 volume_name[16];             /* volume name */
> > >       u32 feature_incompat;
> > > -
> > > +     u32 feature_compat;
> > >       unsigned int mount_opt;
> > >  };
> > >
> > > @@ -426,6 +426,7 @@ static inline void z_erofs_exit_zip_subsystem(void)
> > {}
> > >  #endif       /* !CONFIG_EROFS_FS_ZIP */
> > >
> > >  #define EFSCORRUPTED    EUCLEAN         /* Filesystem is corrupted */
> > > +#define EFSBADCRC    EBADMSG         /* Bad crc found */
> > >
> > >  #endif       /* __EROFS_INTERNAL_H */
> > >
> > > diff --git a/fs/erofs/super.c b/fs/erofs/super.c
> > > index 0e36949..9cda72d 100644
> > > --- a/fs/erofs/super.c
> > > +++ b/fs/erofs/super.c
> > > @@ -9,6 +9,7 @@
> > >  #include <linux/statfs.h>
> > >  #include <linux/parser.h>
> > >  #include <linux/seq_file.h>
> > > +#include <linux/crc32c.h>
> > >  #include "xattr.h"
> > >
> > >  #define CREATE_TRACE_POINTS
> > > @@ -46,6 +47,31 @@ void _erofs_info(struct super_block *sb, const char
> > *function,
> > >       va_end(args);
> > >  }
> > >
> > > +static int erofs_validate_sb_chksum(struct erofs_super_block *dsb,
> > > +                                    struct super_block *sb)
> > > +{
> > > +     u32 disk_chksum, nblocks, crc = 0;
> > > +     void *kaddr;
> > > +     struct page *page;
> > > +     int i;
> > > +
> > > +     disk_chksum = le32_to_cpu(dsb->checksum);
> > > +     nblocks = le32_to_cpu(dsb->chksum_blocks);
> >
> > We cannot write the page data directly since the page cache should be kept
> > in
> > sync with ondisk data (or for read-write fs, if it's claimed as uptodated,
> > and
> > it is modified later,  you should mark it dirty, and do writeback then, but
> > that is not the erofs case.)
> >
> > > +     dsb->checksum = 0;
> > > +     for (i = 0; i < nblocks; i++) {
> > > +             page = erofs_get_meta_page(sb, i);
> > > +             if (IS_ERR(page))
> > > +                     return PTR_ERR(page);
> > > +             kaddr = kmap(page);
> >
> > Here kmap_atomic(page) is better. what I mean is kmap_atomic() in the
> > caller
> > erofs_read_superblock(), it should be replaced to kmap() instead.
> >
> > > +             crc = crc32c(crc, kaddr, EROFS_BLKSIZ);
> > > +             kunmap(page);
> > > +             unlock_page(page);
> >
> > need
> >                 put_page(page);
> >
> >
> > I'm not sure whether I explained quite well, but this patch needs something
> > to do. I'm now working on demonstrating new XZ algorithm and releasing
> > erofs-utils v1.0.
> >
> > You can give more tries or I will help later. :-)
> >
> > Thanks,
> > Gao Xiang
> >
> >
> > > +     }
> > > +     if (crc != disk_chksum)
> > > +             return -EFSBADCRC;
> > > +     return 0;
> > > +}
> > > +
> > >  static void erofs_inode_init_once(void *ptr)
> > >  {
> > >       struct erofs_inode *vi = ptr;
> > > @@ -121,6 +147,13 @@ static int erofs_read_superblock(struct super_block
> > *sb)
> > >               goto out;
> > >       }
> > >
> > > +     if (dsb->feature_compat & EROFS_FEATURE_COMPAT_SB_CHKSUM) {
> > > +             ret = erofs_validate_sb_chksum(dsb, sb);
> > > +             if (ret < 0) {
> > > +                     erofs_err(sb, "super block checksum incorrect");
> > > +                     goto out;
> > > +             }
> > > +     }
> > >       blkszbits = dsb->blkszbits;
> > >       /* 9(512 bytes) + LOG_SECTORS_PER_BLOCK == LOG_BLOCK_SIZE */
> > >       if (blkszbits != LOG_BLOCK_SIZE) {
> > > --
> > > 2.9.3
> > >
> >