Hi, All
arch/x86/kernel/pci-calgary.c is the only user of saved_max_pfn today:
int __init detect_calgary(void)
{
[..]
specified_table_size = determine_tce_table_size((is_kdump_kernel() ?
saved_max_pfn : max_pfn) * PAGE_SIZE);
[..]
}
saved_max_pfn is the real mem size and is calculated by 1st kernel E820
memmap which is passed in by 2nd kernel's boot_params (done by kexec):
saved_max_pfn = e820_end_of_ram_pfn();
After saved_max_pfn has been set, memmap=exactmap will reset the E820
provided by boot_params and use the user defined E820 instead.
Now we want to get rid of memmap=exactmap and directly pass the E820
memmap by boot_params for some reason (eg. exactmap may exceed the cmdline
size and also isn't compatible with kaslr).
However saved_max_pfn becomes the obstacle for obsoleting exactmap.
Because it needs two conditions: first kernel's E820 map and
memmap=exactmap cmdline.
So I'm wondering if it's possible to get rid of saved_max_pfn totally in
calgary code. Or we can get saved_max_pfn using a different way, for
example calculated in 1st kernel and passed in to 2nd kernel by cmdline.
Thanks
WANG Chao
On Tue, Feb 18, 2014 at 11:18 PM, WANG Chao <[email protected]> wrote:
> Hi, All
>
> arch/x86/kernel/pci-calgary.c is the only user of saved_max_pfn today:
>
> int __init detect_calgary(void)
> {
> [..]
> specified_table_size = determine_tce_table_size((is_kdump_kernel() ?
> saved_max_pfn : max_pfn) * PAGE_SIZE);
> [..]
> }
IIUC, the purpose of this code is to reuse the TCE table from the
previous kernel. Thus, it needs to be of the same size as the
pre-kdump kernel. It is using the max_pfn to determine the TCE table
size in the non-kdump case. If there is another way to determine the
size it used before, then I am fine making the change to use that way.
Thanks,
Jon
> saved_max_pfn is the real mem size and is calculated by 1st kernel E820
> memmap which is passed in by 2nd kernel's boot_params (done by kexec):
>
> saved_max_pfn = e820_end_of_ram_pfn();
>
> After saved_max_pfn has been set, memmap=exactmap will reset the E820
> provided by boot_params and use the user defined E820 instead.
>
> Now we want to get rid of memmap=exactmap and directly pass the E820
> memmap by boot_params for some reason (eg. exactmap may exceed the cmdline
> size and also isn't compatible with kaslr).
>
> However saved_max_pfn becomes the obstacle for obsoleting exactmap.
> Because it needs two conditions: first kernel's E820 map and
> memmap=exactmap cmdline.
>
> So I'm wondering if it's possible to get rid of saved_max_pfn totally in
> calgary code. Or we can get saved_max_pfn using a different way, for
> example calculated in 1st kernel and passed in to 2nd kernel by cmdline.
>
> Thanks
> WANG Chao
On Wed, Feb 19, 2014 at 05:04:22PM -0700, Jon Mason wrote:
> On Tue, Feb 18, 2014 at 11:18 PM, WANG Chao <[email protected]> wrote:
> > Hi, All
> >
> > arch/x86/kernel/pci-calgary.c is the only user of saved_max_pfn today:
> >
> > int __init detect_calgary(void)
> > {
> > [..]
> > specified_table_size = determine_tce_table_size((is_kdump_kernel() ?
> > saved_max_pfn : max_pfn) * PAGE_SIZE);
> > [..]
> > }
>
> IIUC, the purpose of this code is to reuse the TCE table from the
> previous kernel. Thus, it needs to be of the same size as the
> pre-kdump kernel. It is using the max_pfn to determine the TCE table
> size in the non-kdump case. If there is another way to determine the
> size it used before, then I am fine making the change to use that way.
How about passing old tce table size on command line to second kernel.
Given the fact that it is specific to calgary only, we can it very
specific. Say calgary_iommu_old_tce_table_sz=<size>.
But we will then need to know the size of TCE table in first kernel. Is
this information exported to user space somewhere?
Thanks
Vivek
Remove [email protected] from CC, this email isn't valid now.
On 02/19/14 at 09:36pm, Vivek Goyal wrote:
> On Wed, Feb 19, 2014 at 05:04:22PM -0700, Jon Mason wrote:
> > On Tue, Feb 18, 2014 at 11:18 PM, WANG Chao <[email protected]> wrote:
> > > Hi, All
> > >
> > > arch/x86/kernel/pci-calgary.c is the only user of saved_max_pfn today:
> > >
> > > int __init detect_calgary(void)
> > > {
> > > [..]
> > > specified_table_size = determine_tce_table_size((is_kdump_kernel() ?
> > > saved_max_pfn : max_pfn) * PAGE_SIZE);
> > > [..]
> > > }
> >
> > IIUC, the purpose of this code is to reuse the TCE table from the
> > previous kernel. Thus, it needs to be of the same size as the
> > pre-kdump kernel. It is using the max_pfn to determine the TCE table
> > size in the non-kdump case. If there is another way to determine the
> > size it used before, then I am fine making the change to use that way.
>
> How about passing old tce table size on command line to second kernel.
> Given the fact that it is specific to calgary only, we can it very
> specific. Say calgary_iommu_old_tce_table_sz=<size>.
Don't need to introduce a new parameter, this is already there:
calgary=[64k,128k,256k,512k,1M,2M,4M,8M]
>
> But we will then need to know the size of TCE table in first kernel. Is
> this information exported to user space somewhere?
If this value isn't exported to userspace or even won't in the future, I think
the table size also can be determined by userspace. We can implement something like the
kernel does in kexec.
The calgary code:
static inline int __init determine_tce_table_size(u64 ram)
{
int ret;
if (specified_table_size != TCE_TABLE_SIZE_UNSPECIFIED)
return specified_table_size;
/*
* Table sizes are from 0 to 7 (TCE_TABLE_SIZE_64K to
* TCE_TABLE_SIZE_8M). Table size 0 has 8K entries and each
* larger table size has twice as many entries, so shift the
* max ram address by 13 to divide by 8K and then look at the
* order of the result to choose between 0-7.
*/
ret = get_order(ram >> 13);
if (ret > TCE_TABLE_SIZE_8M)
ret = TCE_TABLE_SIZE_8M;
return ret;
}
But it still no clear to me how I can determine calgary iommu is in use in 1st kernel.
Thanks
WANG Chao
On 02/19/14 at 05:04pm, Jon Mason wrote:
> On Tue, Feb 18, 2014 at 11:18 PM, WANG Chao <[email protected]> wrote:
> > Hi, All
> >
> > arch/x86/kernel/pci-calgary.c is the only user of saved_max_pfn today:
> >
> > int __init detect_calgary(void)
> > {
> > [..]
> > specified_table_size = determine_tce_table_size((is_kdump_kernel() ?
> > saved_max_pfn : max_pfn) * PAGE_SIZE);
> > [..]
> > }
>
> IIUC, the purpose of this code is to reuse the TCE table from the
> previous kernel. Thus, it needs to be of the same size as the
> pre-kdump kernel. It is using the max_pfn to determine the TCE table
> size in the non-kdump case. If there is another way to determine the
> size it used before, then I am fine making the change to use that way.
>From code the size is from 64K to 8M, saved_max_pfn is needed to get
this. Could it be a fixed size for TCE table? If this can be a fixed
value, E.g 8M, saved_max_pfn will be not needed any more though a
little memory may be wasted if total ram is smaller than 4G.
Baoquan
Thanks
>