From: "Luis R. Rodriguez" <[email protected]>
Here's a v2 series on kernel integration support. This addresses
the expresed concerns over killing the versions file for packaging.
While at it it also addresses avoiding the double prefix issue
by prefixing the things we carry over with the copy trick with
BPAUTO, and removing the BACKPORT prefix all together from the
kernel versions we backport.
Luis R. Rodriguez (13):
backports: move legacy and SmPL patch application into helper
backports: ifdef around module_init() module_exit() for modules
backports: allow for different backport prefix
backports: replace BACKPORT_PWD with BACKPORT_DIR
backports: use BACKPORT_DIR prefix on kconfig sources
backports: update dependencies map file
backports: split Kconfig into Kconfig.package and Kconfig.sources
backports: move version file generation to run earlier
backports: define C code backport version info using CPTCFG_
backports: add backport version parsing for kernel integration
backports: prefix c-file / h-file auto backport with BPAUTO
backports: remove extra BACKPORT_ prefix from kernel versioning
backports: add full kernel integration support
backport/Kconfig | 47 --
backport/Kconfig.integrate | 36 ++
backport/Kconfig.package | 32 ++
backport/Kconfig.sources | 23 +
backport/Makefile | 8 +-
backport/Makefile.build | 4 +-
backport/Makefile.kernel | 19 +-
backport/Makefile.real | 2 +-
backport/backport-include/asm/dma-mapping.h | 4 +-
backport/backport-include/backport/backport.h | 5 +
backport/backport-include/backport/leds-disabled.h | 2 +-
backport/backport-include/linux/module.h | 8 +-
backport/compat/Kconfig | 84 ++--
backport/compat/Makefile | 42 +-
backport/compat/backports.h | 4 +-
backport/compat/main.c | 40 +-
backport/scripts/uninstall.sh | 4 +-
dependencies | 31 +-
devel/doc/kconfig-operation | 13 +-
gentree.py | 487 +++++++++++++--------
.../0001-enable-backports-built-in.patch | 40 ++
lib/bpversion.py | 48 ++
lib/kconfig.py | 164 ++++++-
patches/backport-adjustments/devcoredump.patch | 4 +-
.../media/0002-no_dmabuf/v4l2.patch | 6 +-
25 files changed, 800 insertions(+), 357 deletions(-)
delete mode 100644 backport/Kconfig
create mode 100644 backport/Kconfig.integrate
create mode 100644 backport/Kconfig.package
create mode 100644 backport/Kconfig.sources
create mode 100644 integration-patches/0001-enable-backports/0001-enable-backports-built-in.patch
create mode 100644 lib/bpversion.py
--
2.1.1
From: "Luis R. Rodriguez" <[email protected]>
This allows us to extend how backports uses patches for
different types of applications. This will later be used
for kernel integration support, for example.
This should have no functional change.
Signed-off-by: Luis R. Rodriguez <[email protected]>
---
gentree.py | 279 +++++++++++++++++++++++++++++++------------------------------
1 file changed, 144 insertions(+), 135 deletions(-)
diff --git a/gentree.py b/gentree.py
index 9b4db98..59ae19d 100755
--- a/gentree.py
+++ b/gentree.py
@@ -438,6 +438,149 @@ def upload_release(args, rel_prep, logwrite=lambda x:None):
kup_cmd = "kup put /\n\t\t%s /\n\t\t%s /\n\t\t%s" % (gzip_name, tar_name + '.asc', korg_path)
logwrite("kup-test: skipping cmd: %s" % kup_cmd)
+def apply_patches(args, desc, source_dir, patch_src, target_dir, logwrite=lambda x:None):
+ """
+ Given a path of a directories of patches and SmPL patches apply
+ them on the target directory. If requested refresh patches, or test
+ a specific SmPL patch.
+ """
+ logwrite('Applying patches from %s to %s ...' % (patch_src, target_dir))
+ test_cocci = args.test_cocci or args.profile_cocci
+ test_cocci_found = False
+ patches = []
+ sempatches = []
+ for root, dirs, files in os.walk(os.path.join(source_dir, patch_src)):
+ for f in files:
+ if not test_cocci and f.endswith('.patch'):
+ patches.append(os.path.join(root, f))
+ if f.endswith('.cocci'):
+ if test_cocci:
+ if f not in test_cocci:
+ continue
+ test_cocci_found = True
+ if args.test_cocci:
+ logwrite("Testing Coccinelle SmPL patch: %s" % test_cocci)
+ elif args.profile_cocci:
+ logwrite("Profiling Coccinelle SmPL patch: %s" % test_cocci)
+ sempatches.append(os.path.join(root, f))
+ patches.sort()
+ prefix_len = len(os.path.join(source_dir, patch_src)) + 1
+ for pfile in patches:
+ print_name = pfile[prefix_len:]
+ # read the patch file
+ p = patch.fromfile(pfile)
+ # complain if it's not a patch
+ if not p:
+ raise Exception('No patch content found in %s' % print_name)
+ # leading / seems to be stripped?
+ if 'dev/null' in p.items[0].source:
+ raise Exception('Patches creating files are not supported (in %s)' % print_name)
+ # check if the first file the patch touches exists, if so
+ # assume the patch needs to be applied -- otherwise continue
+ patched_file = '/'.join(p.items[0].source.split('/')[1:])
+ fullfn = os.path.join(target_dir, patched_file)
+ if not os.path.exists(fullfn):
+ if args.verbose:
+ logwrite("Not applying %s, not needed" % print_name)
+ continue
+ if args.verbose:
+ logwrite("Applying patch %s" % print_name)
+
+ if args.refresh:
+ # but for refresh, of course look at all files the patch touches
+ for patchitem in p.items:
+ patched_file = '/'.join(patchitem.source.split('/')[1:])
+ fullfn = os.path.join(target_dir, patched_file)
+ shutil.copyfile(fullfn, fullfn + '.orig_file')
+
+ process = subprocess.Popen(['patch', '-p1'], stdout=subprocess.PIPE,
+ stderr=subprocess.STDOUT, stdin=subprocess.PIPE,
+ close_fds=True, universal_newlines=True,
+ cwd=target_dir)
+ output = process.communicate(input=open(pfile, 'r').read())[0]
+ output = output.split('\n')
+ if output[-1] == '':
+ output = output[:-1]
+ if args.verbose:
+ for line in output:
+ logwrite('> %s' % line)
+ if process.returncode != 0:
+ if not args.verbose:
+ logwrite("Failed to apply changes from %s" % print_name)
+ for line in output:
+ logwrite('> %s' % line)
+ raise Exception('Patch failed')
+
+ if args.refresh:
+ pfilef = open(pfile + '.tmp', 'a')
+ pfilef.write(p.top_header)
+ pfilef.flush()
+ for patchitem in p.items:
+ patched_file = '/'.join(patchitem.source.split('/')[1:])
+ fullfn = os.path.join(target_dir, patched_file)
+ process = subprocess.Popen(['diff', '-p', '-u', patched_file + '.orig_file', patched_file,
+ '--label', 'a/' + patched_file,
+ '--label', 'b/' + patched_file],
+ stdout=pfilef, close_fds=True,
+ universal_newlines=True, cwd=target_dir)
+ process.wait()
+ os.unlink(fullfn + '.orig_file')
+ if not process.returncode in (0, 1):
+ logwrite("Failed to diff to refresh %s" % print_name)
+ pfilef.close()
+ os.unlink(pfile + '.tmp')
+ raise Exception('Refresh failed')
+ pfilef.close()
+ os.rename(pfile + '.tmp', pfile)
+
+ # remove orig/rej files that patch sometimes creates
+ for root, dirs, files in os.walk(target_dir):
+ for f in files:
+ if f[-5:] == '.orig' or f[-4:] == '.rej':
+ os.unlink(os.path.join(root, f))
+ git_debug_snapshot(args, "apply %s patch %s" % (desc, print_name))
+
+ sempatches.sort()
+ prefix_len = len(os.path.join(source_dir, patch_src)) + 1
+
+ for cocci_file in sempatches:
+ # Until Coccinelle picks this up
+ pycocci = os.path.join(source_dir, 'devel/pycocci')
+ cmd = [pycocci, cocci_file]
+ extra_spatch_args = []
+ if args.profile_cocci:
+ cmd.append('--profile-cocci')
+ cmd.append(os.path.abspath(target_dir))
+ print_name = cocci_file[prefix_len:]
+ if args.verbose:
+ logwrite("Applying SmPL patch %s" % print_name)
+ sprocess = subprocess.Popen(cmd,
+ stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
+ close_fds=True, universal_newlines=True,
+ cwd=target_dir)
+ output = sprocess.communicate()[0]
+ sprocess.wait()
+ if sprocess.returncode != 0:
+ logwrite("Failed to process SmPL patch %s" % print_name)
+ raise Exception('SmPL patch failed')
+ output = output.split('\n')
+ if output[-1] == '':
+ output = output[:-1]
+ if args.verbose:
+ for line in output:
+ logwrite('> %s' % line)
+
+ # remove cocci_backup files
+ for root, dirs, files in os.walk(target_dir):
+ for f in files:
+ if f.endswith('.cocci_backup'):
+ os.unlink(os.path.join(root, f))
+ git_debug_snapshot(args, "apply %s SmPL patch %s" % (desc, print_name))
+
+ if test_cocci and test_cocci_found:
+ logwrite('Done!')
+ sys.exit(0)
+
def _main():
# Our binary requirements go here
req = reqs.Req()
@@ -620,141 +763,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
bpcfg.disable_symbols(disable_list)
git_debug_snapshot(args, 'Add automatic backports')
- test_cocci = args.test_cocci or args.profile_cocci
-
- logwrite('Apply patches ...')
- patches = []
- sempatches = []
- for root, dirs, files in os.walk(os.path.join(source_dir, 'patches')):
- for f in files:
- if not test_cocci and f.endswith('.patch'):
- patches.append(os.path.join(root, f))
- if f.endswith('.cocci'):
- if test_cocci:
- if f not in test_cocci:
- continue
- if args.test_cocci:
- logwrite("Testing Coccinelle SmPL patch: %s" % test_cocci)
- elif args.profile_cocci:
- logwrite("Profiling Coccinelle SmPL patch: %s" % test_cocci)
- sempatches.append(os.path.join(root, f))
- patches.sort()
- prefix_len = len(os.path.join(source_dir, 'patches')) + 1
- for pfile in patches:
- print_name = pfile[prefix_len:]
- # read the patch file
- p = patch.fromfile(pfile)
- # complain if it's not a patch
- if not p:
- raise Exception('No patch content found in %s' % print_name)
- # leading / seems to be stripped?
- if 'dev/null' in p.items[0].source:
- raise Exception('Patches creating files are not supported (in %s)' % print_name)
- # check if the first file the patch touches exists, if so
- # assume the patch needs to be applied -- otherwise continue
- patched_file = '/'.join(p.items[0].source.split('/')[1:])
- fullfn = os.path.join(args.outdir, patched_file)
- if not os.path.exists(fullfn):
- if args.verbose:
- logwrite("Not applying %s, not needed" % print_name)
- continue
- if args.verbose:
- logwrite("Applying patch %s" % print_name)
-
- if args.refresh:
- # but for refresh, of course look at all files the patch touches
- for patchitem in p.items:
- patched_file = '/'.join(patchitem.source.split('/')[1:])
- fullfn = os.path.join(args.outdir, patched_file)
- shutil.copyfile(fullfn, fullfn + '.orig_file')
-
- process = subprocess.Popen(['patch', '-p1'], stdout=subprocess.PIPE,
- stderr=subprocess.STDOUT, stdin=subprocess.PIPE,
- close_fds=True, universal_newlines=True,
- cwd=args.outdir)
- output = process.communicate(input=open(pfile, 'r').read())[0]
- output = output.split('\n')
- if output[-1] == '':
- output = output[:-1]
- if args.verbose:
- for line in output:
- logwrite('> %s' % line)
- if process.returncode != 0:
- if not args.verbose:
- logwrite("Failed to apply changes from %s" % print_name)
- for line in output:
- logwrite('> %s' % line)
- return 2
-
- if args.refresh:
- pfilef = open(pfile + '.tmp', 'a')
- pfilef.write(p.top_header)
- pfilef.flush()
- for patchitem in p.items:
- patched_file = '/'.join(patchitem.source.split('/')[1:])
- fullfn = os.path.join(args.outdir, patched_file)
- process = subprocess.Popen(['diff', '-p', '-u', patched_file + '.orig_file', patched_file,
- '--label', 'a/' + patched_file,
- '--label', 'b/' + patched_file],
- stdout=pfilef, close_fds=True,
- universal_newlines=True, cwd=args.outdir)
- process.wait()
- os.unlink(fullfn + '.orig_file')
- if not process.returncode in (0, 1):
- logwrite("Failed to diff to refresh %s" % print_name)
- pfilef.close()
- os.unlink(pfile + '.tmp')
- return 3
- pfilef.close()
- os.rename(pfile + '.tmp', pfile)
-
- # remove orig/rej files that patch sometimes creates
- for root, dirs, files in os.walk(args.outdir):
- for f in files:
- if f[-5:] == '.orig' or f[-4:] == '.rej':
- os.unlink(os.path.join(root, f))
- git_debug_snapshot(args, "apply backport patch %s" % print_name)
-
- sempatches.sort()
- prefix_len = len(os.path.join(source_dir, 'patches')) + 1
-
- for cocci_file in sempatches:
- # Until Coccinelle picks this up
- pycocci = os.path.join(source_dir, 'devel/pycocci')
- cmd = [pycocci, cocci_file]
- extra_spatch_args = []
- if args.profile_cocci:
- cmd.append('--profile-cocci')
- cmd.append(os.path.abspath(args.outdir))
- print_name = cocci_file[prefix_len:]
- if args.verbose:
- logwrite("Applying SmPL patch %s" % print_name)
- sprocess = subprocess.Popen(cmd,
- stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
- close_fds=True, universal_newlines=True,
- cwd=args.outdir)
- output = sprocess.communicate()[0]
- sprocess.wait()
- if sprocess.returncode != 0:
- logwrite("Failed to process SmPL patch %s" % print_name)
- return 3
- output = output.split('\n')
- if output[-1] == '':
- output = output[:-1]
- if args.verbose:
- for line in output:
- logwrite('> %s' % line)
-
- # remove cocci_backup files
- for root, dirs, files in os.walk(args.outdir):
- for f in files:
- if f.endswith('.cocci_backup'):
- os.unlink(os.path.join(root, f))
- git_debug_snapshot(args, "apply backport SmPL patch %s" % print_name)
-
- if test_cocci:
- logwrite('Done!')
- return 0
+ apply_patches(args, "backport", source_dir, 'patches', args.outdir, logwrite)
# some post-processing is required
configtree = kconfig.ConfigTree(os.path.join(args.outdir, 'Kconfig'))
--
2.1.1
From: "Luis R. Rodriguez" <[email protected]>
We only need this for modules.
Signed-off-by: Luis R. Rodriguez <[email protected]>
---
backport/backport-include/linux/module.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/backport/backport-include/linux/module.h b/backport/backport-include/linux/module.h
index 82c96bd..e8f28b1 100644
--- a/backport/backport-include/linux/module.h
+++ b/backport/backport-include/linux/module.h
@@ -25,6 +25,7 @@ extern void backport_dependency_symbol(void);
" using backports " BACKPORTS_VERSION);
#endif
+#ifdef MODULE
#undef module_init
#define module_init(initfn) \
static int __init __init_backport(void) \
@@ -58,6 +59,7 @@ extern void backport_dependency_symbol(void);
rcu_barrier(); \
} \
void cleanup_module(void) __attribute__((alias("__exit_compat")));
+#endif
#if LINUX_VERSION_CODE < KERNEL_VERSION(3,3,0)
#undef param_check_bool
--
2.1.1
From: "Luis R. Rodriguez" <[email protected]>
The way we backport when packaging is to minimize the amount of
changes required by taking advantage of the fact that Kconfig
will treat can read CONFIG_ an environment variable with getenv()
when parsing menu entries. When doing integration we don't want
to do this so instead we'll rely on the CONFIG_BACKPORT prefix.
This requires a bit of work on our current parsers.
This adds some initial code for integration, this doesn't run yet.
Although adjust_backported_configs() runs when packaging right now
this just removes some comments out the top level Kconfig, but
leaves the code in place should it later want to consider using
CONFIG_BACKPORT prefix and the gains of now allowing to build
when a respective symbols is already built in.
As a small enhancement add to each backported local symbol a
dependency to require the user's kernel to not have built-in the
backported symbols when packaging, this allows to get rid of the
checks in checks.h, we make this more generic for all drivers and
subsystems we backport now.
Signed-off-by: Luis R. Rodriguez <[email protected]>
---
backport/Kconfig | 6 ++-
devel/doc/kconfig-operation | 4 +-
gentree.py | 53 ++++++++++++++++++-------
lib/kconfig.py | 94 ++++++++++++++++++++++++++++++++++++++++++++-
4 files changed, 138 insertions(+), 19 deletions(-)
diff --git a/backport/Kconfig b/backport/Kconfig
index b14a268..f63f4fd 100644
--- a/backport/Kconfig
+++ b/backport/Kconfig
@@ -14,11 +14,15 @@ config BACKPORTED_KERNEL_NAME
source Kconfig.kernel
source Kconfig.versions
-# some hacks ...
+# some hacks for when we use backports to generate a package
+# to build modules out of tree.
+#ignore-parser-package
config WIRELESS
def_bool y
+#ignore-parser-package
config NET_CORE
def_bool y
+#ignore-parser-package
config EXPERT
def_bool y
diff --git a/devel/doc/kconfig-operation b/devel/doc/kconfig-operation
index 0245889..f1ecf60 100644
--- a/devel/doc/kconfig-operation
+++ b/devel/doc/kconfig-operation
@@ -63,7 +63,7 @@ being compiled against is older than X.
All together, this allows the correct options to be selected by the user.
-There's one more caveat: At backport configuration time, the CONFIG_
+When using Linux backports as a packaged solution the the CONFIG_
environment variable is set to CPTCFG_ ("compat config", but also chosen
because it has the same length as CONFIG_). This shows up in the .config
file and other places, and in fact in all makefiles and in the code. The
@@ -73,8 +73,6 @@ This allows code to, for example, have "#ifdef CONFIG_PM" which can only
be set or cleared in the kernel, not in the backport configuration. Since
this is needed, a transformation step is done at backport creation time.
-
-
Backport creation for Kconfig
-------------------------------
diff --git a/gentree.py b/gentree.py
index 59ae19d..3d4a8b4 100755
--- a/gentree.py
+++ b/gentree.py
@@ -42,7 +42,7 @@ def read_copy_list(copyfile):
return ret
-def read_dependencies(depfilename):
+def read_dependencies(depfilename, bp_prefix):
"""
Read a (the) dependency file and return the list of
dependencies as a dictionary, mapping a Kconfig symbol
@@ -71,6 +71,9 @@ def read_dependencies(depfilename):
ret[sym].append(kconfig_exp)
else:
sym, dep = item.split()
+ if bp_prefix != 'CPTCFG_':
+ dep_prefix = re.sub(r'^CONFIG_(.*)', r'\1', bp_prefix)
+ sym = dep_prefix + sym
if not sym in ret:
ret[sym] = [dep, ]
else:
@@ -195,7 +198,7 @@ def automatic_backport_mangle_c_file(name):
return name.replace('/', '-')
-def add_automatic_backports(args):
+def add_automatic_backports(args, bp_prefix):
disable_list = []
export = re.compile(r'^EXPORT_SYMBOL(_GPL)?\((?P<sym>[^\)]*)\)')
bpi = kconfig.get_backport_info(os.path.join(args.outdir, 'compat', 'Kconfig'))
@@ -228,9 +231,9 @@ def add_automatic_backports(args):
raise Exception('backporting a module requires a #module-name')
for of in o_files:
mf.write('%s-objs += %s\n' % (module_name, of))
- mf.write('obj-$(CPTCFG_%s) += %s.o\n' % (sym, module_name))
+ mf.write('obj-$(%s%s) += %s.o\n' % (bp_prefix, sym, module_name))
elif symtype == 'bool':
- mf.write('compat-$(CPTCFG_%s) += %s\n' % (sym, ' '.join(o_files)))
+ mf.write('compat-$(%s%s) += %s\n' % (bp_prefix, sym, ' '.join(o_files)))
# finally create the include file
syms = []
@@ -243,14 +246,14 @@ def add_automatic_backports(args):
for f in h_files:
outf = open(os.path.join(args.outdir, 'include', f), 'w')
outf.write('/* Automatically created during backport process */\n')
- outf.write('#ifndef CPTCFG_%s\n' % sym)
+ outf.write('#ifndef %s%s\n' % (bp_prefix, sym))
outf.write('#include_next <%s>\n' % f)
outf.write('#else\n');
for s in syms:
outf.write('#undef %s\n' % s)
outf.write('#define %s LINUX_BACKPORT(%s)\n' % (s, s))
outf.write('#include <%s>\n' % (os.path.dirname(f) + '/backport-' + os.path.basename(f), ))
- outf.write('#endif /* CPTCFG_%s */\n' % sym)
+ outf.write('#endif /* %s%s */\n' % (bp_prefix, sym))
return disable_list
def git_debug_init(args):
@@ -695,6 +698,19 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
gitdebug, verbose, extra_driver, kup, kup_test,
test_cocci, profile_cocci)
rel_prep = None
+ integrate = False
+
+ # When building a package we use CPTCFG as we can rely on the
+ # fact that kconfig treats CONFIG_ as an environment variable
+ # requring less changes on code. For kernel integration we use
+ # the longer CONFIG_BACKPORT given that we'll be sticking to
+ # the kernel symbol namespace, to address that we do a final
+ # search / replace. Technically its possible to rely on the
+ # same prefix for packaging as with kernel integration but
+ # there are already som users of the CPTCFG prefix.
+ bp_prefix = 'CPTCFG_'
+ if integrate:
+ bp_prefix = 'CONFIG_BACKPORT_'
# start processing ...
if (args.kup or args.kup_test):
@@ -724,7 +740,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
sys.exit(1)
copy_list = read_copy_list(args.copy_list)
- deplist = read_dependencies(os.path.join(source_dir, 'dependencies'))
+ deplist = read_dependencies(os.path.join(source_dir, 'dependencies'), bp_prefix)
# validate output directory
check_output_dir(args.outdir, args.clean)
@@ -757,7 +773,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
git_debug_snapshot(args, 'Add driver sources')
- disable_list = add_automatic_backports(args)
+ disable_list = add_automatic_backports(args, bp_prefix)
if disable_list:
bpcfg = kconfig.ConfigTree(os.path.join(args.outdir, 'compat', 'Kconfig'))
bpcfg.disable_symbols(disable_list)
@@ -767,11 +783,19 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
# some post-processing is required
configtree = kconfig.ConfigTree(os.path.join(args.outdir, 'Kconfig'))
+ orig_symbols = configtree.symbols()
+
logwrite('Modify Kconfig tree ...')
configtree.prune_sources(ignore=['Kconfig.kernel', 'Kconfig.versions'])
git_debug_snapshot(args, "prune Kconfig tree")
- configtree.force_tristate_modular()
- git_debug_snapshot(args, "force tristate options modular")
+
+ if not integrate:
+ configtree.force_tristate_modular()
+ git_debug_snapshot(args, "force tristate options modular")
+
+ configtree.adjust_backported_configs(integrate, orig_symbols, bp_prefix)
+ git_debug_snapshot(args, "adjust backports config symbols we port")
+
configtree.modify_selects()
git_debug_snapshot(args, "convert select to depends on")
@@ -828,8 +852,9 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
# rewrite Makefile and source symbols
regexes = []
- for some_symbols in [symbols[i:i + 50] for i in range(0, len(symbols), 50)]:
- r = 'CONFIG_((' + '|'.join([s + '(_MODULE)?' for s in some_symbols]) + ')([^A-Za-z0-9_]|$))'
+ all_symbols = orig_symbols + symbols
+ for some_symbols in [all_symbols[i:i + 50] for i in range(0, len(all_symbols), 50)]:
+ r = 'CONFIG_((?!BACKPORT)(' + '|'.join([s + '(_MODULE)?' for s in some_symbols]) + ')([^A-Za-z0-9_]|$))'
regexes.append(re.compile(r, re.MULTILINE))
for root, dirs, files in os.walk(args.outdir):
# don't go into .git dir (possible debug thing)
@@ -838,7 +863,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
for f in files:
data = open(os.path.join(root, f), 'r').read()
for r in regexes:
- data = r.sub(r'CPTCFG_\1', data)
+ data = r.sub(r'' + bp_prefix + '\\1', data)
data = re.sub(r'\$\(srctree\)', '$(backport_srctree)', data)
data = re.sub(r'-Idrivers', '-I$(backport_srctree)/drivers', data)
fo = open(os.path.join(root, f), 'w')
@@ -884,7 +909,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
# groups -- 50 seemed safer and is still fast)
regexes = []
for some_symbols in [disable_makefile[i:i + 50] for i in range(0, len(disable_makefile), 50)]:
- r = '^([^#].*((CPTCFG|CONFIG)_(' + '|'.join([s for s in some_symbols]) + ')))'
+ r = '^([^#].*((CPTCFG|CONFIG_BACKPORT|CONFIG)_(' + '|'.join([s for s in some_symbols]) + ')))'
regexes.append(re.compile(r, re.MULTILINE))
for f in maketree.get_makefiles():
data = open(f, 'r').read()
diff --git a/lib/kconfig.py b/lib/kconfig.py
index 179121a..6c98b4c 100644
--- a/lib/kconfig.py
+++ b/lib/kconfig.py
@@ -7,9 +7,10 @@ import os, re
src_line = re.compile(r'^\s*source\s+"?(?P<src>[^\s"]*)"?\s*$')
tri_line = re.compile(r'^(?P<spc>\s+)tristate')
bool_line = re.compile(r'^(?P<spc>\s+)bool')
-cfg_line = re.compile(r'^(config|menuconfig)\s+(?P<sym>[^\s]*)')
+cfg_line = re.compile(r'^(?P<opt>config|menuconfig)\s+(?P<sym>[^\s]*)')
sel_line = re.compile(r'^(?P<spc>\s+)select\s+(?P<sym>[^\s]*)\s*$')
backport_line = re.compile(r'^\s+#(?P<key>[ch]-file|module-name)\s*(?P<name>.*)')
+ignore_parse_p = re.compile(r'^\s*#(?P<key>ignore-parser-package)')
class ConfigTree(object):
def __init__(self, rootfile):
@@ -56,6 +57,97 @@ class ConfigTree(object):
outf.write(out)
outf.close()
+ def _mod_kconfig_line(self, l, orig_symbols, bp_prefix):
+ if bp_prefix != 'CPTCFG_':
+ prefix = re.sub(r'^CONFIG_(.*)', r'\1', bp_prefix)
+ for sym in orig_symbols:
+ if sym in l:
+ return re.sub(r' (' + sym + ')', r' ' + prefix + '\\1', l)
+ return l
+
+ def adjust_backported_configs(self, integrate, orig_symbols, bp_prefix):
+ m = None
+ old_l = None
+ ignore_parse_modular = False
+ for nf in self._walk(self.rootfile):
+ out = ''
+ for l in open(os.path.join(self.basedir, nf), 'r'):
+ pm = ignore_parse_p.match(l)
+ if pm:
+ ignore_parse_modular = True
+ continue
+ n = cfg_line.match(l)
+ if n:
+ m = n
+ old_l = l
+ continue
+ # We're now on the second line for the config symbol
+ if m:
+ # Right now this only supports one line hacks
+ if ignore_parse_modular:
+ if not integrate:
+ out += old_l
+ out += l
+ ignore_parse_modular = False
+ m = None
+ continue
+ built_in_sym = m.group('sym')
+ if bp_prefix != 'CPTCFG_':
+ prefix = re.sub(r'^CONFIG_(.*)_', r'\1', bp_prefix)
+ built_in_sym = re.sub(r'' + prefix + '(.*)', r'\1', m.group('sym'))
+ else:
+ prefix = 'CPTCFG'
+ # These are things that we carry as part of our backports
+ # module or things we automatically copy over into our
+ # backports module.
+ if prefix in m.group('sym'):
+ out += old_l
+ out += l
+ # For modular solutions This might still be possible if
+ # we use backport_ prefixes for our own symbols but it
+ # seems that folks now don't want these prefixes so
+ # restrict to ensure what we replace will not be
+ # available built-in as we are reviewing phasing that
+ # usage out.
+ if not integrate:
+ x = 0
+ # XXX: for we won't address this, as this needs
+ # review and getting rid of the prefixes, otherwise
+ # its OK. Some folks may *want* to replace built-in
+ # old symbols with some modular hack when they
+ # know its safe, for instance.
+ # out += "\tdepends on %s!=y\n" % (built_in_sym)
+ else:
+ # For backports kernel integration solutions we
+ # allow the backported solution from future kernels
+ # to replace the kernel solution you on your
+ # original tree
+ # XXX: only do this for symbols that have C files
+ # depending on it
+ out += "\tdepends on !%s\n" % (built_in_sym)
+ else:
+ # First rewrite the upstream symbol with our prefix if
+ # needed
+ if bp_prefix != 'CPTCFG_':
+ out += m.group('opt') + ' ' + prefix + '_' + m.group('sym') + '\n'
+ out += l
+ # This can be done when not using the CPTCFG CONFIG_
+ # variable override, so it uses the checks.h but
+ # that's a reactive measure.
+ if not integrate:
+ out += "\tdepends on %s!=y\n" % (built_in_sym)
+ else:
+ out += "\tdepends on !%s\n" % (built_in_sym)
+ else:
+ out += m.group('opt') + ' ' + m.group('sym') + '\n'
+ out += l
+ m = None
+ else:
+ out += self._mod_kconfig_line(l, orig_symbols, bp_prefix)
+ outf = open(os.path.join(self.basedir, nf), 'w')
+ outf.write(out)
+ outf.close()
+
def symbols(self):
syms = []
for nf in self._walk(self.rootfile):
--
2.1.1
From: "Luis R. Rodriguez" <[email protected]>
This will also be used for built-in kernel integration, by
using this we can share more code with the built-in kernel
approach.
Reviewed-by: Johannes Berg <[email protected]>
Signed-off-by: Luis R. Rodriguez <[email protected]>
---
backport/Makefile | 6 +++---
backport/Makefile.build | 4 ++--
backport/Makefile.real | 2 +-
backport/scripts/uninstall.sh | 4 ++--
4 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/backport/Makefile b/backport/Makefile
index ea3982b..fcf2f01 100644
--- a/backport/Makefile
+++ b/backport/Makefile
@@ -6,7 +6,7 @@ ifeq ($(KERNELRELEASE),)
MAKEFLAGS += --no-print-directory
SHELL := /bin/bash
-BACKPORT_PWD := $(shell pwd)
+BACKPORT_DIR := $(shell pwd)
KMODDIR ?= updates
ifneq ($(origin KLIB), undefined)
@@ -20,7 +20,7 @@ KERNEL_CONFIG := $(KLIB_BUILD)/.config
KERNEL_MAKEFILE := $(KLIB_BUILD)/Makefile
CONFIG_MD5 := $(shell md5sum $(KERNEL_CONFIG) 2>/dev/null | sed 's/\s.*//')
-export KLIB KLIB_BUILD BACKPORT_PWD KMODDIR KMODPATH_ARG
+export KLIB KLIB_BUILD BACKPORT_DIR KMODDIR KMODPATH_ARG
# disable built-in rules for this file
.SUFFIXES:
@@ -159,5 +159,5 @@ help: defconfig-help
@echo ""
@echo "Execute "make" or "make all" to build all targets marked with [*]"
else
-include $(BACKPORT_PWD)/Makefile.kernel
+include $(BACKPORT_DIR)/Makefile.kernel
endif
diff --git a/backport/Makefile.build b/backport/Makefile.build
index d209041..a848b37 100644
--- a/backport/Makefile.build
+++ b/backport/Makefile.build
@@ -3,8 +3,8 @@ export
.PHONY: modules
modules:
- @$(MAKE) -C $(KLIB_BUILD) M=$(BACKPORT_PWD) modules
+ @$(MAKE) -C $(KLIB_BUILD) M=$(BACKPORT_DIR) modules
.PHONY: clean
clean:
- @$(MAKE) -C $(KLIB_BUILD) M=$(BACKPORT_PWD) clean
+ @$(MAKE) -C $(KLIB_BUILD) M=$(BACKPORT_DIR) clean
diff --git a/backport/Makefile.real b/backport/Makefile.real
index f60d5ca..a0f4916 100644
--- a/backport/Makefile.real
+++ b/backport/Makefile.real
@@ -89,7 +89,7 @@ modules: backport-include/backport/autoconf.h
.PHONY: install
install: modules
- @$(MAKE) -C $(KLIB_BUILD) M=$(BACKPORT_PWD) \
+ @$(MAKE) -C $(KLIB_BUILD) M=$(BACKPORT_DIR) \
INSTALL_MOD_DIR=$(KMODDIR) $(KMODPATH_ARG) \
modules_install
@./scripts/blacklist.sh $(KLIB)/ $(KLIB)/$(KMODDIR)
diff --git a/backport/scripts/uninstall.sh b/backport/scripts/uninstall.sh
index 1eb42b0..b46d3fe 100755
--- a/backport/scripts/uninstall.sh
+++ b/backport/scripts/uninstall.sh
@@ -10,8 +10,8 @@ else
compr=""
fi
-for driver in $(find ${BACKPORT_PWD} -type f -name *.ko); do
- mod_name=${driver/${BACKPORT_PWD}/${KLIB}${KMODDIR}}${compr}
+for driver in $(find ${BACKPORT_DIR} -type f -name *.ko); do
+ mod_name=${driver/${BACKPORT_DIR}/${KLIB}${KMODDIR}}${compr}
echo " uninstall" $mod_name
rm -f $mod_name
done
--
2.1.1
From: "Luis R. Rodriguez" <[email protected]>
During development of kernel integration support using CONFIG_BACKPORT
was evaluated as a prefix over CPTCFG even for packaging backports,
for some reason this change lifted some restrictions one some device
drivers which was present before and as such requires some changes to
the dependencies map file to ensure correct compilation for respective
kernel versions.
Signed-off-by: Luis R. Rodriguez <[email protected]>
---
dependencies | 31 +++++++++++++++++++++++--------
1 file changed, 23 insertions(+), 8 deletions(-)
diff --git a/dependencies b/dependencies
index 4964aa6..64903dc 100644
--- a/dependencies
+++ b/dependencies
@@ -16,6 +16,8 @@ USB_GSPCA_OV534 3.5
# we only support on >= 3.4 give that reguatory itself depends
# on -EPROBE_DEFER.
SOC_CAMERA 3.4
+VIDEO_S5K5BAF 3.4
+VIDEO_S5K4ECGX 3.4
# 49920bc6 and 1003cab8 and while it seems there is a
# one to one map DMA_FROM_DEVICE to DMA_DEV_TO_MEM
# I can't verify this fully yet.
@@ -29,16 +31,29 @@ DVB_USB_RTL28XXU 3.4
# missing linux/gpio/consumer.h
VIDEO_ADV7604 3.13
# missing function clk_prepare_enable(), clk_disable_unprepare() and devm_regulator_bulk_get()
-VIDEO_MT9P031 3.3
-VIDEO_MT9T001 3.3
VIDEO_MT9V032 3.3
-VIDEO_NOON010PC30 3.3
-VIDEO_S5K6A3 3.3
VIDEO_S5K6AA 3.3
-VIDEO_S5K5BAF 3.3
-VIDEO_S5K4ECGX 3.3
-VIDEO_M5MOLS 3.3
-VIDEO_S5C73M3 3.3
+
+# missing devm_regulator_bulk_get
+VIDEO_MT9T001 3.5
+VIDEO_NOON010PC30 3.5
+VIDEO_MT9P031 3.5
+
+# missing of_property_read_bool() added via fa4d34cc on v3.4-rc1~71, backport
+# that attomically and you can enable this for older kernels.
+VIDEO_ADV7343 3.4
+
+# for whoever wants to backport this: GPIOF_EXPORT was redefined via commit
+# 79a9becd on v3.13-rc1~120^2~15, before this it was redefined via commit
+# f567fde2 on v3.5-rc7~32^2~4 and finally added via commit fc3a1f04 on
+# v3.5-rc1~93^2~25. We don't address any of these changes yet so we require
+# the latest interpretation.
+VIDEO_S5K6AA 3.13
+VIDEO_S5K6A3 3.13
+
+# missing linux/sizes.h
+VIDEO_M5MOLS 3.6
+VIDEO_S5C73M3 3.6
# missing function devm_regmap_init_i2c()
VIDEO_LM3646 3.3
--
2.1.1
From: "Luis R. Rodriguez" <[email protected]>
This will allow us to do less work for built-in integration support.
This requires a bit of self evaluation of the variable within our
kconfig library, ideally we'd have support for groking all variables
defined within the Kconfig setup but that requires quite a lot more
work as it means also parsing Makefiles and inheriting these definitions
within config symbols when used. Since we only define one right now and
its used for built-in support we deal with it ourselves for now.
Please consider the complexity of adding others, it doesn't seem like
there would be much need for others though. If others wanted to use a
different BACKPORT_DIR path other than 'backports' that would require
tweaking here, but we'll start by assuming no one will want to do that.
Signed-off-by: Luis R. Rodriguez <[email protected]>
---
backport/Kconfig | 37 ++++++++++++++++++++-----------------
lib/kconfig.py | 53 ++++++++++++++++++++++++++++++++++++++++++-----------
2 files changed, 62 insertions(+), 28 deletions(-)
diff --git a/backport/Kconfig b/backport/Kconfig
index f63f4fd..c523323 100644
--- a/backport/Kconfig
+++ b/backport/Kconfig
@@ -1,5 +1,8 @@
mainmenu "Linux Backports from $BACKPORTED_KERNEL_NAME $BACKPORTED_KERNEL_VERSION (with backports $BACKPORTS_VERSION)"
+config BACKPORT_DIR
+ string
+ option env="BACKPORT_DIR"
config BACKPORTS_VERSION
string
option env="BACKPORTS_VERSION"
@@ -11,8 +14,8 @@ config BACKPORTED_KERNEL_NAME
option env="BACKPORTED_KERNEL_NAME"
# these will be generated
-source Kconfig.kernel
-source Kconfig.versions
+source "$BACKPORT_DIR/Kconfig.kernel"
+source "$BACKPORT_DIR/Kconfig.versions"
# some hacks for when we use backports to generate a package
# to build modules out of tree.
@@ -27,25 +30,25 @@ config EXPERT
def_bool y
# this has the configuration for the backport code
-source compat/Kconfig
+source "$BACKPORT_DIR/compat/Kconfig"
# these are copied from the kernel
-source net/wireless/Kconfig
-source net/mac80211/Kconfig
-source net/bluetooth/Kconfig
-source drivers/net/wireless/Kconfig
-source drivers/net/ethernet/Kconfig
-source drivers/net/usb/Kconfig
+source "$BACKPORT_DIR/net/wireless/Kconfig"
+source "$BACKPORT_DIR/net/mac80211/Kconfig"
+source "$BACKPORT_DIR/net/bluetooth/Kconfig"
+source "$BACKPORT_DIR/drivers/net/wireless/Kconfig"
+source "$BACKPORT_DIR/drivers/net/ethernet/Kconfig"
+source "$BACKPORT_DIR/drivers/net/usb/Kconfig"
-source drivers/ssb/Kconfig
-source drivers/bcma/Kconfig
+source "$BACKPORT_DIR/drivers/ssb/Kconfig"
+source "$BACKPORT_DIR/drivers/bcma/Kconfig"
-source net/nfc/Kconfig
+source "$BACKPORT_DIR/net/nfc/Kconfig"
-source drivers/media/Kconfig
+source "$BACKPORT_DIR/drivers/media/Kconfig"
-source net/ieee802154/Kconfig
-source net/mac802154/Kconfig
-source drivers/net/ieee802154/Kconfig
+source "$BACKPORT_DIR/net/ieee802154/Kconfig"
+source "$BACKPORT_DIR/net/mac802154/Kconfig"
+source "$BACKPORT_DIR/drivers/net/ieee802154/Kconfig"
-source drivers/usb/class/Kconfig
+source "$BACKPORT_DIR/drivers/usb/class/Kconfig"
diff --git a/lib/kconfig.py b/lib/kconfig.py
index 6c98b4c..430b642 100644
--- a/lib/kconfig.py
+++ b/lib/kconfig.py
@@ -5,6 +5,7 @@
import os, re
src_line = re.compile(r'^\s*source\s+"?(?P<src>[^\s"]*)"?\s*$')
+bk_src_line = re.compile(r'^\s*source\s+"?\$BACKPORT_DIR/(?P<src>[^\s"]*)"?\s*$')
tri_line = re.compile(r'^(?P<spc>\s+)tristate')
bool_line = re.compile(r'^(?P<spc>\s+)bool')
cfg_line = re.compile(r'^(?P<opt>config|menuconfig)\s+(?P<sym>[^\s]*)')
@@ -21,23 +22,53 @@ class ConfigTree(object):
yield f
for l in open(os.path.join(self.basedir, f), 'r'):
m = src_line.match(l)
- if m and os.path.exists(os.path.join(self.basedir, m.group('src'))):
- for i in self._walk(m.group('src')):
- yield i
+ if m:
+ bm = bk_src_line.match(l)
+ if bm:
+ if os.path.exists(os.path.join(self.basedir, bm.group('src'))):
+ for i in self._walk(os.path.join(self.basedir, bm.group('src'))):
+ yield i
+ elif os.path.exists(os.path.join(self.basedir, 'backports/' + bm.group('src'))):
+ for i in self._walk(os.path.join(self.basedir, 'backports/' + bm.group('src'))):
+ yield i
+ else:
+ if os.path.exists(os.path.join(self.basedir, m.group('src'))):
+ for i in self._walk(m.group('src')):
+ yield i
def _prune_sources(self, f, ignore):
for nf in self._walk(f):
out = ''
for l in open(os.path.join(self.basedir, nf), 'r'):
- m = src_line.match(l)
- if not m:
- out += l
- continue
- src = m.group('src')
- if src in ignore or os.path.exists(os.path.join(self.basedir, src)):
- out += l
+ bm = bk_src_line.match(l)
+ if bm:
+ bp_src = bm.group('src')
+ if bp_src in ignore or \
+ os.path.exists(os.path.join(self.basedir, bp_src)) or \
+ os.path.exists(os.path.join(self.basedir, 'backports/' + bp_src)):
+ out += l
+ else:
+ out += '#' + l
else:
- out += '#' + l
+ m = src_line.match(l)
+ # we should consider disallowing these as it could mean
+ # someone forgot to add the BACKPORT_DIR prefix to
+ # the kconfig source entries which we will need to
+ # support built-in integration.
+ if not m:
+ out += l
+ continue
+ src = m.group('src')
+ if src in ignore or os.path.exists(os.path.join(self.basedir, src)):
+ if 'backports' in self.basedir and 'BACKPORT_DIR' not in bp_src:
+ out += 'source "$BACKPORT_DIR/' + src + '"\n'
+ else:
+ out += l
+ else:
+ if 'backports' in self.basedir and 'BACKPORT_DIR' not in bp_src:
+ out += '# source "$BACKPORT_DIR/' + src + '"\n'
+ else:
+ out += '#' + l
outf = open(os.path.join(self.basedir, nf), 'w')
outf.write(out)
outf.close()
--
2.1.1
From: "Luis R. Rodriguez" <[email protected]>
This splits up the Kconfig from things which are needed due to
our copy of code from the kernel and things which we need
specifically only for packaging backports. We will later then
share the Kconfig.souce for instance for kernel integration.
While at it, split up the list of files that we need to copy
into what we know could be shared for integration. The the
case of backport packaging Kconfig.package will be copied to
the package's Kconfig when building the package with gentree.py.
Signed-off-by: Luis R. Rodriguez <[email protected]>
---
backport/Kconfig | 54 ------------------------------------------------
backport/Kconfig.package | 32 ++++++++++++++++++++++++++++
backport/Kconfig.sources | 23 +++++++++++++++++++++
gentree.py | 22 +++++++++++++++++---
4 files changed, 74 insertions(+), 57 deletions(-)
delete mode 100644 backport/Kconfig
create mode 100644 backport/Kconfig.package
create mode 100644 backport/Kconfig.sources
diff --git a/backport/Kconfig b/backport/Kconfig
deleted file mode 100644
index c523323..0000000
--- a/backport/Kconfig
+++ /dev/null
@@ -1,54 +0,0 @@
-mainmenu "Linux Backports from $BACKPORTED_KERNEL_NAME $BACKPORTED_KERNEL_VERSION (with backports $BACKPORTS_VERSION)"
-
-config BACKPORT_DIR
- string
- option env="BACKPORT_DIR"
-config BACKPORTS_VERSION
- string
- option env="BACKPORTS_VERSION"
-config BACKPORTED_KERNEL_VERSION
- string
- option env="BACKPORTED_KERNEL_VERSION"
-config BACKPORTED_KERNEL_NAME
- string
- option env="BACKPORTED_KERNEL_NAME"
-
-# these will be generated
-source "$BACKPORT_DIR/Kconfig.kernel"
-source "$BACKPORT_DIR/Kconfig.versions"
-
-# some hacks for when we use backports to generate a package
-# to build modules out of tree.
-#ignore-parser-package
-config WIRELESS
- def_bool y
-#ignore-parser-package
-config NET_CORE
- def_bool y
-#ignore-parser-package
-config EXPERT
- def_bool y
-
-# this has the configuration for the backport code
-source "$BACKPORT_DIR/compat/Kconfig"
-
-# these are copied from the kernel
-source "$BACKPORT_DIR/net/wireless/Kconfig"
-source "$BACKPORT_DIR/net/mac80211/Kconfig"
-source "$BACKPORT_DIR/net/bluetooth/Kconfig"
-source "$BACKPORT_DIR/drivers/net/wireless/Kconfig"
-source "$BACKPORT_DIR/drivers/net/ethernet/Kconfig"
-source "$BACKPORT_DIR/drivers/net/usb/Kconfig"
-
-source "$BACKPORT_DIR/drivers/ssb/Kconfig"
-source "$BACKPORT_DIR/drivers/bcma/Kconfig"
-
-source "$BACKPORT_DIR/net/nfc/Kconfig"
-
-source "$BACKPORT_DIR/drivers/media/Kconfig"
-
-source "$BACKPORT_DIR/net/ieee802154/Kconfig"
-source "$BACKPORT_DIR/net/mac802154/Kconfig"
-source "$BACKPORT_DIR/drivers/net/ieee802154/Kconfig"
-
-source "$BACKPORT_DIR/drivers/usb/class/Kconfig"
diff --git a/backport/Kconfig.package b/backport/Kconfig.package
new file mode 100644
index 0000000..c507de4
--- /dev/null
+++ b/backport/Kconfig.package
@@ -0,0 +1,32 @@
+mainmenu "Backports from $BACKPORTED_KERNEL_NAME $BACKPORTED_KERNEL_VERSION (backports $BACKPORTS_VERSION)"
+
+config BACKPORT_DIR
+ string
+ option env="BACKPORT_DIR"
+config BACKPORTS_VERSION
+ string
+ option env="BACKPORTS_VERSION"
+config BACKPORTED_KERNEL_VERSION
+ string
+ option env="BACKPORTED_KERNEL_VERSION"
+config BACKPORTED_KERNEL_NAME
+ string
+ option env="BACKPORTED_KERNEL_NAME"
+
+# some hacks for when we use backports to generate a package
+# to build modules out of tree.
+#ignore-parser-package
+config WIRELESS
+ def_bool y
+#ignore-parser-package
+config NET_CORE
+ def_bool y
+#ignore-parser-package
+config EXPERT
+ def_bool y
+
+source "$BACKPORT_DIR/Kconfig.sources"
+
+# these will be generated
+source "$BACKPORT_DIR/Kconfig.kernel"
+source "$BACKPORT_DIR/Kconfig.versions"
diff --git a/backport/Kconfig.sources b/backport/Kconfig.sources
new file mode 100644
index 0000000..cdf993c
--- /dev/null
+++ b/backport/Kconfig.sources
@@ -0,0 +1,23 @@
+# this has the configuration for the backport code
+source "$BACKPORT_DIR/compat/Kconfig"
+
+# these are copied from the kernel
+source "$BACKPORT_DIR/net/wireless/Kconfig"
+source "$BACKPORT_DIR/net/mac80211/Kconfig"
+source "$BACKPORT_DIR/net/bluetooth/Kconfig"
+source "$BACKPORT_DIR/drivers/net/wireless/Kconfig"
+source "$BACKPORT_DIR/drivers/net/ethernet/Kconfig"
+source "$BACKPORT_DIR/drivers/net/usb/Kconfig"
+
+source "$BACKPORT_DIR/drivers/ssb/Kconfig"
+source "$BACKPORT_DIR/drivers/bcma/Kconfig"
+
+source "$BACKPORT_DIR/net/nfc/Kconfig"
+
+source "$BACKPORT_DIR/drivers/media/Kconfig"
+
+source "$BACKPORT_DIR/net/ieee802154/Kconfig"
+source "$BACKPORT_DIR/net/mac802154/Kconfig"
+source "$BACKPORT_DIR/drivers/net/ieee802154/Kconfig"
+
+source "$BACKPORT_DIR/drivers/usb/class/Kconfig"
diff --git a/gentree.py b/gentree.py
index 3d4a8b4..860bb71 100755
--- a/gentree.py
+++ b/gentree.py
@@ -746,11 +746,27 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
check_output_dir(args.outdir, args.clean)
# do the copy
+ backport_package_files = [(x, x) for x in [
+ 'Makefile',
+ 'kconf/',
+ 'Makefile.real',
+ 'Makefile.kernel',
+ 'scripts/',
+ '.blacklist.map',
+ '.gitignore',
+ 'Makefile.build'] ]
+ backport_package_files += [
+ ('Kconfig.package', 'Kconfig'),
+ ]
backport_files = [(x, x) for x in [
- 'Kconfig', 'Makefile', 'Makefile.build', 'Makefile.kernel', '.gitignore',
- 'Makefile.real', 'compat/', 'backport-include/', 'kconf/',
- 'scripts/', '.blacklist.map',
+ 'Kconfig.sources',
+ 'compat/',
+ 'backport-include/',
]]
+
+ if not integrate:
+ backport_files += backport_package_files
+
if not args.git_revision:
logwrite('Copy original source files ...')
else:
--
2.1.1
From: "Luis R. Rodriguez" <[email protected]>
The way we'll define backports versioning information for
kernel integration is slightly different, we'll rely completely
on Kconfig for the job, but in the end share the same C code
defines.
Signed-off-by: Luis R. Rodriguez <[email protected]>
---
gentree.py | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/gentree.py b/gentree.py
index 1b6dc89..55abe6d 100755
--- a/gentree.py
+++ b/gentree.py
@@ -807,6 +807,24 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
f.write('BACKPORTS_GIT_TRACKED="backport tracker ID: $(shell git rev-parse HEAD 2>/dev/null || echo \'not built in git tree\')"\n')
f.close()
git_debug_snapshot(args, "generate kernel version info file: version")
+ else:
+ kconf_regexes = [
+ (re.compile(r'.*(?P<key>%%BACKPORT_DIR%%)'), '%%BACKPORT_DIR%%', 'backports/'),
+ (re.compile(r'.*(?P<key>%%BACKPORTS_VERSION%%).*'), '%%BACKPORTS_VERSION%%', backports_version),
+ (re.compile(r'.*(?P<key>%%BACKPORTED_KERNEL_VERSION%%).*'), '%%BACKPORTED_KERNEL_VERSION%%', kernel_version),
+ (re.compile(r'.*(?P<key>%%BACKPORTED_KERNEL_NAME%%).*'), '%%BACKPORTED_KERNEL_NAME%%', args.base_name),
+ ]
+ out = ''
+ for l in open(os.path.join(args.outdir, 'Kconfig'), 'r'):
+ for r in kconf_regexes:
+ m = r[0].match(l)
+ if m:
+ l = re.sub(r'(' + r[1] + ')', r'' + r[2] + '', l)
+ out += l
+ outf = open(os.path.join(args.outdir, 'Kconfig'), 'w')
+ outf.write(out)
+ outf.close()
+ git_debug_snapshot(args, "modify top level backports/Kconfig with backports identity")
disable_list = add_automatic_backports(args, bp_prefix)
if disable_list:
--
2.1.1
From: "Luis R. Rodriguez" <[email protected]>
In order to help unify the naming scheme for shared
backports versioning information rely on the CPTCFG_
prefix, when integration support gets added that will
translate to the respective CONFIG_BACKPORT_ prefix.
Kconfig opt env entries don't get propagated out, so
we need to define these ourselves. This leaves all
other names in place for packaging and just focuses
on sharing on the C / header code.
Signed-off-by: Luis R. Rodriguez <[email protected]>
---
backport/Makefile.kernel | 6 +++---
backport/backport-include/linux/module.h | 6 +++---
backport/compat/main.c | 32 ++++++++++++++++----------------
3 files changed, 22 insertions(+), 22 deletions(-)
diff --git a/backport/Makefile.kernel b/backport/Makefile.kernel
index dcb2ba7..a63184b 100644
--- a/backport/Makefile.kernel
+++ b/backport/Makefile.kernel
@@ -10,9 +10,9 @@ NOSTDINC_FLAGS := \
-I$(M)/include/uapi \
-include $(M)/backport-include/backport/backport.h \
$(call backport-cc-disable-warning, unused-but-set-variable) \
- -DBACKPORTS_VERSION=\"$(BACKPORTS_VERSION)\" \
- -DBACKPORTED_KERNEL_VERSION=\"$(BACKPORTED_KERNEL_VERSION)\" \
- -DBACKPORTED_KERNEL_NAME=\"$(BACKPORTED_KERNEL_NAME)\" \
+ -DCPTCFG_VERSION=\"$(BACKPORTS_VERSION)\" \
+ -DCPTCFG_KERNEL_VERSION=\"$(BACKPORTED_KERNEL_VERSION)\" \
+ -DCPTCFG_KERNEL_NAME=\"$(BACKPORTED_KERNEL_NAME)\" \
$(BACKPORTS_GIT_TRACKER_DEF) \
$(CFLAGS)
diff --git a/backport/backport-include/linux/module.h b/backport/backport-include/linux/module.h
index e8f28b1..8870abb 100644
--- a/backport/backport-include/linux/module.h
+++ b/backport/backport-include/linux/module.h
@@ -20,9 +20,9 @@ extern void backport_dependency_symbol(void);
#define BACKPORT_MOD_VERSIONS MODULE_VERSION(BACKPORTS_GIT_TRACKED);
#else
#define BACKPORT_MOD_VERSIONS \
- MODULE_VERSION("backported from " BACKPORTED_KERNEL_NAME \
- " (" BACKPORTED_KERNEL_VERSION ")" \
- " using backports " BACKPORTS_VERSION);
+ MODULE_VERSION("backported from " CPTCFG_KERNEL_NAME \
+ " (" CPTCFG_KERNEL_VERSION ")" \
+ " using backports " CPTCFG_VERSION);
#endif
#ifdef MODULE
diff --git a/backport/compat/main.c b/backport/compat/main.c
index d3f8944..04ebbfd 100644
--- a/backport/compat/main.c
+++ b/backport/compat/main.c
@@ -8,40 +8,40 @@ MODULE_AUTHOR("Luis R. Rodriguez");
MODULE_DESCRIPTION("Kernel backport module");
MODULE_LICENSE("GPL");
-#ifndef BACKPORTED_KERNEL_NAME
-#error "You need a BACKPORTED_KERNEL_NAME"
+#ifndef CPTCFG_KERNEL_NAME
+#error "You need a CPTCFG_KERNEL_NAME"
#endif
-#ifndef BACKPORTED_KERNEL_VERSION
-#error "You need a BACKPORTED_KERNEL_VERSION"
+#ifndef CPTCFG_KERNEL_VERSION
+#error "You need a CPTCFG_KERNEL_VERSION"
#endif
-#ifndef BACKPORTS_VERSION
-#error "You need a BACKPORTS_VERSION"
+#ifndef CPTCFG_VERSION
+#error "You need a CPTCFG_VERSION"
#endif
-static char *backported_kernel_name = BACKPORTED_KERNEL_NAME;
+static char *backported_kernel_name = CPTCFG_KERNEL_NAME;
module_param(backported_kernel_name, charp, 0400);
MODULE_PARM_DESC(backported_kernel_name,
- "The kernel tree name that was used for this backport (" BACKPORTED_KERNEL_NAME ")");
+ "The kernel tree name that was used for this backport (" CPTCFG_KERNEL_NAME ")");
-#ifdef BACKPORTS_GIT_TRACKED
+#ifdef BACKPORTS_GIT_TRACKED
static char *backports_tracker_id = BACKPORTS_GIT_TRACKED;
module_param(backports_tracker_id, charp, 0400);
MODULE_PARM_DESC(backports_tracker_id,
"The version of the tree containing this backport (" BACKPORTS_GIT_TRACKED ")");
#else
-static char *backported_kernel_version = BACKPORTED_KERNEL_VERSION;
-static char *backports_version = BACKPORTS_VERSION;
+static char *backported_kernel_version = CPTCFG_KERNEL_VERSION;
+static char *backports_version = CPTCFG_VERSION;
module_param(backported_kernel_version, charp, 0400);
MODULE_PARM_DESC(backported_kernel_version,
- "The kernel version that was used for this backport (" BACKPORTED_KERNEL_VERSION ")");
+ "The kernel version that was used for this backport (" CPTCFG_KERNEL_VERSION ")");
module_param(backports_version, charp, 0400);
MODULE_PARM_DESC(backports_version,
- "The git version of the backports tree used to generate this backport (" BACKPORTS_VERSION ")");
+ "The git version of the backports tree used to generate this backport (" CPTCFG_VERSION ")");
#endif
@@ -63,15 +63,15 @@ static int __init backport_init(void)
return ret;
}
- printk(KERN_INFO "Loading modules backported from " BACKPORTED_KERNEL_NAME
+ printk(KERN_INFO "Loading modules backported from " CPTCFG_KERNEL_NAME
#ifndef BACKPORTS_GIT_TRACKED
- " version " BACKPORTED_KERNEL_VERSION
+ " version " CPTCFG_KERNEL_VERSION
#endif
"\n");
#ifdef BACKPORTS_GIT_TRACKED
printk(KERN_INFO BACKPORTS_GIT_TRACKED "\n");
#else
- printk(KERN_INFO "Backport generated by backports.git " BACKPORTS_VERSION "\n");
+ printk(KERN_INFO "Backport generated by backports.git " CPTCFG_VERSION "\n");
#endif
return 0;
--
2.1.1
From: "Luis R. Rodriguez" <[email protected]>
This enables support for using the backports project
to integrate device drivers from a future version of Linux
into an older version of Linux. What you end up seeing is
a backports submenu when configuring your kernel and the
ability to select specific device drivers from subsystems
supported through the Linux backports project.
At this time enabling one device driver from a future version
of Linux will require using only the latest version of the
subsystem modules and other subsystem drivers. For example
enabling cfg80211 and mac80211 from a future version of Linux
will require you to only use future version of the respective
device drivers. In order to enable the backported version of
802.11 drivers for example, you will have to enable first:
Networking support -->
Wireless -->
But under that menu disable all options, then jump to the backports
submenu to now enable:
Backports -->
cfg80211
mac80211
Wireless LAN --->
etc
You build these device drivers modular or built-in to the kernel.
Integration support requires only slight modifications to the original
kernel sources, one to the top level Kconfig to add our entry, and also
the top level Makefile to enable backports code to be part of the
built-in vmlinux.
Support for integration takes advantage over the existing infrastructure
added by Johannes to keep track of each indvidual change done by the
backports infrastructure if --gitdebug is used.
1 3.0.101 [ OK ]
2 3.1.10 [ OK ]
3 3.2.62 [ OK ]
4 3.3.8 [ OK ]
5 3.4.104 [ OK ]
6 3.5.7 [ OK ]
7 3.6.11 [ OK ]
8 3.7.10 [ OK ]
9 3.8.13 [ OK ]
10 3.9.11 [ OK ]
11 3.10.58 [ OK ]
12 3.11.10 [ OK ]
13 3.12.31 [ OK ]
14 3.13.11 [ OK ]
15 3.14.22 [ OK ]
16 3.15.10 [ OK ]
17 3.16.6 [ OK ]
18 3.17.1 [ OK ]
19 3.18-rc1 [ OK ]
real 3m17.573s
user 67m33.172s
sys 18m27.596s
Signed-off-by: Luis R. Rodriguez <[email protected]>
---
backport/Kconfig.integrate | 36 +++++++++++
backport/Makefile.kernel | 13 ++++
backport/backport-include/backport/backport.h | 5 ++
backport/compat/Makefile | 4 ++
backport/compat/main.c | 8 ++-
devel/doc/kconfig-operation | 5 ++
gentree.py | 69 +++++++++++++++++++---
.../0001-enable-backports-built-in.patch | 40 +++++++++++++
lib/bpversion.py | 48 +++++++++++++++
lib/kconfig.py | 17 ++++++
10 files changed, 236 insertions(+), 9 deletions(-)
create mode 100644 backport/Kconfig.integrate
create mode 100644 integration-patches/0001-enable-backports/0001-enable-backports-built-in.patch
create mode 100644 lib/bpversion.py
diff --git a/backport/Kconfig.integrate b/backport/Kconfig.integrate
new file mode 100644
index 0000000..f64a3f6
--- /dev/null
+++ b/backport/Kconfig.integrate
@@ -0,0 +1,36 @@
+config BACKPORT_INTEGRATE
+ bool
+ def_bool y
+
+config BACKPORT_DIR
+ string
+ default "%%BACKPORT_DIR%%"
+
+config BACKPORT_VERSION
+ string
+ default "%%BACKPORTS_VERSION%%"
+
+config BACKPORT_KERNEL_VERSION
+ string
+ default "%%BACKPORTED_KERNEL_VERSION%%"
+
+config BACKPORT_KERNEL_NAME
+ string
+ default "%%BACKPORTED_KERNEL_NAME%%"
+
+menuconfig BACKPORT_LINUX
+ bool "Backport %%BACKPORTED_KERNEL_NAME%% %%BACKPORTED_KERNEL_VERSION%% (backports %%BACKPORTS_VERSION%%)"
+ default n
+ ---help---
+ Enabling this will let give you the opportunity to use features and
+ drivers backported from %%BACKPORTED_KERNEL_NAME%% %%BACKPORTED_KERNEL_VERSION%%
+ on the kernel your are using. This is experimental and you should
+ say no unless you'd like to help test things or want to help debug
+ this should we run into any issues.
+
+if BACKPORT_LINUX
+
+source "$BACKPORT_DIR/Kconfig.versions"
+source "$BACKPORT_DIR/Kconfig.sources"
+
+endif # BACKPORT_LINUX
diff --git a/backport/Makefile.kernel b/backport/Makefile.kernel
index a63184b..c57b2d3 100644
--- a/backport/Makefile.kernel
+++ b/backport/Makefile.kernel
@@ -1,3 +1,4 @@
+ifeq ($(CONFIG_BACKPORT_INTEGRATE),)
# Since 2.6.21, try-run is available, but cc-disable-warning
# was only added later, so we add it here ourselves:
backport-cc-disable-warning = $(call try-run,\
@@ -17,6 +18,18 @@ NOSTDINC_FLAGS := \
$(CFLAGS)
export backport_srctree = $(M)
+else
+export BACKPORT_DIR = backports/
+export backport_srctree = $(BACKPORT_DIR)
+NOSTDINC_FLAGS := \
+ -I$(BACKPORT_DIR)/backport-include/ \
+ -I$(BACKPORT_DIR)/backport-include/uapi \
+ -I$(BACKPORT_DIR)/include/ \
+ -I$(BACKPORT_DIR)/include/uapi \
+ -include $(BACKPORT_DIR)/backport-include/backport/backport.h \
+ $(CFLAGS)
+endif
+
obj-y += compat/
diff --git a/backport/backport-include/backport/backport.h b/backport/backport-include/backport/backport.h
index 7cf21aa..d1d3b10 100644
--- a/backport/backport-include/backport/backport.h
+++ b/backport/backport-include/backport/backport.h
@@ -1,11 +1,16 @@
#ifndef __BACKPORT_H
#define __BACKPORT_H
+#include <generated/autoconf.h>
+#ifndef CONFIG_BACKPORT_INTEGRATE
#include <backport/autoconf.h>
+#endif
#include <linux/kconfig.h>
#ifndef __ASSEMBLY__
#define LINUX_BACKPORT(__sym) backport_ ##__sym
+#ifndef CONFIG_BACKPORT_INTEGRATE
#include <backport/checks.h>
#endif
+#endif
#endif /* __BACKPORT_H */
diff --git a/backport/compat/Makefile b/backport/compat/Makefile
index e787763..f14b516 100644
--- a/backport/compat/Makefile
+++ b/backport/compat/Makefile
@@ -1,5 +1,9 @@
ccflags-y += -I$(src)
+ifeq ($(CONFIG_BACKPORT_INTEGRATE),)
obj-m += compat.o
+else
+obj-y += compat.o
+endif
compat-y += main.o
# Kernel backport compatibility code
diff --git a/backport/compat/main.c b/backport/compat/main.c
index 04ebbfd..5d45e3d 100644
--- a/backport/compat/main.c
+++ b/backport/compat/main.c
@@ -71,8 +71,14 @@ static int __init backport_init(void)
#ifdef BACKPORTS_GIT_TRACKED
printk(KERN_INFO BACKPORTS_GIT_TRACKED "\n");
#else
+
+#ifdef CONFIG_BACKPORT_INTEGRATE
+ printk(KERN_INFO "Backport integrated by backports.git " CPTCFG_VERSION "\n");
+#else
printk(KERN_INFO "Backport generated by backports.git " CPTCFG_VERSION "\n");
-#endif
+#endif /* CONFIG_BACKPORT_INTEGRATE */
+
+#endif /* BACKPORTS_GIT_TRACKED */
return 0;
}
diff --git a/devel/doc/kconfig-operation b/devel/doc/kconfig-operation
index ddb4de7..35e198c 100644
--- a/devel/doc/kconfig-operation
+++ b/devel/doc/kconfig-operation
@@ -73,6 +73,11 @@ This allows code to, for example, have "#ifdef CONFIG_PM" which can only
be set or cleared in the kernel, not in the backport configuration. Since
this is needed, a transformation step is done at backport creation time.
+When using Linux backports to integrate into an existing Linux tree
+the CONFIG_BACKPORT_ prefix is used, this allows a CONFIG_BACKPORT_
+symbol to depend on the non-backported respective symbol to be selected
+allowing these to be mutually exclusive.
+
Backport creation for Kconfig
-------------------------------
diff --git a/gentree.py b/gentree.py
index 989b6ea..6b10857 100755
--- a/gentree.py
+++ b/gentree.py
@@ -16,6 +16,7 @@ from lib import bpgpg as gpg
from lib import bpkup as kup
from lib.tempdir import tempdir
from lib import bpreqs as reqs
+from lib import bpversion as gen_version
def read_copy_list(copyfile):
"""
@@ -607,6 +608,9 @@ def _main():
'and we use git ls-tree to get the files.')
parser.add_argument('--clean', const=True, default=False, action="store_const",
help='Clean output directory instead of erroring if it isn\'t empty')
+ parser.add_argument('--integrate', const=True, default=False, action="store_const",
+ help='Integrate a future backported kernel solution into ' +
+ 'an older kernel tree source directory.')
parser.add_argument('--refresh', const=True, default=False, action="store_const",
help='Refresh patches as they are applied, the source dir will be modified!')
parser.add_argument('--base-name', metavar='<name>', type=str, default='Linux',
@@ -637,13 +641,18 @@ def _main():
'of changes done by Coccinelle.')
args = parser.parse_args()
+ if args.integrate:
+ args.outdir = os.path.join(args.outdir, 'backports/')
+
def logwrite(msg):
sys.stdout.write(msg)
sys.stdout.write('\n')
sys.stdout.flush()
return process(args.kerneldir, args.outdir, args.copy_list,
- git_revision=args.git_revision, clean=args.clean,
+ git_revision=args.git_revision,
+ integrate=args.integrate,
+ clean=args.clean,
refresh=args.refresh, base_name=args.base_name,
gitdebug=args.gitdebug, verbose=args.verbose,
extra_driver=args.extra_driver,
@@ -654,6 +663,7 @@ def _main():
logwrite=logwrite)
def process(kerneldir, outdir, copy_list_file, git_revision=None,
+ integrate=False,
clean=False, refresh=False, base_name="Linux", gitdebug=False,
verbose=False, extra_driver=[], kup=False,
kup_test=False,
@@ -663,7 +673,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
git_tracked_version=False):
class Args(object):
def __init__(self, kerneldir, outdir, copy_list_file,
- git_revision, clean, refresh, base_name,
+ git_revision, integrate, clean, refresh, base_name,
gitdebug, verbose, extra_driver, kup,
kup_test,
test_cocci,
@@ -672,6 +682,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
self.outdir = outdir
self.copy_list = copy_list_file
self.git_revision = git_revision
+ self.integrate = integrate
self.clean = clean
self.refresh = refresh
self.base_name = base_name
@@ -694,11 +705,15 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
logwrite('Validated tree: %s' % tree)
args = Args(kerneldir, outdir, copy_list_file,
- git_revision, clean, refresh, base_name,
+ git_revision, integrate, clean, refresh, base_name,
gitdebug, verbose, extra_driver, kup, kup_test,
test_cocci, profile_cocci)
rel_prep = None
- integrate = False
+
+ if args.integrate:
+ if args.kup_test or args.test_cocci or args.profile_cocci or args.refresh:
+ logwrite('Cannot use integration with:\n\tkup_test\n\ttest_cocci\n\tprofile_cocci\n\trefresh\n');
+ sys.exit(1)
# When building a package we use CPTCFG as we can rely on the
# fact that kconfig treats CONFIG_ as an environment variable
@@ -746,6 +761,10 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
check_output_dir(args.outdir, args.clean)
# do the copy
+ backport_integrate_files = [
+ ('Makefile.kernel', 'Makefile'),
+ ('Kconfig.integrate', 'Kconfig'),
+ ]
backport_package_files = [(x, x) for x in [
'Makefile',
'kconf/',
@@ -764,8 +783,10 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
'backport-include/',
]]
- if not integrate:
+ if not args.integrate:
backport_files += backport_package_files
+ else:
+ backport_files += backport_integrate_files
if not args.git_revision:
logwrite('Copy original source files ...')
@@ -774,7 +795,8 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
copy_files(os.path.join(source_dir, 'backport'), backport_files, args.outdir)
- git_debug_init(args)
+ if not args.integrate:
+ git_debug_init(args)
if not args.git_revision:
copy_files(args.kerneldir, copy_list, args.outdir)
@@ -834,6 +856,21 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
apply_patches(args, "backport", source_dir, 'patches', args.outdir, logwrite)
+ # Kernel integration requires Kconfig.versions already generated for you,
+ # we cannot do this for a package as we have no idea what kernel folks
+ # will be using.
+ if args.integrate:
+ kver = gen_version.kernelversion(os.path.join(args.outdir, "../"))
+ rel_specs = gen_version.get_rel_spec_stable(kver)
+ if not rel_specs:
+ logwrite('Cannot parse source kernel version, update parser')
+ sys.exit(1)
+ data = gen_version.genkconfig_versions(rel_specs)
+ fo = open(os.path.join(args.outdir, 'Kconfig.versions'), 'w')
+ fo.write(data)
+ fo.close()
+ git_debug_snapshot(args, "generate kernel version requirement Kconfig file")
+
# some post-processing is required
configtree = kconfig.ConfigTree(os.path.join(args.outdir, 'Kconfig'))
orig_symbols = configtree.symbols()
@@ -854,7 +891,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
symbols = configtree.symbols()
- if not integrate:
+ if not args.integrate:
# write local symbol list -- needed during build
f = open(os.path.join(args.outdir, '.local-symbols'), 'w')
for sym in symbols:
@@ -892,6 +929,8 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
for some_symbols in [all_symbols[i:i + 50] for i in range(0, len(all_symbols), 50)]:
r = 'CONFIG_((?!BACKPORT)(' + '|'.join([s + '(_MODULE)?' for s in some_symbols]) + ')([^A-Za-z0-9_]|$))'
regexes.append(re.compile(r, re.MULTILINE))
+ # Replace default prefix
+ reg = re.compile(r'(CPTCFG_)', re.MULTILINE)
for root, dirs, files in os.walk(args.outdir):
# don't go into .git dir (possible debug thing)
if '.git' in dirs:
@@ -900,6 +939,8 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
data = open(os.path.join(root, f), 'r').read()
for r in regexes:
data = r.sub(r'' + bp_prefix + '\\1', data)
+ if bp_prefix != 'CPTCFG_':
+ data = reg.sub(r'' + bp_prefix + '', data)
data = re.sub(r'\$\(srctree\)', '$(backport_srctree)', data)
data = re.sub(r'-Idrivers', '-I$(backport_srctree)/drivers', data)
fo = open(os.path.join(root, f), 'w')
@@ -909,7 +950,10 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
git_debug_snapshot(args, "rename config symbol / srctree usage")
# disable unbuildable Kconfig symbols and stuff Makefiles that doesn't exist
- maketree = make.MakeTree(os.path.join(args.outdir, 'Makefile.kernel'))
+ if args.integrate:
+ maketree = make.MakeTree(os.path.join(args.outdir, 'Makefile'))
+ else:
+ maketree = make.MakeTree(os.path.join(args.outdir, 'Makefile.kernel'))
disable_kconfig = []
disable_makefile = []
for sym in maketree.get_impossible_symbols():
@@ -956,6 +1000,15 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
fo.close()
git_debug_snapshot(args, "disable unsatisfied Makefile parts")
+ if args.integrate:
+ f = open(os.path.join(args.outdir, '../Kconfig'), 'a')
+ f.write('source "backports/Kconfig"\n')
+ f.close()
+ git_debug_snapshot(args, "hooked backport to top level Kconfig")
+
+ apply_patches(args, "integration", source_dir, 'integration-patches/',
+ os.path.join(args.outdir, '../'), logwrite)
+
if (args.kup or args.kup_test):
req = reqs.Req()
req.kup()
diff --git a/integration-patches/0001-enable-backports/0001-enable-backports-built-in.patch b/integration-patches/0001-enable-backports/0001-enable-backports-built-in.patch
new file mode 100644
index 0000000..d66b203
--- /dev/null
+++ b/integration-patches/0001-enable-backports/0001-enable-backports-built-in.patch
@@ -0,0 +1,40 @@
+Allow backports to be integrated into vmlinux.
+
+diff --git a/Makefile b/Makefile
+index 6d1e304..de26b18 100644
+--- a/Makefile
++++ b/Makefile
+@@ -542,6 +542,7 @@ scripts: scripts_basic include/config/auto.conf include/config/tristate.conf \
+ $(Q)$(MAKE) $(build)=$(@)
+
+ # Objects we will link into vmlinux / subdirs we need to visit
++backports-y := backports/
+ init-y := init/
+ drivers-y := drivers/ sound/ firmware/
+ net-y := net/
+@@ -820,13 +821,16 @@ core-y += kernel/ mm/ fs/ ipc/ security/ crypto/ block/
+
+ vmlinux-dirs := $(patsubst %/,%,$(filter %/, $(init-y) $(init-m) \
+ $(core-y) $(core-m) $(drivers-y) $(drivers-m) \
++ $(backports-y) $(backports-m) \
+ $(net-y) $(net-m) $(libs-y) $(libs-m)))
+
+ vmlinux-alldirs := $(sort $(vmlinux-dirs) $(patsubst %/,%,$(filter %/, \
+ $(init-n) $(init-) \
+ $(core-n) $(core-) $(drivers-n) $(drivers-) \
++ $(backports-n) $(backports-) \
+ $(net-n) $(net-) $(libs-n) $(libs-))))
+
++backports-y := $(patsubst %/, %/built-in.o, $(backports-y))
+ init-y := $(patsubst %/, %/built-in.o, $(init-y))
+ core-y := $(patsubst %/, %/built-in.o, $(core-y))
+ drivers-y := $(patsubst %/, %/built-in.o, $(drivers-y))
+@@ -837,7 +841,7 @@ libs-y := $(libs-y1) $(libs-y2)
+
+ # Externally visible symbols (used by link-vmlinux.sh)
+ export KBUILD_VMLINUX_INIT := $(head-y) $(init-y)
+-export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y) $(drivers-y) $(net-y)
++export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y) $(drivers-y) $(net-y) $(backports-y)
+ export KBUILD_LDS := arch/$(SRCARCH)/kernel/vmlinux.lds
+ export LDFLAGS_vmlinux
+ # used by scripts/pacmage/Makefile
diff --git a/lib/bpversion.py b/lib/bpversion.py
new file mode 100644
index 0000000..aefdcf0
--- /dev/null
+++ b/lib/bpversion.py
@@ -0,0 +1,48 @@
+import subprocess, os, re
+
+class VersionError(Exception):
+ pass
+class ExecutionError(VersionError):
+ def __init__(self, errcode):
+ self.error_code = errcode
+
+def _check(process):
+ if process.returncode != 0:
+ raise ExecutionError(process.returncode)
+
+def get_rel_spec_stable(rel):
+ """
+ Returns release specs for a linux-stable backports based release.
+ """
+ m = None
+ if ("rc" in rel):
+ m = re.match(r"v*(?P<VERSION>\d+)\.+" \
+ "(?P<PATCHLEVEL>\d+)[.]*" \
+ "(?P<SUBLEVEL>\d*)" \
+ "[-rc]+(?P<RC_VERSION>\d+)",
+ rel)
+ else:
+ m = re.match(r"(?P<VERSION>\d+)\.+" \
+ "(?P<PATCHLEVEL>\d+)[.]*" \
+ "(?P<SUBLEVEL>\d*)",
+ rel)
+ if (not m):
+ return m
+ return m.groupdict()
+
+def kernelversion(tree):
+ cmd = ['make', '--no-print-directory', '-C', tree, 'kernelversion' ]
+ process = subprocess.Popen(cmd,
+ stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
+ close_fds=True, universal_newlines=True)
+ stdout = process.communicate()[0]
+ process.wait()
+ _check(process)
+ return stdout.strip()
+
+def genkconfig_versions(rel_specs):
+ data = ''
+ for i in range(int(rel_specs['PATCHLEVEL']) + 1, 99):
+ data += "config BACKPORT_KERNEL_%s_%s\n" % (rel_specs['VERSION'], i)
+ data += " def_bool y\n"
+ return data
diff --git a/lib/kconfig.py b/lib/kconfig.py
index 430b642..fd11ea1 100644
--- a/lib/kconfig.py
+++ b/lib/kconfig.py
@@ -12,6 +12,10 @@ cfg_line = re.compile(r'^(?P<opt>config|menuconfig)\s+(?P<sym>[^\s]*)')
sel_line = re.compile(r'^(?P<spc>\s+)select\s+(?P<sym>[^\s]*)\s*$')
backport_line = re.compile(r'^\s+#(?P<key>[ch]-file|module-name)\s*(?P<name>.*)')
ignore_parse_p = re.compile(r'^\s*#(?P<key>ignore-parser-package)')
+kernel_version_config = re.compile(r'^(?P<opt>config)\s+(?P<sym>BACKPORT_KERNEL_[\d])')
+ignore_bp_syms = 'BACKPORT_DIR|BACKPORT_VERSION|BACKPORT_KERNEL_VERSION|BACKPORT_KERNEL_NAME|BACKPORT_LINUX|BACKPORT_INTEGRATE'
+ignore_syms = re.compile(r'^(?P<opt>config|menuconfig)\s+(?P<sym>' + ignore_bp_syms + ')')
+ignore_cond_syms = re.compile(r'^(?P<cond>if|endif)\s*#*\s*(?P<sym>' + ignore_bp_syms + ')')
class ConfigTree(object):
def __init__(self, rootfile):
@@ -107,6 +111,19 @@ class ConfigTree(object):
if pm:
ignore_parse_modular = True
continue
+ vm = kernel_version_config.match(l)
+ if vm:
+ out += l
+ continue
+ cm = ignore_cond_syms.match(l)
+ if cm:
+ out += l
+ ignore_parse_modular = False
+ continue
+ ig = ignore_syms.match(l)
+ if ig:
+ out += l
+ continue
n = cfg_line.match(l)
if n:
m = n
--
2.1.1
From: "Luis R. Rodriguez" <[email protected]>
The CPTCFG_ prefix already implies backport, when integration
is used we'd end up with a double BACKPORT_ prefix, so just
remove the existing one as its not needed.
Signed-off-by: Luis R. Rodriguez <[email protected]>
---
backport/Makefile | 2 +-
backport/compat/Kconfig | 10 +++----
backport/compat/Makefile | 34 +++++++++++-----------
devel/doc/kconfig-operation | 6 ++--
gentree.py | 2 +-
.../media/0002-no_dmabuf/v4l2.patch | 6 ++--
6 files changed, 30 insertions(+), 30 deletions(-)
diff --git a/backport/Makefile b/backport/Makefile
index fcf2f01..c21b813 100644
--- a/backport/Makefile
+++ b/backport/Makefile
@@ -93,7 +93,7 @@ mrproper:
print=0 ;\
for v in $$kvers ; do \
if [ "$$print" = "1" ] ; then \
- echo config BACKPORT_KERNEL_$$(echo $$v | tr . _) ;\
+ echo config KERNEL_$$(echo $$v | tr . _) ;\
echo " def_bool y" ;\
fi ;\
if [ "$$v" = "$$kver" ] ; then print=1 ; fi ;\
diff --git a/backport/compat/Kconfig b/backport/compat/Kconfig
index 06ef8d5..aadf246 100644
--- a/backport/compat/Kconfig
+++ b/backport/compat/Kconfig
@@ -16,14 +16,14 @@
#
# # not possible on kernel < X.Y, build will fail if any
# # drivers are allowed to build on kernels < X.Y
-# depends on BACKPORT_KERNEL_X_Y
+# depends on KERNEL_X_Y
#
# # don't build the backport code if FOO is in the kernel
# # already, but only if the kernel version is also >= X.Z;
# # this is an example of backporting where the version of
# # the FOO subsystem that we need is only available from
# # kernel version X.Z
-# depends on !FOO || BACKPORT_KERNEL_X_Z
+# depends on !FOO || KERNEL_X_Z
#
# # build if driver needs it (it selects BPAUTO_FOO)
# default m if BPAUTO_FOO
@@ -43,7 +43,7 @@
#
# config BPAUTO_BUILD_KFIFO
# def_bool y
-# depends on BACKPORT_KERNEL_2_6_36
+# depends on KERNEL_2_6_36
#
#
# C) similarly, a kconfig symbol for an option, e.g.
@@ -83,9 +83,9 @@ config BPAUTO_BUILD_DMA_SHARED_HELPERS
depends on HAS_DMA
# Build on other kernels kernels < 3.9 if HAVE_GENERIC_DMA_COHERENT is
# not set. Kernels >= 3.8 have this if HAS_DMA is set.
- depends on (!HAVE_GENERIC_DMA_COHERENT || BACKPORT_KERNEL_3_9)
+ depends on (!HAVE_GENERIC_DMA_COHERENT || KERNEL_3_9)
# Always build if on 3.3 - 3.5
- default y if (BACKPORT_KERNEL_3_4 || BACKPORT_KERNEL_3_5 || BACKPORT_KERNEL_3_6)
+ default y if (KERNEL_3_4 || KERNEL_3_5 || KERNEL_3_6)
# build for testing
default y if BPAUTO_USERSEL_BUILD_ALL
diff --git a/backport/compat/Makefile b/backport/compat/Makefile
index 0dd69fe..e787763 100644
--- a/backport/compat/Makefile
+++ b/backport/compat/Makefile
@@ -3,23 +3,23 @@ obj-m += compat.o
compat-y += main.o
# Kernel backport compatibility code
-compat-$(CPTCFG_BACKPORT_KERNEL_3_0) += compat-3.0.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_1) += compat-3.1.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_2) += backport-3.2.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_3) += compat-3.3.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_4) += compat-3.4.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_5) += compat-3.5.o user_namespace.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_6) += compat-3.6.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_7) += compat-3.7.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_8) += compat-3.8.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_9) += compat-3.9.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_10) += backport-3.10.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_12) += backport-3.12.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_13) += backport-3.13.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_14) += backport-3.14.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_15) += backport-3.15.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_17) += backport-3.17.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_18) += backport-3.18.o
+compat-$(CPTCFG_KERNEL_3_0) += compat-3.0.o
+compat-$(CPTCFG_KERNEL_3_1) += compat-3.1.o
+compat-$(CPTCFG_KERNEL_3_2) += backport-3.2.o
+compat-$(CPTCFG_KERNEL_3_3) += compat-3.3.o
+compat-$(CPTCFG_KERNEL_3_4) += compat-3.4.o
+compat-$(CPTCFG_KERNEL_3_5) += compat-3.5.o user_namespace.o
+compat-$(CPTCFG_KERNEL_3_6) += compat-3.6.o
+compat-$(CPTCFG_KERNEL_3_7) += compat-3.7.o
+compat-$(CPTCFG_KERNEL_3_8) += compat-3.8.o
+compat-$(CPTCFG_KERNEL_3_9) += compat-3.9.o
+compat-$(CPTCFG_KERNEL_3_10) += backport-3.10.o
+compat-$(CPTCFG_KERNEL_3_12) += backport-3.12.o
+compat-$(CPTCFG_KERNEL_3_13) += backport-3.13.o
+compat-$(CPTCFG_KERNEL_3_14) += backport-3.14.o
+compat-$(CPTCFG_KERNEL_3_15) += backport-3.15.o
+compat-$(CPTCFG_KERNEL_3_17) += backport-3.17.o
+compat-$(CPTCFG_KERNEL_3_18) += backport-3.18.o
compat-$(CPTCFG_BPAUTO_BUILD_CRYPTO_CCM) += crypto-ccm.o
compat-$(CPTCFG_BPAUTO_BUILD_DMA_SHARED_HELPERS) += dma-shared-helpers.o
diff --git a/devel/doc/kconfig-operation b/devel/doc/kconfig-operation
index f1ecf60..ddb4de7 100644
--- a/devel/doc/kconfig-operation
+++ b/devel/doc/kconfig-operation
@@ -53,12 +53,12 @@ The second file (Kconfig.versions) is generated from just the version of
the kernel and also contains invisible and unselectable boolean options
like
-config BACKPORT_KERNEL_X
+config KERNEL_X
def_bool y
to indicate that APIs introduced in kernel version X (e.g. 3_3 for 3.3
or 2_6_24 for 2.6.24) must be backported. Essentially, the presence of
-a symbol BACKPORT_KERNEL_X indicates that the kernel that the code is
+a symbol KERNEL_X indicates that the kernel that the code is
being compiled against is older than X.
All together, this allows the correct options to be selected by the user.
@@ -121,7 +121,7 @@ These are the problems and their solutions:
requires at least 2.6.29 due to the shash crypto code. This cannot be
determined easily automatically, so the 'dependencies' file is read
and options that are listed there are rewritten to include
- depends on !BACKPORT_KERNEL_X_Y
+ depends on !KERNEL_X_Y
(where X_Y is the listed kernel version), which makes them available
only on kernels >= X.Y.
diff --git a/gentree.py b/gentree.py
index e0618e5..989b6ea 100755
--- a/gentree.py
+++ b/gentree.py
@@ -930,7 +930,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
elif (dep == "DISABLE"):
new.append('BACKPORT_DISABLED_KCONFIG_OPTION')
else:
- new.append('!BACKPORT_KERNEL_%s' % dep.replace('.', '_'))
+ new.append('!KERNEL_%s' % dep.replace('.', '_'))
deplist[sym] = new
configtree.add_dependencies(deplist)
git_debug_snapshot(args, "add kernel version dependencies")
diff --git a/patches/collateral-evolutions/media/0002-no_dmabuf/v4l2.patch b/patches/collateral-evolutions/media/0002-no_dmabuf/v4l2.patch
index 02d042a..7f88c3e 100644
--- a/patches/collateral-evolutions/media/0002-no_dmabuf/v4l2.patch
+++ b/patches/collateral-evolutions/media/0002-no_dmabuf/v4l2.patch
@@ -5,7 +5,7 @@
# Used by drivers that need Videobuf2 modules
config VIDEOBUF2_CORE
- select DMA_SHARED_BUFFER
-+ select DMA_SHARED_BUFFER if !BACKPORT_KERNEL_3_5
++ select DMA_SHARED_BUFFER if !KERNEL_3_5
tristate
config VIDEOBUF2_MEMOPS
@@ -14,14 +14,14 @@
select VIDEOBUF2_CORE
select VIDEOBUF2_MEMOPS
- select DMA_SHARED_BUFFER
-+ select DMA_SHARED_BUFFER if !BACKPORT_KERNEL_3_5
++ select DMA_SHARED_BUFFER if !KERNEL_3_5
config VIDEOBUF2_VMALLOC
tristate
select VIDEOBUF2_CORE
select VIDEOBUF2_MEMOPS
- select DMA_SHARED_BUFFER
-+ select DMA_SHARED_BUFFER if !BACKPORT_KERNEL_3_5
++ select DMA_SHARED_BUFFER if !KERNEL_3_5
config VIDEOBUF2_DMA_SG
tristate
--
2.1.1
From: "Luis R. Rodriguez" <[email protected]>
Things we backport are now prefixed with BACKPORT_, when we copy
over libraries with the c-files / h-files trick on the compat/Kconfig
trick we prefix these with BACKPORT_ already so in order to help
distinguish them use BACKPORT_BPAUTO_ for them and prevent a double
BACKPORT_ prefix.
Signed-off-by: Luis R. Rodriguez <[email protected]>
---
backport/backport-include/asm/dma-mapping.h | 4 +-
backport/backport-include/backport/leds-disabled.h | 2 +-
backport/compat/Kconfig | 74 +++++++++++-----------
backport/compat/Makefile | 4 +-
backport/compat/backports.h | 4 +-
gentree.py | 2 +-
patches/backport-adjustments/devcoredump.patch | 4 +-
7 files changed, 47 insertions(+), 47 deletions(-)
diff --git a/backport/backport-include/asm/dma-mapping.h b/backport/backport-include/asm/dma-mapping.h
index 844fe3b..b73b7da 100644
--- a/backport/backport-include/asm/dma-mapping.h
+++ b/backport/backport-include/asm/dma-mapping.h
@@ -3,12 +3,12 @@
#include_next <asm/dma-mapping.h>
#include <linux/version.h>
-#if defined(CPTCFG_BACKPORT_BUILD_DMA_SHARED_HELPERS)
+#if defined(CPTCFG_BPAUTO_BUILD_DMA_SHARED_HELPERS)
#define dma_common_get_sgtable LINUX_BACKPORT(dma_common_get_sgtable)
int
dma_common_get_sgtable(struct device *dev, struct sg_table *sgt,
void *cpu_addr, dma_addr_t dma_addr, size_t size);
-#endif /* defined(CPTCFG_BACKPORT_BUILD_DMA_SHARED_HELPERS) */
+#endif /* defined(CPTCFG_BPAUTO_BUILD_DMA_SHARED_HELPERS) */
#if LINUX_VERSION_CODE < KERNEL_VERSION(3,6,0)
diff --git a/backport/backport-include/backport/leds-disabled.h b/backport/backport-include/backport/leds-disabled.h
index 501f2a0..156d7fa 100644
--- a/backport/backport-include/backport/leds-disabled.h
+++ b/backport/backport-include/backport/leds-disabled.h
@@ -8,7 +8,7 @@
* allows compilation.
*/
-#ifdef CPTCFG_BACKPORT_BUILD_LEDS
+#ifdef CPTCFG_BPAUTO_BUILD_LEDS
#include <linux/list.h>
#include <linux/spinlock.h>
#include <linux/rwsem.h>
diff --git a/backport/compat/Kconfig b/backport/compat/Kconfig
index 49699bd..06ef8d5 100644
--- a/backport/compat/Kconfig
+++ b/backport/compat/Kconfig
@@ -1,16 +1,16 @@
#
# backport Kconfig
#
-# Some options are user-selectable ("BACKPORT_USERSEL_*")
+# Some options are user-selectable ("BPAUTO_USERSEL_*")
#
# Most options, however, follow a few different schemes:
#
# A) An option that is selected by drivers ("select FOO") will be
-# changed to "select BACKPORT_FOO" (if the option BACKPORT_FOO
-# exists). The option BACKPORT_FOO then controls setting of the
-# BACKPORT_BUILD_FOO option, which is a module, like this:
+# changed to "select BPAUTO_FOO" (if the option BPAUTO_FOO
+# exists). The option BPAUTO_FOO then controls setting of the
+# BPAUTO_BUILD_FOO option, which is a module, like this:
#
-# config BACKPORT_BUILD_FOO
+# config BPAUTO_BUILD_FOO
# tristate
# # or bool
#
@@ -25,13 +25,13 @@
# # kernel version X.Z
# depends on !FOO || BACKPORT_KERNEL_X_Z
#
-# # build if driver needs it (it selects BACKPORT_FOO)
-# default m if BACKPORT_FOO
+# # build if driver needs it (it selects BPAUTO_FOO)
+# default m if BPAUTO_FOO
#
-# # or for build-testing (BACKPORT_USERSEL_BUILD_ALL is enabled)
-# default m if BACKPORT_USERSEL_BUILD_ALL
+# # or for build-testing (BPAUTO_USERSEL_BUILD_ALL is enabled)
+# default m if BPAUTO_USERSEL_BUILD_ALL
#
-# config BACKPORT_FOO
+# config BPAUTO_FOO
# bool
#
# This only works as-is if the kernel code is usable on any version,
@@ -41,44 +41,44 @@
# B) An option for code always present on some kernels (e.g. KFIFO).
# This simply depends on/sets the default based on the version:
#
-# config BACKPORT_BUILD_KFIFO
+# config BPAUTO_BUILD_KFIFO
# def_bool y
# depends on BACKPORT_KERNEL_2_6_36
#
#
# C) similarly, a kconfig symbol for an option, e.g.
-# BACKPORT_OPTION_SOME_FIX (no examples provided) check git log
+# BPAUTO_OPTION_SOME_FIX (no examples provided) check git log
#
#
# Variations are obviously possible.
#
-config BACKPORT_BUILD_CORDIC
+config BPAUTO_BUILD_CORDIC
tristate
depends on !CORDIC
- default m if BACKPORT_CORDIC
- default m if BACKPORT_USERSEL_BUILD_ALL
+ default m if BPAUTO_CORDIC
+ default m if BPAUTO_USERSEL_BUILD_ALL
#module-name cordic
#c-file lib/cordic.c
-config BACKPORT_CORDIC
+config BPAUTO_CORDIC
bool
-config BACKPORT_BUILD_AVERAGE
+config BPAUTO_BUILD_AVERAGE
bool
depends on !AVERAGE
- default y if BACKPORT_USERSEL_BUILD_ALL
- default y if BACKPORT_AVERAGE
+ default y if BPAUTO_USERSEL_BUILD_ALL
+ default y if BPAUTO_AVERAGE
#h-file linux/average.h
#c-file lib/average.c
-config BACKPORT_AVERAGE
+config BPAUTO_AVERAGE
bool
-config BACKPORT_MII
+config BPAUTO_MII
bool
-config BACKPORT_BUILD_DMA_SHARED_HELPERS
+config BPAUTO_BUILD_DMA_SHARED_HELPERS
bool
depends on HAS_DMA
# Build on other kernels kernels < 3.9 if HAVE_GENERIC_DMA_COHERENT is
@@ -87,25 +87,25 @@ config BACKPORT_BUILD_DMA_SHARED_HELPERS
# Always build if on 3.3 - 3.5
default y if (BACKPORT_KERNEL_3_4 || BACKPORT_KERNEL_3_5 || BACKPORT_KERNEL_3_6)
# build for testing
- default y if BACKPORT_USERSEL_BUILD_ALL
+ default y if BPAUTO_USERSEL_BUILD_ALL
-config BACKPORT_BUILD_LEDS
+config BPAUTO_BUILD_LEDS
bool
depends on !NEW_LEDS || LEDS_CLASS=n || !LEDS_TRIGGERS
- default y if BACKPORT_NEW_LEDS
- default y if BACKPORT_LEDS_CLASS
- default y if BACKPORT_LEDS_TRIGGERS
+ default y if BPAUTO_NEW_LEDS
+ default y if BPAUTO_LEDS_CLASS
+ default y if BPAUTO_LEDS_TRIGGERS
-config BACKPORT_NEW_LEDS
+config BPAUTO_NEW_LEDS
bool
-config BACKPORT_LEDS_CLASS
+config BPAUTO_LEDS_CLASS
bool
-config BACKPORT_LEDS_TRIGGERS
+config BPAUTO_LEDS_TRIGGERS
bool
-config BACKPORT_USERSEL_BUILD_ALL
+config BPAUTO_USERSEL_BUILD_ALL
bool "Build all compat code"
help
This option selects all the compat code options
@@ -114,24 +114,24 @@ config BACKPORT_USERSEL_BUILD_ALL
It's only really useful for compat testing, so
you probably shouldn't enable it.
-config BACKPORT_CRYPTO_CCM
+config BPAUTO_CRYPTO_CCM
depends on CRYPTO_AEAD
depends on CRYPTO_CTR
bool
-config BACKPORT_BUILD_CRYPTO_CCM
+config BPAUTO_BUILD_CRYPTO_CCM
bool
default n if CRYPTO_CCM
- default y if BACKPORT_CRYPTO_CCM
+ default y if BPAUTO_CRYPTO_CCM
#c-file crypto/ccm.c
-config BACKPORT_WANT_DEV_COREDUMP
+config BPAUTO_WANT_DEV_COREDUMP
bool
-config BACKPORT_BUILD_WANT_DEV_COREDUMP
+config BPAUTO_BUILD_WANT_DEV_COREDUMP
bool
default n if DEV_COREDUMP
default n if DISABLE_DEV_COREDUMP
- default y if BACKPORT_WANT_DEV_COREDUMP
+ default y if BPAUTO_WANT_DEV_COREDUMP
#h-file linux/devcoredump.h
#c-file drivers/base/devcoredump.c
diff --git a/backport/compat/Makefile b/backport/compat/Makefile
index 6d210b0..0dd69fe 100644
--- a/backport/compat/Makefile
+++ b/backport/compat/Makefile
@@ -21,5 +21,5 @@ compat-$(CPTCFG_BACKPORT_KERNEL_3_15) += backport-3.15.o
compat-$(CPTCFG_BACKPORT_KERNEL_3_17) += backport-3.17.o
compat-$(CPTCFG_BACKPORT_KERNEL_3_18) += backport-3.18.o
-compat-$(CPTCFG_BACKPORT_BUILD_CRYPTO_CCM) += crypto-ccm.o
-compat-$(CPTCFG_BACKPORT_BUILD_DMA_SHARED_HELPERS) += dma-shared-helpers.o
+compat-$(CPTCFG_BPAUTO_BUILD_CRYPTO_CCM) += crypto-ccm.o
+compat-$(CPTCFG_BPAUTO_BUILD_DMA_SHARED_HELPERS) += dma-shared-helpers.o
diff --git a/backport/compat/backports.h b/backport/compat/backports.h
index c9094ac..323c908 100644
--- a/backport/compat/backports.h
+++ b/backport/compat/backports.h
@@ -3,7 +3,7 @@
#include <linux/version.h>
-#ifdef CPTCFG_BACKPORT_BUILD_CRYPTO_CCM
+#ifdef CPTCFG_BPAUTO_BUILD_CRYPTO_CCM
int crypto_ccm_module_init(void);
void crypto_ccm_module_exit(void);
#else
@@ -13,7 +13,7 @@ static inline void crypto_ccm_module_exit(void)
{}
#endif
-#ifdef CPTCFG_BACKPORT_BUILD_WANT_DEV_COREDUMP
+#ifdef CPTCFG_BPAUTO_WANT_DEV_COREDUMP
int devcoredump_init(void);
void devcoredump_exit(void);
#else
diff --git a/gentree.py b/gentree.py
index 55abe6d..e0618e5 100755
--- a/gentree.py
+++ b/gentree.py
@@ -205,7 +205,7 @@ def add_automatic_backports(args, bp_prefix):
configtree = kconfig.ConfigTree(os.path.join(args.outdir, 'Kconfig'))
all_selects = configtree.all_selects()
for sym, vals in bpi.items():
- if sym.startswith('BACKPORT_BUILD_'):
+ if sym.startswith('BPAUTO_BUILD_'):
if not sym[15:] in all_selects:
disable_list.append(sym)
continue
diff --git a/patches/backport-adjustments/devcoredump.patch b/patches/backport-adjustments/devcoredump.patch
index 3378884..3bf35f9 100644
--- a/patches/backport-adjustments/devcoredump.patch
+++ b/patches/backport-adjustments/devcoredump.patch
@@ -107,7 +107,7 @@ index c0a360e99f64..da20e61f6c06 100644
#include <linux/vmalloc.h>
-#ifdef CONFIG_DEV_COREDUMP
-+#ifdef CPTCFG_BACKPORT_BUILD_WANT_DEV_COREDUMP
++#ifdef CPTCFG_BPAUTO_WANT_DEV_COREDUMP
void dev_coredumpv(struct device *dev, const void *data, size_t datalen,
gfp_t gfp);
@@ -116,6 +116,6 @@ index c0a360e99f64..da20e61f6c06 100644
free(data);
}
-#endif /* CONFIG_DEV_COREDUMP */
-+#endif /* CPTCFG_BACKPORT_BUILD_WANT_DEV_COREDUMP */
++#endif /* CPTCFG_BPAUTO_WANT_DEV_COREDUMP */
#endif /* __DEVCOREDUMP_H */
--
2.1.1
From: "Luis R. Rodriguez" <[email protected]>
The version file contains:
* BACKPORTS_VERSION
* BACKPORTED_KERNEL_VERSION
* BACKPORTED_KERNEL_NAME
This file is used by the Makefile to export variable
definitions used to identify the backport. For kernel
integration we'll be using a generated Kconfig file
instead and since these are needed when processing
kconfig logic this will need to be defined earlier.
Just move the generation of the version file for packaging
up early to match where we will generate the Kconfig with
the backports versioning identification.
This change has no real functional change other than the
move of some code running earlier.
Signed-off-by: Luis R. Rodriguez <[email protected]>
---
gentree.py | 50 ++++++++++++++++++++++++++------------------------
1 file changed, 26 insertions(+), 24 deletions(-)
diff --git a/gentree.py b/gentree.py
index 860bb71..1b6dc89 100755
--- a/gentree.py
+++ b/gentree.py
@@ -789,6 +789,25 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
git_debug_snapshot(args, 'Add driver sources')
+ if git_tracked_version:
+ backports_version = "(see git)"
+ kernel_version = "(see git)"
+ else:
+ backports_version = git.describe(tree=source_dir, extra_args=['--long'])
+ kernel_version = git.describe(rev=args.git_revision or 'HEAD',
+ tree=args.kerneldir,
+ extra_args=['--long'])
+
+ if not integrate:
+ f = open(os.path.join(args.outdir, 'versions'), 'w')
+ f.write('BACKPORTS_VERSION="%s"\n' % backports_version)
+ f.write('BACKPORTED_KERNEL_VERSION="%s"\n' % kernel_version)
+ f.write('BACKPORTED_KERNEL_NAME="%s"\n' % args.base_name)
+ if git_tracked_version:
+ f.write('BACKPORTS_GIT_TRACKED="backport tracker ID: $(shell git rev-parse HEAD 2>/dev/null || echo \'not built in git tree\')"\n')
+ f.close()
+ git_debug_snapshot(args, "generate kernel version info file: version")
+
disable_list = add_automatic_backports(args, bp_prefix)
if disable_list:
bpcfg = kconfig.ConfigTree(os.path.join(args.outdir, 'compat', 'Kconfig'))
@@ -815,32 +834,15 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
configtree.modify_selects()
git_debug_snapshot(args, "convert select to depends on")
- # write the versioning file
- if git_tracked_version:
- backports_version = "(see git)"
- kernel_version = "(see git)"
- else:
- backports_version = git.describe(tree=source_dir, extra_args=['--long'])
- kernel_version = git.describe(rev=args.git_revision or 'HEAD',
- tree=args.kerneldir,
- extra_args=['--long'])
- f = open(os.path.join(args.outdir, 'versions'), 'w')
- f.write('BACKPORTS_VERSION="%s"\n' % backports_version)
- f.write('BACKPORTED_KERNEL_VERSION="%s"\n' % kernel_version)
- f.write('BACKPORTED_KERNEL_NAME="%s"\n' % args.base_name)
- if git_tracked_version:
- f.write('BACKPORTS_GIT_TRACKED="backport tracker ID: $(shell git rev-parse HEAD 2>/dev/null || echo \'not built in git tree\')"\n')
- f.close()
-
symbols = configtree.symbols()
- # write local symbol list -- needed during build
- f = open(os.path.join(args.outdir, '.local-symbols'), 'w')
- for sym in symbols:
- f.write('%s=\n' % sym)
- f.close()
-
- git_debug_snapshot(args, "add versions/symbols files")
+ if not integrate:
+ # write local symbol list -- needed during build
+ f = open(os.path.join(args.outdir, '.local-symbols'), 'w')
+ for sym in symbols:
+ f.write('%s=\n' % sym)
+ f.close()
+ git_debug_snapshot(args, "add symbols files")
# add defconfigs that we want
defconfigs_dir = os.path.join(source_dir, 'backport', 'defconfigs')
--
2.1.1
On Tue, 2014-11-04 at 19:18 -0800, Luis R. Rodriguez wrote:
> @@ -71,6 +71,9 @@ def read_dependencies(depfilename):
> ret[sym].append(kconfig_exp)
> else:
> sym, dep = item.split()
> + if bp_prefix != 'CPTCFG_':
> + dep_prefix = re.sub(r'^CONFIG_(.*)', r'\1', bp_prefix)
> + sym = dep_prefix + sym
I'm not sure I like the way you're framing this here. For one, you could
do the re.sub() outside the loop (minimal performance optimisation).
The more interesting part though is that you're parsing the prefix, I
don't think that's a good idea because it breaks making the prefix
generic.
IMHO this would be better handled in the code that uses the return value
to add things to the Kconfig dependencies, there you could just go
if integrate:
deplist[sym] = ["BACKPORT_" + x for x in new]
else:
deplist[sym] = new
or so.
That would probably belong into another patch though.
> @@ -724,7 +740,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
> sys.exit(1)
>
> copy_list = read_copy_list(args.copy_list)
> - deplist = read_dependencies(os.path.join(source_dir, 'dependencies'))
> + deplist = read_dependencies(os.path.join(source_dir, 'dependencies'), bp_prefix)
Another way to look at it would be to say that reading a file shouldn't
modify the data :-)
> # rewrite Makefile and source symbols
> regexes = []
> - for some_symbols in [symbols[i:i + 50] for i in range(0, len(symbols), 50)]:
> - r = 'CONFIG_((' + '|'.join([s + '(_MODULE)?' for s in some_symbols]) + ')([^A-Za-z0-9_]|$))'
I'm not even really sure any more what this was meant to do?
> + all_symbols = orig_symbols + symbols
> + for some_symbols in [all_symbols[i:i + 50] for i in range(0, len(all_symbols), 50)]:
> + r = 'CONFIG_((?!BACKPORT)(' + '|'.join([s + '(_MODULE)?' for s in some_symbols]) + ')([^A-Za-z0-9_]|$))'
But this seems odd, why would you have BACKPORT_ already in some
existing Makefile? It doesn't seem like that would ever happen, so it
seems you could and should drop this change.
> @@ -838,7 +863,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
> for f in files:
> data = open(os.path.join(root, f), 'r').read()
> for r in regexes:
> - data = r.sub(r'CPTCFG_\1', data)
> + data = r.sub(r'' + bp_prefix + '\\1', data)
technically, that should be re.escape(bp_prefix)
(btw, it might be clearer if you used %s instead of +'ing the bp_prefix
in)
> for some_symbols in [disable_makefile[i:i + 50] for i in range(0, len(disable_makefile), 50)]:
> - r = '^([^#].*((CPTCFG|CONFIG)_(' + '|'.join([s for s in some_symbols]) + ')))'
> + r = '^([^#].*((CPTCFG|CONFIG_BACKPORT|CONFIG)_(' + '|'.join([s for s in some_symbols]) + ')))'
should just use bp_prefix instead of the various options
> -cfg_line = re.compile(r'^(config|menuconfig)\s+(?P<sym>[^\s]*)')
> +cfg_line = re.compile(r'^(?P<opt>config|menuconfig)\s+(?P<sym>[^\s]*)')
> sel_line = re.compile(r'^(?P<spc>\s+)select\s+(?P<sym>[^\s]*)\s*$')
> backport_line = re.compile(r'^\s+#(?P<key>[ch]-file|module-name)\s*(?P<name>.*)')
> +ignore_parse_p = re.compile(r'^\s*#(?P<key>ignore-parser-package)')
Since you're later splitting it into separate files, maybe you should
just ignore a whole file instead of having to annotate each symbol?
That'd be easier to maintain (and easier to parse as well :) )
> + def _mod_kconfig_line(self, l, orig_symbols, bp_prefix):
> + if bp_prefix != 'CPTCFG_':
> + prefix = re.sub(r'^CONFIG_(.*)', r'\1', bp_prefix)
Another case like above ... maybe you should have bp_prefix and
bp_kconf_prefix separately. Actually that seems like a good idea.
bp_kconf_prefix is empty for the backport package case, so you could add
it in unconditionally, and bp_prefix would be CONFIG_ or CPTCFG_ for the
two cases. Yes, I think that would make a lot of sense and allow you to
get rid of this regular expression magic while making the code easier to
read/understand.
> + def adjust_backported_configs(self, integrate, orig_symbols, bp_prefix):
This is only used for integrated though, no?
johannes
On Tue, 2014-11-04 at 19:18 -0800, Luis R. Rodriguez wrote:
> @@ -21,23 +22,53 @@ class ConfigTree(object):
> yield f
> for l in open(os.path.join(self.basedir, f), 'r'):
> m = src_line.match(l)
> - if m and os.path.exists(os.path.join(self.basedir, m.group('src'))):
> - for i in self._walk(m.group('src')):
> - yield i
> + if m:
> + bm = bk_src_line.match(l)
> + if bm:
> + if os.path.exists(os.path.join(self.basedir, bm.group('src'))):
> + for i in self._walk(os.path.join(self.basedir, bm.group('src'))):
> + yield i
> + elif os.path.exists(os.path.join(self.basedir, 'backports/' + bm.group('src'))):
> + for i in self._walk(os.path.join(self.basedir, 'backports/' + bm.group('src'))):
> + yield i
> + else:
> + if os.path.exists(os.path.join(self.basedir, m.group('src'))):
> + for i in self._walk(m.group('src')):
> + yield i
Shouldn't this depend on "integrate" rather than on existence?
Ah, this is what you were alluding to in the commit log? The fact that
you hardcode "backports/" into this? IMHO it would make more sense to
pass in the "base" directory (either "backports/" or "") though, instead
of making *that* depend on the existence of the directory as well.
Remember, the existence check here serves to remove includes that cannot
be satisfied; your existence check mixes in the differentiation between
packaged and integrated, which doesn't seem right.
Regardless of whether you make the "backports/" prefix configurable or
not, you should pass it to this library function and use it
unconditionally instead of trying to determine package vs. integrated
from the existence of directories.
> def _prune_sources(self, f, ignore):
> for nf in self._walk(f):
> out = ''
> for l in open(os.path.join(self.basedir, nf), 'r'):
> - m = src_line.match(l)
> - if not m:
> - out += l
> - continue
> - src = m.group('src')
> - if src in ignore or os.path.exists(os.path.join(self.basedir, src)):
> - out += l
> + bm = bk_src_line.match(l)
> + if bm:
> + bp_src = bm.group('src')
> + if bp_src in ignore or \
> + os.path.exists(os.path.join(self.basedir, bp_src)) or \
> + os.path.exists(os.path.join(self.basedir, 'backports/' + bp_src)):
same here.
johannes
On Tue, 2014-11-04 at 19:18 -0800, Luis R. Rodriguez wrote:
> From: "Luis R. Rodriguez" <[email protected]>
>
> During development of kernel integration support using CONFIG_BACKPORT
> was evaluated as a prefix over CPTCFG even for packaging backports,
> for some reason this change lifted some restrictions one some device
> drivers which was present before and as such requires some changes to
> the dependencies map file to ensure correct compilation for respective
> kernel versions.
This is confusing ... I think what you're trying to say is that you
noticed it because your integration system was broken and not honouring
restrictions properly?
johannes
On Tue, 2014-11-04 at 19:18 -0800, Luis R. Rodriguez wrote:
> +++ b/backport/Kconfig.package
> @@ -0,0 +1,32 @@
> +mainmenu "Backports from $BACKPORTED_KERNEL_NAME $BACKPORTED_KERNEL_VERSION (backports $BACKPORTS_VERSION)"
> +
> +config BACKPORT_DIR
> + string
> + option env="BACKPORT_DIR"
> +config BACKPORTS_VERSION
> + string
> + option env="BACKPORTS_VERSION"
> +config BACKPORTED_KERNEL_VERSION
> + string
> + option env="BACKPORTED_KERNEL_VERSION"
> +config BACKPORTED_KERNEL_NAME
> + string
> + option env="BACKPORTED_KERNEL_NAME"
> +
> +# some hacks for when we use backports to generate a package
> +# to build modules out of tree.
> +#ignore-parser-package
Here you're splitting it, and I think you should just add something to
this file to ignore it instead of the "#ignore-parser-package" thing.
johannes
On Tue, 2014-11-04 at 19:18 -0800, Luis R. Rodriguez wrote:
> From: "Luis R. Rodriguez" <[email protected]>
>
> In order to help unify the naming scheme for shared
> backports versioning information rely on the CPTCFG_
> prefix, when integration support gets added that will
> translate to the respective CONFIG_BACKPORT_ prefix.
> Kconfig opt env entries don't get propagated out, so
> we need to define these ourselves. This leaves all
> other names in place for packaging and just focuses
> on sharing on the C / header code.
What difference does this make? It'll break some scripting that we have
for sure (assuming the BACKPORTED_ prefix), so naturally I'd like to see
why it is necessary.
johannes
On Wed, Nov 05, 2014 at 08:46:36AM +0100, Johannes Berg wrote:
> On Tue, 2014-11-04 at 19:18 -0800, Luis R. Rodriguez wrote:
>
> > @@ -71,6 +71,9 @@ def read_dependencies(depfilename):
> > ret[sym].append(kconfig_exp)
> > else:
> > sym, dep = item.split()
> > + if bp_prefix != 'CPTCFG_':
> > + dep_prefix = re.sub(r'^CONFIG_(.*)', r'\1', bp_prefix)
> > + sym = dep_prefix + sym
>
> I'm not sure I like the way you're framing this here. For one, you could
> do the re.sub() outside the loop (minimal performance optimisation).
Indeed.
> The more interesting part though is that you're parsing the prefix, I
> don't think that's a good idea because it breaks making the prefix
> generic.
Makes me wonder if we should even bother really making the prefix configurable,
once we do this we'll have to support it. The code would be a lot simpler if
we only had two prefixes.
> IMHO this would be better handled in the code that uses the return value
> to add things to the Kconfig dependencies, there you could just go
> if integrate:
> deplist[sym] = ["BACKPORT_" + x for x in new]
> else:
> deplist[sym] = new
I like it, thanks.
I will note how you still provided "BACKPORT_" here rather than the prefix,
that's why I did the sub thing, but I'm more inclined to remove the dynamic
nature of the prefix for integration. Not sure why that'd be a good idea
and could only make things harder to support / change in the future as
we are learning with CPTCFG_.
Thoughts?
> That would probably belong into another patch though.
OK
> > # rewrite Makefile and source symbols
> > regexes = []
> > - for some_symbols in [symbols[i:i + 50] for i in range(0, len(symbols), 50)]:
> > - r = 'CONFIG_((' + '|'.join([s + '(_MODULE)?' for s in some_symbols]) + ')([^A-Za-z0-9_]|$))'
>
> I'm not even really sure any more what this was meant to do?
>From what I can understand this looks iterates over the symbols list 50 at a time,
then for each it uses that 50 batch make a huge fucking or statement or possible
configs that might be possible.
> > + all_symbols = orig_symbols + symbols
> > + for some_symbols in [all_symbols[i:i + 50] for i in range(0, len(all_symbols), 50)]:
> > + r = 'CONFIG_((?!BACKPORT)(' + '|'.join([s + '(_MODULE)?' for s in some_symbols]) + ')([^A-Za-z0-9_]|$))'
>
> But this seems odd, why would you have BACKPORT_ already in some
> existing Makefile?
Yeah that I think we can ignore now the other internal stuff was addressed,
that was to account for the bpauto and kernel version things.
> It doesn't seem like that would ever happen, so it
> seems you could and should drop this change.
I think that's true now. Nuked and will test at the end.
> > @@ -838,7 +863,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
> > for f in files:
> > data = open(os.path.join(root, f), 'r').read()
> > for r in regexes:
> > - data = r.sub(r'CPTCFG_\1', data)
> > + data = r.sub(r'' + bp_prefix + '\\1', data)
>
> technically, that should be re.escape(bp_prefix)
We want to support bp_prefix having a regexp ? Sorry I didn't get that.
> (btw, it might be clearer if you used %s instead of +'ing the bp_prefix
> in)
Wasn't quite sure how'd well that'd look with the r'' prefix thing, and
still not sure, r.sub(r"%s\\1" % bp_prefix, data) ? If so that looks rather
like hell to me.
> > for some_symbols in [disable_makefile[i:i + 50] for i in range(0, len(disable_makefile), 50)]:
> > - r = '^([^#].*((CPTCFG|CONFIG)_(' + '|'.join([s for s in some_symbols]) + ')))'
> > + r = '^([^#].*((CPTCFG|CONFIG_BACKPORT|CONFIG)_(' + '|'.join([s for s in some_symbols]) + ')))'
>
> should just use bp_prefix instead of the various options
OK will try that.
> > -cfg_line = re.compile(r'^(config|menuconfig)\s+(?P<sym>[^\s]*)')
> > +cfg_line = re.compile(r'^(?P<opt>config|menuconfig)\s+(?P<sym>[^\s]*)')
> > sel_line = re.compile(r'^(?P<spc>\s+)select\s+(?P<sym>[^\s]*)\s*$')
> > backport_line = re.compile(r'^\s+#(?P<key>[ch]-file|module-name)\s*(?P<name>.*)')
> > +ignore_parse_p = re.compile(r'^\s*#(?P<key>ignore-parser-package)')
>
> Since you're later splitting it into separate files, maybe you should
> just ignore a whole file instead of having to annotate each symbol?
> That'd be easier to maintain (and easier to parse as well :) )
Neat yea good idea.
> > + def _mod_kconfig_line(self, l, orig_symbols, bp_prefix):
> > + if bp_prefix != 'CPTCFG_':
> > + prefix = re.sub(r'^CONFIG_(.*)', r'\1', bp_prefix)
>
> Another case like above ... maybe you should have bp_prefix and
> bp_kconf_prefix separately. Actually that seems like a good idea.
> bp_kconf_prefix is empty for the backport package case, so you could add
> it in unconditionally, and bp_prefix would be CONFIG_
You mean CONFIG_BACKPORT_ ?
> or CPTCFG_ for the
> two cases. Yes, I think that would make a lot of sense and allow you to
> get rid of this regular expression magic while making the code easier to
> read/understand.
Once this is clear sure, I do prefer it but only once we evaluate if we
really need to make the prefixes configurable.
> > + def adjust_backported_configs(self, integrate, orig_symbols, bp_prefix):
>
> This is only used for integrated though, no?
Right now yes, but I'm hinting that perhaps it should also be used for
packaging since it deals with negating a symbol if its built-in on
the kernel already. There should be other ways to do this for packaging,
the checks.h does it but that's just for two modules, we should be doing
this for much other symbols as well.
Luis
On Wed, 2014-11-05 at 10:16 +0100, Luis R. Rodriguez wrote:
> > IMHO this would be better handled in the code that uses the return value
> > to add things to the Kconfig dependencies, there you could just go
> > if integrate:
> > deplist[sym] = ["BACKPORT_" + x for x in new]
> > else:
> > deplist[sym] = new
>
> I like it, thanks.
>
> I will note how you still provided "BACKPORT_" here rather than the prefix,
> that's why I did the sub thing, but I'm more inclined to remove the dynamic
> nature of the prefix for integration. Not sure why that'd be a good idea
> and could only make things harder to support / change in the future as
> we are learning with CPTCFG_.
>
> Thoughts?
I think the splitting of bp_kconf_prefix and bp_prefix I suggested would
help. Now that I look at it again, the names don't really make sense
though.
> > > @@ -838,7 +863,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
> > > for f in files:
> > > data = open(os.path.join(root, f), 'r').read()
> > > for r in regexes:
> > > - data = r.sub(r'CPTCFG_\1', data)
> > > + data = r.sub(r'' + bp_prefix + '\\1', data)
> >
> > technically, that should be re.escape(bp_prefix)
>
> We want to support bp_prefix having a regexp ? Sorry I didn't get that.
No, I mean if bp_prefix were to contain some special character like [.
This can't actually happen though.
> > (btw, it might be clearer if you used %s instead of +'ing the bp_prefix
> > in)
>
> Wasn't quite sure how'd well that'd look with the r'' prefix thing, and
> still not sure, r.sub(r"%s\\1" % bp_prefix, data) ? If so that looks rather
> like hell to me.
Ok sure :) + is fine with me.
> > > + def _mod_kconfig_line(self, l, orig_symbols, bp_prefix):
> > > + if bp_prefix != 'CPTCFG_':
> > > + prefix = re.sub(r'^CONFIG_(.*)', r'\1', bp_prefix)
> >
> > Another case like above ... maybe you should have bp_prefix and
> > bp_kconf_prefix separately. Actually that seems like a good idea.
> > bp_kconf_prefix is empty for the backport package case, so you could add
> > it in unconditionally, and bp_prefix would be CONFIG_
>
> You mean CONFIG_BACKPORT_ ?
No, I did mean CONFIG_. One prefix for kconfig, and one prefix for
"additional renaming".
The names I gave them here were really bad though. Maybe
bp_prefix = "CONFIG_"
additional_prefix = "BACKPORT_"
(or bp_prefix = "CPTCFG_" / additional_prefix = "")
or so?
> > or CPTCFG_ for the
> > two cases. Yes, I think that would make a lot of sense and allow you to
> > get rid of this regular expression magic while making the code easier to
> > read/understand.
>
> Once this is clear sure, I do prefer it but only once we evaluate if we
> really need to make the prefixes configurable.
Yeah I guess we don't really, but I'd also hate to hardcode BACKPORT_
everywhere?
> > > + def adjust_backported_configs(self, integrate, orig_symbols, bp_prefix):
> >
> > This is only used for integrated though, no?
>
> Right now yes, but I'm hinting that perhaps it should also be used for
> packaging since it deals with negating a symbol if its built-in on
> the kernel already. There should be other ways to do this for packaging,
> the checks.h does it but that's just for two modules, we should be doing
> this for much other symbols as well.
Yeah that might be worthwhile.
johannes
On Wed, Nov 05, 2014 at 10:22:44AM +0100, Johannes Berg wrote:
> On Wed, 2014-11-05 at 10:16 +0100, Luis R. Rodriguez wrote:
>
> > > IMHO this would be better handled in the code that uses the return value
> > > to add things to the Kconfig dependencies, there you could just go
> > > if integrate:
> > > deplist[sym] = ["BACKPORT_" + x for x in new]
> > > else:
> > > deplist[sym] = new
> >
> > I like it, thanks.
> >
> > I will note how you still provided "BACKPORT_" here rather than the prefix,
> > that's why I did the sub thing, but I'm more inclined to remove the dynamic
> > nature of the prefix for integration. Not sure why that'd be a good idea
> > and could only make things harder to support / change in the future as
> > we are learning with CPTCFG_.
> >
> > Thoughts?
>
> I think the splitting of bp_kconf_prefix and bp_prefix I suggested would
> help. Now that I look at it again, the names don't really make sense
> though.
>
> > > > @@ -838,7 +863,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
> > > > for f in files:
> > > > data = open(os.path.join(root, f), 'r').read()
> > > > for r in regexes:
> > > > - data = r.sub(r'CPTCFG_\1', data)
> > > > + data = r.sub(r'' + bp_prefix + '\\1', data)
> > >
> > > technically, that should be re.escape(bp_prefix)
> >
> > We want to support bp_prefix having a regexp ? Sorry I didn't get that.
>
> No, I mean if bp_prefix were to contain some special character like [.
> This can't actually happen though.
OK if that can't happen then I don't see the point.
> > > (btw, it might be clearer if you used %s instead of +'ing the bp_prefix
> > > in)
> >
> > Wasn't quite sure how'd well that'd look with the r'' prefix thing, and
> > still not sure, r.sub(r"%s\\1" % bp_prefix, data) ? If so that looks rather
> > like hell to me.
>
> Ok sure :) + is fine with me.
:)
> > > > + def _mod_kconfig_line(self, l, orig_symbols, bp_prefix):
> > > > + if bp_prefix != 'CPTCFG_':
> > > > + prefix = re.sub(r'^CONFIG_(.*)', r'\1', bp_prefix)
> > >
> > > Another case like above ... maybe you should have bp_prefix and
> > > bp_kconf_prefix separately. Actually that seems like a good idea.
> > > bp_kconf_prefix is empty for the backport package case, so you could add
> > > it in unconditionally, and bp_prefix would be CONFIG_
> >
> > You mean CONFIG_BACKPORT_ ?
>
> No, I did mean CONFIG_. One prefix for kconfig, and one prefix for
> "additional renaming".
>
> The names I gave them here were really bad though. Maybe
> bp_prefix = "CONFIG_"
> additional_prefix = "BACKPORT_"
>
> (or bp_prefix = "CPTCFG_" / additional_prefix = "")
>
> or so?
How about:
if integrate:
kconfig_prefix = "CONFIG_"
project_prefix = "BACKPORT_"
else:
kconfig_prefix = "CPTCFG_"
# implied by kconfig_prefix
project_prefix = ""
full_prefix = kconfig_prefix + project_prefix
But note that we special case things all over for when the bp_prefix is 'CPTCFG'
and the reason is that it is used when the kconfig getenv trick is used. I suppose
we can infer that the kconfig trick is used when kconfig_prefix != 'CONFIG_' though.
But the question still stands: why do we want to make these configurable? As it
stands I actually hard code things mostly, I can clean this up but would prefer
to deal with just the above.
Also note that in some places we use BACKPORT vs BACKPORT_ so a sub or another
variable would be needed.
> > > or CPTCFG_ for the
> > > two cases. Yes, I think that would make a lot of sense and allow you to
> > > get rid of this regular expression magic while making the code easier to
> > > read/understand.
> >
> > Once this is clear sure, I do prefer it but only once we evaluate if we
> > really need to make the prefixes configurable.
>
> Yeah I guess we don't really, but I'd also hate to hardcode BACKPORT_
> everywhere?
As it stands in the v2 series packaging gets no BACKPORT_ prefix as the
kconfig getenv() CTPCFG_ trick is used, however for integration we
do add the prefix everywhere as we are carrying code into the kernel,
we then use this to easily make this symbol depend on the non backported
respective symbol making them mutually exclusive. Because of these two
things I think a BACKPORT_ prefix for things we carry in is proper. This
applies to the BACKPORT_BPAUTO and versioning entries, BACKPORT_KERNEL_3_18
and so on.
I am not sure what other prefix we could possibly use here for integration.
> > > > + def adjust_backported_configs(self, integrate, orig_symbols, bp_prefix):
> > >
> > > This is only used for integrated though, no?
> >
> > Right now yes, but I'm hinting that perhaps it should also be used for
> > packaging since it deals with negating a symbol if its built-in on
> > the kernel already. There should be other ways to do this for packaging,
> > the checks.h does it but that's just for two modules, we should be doing
> > this for much other symbols as well.
>
> Yeah that might be worthwhile.
If CPTCFG was dropped, as I had orignally proposed this would be shared :)
Luis
On Wed, Nov 05, 2014 at 08:51:45AM +0100, Johannes Berg wrote:
> On Tue, 2014-11-04 at 19:18 -0800, Luis R. Rodriguez wrote:
>
> > @@ -21,23 +22,53 @@ class ConfigTree(object):
> > yield f
> > for l in open(os.path.join(self.basedir, f), 'r'):
> > m = src_line.match(l)
> > - if m and os.path.exists(os.path.join(self.basedir, m.group('src'))):
> > - for i in self._walk(m.group('src')):
> > - yield i
> > + if m:
> > + bm = bk_src_line.match(l)
> > + if bm:
> > + if os.path.exists(os.path.join(self.basedir, bm.group('src'))):
> > + for i in self._walk(os.path.join(self.basedir, bm.group('src'))):
> > + yield i
> > + elif os.path.exists(os.path.join(self.basedir, 'backports/' + bm.group('src'))):
> > + for i in self._walk(os.path.join(self.basedir, 'backports/' + bm.group('src'))):
> > + yield i
> > + else:
> > + if os.path.exists(os.path.join(self.basedir, m.group('src'))):
> > + for i in self._walk(m.group('src')):
> > + yield i
>
> Shouldn't this depend on "integrate" rather than on existence?
>
> Ah, this is what you were alluding to in the commit log? The fact that
> you hardcode "backports/" into this? IMHO it would make more sense to
> pass in the "base" directory (either "backports/" or "") though, instead
> of making *that* depend on the existence of the directory as well.
>
> Remember, the existence check here serves to remove includes that cannot
> be satisfied; your existence check mixes in the differentiation between
> packaged and integrated, which doesn't seem right.
>
> Regardless of whether you make the "backports/" prefix configurable or
> not, you should pass it to this library function and use it
> unconditionally instead of trying to determine package vs. integrated
> from the existence of directories.
Fair enough.
> > def _prune_sources(self, f, ignore):
> > for nf in self._walk(f):
> > out = ''
> > for l in open(os.path.join(self.basedir, nf), 'r'):
> > - m = src_line.match(l)
> > - if not m:
> > - out += l
> > - continue
> > - src = m.group('src')
> > - if src in ignore or os.path.exists(os.path.join(self.basedir, src)):
> > - out += l
> > + bm = bk_src_line.match(l)
> > + if bm:
> > + bp_src = bm.group('src')
> > + if bp_src in ignore or \
> > + os.path.exists(os.path.join(self.basedir, bp_src)) or \
> > + os.path.exists(os.path.join(self.basedir, 'backports/' + bp_src)):
>
> same here.
Yeah I get it, good points.
This does mean that bp_prefix topic *can* also be tied down
with this other directory prefix as a form of 'builder' for
integration. Making the prefix configurable would make sense
then only if also making the directory prefix should be
configurable.
I think we're better off right now with just supporting two
approaches with their own directory prefix, and prefixes
for variables.
Luis
On Wed, Nov 05, 2014 at 08:54:11AM +0100, Johannes Berg wrote:
> On Tue, 2014-11-04 at 19:18 -0800, Luis R. Rodriguez wrote:
> > From: "Luis R. Rodriguez" <[email protected]>
> >
> > During development of kernel integration support using CONFIG_BACKPORT
> > was evaluated as a prefix over CPTCFG even for packaging backports,
> > for some reason this change lifted some restrictions one some device
> > drivers which was present before and as such requires some changes to
> > the dependencies map file to ensure correct compilation for respective
> > kernel versions.
>
> This is confusing ... I think what you're trying to say is that you
> noticed it because your integration system was broken and not honouring
> restrictions properly?
No I verified each reported case I got and verified the issues found
were valid. For some reason some of these drivers were not allowed to
compile on some older versions, and from what me and Hauke could tell
they should, but they weren't. So the new annotations on requirements
are valid.
Luis
On Wed, Nov 05, 2014 at 08:55:33AM +0100, Johannes Berg wrote:
> On Tue, 2014-11-04 at 19:18 -0800, Luis R. Rodriguez wrote:
>
> > +++ b/backport/Kconfig.package
> > @@ -0,0 +1,32 @@
> > +mainmenu "Backports from $BACKPORTED_KERNEL_NAME $BACKPORTED_KERNEL_VERSION (backports $BACKPORTS_VERSION)"
> > +
> > +config BACKPORT_DIR
> > + string
> > + option env="BACKPORT_DIR"
> > +config BACKPORTS_VERSION
> > + string
> > + option env="BACKPORTS_VERSION"
> > +config BACKPORTED_KERNEL_VERSION
> > + string
> > + option env="BACKPORTED_KERNEL_VERSION"
> > +config BACKPORTED_KERNEL_NAME
> > + string
> > + option env="BACKPORTED_KERNEL_NAME"
> > +
> > +# some hacks for when we use backports to generate a package
> > +# to build modules out of tree.
> > +#ignore-parser-package
>
> Here you're splitting it, and I think you should just add something to
> this file to ignore it instead of the "#ignore-parser-package" thing.
Agreed, done.
Luis
On Wed, Nov 05, 2014 at 08:57:25AM +0100, Johannes Berg wrote:
> On Tue, 2014-11-04 at 19:18 -0800, Luis R. Rodriguez wrote:
> > From: "Luis R. Rodriguez" <[email protected]>
> >
> > In order to help unify the naming scheme for shared
> > backports versioning information rely on the CPTCFG_
> > prefix, when integration support gets added that will
> > translate to the respective CONFIG_BACKPORT_ prefix.
> > Kconfig opt env entries don't get propagated out, so
> > we need to define these ourselves. This leaves all
> > other names in place for packaging and just focuses
> > on sharing on the C / header code.
>
> What difference does this make? It'll break some scripting that we have
> for sure (assuming the BACKPORTED_ prefix), so naturally I'd like to see
> why it is necessary.
Sure, let me explain. So if we don't unify we will have to end up with defines
for some packaging version scheme to another. The approach I took here was to
minimize impact on on userspace side generation side of things and only
affect the target C code by modifying the Makefile to define variables
we can share. That's pretty much it. I ended up defining things with
CPTCFG_ as that will get morphed to the other bp_prefix later for us
when integrating. That lets us share it.
Addressing this on scripts that do rely on touching C / H files should
just be a matter of doing a direct translation to 3 variables.
Luis
On Wed, 2014-11-05 at 20:42 +0100, Luis R. Rodriguez wrote:
> > No, I mean if bp_prefix were to contain some special character like [.
> > This can't actually happen though.
>
> OK if that can't happen then I don't see the point.
Where by "can't happen" I mean that Kconfig probably wouldn't be happy
about it :-)
> How about:
>
> if integrate:
> kconfig_prefix = "CONFIG_"
> project_prefix = "BACKPORT_"
> else:
> kconfig_prefix = "CPTCFG_"
> # implied by kconfig_prefix
> project_prefix = ""
> full_prefix = kconfig_prefix + project_prefix
Yeah, this seems like decent naming.
> But note that we special case things all over for when the bp_prefix is 'CPTCFG'
> and the reason is that it is used when the kconfig getenv trick is used. I suppose
> we can infer that the kconfig trick is used when kconfig_prefix != 'CONFIG_' though.
> But the question still stands: why do we want to make these configurable? As it
> stands I actually hard code things mostly, I can clean this up but would prefer
> to deal with just the above.
I'm not sure where that's really the case? The one thing you did was use
CPTCFG_ in some places like the version, but that was independent of
this and could just be hardcoded there?
> Also note that in some places we use BACKPORT vs BACKPORT_ so a sub or another
> variable would be needed.
I also didn't see that, why would that ever be needed?
> As it stands in the v2 series packaging gets no BACKPORT_ prefix as the
> kconfig getenv() CTPCFG_ trick is used, however for integration we
> do add the prefix everywhere as we are carrying code into the kernel,
> we then use this to easily make this symbol depend on the non backported
> respective symbol making them mutually exclusive. Because of these two
> things I think a BACKPORT_ prefix for things we carry in is proper. This
> applies to the BACKPORT_BPAUTO and versioning entries, BACKPORT_KERNEL_3_18
> and so on.
Sure.
> I am not sure what other prefix we could possibly use here for integration.
Well, in theory you could imagine having two people backport different
subsystems and trying to integrate them both into a backport kernel ...
seems unlikely and will conflict on other levels (e.g. compat.ko) but
still.
Anyway, I don't really care if you don't make this configurable, but I
don't like the way you're doing string manipulation to figure out what
you want :-) That was really the reason for suggesting to split the
existing prefix variable into two pieces. You have it today, after all,
in theory you could have gotten rid of it completely and just pass the
"integrate" flag around.
johannes
On Wed, 2014-11-05 at 21:11 +0100, Luis R. Rodriguez wrote:
> This does mean that bp_prefix topic *can* also be tied down
> with this other directory prefix as a form of 'builder' for
> integration. Making the prefix configurable would make sense
> then only if also making the directory prefix should be
> configurable.
Indeed.
> I think we're better off right now with just supporting two
> approaches with their own directory prefix, and prefixes
> for variables.
Sure, that's fine. I have no issues with either, but I'd like to see the
two cases actually combined and separated, in the sense that you don't
have magic code that tries both and just succeeds on one, but only have
code that tries the right thing.
johannes
On Wed, 2014-11-05 at 21:13 +0100, Luis R. Rodriguez wrote:
> On Wed, Nov 05, 2014 at 08:54:11AM +0100, Johannes Berg wrote:
> > On Tue, 2014-11-04 at 19:18 -0800, Luis R. Rodriguez wrote:
> > > From: "Luis R. Rodriguez" <[email protected]>
> > >
> > > During development of kernel integration support using CONFIG_BACKPORT
> > > was evaluated as a prefix over CPTCFG even for packaging backports,
> > > for some reason this change lifted some restrictions one some device
> > > drivers which was present before and as such requires some changes to
> > > the dependencies map file to ensure correct compilation for respective
> > > kernel versions.
> >
> > This is confusing ... I think what you're trying to say is that you
> > noticed it because your integration system was broken and not honouring
> > restrictions properly?
>
> No I verified each reported case I got and verified the issues found
> were valid. For some reason some of these drivers were not allowed to
> compile on some older versions, and from what me and Hauke could tell
> they should, but they weren't. So the new annotations on requirements
> are valid.
Right, ok, it's just a roundabout way of describing that you audited the
dependencies and found some unnecessary ones :-)
johannes
On Wed, 2014-11-05 at 21:29 +0100, Luis R. Rodriguez wrote:
> > What difference does this make? It'll break some scripting that we have
> > for sure (assuming the BACKPORTED_ prefix), so naturally I'd like to see
> > why it is necessary.
>
> Sure, let me explain. So if we don't unify we will have to end up with defines
> for some packaging version scheme to another. The approach I took here was to
> minimize impact on on userspace side generation side of things and only
> affect the target C code by modifying the Makefile to define variables
> we can share. That's pretty much it. I ended up defining things with
> CPTCFG_ as that will get morphed to the other bp_prefix later for us
> when integrating. That lets us share it.
>
> Addressing this on scripts that do rely on touching C / H files should
> just be a matter of doing a direct translation to 3 variables.
In this particular case I'm not really sure I see why it needs to be
morphed at all?
Anyway, I realized that the whole thing doesn't matter as much to me as
I thought it does, we just have to adjust the one place that changes our
versions file.
johannes
Andi, some reference to some of your old module namespace work below for
a different use case.
On Wed, Nov 05, 2014 at 10:17:38PM +0100, Johannes Berg wrote:
> On Wed, 2014-11-05 at 20:42 +0100, Luis R. Rodriguez wrote:
>
> > > No, I mean if bp_prefix were to contain some special character like [.
> > > This can't actually happen though.
> >
> > OK if that can't happen then I don't see the point.
>
> Where by "can't happen" I mean that Kconfig probably wouldn't be happy
> about it :-)
OK but bp_prefix is also used with a regular print several times and not
just for regexp so this would break either way. To be safe I'll add the
re.escape(bp_prefix) though.
> > How about:
> >
> > if integrate:
> > kconfig_prefix = "CONFIG_"
> > project_prefix = "BACKPORT_"
> > else:
> > kconfig_prefix = "CPTCFG_"
> > # implied by kconfig_prefix
> > project_prefix = ""
> > full_prefix = kconfig_prefix + project_prefix
>
> Yeah, this seems like decent naming.
>
> > But note that we special case things all over for when the bp_prefix is 'CPTCFG'
> > and the reason is that it is used when the kconfig getenv trick is used. I suppose
> > we can infer that the kconfig trick is used when kconfig_prefix != 'CONFIG_' though.
> > But the question still stands: why do we want to make these configurable? As it
> > stands I actually hard code things mostly, I can clean this up but would prefer
> > to deal with just the above.
>
> I'm not sure where that's really the case? The one thing you did was use
> CPTCFG_ in some places like the version, but that was independent of
> this and could just be hardcoded there?
In later code I used CPTCFG_ instead of 'not integrate' but this also carries an
implied 'you are using kconfig trick', there were a few cases but surely I can
generalize this with the above and as I mentioned inferring the kconfig trick was
used if needed (can't think off the top of my head where but do think I needed
this). Again though I'd prefer to think more about the *reason* for why we want
congigurability of the prefix Vs thinking about solving the problems for it
right now. My concerns is once code is baked we'll have to support it. Since
people are stupid I expect things to be abused in the most crazy ways possible
here and my fear is of some of these use case to come and haunt us. Worse thing
I can think of is folks doing double integration with two versions of backports,
say backports-3.18, backports-3.19. Obviously this is grotesque and having code
in place to allow for this is begging for abuse.
> > Also note that in some places we use BACKPORT vs BACKPORT_ so a sub or another
> > variable would be needed.
>
> I also didn't see that, why would that ever be needed?
lib/kconfig.py uses it to negate the built-in symbol for integration for instance
but come to think of it I think we can do this without the 'BACKPORT' and just
use 'BACKPORT_' and no sub'ing, will try...
> > As it stands in the v2 series packaging gets no BACKPORT_ prefix as the
> > kconfig getenv() CTPCFG_ trick is used, however for integration we
> > do add the prefix everywhere as we are carrying code into the kernel,
> > we then use this to easily make this symbol depend on the non backported
> > respective symbol making them mutually exclusive. Because of these two
> > things I think a BACKPORT_ prefix for things we carry in is proper. This
> > applies to the BACKPORT_BPAUTO and versioning entries, BACKPORT_KERNEL_3_18
> > and so on.
>
> Sure.
>
> > I am not sure what other prefix we could possibly use here for integration.
>
> Well, in theory you could imagine having two people backport different
> subsystems and trying to integrate them both into a backport kernel ...
> seems unlikely and will conflict on other levels (e.g. compat.ko) but
> still.
Seems you've though about it too :), indeed sharing compat would be an issue
that would need to be addressed if that is to be a desirable option on
backports.
You know, this use case seems unavoidable so I'll just proceed with the
configurability of it. But note that it seems we're both in agreement
that right now what you described requires more work before in any way
shape or form folks start using it for the exact purpose you described.
I also think Andi Kleen's module namespace might be a much better solution to
that problem [0], we'd just have to carry the code ourselves as I am not
sure if this is ever going to get upstream as it was originally nacked.
[0] https://backports.wiki.kernel.org/index.php/Documentation/backports/hacking/todo#Module_namespaces
> Anyway, I don't really care if you don't make this configurable, but I
> don't like the way you're doing string manipulation to figure out what
> you want :-) That was really the reason for suggesting to split the
> existing prefix variable into two pieces. You have it today, after all,
> in theory you could have gotten rid of it completely and just pass the
> "integrate" flag around.
Yeah point taken. I'll go with the above last proposed variables and
try to remove the other use cases.
Luis
On Wed, Nov 05, 2014 at 10:19:14PM +0100, Johannes Berg wrote:
> On Wed, 2014-11-05 at 21:11 +0100, Luis R. Rodriguez wrote:
>
> > This does mean that bp_prefix topic *can* also be tied down
> > with this other directory prefix as a form of 'builder' for
> > integration. Making the prefix configurable would make sense
> > then only if also making the directory prefix should be
> > configurable.
>
> Indeed.
>
> > I think we're better off right now with just supporting two
> > approaches with their own directory prefix, and prefixes
> > for variables.
>
> Sure, that's fine. I have no issues with either, but I'd like to see the
> two cases actually combined and separated, in the sense that you don't
> have magic code that tries both and just succeeds on one, but only have
> code that tries the right thing.
Fair enough.
Luis
On Wed, Nov 05, 2014 at 10:20:01PM +0100, Johannes Berg wrote:
> On Wed, 2014-11-05 at 21:13 +0100, Luis R. Rodriguez wrote:
> > On Wed, Nov 05, 2014 at 08:54:11AM +0100, Johannes Berg wrote:
> > > On Tue, 2014-11-04 at 19:18 -0800, Luis R. Rodriguez wrote:
> > > > From: "Luis R. Rodriguez" <[email protected]>
> > > >
> > > > During development of kernel integration support using CONFIG_BACKPORT
> > > > was evaluated as a prefix over CPTCFG even for packaging backports,
> > > > for some reason this change lifted some restrictions one some device
> > > > drivers which was present before and as such requires some changes to
> > > > the dependencies map file to ensure correct compilation for respective
> > > > kernel versions.
> > >
> > > This is confusing ... I think what you're trying to say is that you
> > > noticed it because your integration system was broken and not honouring
> > > restrictions properly?
> >
> > No I verified each reported case I got and verified the issues found
> > were valid. For some reason some of these drivers were not allowed to
> > compile on some older versions, and from what me and Hauke could tell
> > they should, but they weren't. So the new annotations on requirements
> > are valid.
>
> Right, ok, it's just a roundabout way of describing that you audited the
> dependencies and found some unnecessary ones :-)
But I didn't do anything other than use a new prefix, and its not clear to
me why this happened, the only thing that comes to mind is perhaps the use
of the orig_symbols thing.
Luis
On Wed, Nov 05, 2014 at 10:20:58PM +0100, Johannes Berg wrote:
> On Wed, 2014-11-05 at 21:29 +0100, Luis R. Rodriguez wrote:
>
> > > What difference does this make? It'll break some scripting that we have
> > > for sure (assuming the BACKPORTED_ prefix), so naturally I'd like to see
> > > why it is necessary.
> >
> > Sure, let me explain. So if we don't unify we will have to end up with defines
> > for some packaging version scheme to another. The approach I took here was to
> > minimize impact on on userspace side generation side of things and only
> > affect the target C code by modifying the Makefile to define variables
> > we can share. That's pretty much it. I ended up defining things with
> > CPTCFG_ as that will get morphed to the other bp_prefix later for us
> > when integrating. That lets us share it.
> >
> > Addressing this on scripts that do rely on touching C / H files should
> > just be a matter of doing a direct translation to 3 variables.
>
> In this particular case I'm not really sure I see why it needs to be
> morphed at all?
>
> Anyway, I realized that the whole thing doesn't matter as much to me as
> I thought it does, we just have to adjust the one place that changes our
> versions file.
If you don't mind the collateral I hope you'd trust me that the alternative
is ugglier. I will note now, since we'll prbably forget later, that this
will also require quite a bit of thought when and if multiple integrations
will be supported. For now sharing the versioning info for packaging and
integration is at least making the code generic between the two.
Luis
> You know, this use case seems unavoidable so I'll just proceed with the
> configurability of it. But note that it seems we're both in agreement
> that right now what you described requires more work before in any way
> shape or form folks start using it for the exact purpose you described.
> I also think Andi Kleen's module namespace might be a much better solution to
> that problem [0], we'd just have to carry the code ourselves as I am not
Just to clarify. My name spaces didn't really provide different
name spaces (like C++), it just added a ACL to exports to make it possible
to do "export only to module N". But the name space itself
was still flat
On the linker side doing real name spaces is probably not too hard,
but it would be difficult for linked in code without special linker
support.
> sure if this is ever going to get upstream as it was originally nacked.
May need to revisit that.
-Andi
On Wed, Nov 5, 2014 at 3:09 PM, Andi Kleen <[email protected]> wrote:
>> You know, this use case seems unavoidable so I'll just proceed with the
>> configurability of it. But note that it seems we're both in agreement
>> that right now what you described requires more work before in any way
>> shape or form folks start using it for the exact purpose you described.
>> I also think Andi Kleen's module namespace might be a much better solution to
>> that problem [0], we'd just have to carry the code ourselves as I am not
>
> Just to clarify. My name spaces didn't really provide different
> name spaces (like C++), it just added a ACL to exports to make it possible
> to do "export only to module N". But the name space itself
> was still flat
Indeed that is how I understood it and the use case in mind for
backporting different subsystems from different future versions of
Linux to an older one might be a use case here (although insane).
> On the linker side doing real name spaces is probably not too hard,
> but it would be difficult for linked in code without special linker
> support.
Good to know, perhaps useful for folks who really do consider
embarking on this strategy with backports (although I don't recommend
it).
>> sure if this is ever going to get upstream as it was originally nacked.
>
> May need to revisit that.
The only thing that was not clear to me from reviewing the module
namespace stuff a while ago was the original intent, but I confess I
actually only looked at the technical details to see if it was
applicable to the backports case, do you recall the original
motivation ?
Luis
> The only thing that was not clear to me from reviewing the module
> namespace stuff a while ago was the original intent, but I confess I
> actually only looked at the technical details to see if it was
> applicable to the backports case, do you recall the original
> motivation ?
The original intent was to provide a cleaner export symbol API.
There are roughly two classes of exported symbols:
- Generally usable, well designed, interfaces intended to be used by multiple
modules (e.g. functions for drivers to register themselves or
library functions)
- Special purpose exports that are only really for a single
module, but are not intended as a general interface. These interfaces
are usually not really clean and reusable.
The idea was to mark the later special purpose exports with the name of the
module that is supposed to them. They wouldn't be available
to everybody else and wouldn't become part of the general
kernel module API.
-Andi