2011-02-01 23:28:15

by Ulf Magnusson

Subject: [PATCH] [ANNOUNCE] kconfig: Kconfiglib: a flexible Python Kconfig parser

Hi,

This is the initial release of Kconfiglib: a Python library for
scripting, debugging, and extracting information from Kconfig-based
configuration systems. It can be used to programmatically generate a
.config when the '*conf' tools are too inflexible, to quickly find out
interesting information about a Kconfig configuration such as dependency
relations between symbols and where undefined symbols are referenced,
and in applications that need to parse and extract information from
Kconfig files.

For a much longer introduction including multiple examples, see
arch/kconfig/kconfiglib.py.

Have fun!

Signed-off-by: Ulf Magnusson <[email protected]>
---
Convenience links:

Documentation, generated from kconfiglib.py with pydoc -w:
http://dl.dropbox.com/u/10406197/kconfiglib.html

Examples as separate files:
http://dl.dropbox.com/u/10406197/kconfiglib-examples.tar.gz


The patch should be preferably be applied to a recent kernel, i.e. Linus's
(2.6.38-rc3 at the time of writing). Due to recent Kconfig changes, the
kconfigtest.py test suite - which compares output character-for-character -
will indicate failure on older (a few months old) kernels versions even though
the outputs are functionally equivalent.

Documentation/kbuild/kconfig-language.txt | 5 +
Documentation/kbuild/kconfig.txt | 8 +
README | 13 +
scripts/kconfig/Makefile | 26 +-
scripts/kconfig/kconfiglib.py | 3918 +++++++++++++++++++++++++++++
scripts/kconfig/kconfigtest.py | 396 +++
6 files changed, 4365 insertions(+), 1 deletions(-)
create mode 100644 scripts/kconfig/kconfiglib.py
create mode 100644 scripts/kconfig/kconfigtest.py

diff --git a/Documentation/kbuild/kconfig-language.txt b/Documentation/kbuild/kconfig-language.txt
index b507d61..63d63ba 100644
--- a/Documentation/kbuild/kconfig-language.txt
+++ b/Documentation/kbuild/kconfig-language.txt
@@ -381,3 +381,8 @@ config FOO

limits FOO to module (=m) or disabled (=n).

+Kconfiglib
+~~~~~~~~~~
+If you need to programmatically generate a .config or parse Kconfig files,
+Kconfiglib might come handy. See 'scriptconfig' and 'iscriptconfig' in
+'make help' and scripts/kconfig/kconfiglib.py.
diff --git a/Documentation/kbuild/kconfig.txt b/Documentation/kbuild/kconfig.txt
index cca46b1..73cf58f 100644
--- a/Documentation/kbuild/kconfig.txt
+++ b/Documentation/kbuild/kconfig.txt
@@ -195,4 +195,12 @@ Searching in gconfig:
however, gconfig does have a few more viewing choices than
xconfig does.

+======================================================================
+scriptconfig
+--------------------------------------------------
+
+If you need to programmatically generate a .config or parse Kconfig files,
+Kconfiglib might come handy. See 'scriptconfig' and 'iscriptconfig' in
+'make help' and scripts/kconfig/kconfiglib.py.
+
###
diff --git a/README b/README
index 1b81d28..bb5e68f 100644
--- a/README
+++ b/README
@@ -196,6 +196,19 @@ CONFIGURING the kernel:
values to 'n' as much as possible.
"make randconfig" Create a ./.config file by setting symbol
values to random values.
+ "make scriptconfig SCRIPT=<path to script>" Run a Kconfiglib
+ script (see scripts/kconfig/kconfiglib.py). This
+ can be used to programatically generate a
+ ./.config, and for applications that need to
+ extract information from Kconfig files.
+ "make iscriptconfig" Launch an interactive Python shell
+ for running Kconfiglib on the architecture's
+ Kconfig configuration. The kconfiglib and sys
+ (for sys.argv[1] - the base Kconfig file) modules
+ will be imported automatically, and a Config
+ instance 'c' will be created for the architecture
+ (using c = kconfiglib.Config(sys.argv[1])).
+

You can find more information on using the Linux kernel config tools
in Documentation/kbuild/kconfig.txt.
diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
index 368ae30..cfffe87 100644
--- a/scripts/kconfig/Makefile
+++ b/scripts/kconfig/Makefile
@@ -3,7 +3,7 @@
# These targets are used from top-level makefile

PHONY += oldconfig xconfig gconfig menuconfig config silentoldconfig update-po-config \
- localmodconfig localyesconfig
+ localmodconfig localyesconfig scriptconfig iscriptconfig kconfiglibtestconfig

ifdef KBUILD_KCONFIG
Kconfig := $(KBUILD_KCONFIG)
@@ -33,6 +33,28 @@ silentoldconfig: $(obj)/conf
$(Q)mkdir -p include/generated
$< --$@ $(Kconfig)

+ifneq ($(filter scriptconfig iscriptconfig,$(MAKECMDGOALS)),)
+PYTHONCMD ?= python
+endif
+
+scriptconfig:
+ $(Q)if [ ! -n "$(SCRIPT)" ]; then \
+ echo 'No script argument provided; use "make scriptconfig SCRIPT=<path to script>".'; \
+ else \
+ PYTHONPATH="$(obj):$$PYTHONPATH" "$(PYTHONCMD)" "$(SCRIPT)" "$(Kconfig)"; \
+ fi
+
+iscriptconfig:
+ $(Q)PYTHONPATH="$(obj):$$PYTHONPATH" "$(PYTHONCMD)" -i -c \
+ "import kconfiglib; \
+ import sys; \
+ c = kconfiglib.Config(sys.argv[1]); \
+ print \"A Config instance 'c' for the architecture ({0}) has been created.\".format(c.get_arch())" "$(Kconfig)"
+
+# Used by kconfigtest.py to prevent an 'option defconfig' .config from being loaded
+kconfiglibtestconfig: $(obj)/conf
+ $(Q)$< --defconfig=.config $(Kconfig)
+
# if no path is given, then use src directory to find file
ifdef LSMOD
LSMOD_F := $(LSMOD)
@@ -139,6 +161,8 @@ help:
@echo ' randconfig - New config with random answer to all options'
@echo ' listnewconfig - List new options'
@echo ' oldnoconfig - Same as silentoldconfig but set new symbols to n (unset)'
+ @echo ' scriptconfig - Run a kconfiglib script (see scripts/kconfig/kconfiglib.py)'
+ @echo ' iscriptconfig - Launch interactive Python shell for using kconfiglib'

# lxdialog stuff
check-lxdialog := $(srctree)/$(src)/lxdialog/check-lxdialog.sh
diff --git a/scripts/kconfig/kconfiglib.py b/scripts/kconfig/kconfiglib.py
new file mode 100644
index 0000000..84e70c3
--- /dev/null
+++ b/scripts/kconfig/kconfiglib.py
@@ -0,0 +1,3918 @@
+# This is Kconfiglib, a Python library for scripting, debugging, and extracing
+# information from Kconfig-based configuration systems. To view the
+# documentation, run
+#
+# $ pydoc kconfiglib
+#
+# or, if you prefer HTML,
+#
+# $ pydoc -w kconfiglib
+#
+# By Ulf "Ulfalizer" Magnusson.
+
+"""
+Kconfiglib is a Python library for scripting, debugging, and extracting
+information from Kconfig-based configuration systems. Features include the
+following:
+
+ - Symbol values and properties can be looked up and values assigned
+ programmatically.
+ - .config files can be read and written.
+ - Expressions can be evaluated in the context of a Kconfig configuration.
+ - Relations between symbols can be quickly determined, such as finding all
+ symbols that reference a particular symbol.
+ - Symbol values are automatically cached and reevaluated only when needed.
+ - Highly compatible with the C Kconfig implementation:
+ * Generates a .config file that is character-for-character identical (modulo
+ header) to the one generated by the C implementation (mconf) when fed each
+ architecture/defconfig pair in the kernel and asked to generate a
+ configuration. This includes nonsensical pairings such as powerpc with
+ arch/x86/configs/x86_64_defconfig, for a total of 9096 combinations as of
+ Linux 2.6.38-rc3.
+ * Also generates a .config that is character-for-character identical to the
+ one generated by mconf for all architectures when no .config is supplied.
+ * The 'make allyesconfig' and 'make allnoconfig' implementations below
+ generate output character-for-character identical to the C implementation
+ for all architectures.
+
+ See scripts/kconfig/kconfigtest.py for the Kconfiglib test suite.
+
+For the Linux kernel, scripts should be run with
+
+$ make scriptconfig SCRIPT=<path to script>
+
+to ensure that the environment (SRCARCH, ARCH, and KERNELVERSION) is set up
+correctly. Alternative architectures can be specified like for other 'make
+*config' targets:
+
+$ make scriptconfig ARCH=mips SCRIPT=<path to script>
+
+The script will receive the name of the Kconfig file to load in sys.argv[1] (as
+of Linux 2.6.38-rc3 this is always "Kconfig" from the kernel top-level
+directory).
+
+To get an interactive Python prompt with Kconfiglib preloaded, use
+
+$ make iscriptconfig ARCH=<architecture>
+
+Kconfiglib requires Python 2. For (i)scriptconfig the command to run the Python
+interpreter can be passed in the environment variable PYTHONCMD (defaults to
+'python').
+
+Learning to use the library is probably easiest by looking at a few examples,
+reading the documentation, and experimenting. The API is designed to be
+intuitive and easy to use.
+
+===============================================================================
+Example 1: loading a configuration and a .config and printing information about
+a symbol.
+
+import kconfiglib
+import sys
+
+# Create a Config object representing a Kconfig configuration (any number of
+# these can be created -- the library has no global state).
+conf = kconfiglib.Config(sys.argv[1])
+
+# Load values from a .config file.
+conf.load_config("arch/x86/configs/i386_defconfig")
+
+# Print some information about a symbol (the Config class implements
+# __getitem__() to provide a handy syntax for getting symbols).
+print conf["SERIAL_UARTLITE_CONSOLE"]
+
+Output for ARCH=i386:
+Symbol SERIAL_UARTLITE_CONSOLE
+Type : bool
+Value : "n"
+User value : (no user value)
+Visibility : "n"
+Is choice item : false
+Is defined : true
+Is from env. : false
+Is special : false
+Prompts:
+ "Support for console on Xilinx uartlite serial port" if SERIAL_UARTLITE = y (value: "n")
+Default values:
+ (no default values)
+Selects:
+ SERIAL_CORE_CONSOLE if SERIAL_UARTLITE = y (value: "n")
+Reverse dependencies:
+ (no reverse dependencies)
+Additional dependencies from enclosing menus and if's:
+ HAS_IOMEM (value: "y")
+Locations: drivers/serial/Kconfig:913
+
+===============================================================================
+Example 2: evaluate an expression in the context of a configuration (here we
+ could load a .config as well).
+
+import kconfiglib
+import sys
+
+conf = kconfiglib.Config(sys.argv[1])
+print conf.eval("(TRACE_IRQFLAGS_SUPPORT || PPC32) && STACKTRACE_SUPPORT")
+
+Output for ARCH=mips:
+y
+
+===============================================================================
+Example 3: print the names of all symbols that are referenced but never defined
+ in the current configuration together with the locations where they
+ are referenced. Integers being included in the list is not a bug, as
+ these need to be treated as symbols per the design of Kconfig.
+
+import kconfiglib
+import sys
+
+conf = kconfiglib.Config(sys.argv[1])
+
+for sym in conf.get_symbols():
+ if not sym.is_defined():
+ print sym.get_name()
+ for (filename, linenr) in sym.get_ref_locations():
+ print " {0}:{1}".format(filename, linenr)
+
+Output for ARCH=i386:
+MACH_NET2BIG_V2
+ drivers/leds/Kconfig:365
+MACH_OMAP3517EVM
+ sound/soc/omap/Kconfig:84
+ARCH_OMAP16XX
+ drivers/pcmcia/Kconfig:271
+ drivers/char/hw_random/Kconfig:117
+ drivers/watchdog/Kconfig:210
+ drivers/rtc/Kconfig:694
+SOC_JZ4740
+ sound/soc/codecs/Kconfig:30
+MACH_AT91SAM9261EK
+ drivers/video/Kconfig:1055
+PPC_83xx
+ drivers/mtd/nand/Kconfig:463
+ drivers/watchdog/Kconfig:951
+ drivers/usb/Kconfig:58
+ drivers/edac/Kconfig:221
+PPC_PSERIES
+ init/Kconfig:999
+ drivers/pci/hotplug/Kconfig:146
+ drivers/scsi/Kconfig:964
+ drivers/scsi/Kconfig:975
+ drivers/scsi/Kconfig:989
+ drivers/net/Kconfig:1353
+ drivers/serial/Kconfig:1261
+ drivers/char/Kconfig:628
+ drivers/char/Kconfig:712
+ drivers/char/Kconfig:729
+... (lots more)
+
+===============================================================================
+Example 4: print the names of all symbols that reference a particular symbol
+ (there's also a method get_selected_symbols() for determining
+ just selection relations).
+
+import kconfiglib
+import sys
+
+conf = kconfiglib.Config(sys.argv[1])
+
+x86 = conf["X86"]
+for sym in conf:
+ if x86 in sym.get_referenced_symbols():
+ print sym.get_name()
+
+Output for ARCH=i386:
+AUDITSYSCALL
+PCSPKR_PLATFORM
+OPROFILE_EVENT_MULTIPLEX
+GCOV_PROFILE_ALL
+SCHED_OMIT_FRAME_POINTER
+MEMORY_HOTPLUG
+PM_TRACE_RTC
+ACPI
+ACPI_AC
+ACPI_BATTERY
+ACPI_VIDEO
+ACPI_PROCESSOR_AGGREGATOR
+ACPI_NUMA
+X86_PM_TIMER
+ACPI_SBS
+ACPI_APEI
+ACPI_APEI_GHES
+INTEL_IDLE
+XEN_PCIDEV_FRONTEND
+EISA_VLB_PRIMING
+EISA_VIRTUAL_ROOT
+HOTPLUG_PCI_COMPAQ
+HOTPLUG_PCI_IBM
+HOTPLUG_PCI_CPCI_ZT5550
+HOTPLUG_PCI_CPCI_GENERIC
+MTD_SC520CDP
+... (lots more)
+
+===============================================================================
+Example 5: print a tree of all items in the configuration.
+
+import kconfiglib
+import sys
+
+def print_with_indent(s, indent):
+ print (" " * indent) + s
+
+def print_items(items, indent):
+ for item in items:
+ if item.is_symbol():
+ print_with_indent("config {0}".format(item.get_name()), indent)
+ elif item.is_menu():
+ print_with_indent('menu "{0}"'.format(item.get_title()), indent)
+ print_items(item.get_items(), indent + 2)
+ elif item.is_choice():
+ print_with_indent('choice', indent)
+ print_items(item.get_items(), indent + 2)
+ elif item.is_comment():
+ print_with_indent('comment "{0}"'.format(item.get_text()), indent)
+
+conf = kconfiglib.Config(sys.argv[1])
+print_items(conf.get_top_level_items(), 0)
+
+Output for ARCH=i386:
+...
+config ARCH
+config KERNELVERSION
+config CONSTRUCTORS
+config HAVE_IRQ_WORK
+config IRQ_WORK
+menu "General setup"
+ config EXPERIMENTAL
+ config BROKEN
+ config BROKEN_ON_SMP
+ config LOCK_KERNEL
+ config INIT_ENV_ARG_LIMIT
+ config CROSS_COMPILE
+ config LOCALVERSION
+ config LOCALVERSION_AUTO
+ config HAVE_KERNEL_GZIP
+ config HAVE_KERNEL_BZIP2
+ config HAVE_KERNEL_LZMA
+ config HAVE_KERNEL_XZ
+ config HAVE_KERNEL_LZO
+ choice
+ config KERNEL_GZIP
+ config KERNEL_BZIP2
+ config KERNEL_LZMA
+ config KERNEL_XZ
+ config KERNEL_LZO
+ config SWAP
+ config SYSVIPC
+ config SYSVIPC_SYSCTL
+ config POSIX_MQUEUE
+ config POSIX_MQUEUE_SYSCTL
+ config BSD_PROCESS_ACCT
+ config BSD_PROCESS_ACCT_V3
+...
+
+===============================================================================
+Example 6: this does the same thing as entering "make menuconfig" and
+ immediately saving and exiting.
+
+import kconfiglib
+import os
+import sys
+
+conf = kconfiglib.Config(sys.argv[1])
+
+if os.path.exists(".config"):
+ conf.load_config(".config")
+else:
+ defconfig = conf.get_defconfig_filename()
+ if defconfig is not None:
+ print "Using " + defconfig
+ conf.load_config(defconfig)
+
+conf.write_config(".config")
+
+===============================================================================
+Example 7: as a more complex example, this is a reimplementation of allnoconfig
+ (verified to produce identical output to 'make allnoconfig' for all
+ ARCHes). The looping is done in case setting one symbol to "n"
+ allows other symbols to be set to "n" (due to dependencies).
+
+import kconfiglib
+import sys
+
+conf = kconfiglib.Config(sys.argv[1])
+
+while True:
+ done = True
+
+ for sym in conf:
+ # Choices take care of themselves for allnoconfig, so we only need to
+ # worry about non-choice symbols
+ if not sym.is_choice_item():
+ lower_bound = sym.get_lower_bound()
+
+ # If we can assign a lower value to the symbol (where "n", "m" and
+ # "y" are ordered from lowest to highest), then do so.
+ # lower_bound() returns None for symbols whose values cannot
+ # (currently) be changed, as well as for non-bool, non-tristate
+ # symbols.
+ if lower_bound is not None and \\
+ kconfiglib.tri_less(lower_bound, sym.calc_value()):
+
+ sym.set_value(lower_bound)
+
+ # We just changed the value of some symbol. As this may effect
+ # other symbols, keep going.
+ done = False
+
+ if done:
+ break
+
+conf.write_config(".config")
+
+===============================================================================
+Example 8: here's allyesconfig (also verified), which is a bit more involved
+ as we need to handle choices in two different modes ("y", i.e.
+ one-is-y-rest-is-n, and "m", i.e.
+ any-number-of-symbols-are-m-rest-are-n). The looping is in case
+ setting one symbol to "y" (or "m") allows the value of other symbols
+ to be raised.
+
+import kconfiglib
+import sys
+
+conf = kconfiglib.Config(sys.argv[1])
+
+# Get a list of all symbols that are not in choices
+non_choice_syms = [sym for sym in conf.get_symbols() if
+ not sym.is_choice_item()]
+
+while True:
+ done = True
+
+ # Handle symbols outside of choices
+
+ for sym in non_choice_syms:
+ upper_bound = sym.get_upper_bound()
+
+ # See corresponding comment for allnoconfig implementation
+ if upper_bound is not None and \\
+ kconfiglib.tri_less(sym.calc_value(), upper_bound):
+ sym.set_value(upper_bound)
+ done = False
+
+ # Handle symbols within choices
+
+ for choice in conf.get_choices():
+
+ # Handle choices whose visibility allow them to be in "y" mode
+
+ if choice.get_visibility() == "y":
+ selection = choice.get_selection_from_defaults()
+ if selection is not None and \\
+ selection is not choice.get_user_selection():
+ selection.set_value("y")
+ done = False
+
+ # Handle choices whose visibility only allow them to be in "m" mode
+
+ elif choice.get_visibility() == "m":
+ for sym in choice.get_items():
+ if sym.calc_value() != "m" and \\
+ sym.get_upper_bound() != "n":
+ sym.set_value("m")
+ done = False
+
+
+ if done:
+ break
+
+conf.write_config(".config")
+
+
+Credits: written by Ulf "Ulfalizer" Magnusson
+
+Send bug reports, suggestions (any missing APIs that would make your life
+easier?) and other feedback to [email protected] ."""
+
+# If you have Psyco installed (32-bit installations, Python <= 2.6 only),
+# setting this to True (right here, not at runtime) might give a nice speedup
+# (22% faster for parsing arch/x86/Kconfig and 58% faster for evaluating all
+# symbols in it without a .config on my Core Duo).
+use_psyco = False
+
+import os
+import re
+import string
+import sys
+
+class Config():
+
+ """Represents a Kconfig configuration."""
+
+ #
+ # Public interface
+ #
+
+ def __init__(self,
+ filename = "Kconfig",
+ base_dir = ".",
+ print_warnings = True,
+ print_undef_assign = False):
+ """Creates a new Config object, representing a Kconfig configuration.
+ Raises Kconfig_Syntax_Error on syntax errors.
+
+ filename (default: "Kconfig") -- The base Kconfig file of the
+ configuration. For the Linux kernel, this should usually be be
+ "Kconfig" from the top-level directory, as environment
+ variables will make sure the right Kconfig is included from
+ there (usually arch/<architecture/Kconfig). If you are using
+ kconfiglib via 'make scriptconfig' the filename of the
+ correct Kconfig will be in sys.argv[1].
+
+ base_dir (default: ".") -- The base directory relative to which
+ 'source' statements within Kconfig files will work. For Linux
+ this should be the top-level directory of the kernel tree.
+
+ print_warnings (default: True) -- Set to True if warnings related to
+ this configuration should be printed to stderr. This can
+ be changed later with Config.set_print_warnings(). It is
+ provided as a constructor argument since warnings might
+ be generated during parsing.
+
+ print_undef_assign (default: False) -- Set to True if informational
+ messages related to assignments to undefined symbols
+ should be printed to stderr for this configuration.
+ Can be changed later with
+ Config.set_print_undef_assign()."""
+
+ # The set of all symbols, indexed by name (a string)
+ self.syms = {}
+
+ # The set of all defined symbols in the configuration in the order they
+ # appear in the Kconfig files. This excludes the special symbols n, m,
+ # and y as well as symbols that are referenced but never defined.
+ self.kconfig_syms = []
+
+ # The set of all named choices (yes, choices can have names), indexed
+ # by name (a string)
+ self.named_choices = {}
+
+ def register_special_symbol(type, name, value):
+ sym = Symbol()
+ sym.is_special_ = True
+ sym.is_defined_ = True
+ sym.config = self
+ sym.name = name
+ sym.type = type
+ sym.cached_value = value
+ self.syms[name] = sym
+
+ # The special symbols n, m and y, used as shorthand for "n", "m" and
+ # "y"
+ register_special_symbol(TRISTATE, "n", "n")
+ register_special_symbol(TRISTATE, "m", "m")
+ register_special_symbol(TRISTATE, "y", "y")
+
+ self.m = self.syms["m"]
+
+ # Maps a symbol to its directly dependent symbols (any symbol whose
+ # prompt, default value or reverse dependency expressions directly
+ # depend on the symbol)
+ self.dep = {}
+
+ # The symbol with "option defconfig_list" set, containing a list of
+ # default .config files
+ self.defconfig_sym = None
+
+ # See Symbol.get_arch()
+ self.arch = os.environ.get("ARCH")
+
+ self.filename = filename
+ self.base_dir = _strip_trailing_slash(base_dir)
+
+ # The 'mainmenu' text
+ self.mainmenu_text = None
+
+ # The filename of the most recently loaded .config file
+ self.config_filename = None
+
+ # The textual header of the most recently loaded .config, uncommented
+ self.config_header = None
+
+ self.print_warnings = print_warnings
+ self.print_undef_assign = print_undef_assign
+
+ # Lists containing all choices, menus and comments in the configuration
+
+ self.choices = []
+ self.menus = []
+ self.comments = []
+
+ # For parsing routines that stop when finding a line belonging to a
+ # different construct, these holds that line and the tokenized version
+ # of that line. The purpose is to avoid having to retokenize the line,
+ # which is inefficient and causes problems when recording references to
+ # symbols.
+ self.end_line = None
+ self.end_line_tokens = None
+
+ # See the comment in _parse_expr().
+ self.parse_expr_cur_sym_or_choice = None
+ self.parse_expr_line = None
+ self.parse_expr_filename = None
+ self.parse_expr_linenr = None
+ self.parse_expr_transform_m = None
+
+ # Parse the Kconfig files
+ self.top_block = self._parse_file(filename, None, None, None)
+
+ # Build self.dep
+ self._build_dep()
+
+ def load_config(self, filename, reset = True):
+ """Loads symbol values from a file in the familiar .config format.
+
+ filename -- The .config file to load.
+
+ reset (default: True) -- True if the configuration should replace
+ the old configuration; False if it should add to it."""
+
+ def warn_override(filename, linenr, name, old_user_val, new_user_val):
+ self._warn("overriding the value of {0}. "
+ 'Old value: "{1}", new value: "{2}".'
+ .format(name, old_user_val, new_user_val),
+ filename,
+ linenr)
+
+ # Put this first so that a missing file doesn't screw up our state
+ line_feeder = _FileFeed(_get_lines(filename), filename)
+
+ self.config_filename = filename
+
+ if reset:
+ self.reset()
+
+ # Invalidate everything. This is usually faster than finding the
+ # minimal set of symbols that needs to be invalidated, as nearly all
+ # symbols will tend to be affected anyway.
+ self._invalidate_all()
+
+ # Read header
+
+ self.config_header = None
+
+ def is_header_line(line):
+ return line.startswith("#") and \
+ not re.match(unset_re, line)
+
+ first_line = line_feeder.get_next()
+
+ if first_line is None:
+ return
+
+ if not is_header_line(first_line):
+ line_feeder.go_back()
+ else:
+ self.config_header = first_line[1:]
+
+ # Read remaining header lines
+ while True:
+ line = line_feeder.get_next()
+
+ if line is None:
+ break
+
+ if not is_header_line(line):
+ line_feeder.go_back()
+ break
+
+ self.config_header += line[1:]
+
+ # Remove trailing newline
+ if self.config_header.endswith("\n"):
+ self.config_header = self.config_header[:-1]
+
+ # Read assignments
+
+ filename = line_feeder.get_filename()
+
+ while True:
+ line = line_feeder.get_next()
+ if line is None:
+ return
+
+ linenr = line_feeder.get_linenr()
+
+ line = line.strip()
+
+ set_re_match = re.match(set_re, line)
+ if set_re_match:
+ name, val = set_re_match.groups()
+ val = _strip_quotes(val, line, filename, linenr)
+ if name in self.syms:
+ sym = self.syms[name]
+
+ old_user_val = sym.get_user_value()
+ if old_user_val is not None:
+ warn_override(filename, linenr, name, old_user_val, val)
+
+ if sym.is_choice_item():
+ user_mode = sym.get_parent()._get_user_mode()
+ if user_mode is not None and user_mode != val:
+ self._warn("assignment to {0} changes mode of containing "
+ 'choice from "{1}" to "{2}".'
+ .format(name, val, user_mode),
+ filename,
+ linenr)
+
+ sym._set_value_no_invalidate(val, True)
+
+ else:
+ self._undef_assign('attempt to assign the value "{0}" to the '
+ "undefined symbol {1}."
+ .format(val, name),
+ filename,
+ linenr)
+ continue
+
+ unset_re_match = re.match(unset_re, line)
+ if unset_re_match:
+ name = unset_re_match.group(1)
+ if name in self.syms:
+ sym = self.syms[name]
+
+ old_user_val = sym.get_user_value()
+ if old_user_val is not None:
+ warn_override(filename, linenr, name, old_user_val, "n")
+
+ sym._set_value_no_invalidate("n", True)
+
+ def write_config(self, filename, header = None):
+ """Writes out symbol values in the familiar .config format.
+
+ filename -- The filename under which to save the configuration.
+
+ header (default: None) -- A textual header that will appear at the
+ beginning of the file, with each line commented out
+ automatically. None means no header."""
+
+ # already_written is set when _make_conf() is called on a symbol, so
+ # that symbols defined in multiple locations only get one entry in the
+ # .config. We need to reset it prior to writing out a new .config.
+ for sym in self.syms.itervalues():
+ sym.already_written = False
+
+ with open(filename, "w") as f:
+ # Write header
+ if header is not None:
+ f.write(_comment(header))
+ f.write("\n")
+
+ # Write configuration
+ f.write("\n".join(self.top_block._make_conf()))
+ f.write("\n")
+
+ def get_kconfig_filename(self):
+ """Returns the name of the (base) kconfig file this configuration was
+ loaded from."""
+ return self.filename
+
+ def get_arch(self):
+ """Mostly Linux specific. Returns the value the environment variable
+ ARCH had at the time the Config instance was created, or None if ARCH
+ was not defined. This corresponds to the architecture, with values such
+ as "i386" or "mips"."""
+ return self.arch
+
+ def get_mainmenu_text(self):
+ """Returns the text of the 'mainmenu' statement (with environment
+ variables expanded to the value they had when the Config was created),
+ or None if the configuration has no 'mainmenu' statement."""
+ return self.mainmenu_text
+
+ def get_defconfig_filename(self):
+ """Returns the name of the defconfig file, which is the first
+ existing file in the list given in a symbol having 'option
+ defconfig_list' set. $-references to environment variables will be
+ expanded. Returns None in case of no defconfig file. Setting 'option
+ defconfig_list' on multiple symbols currently results in undefined
+ behavior."""
+
+ if self.defconfig_sym is None:
+ return None
+
+ for (filename, cond_expr) in self.defconfig_sym.def_exprs:
+ cond_val = self._eval_expr(cond_expr)
+ if cond_val == "y":
+ f = os.path.expandvars(filename)
+ if os.path.exists(f):
+ return f
+
+ return None
+
+ def get_symbol(self, name):
+ """Returns the symbol with name 'name', or None if no such symbol
+ appears in the configuration. An alternative shorthand is conf[name],
+ where conf is a Config instance, though that will instead raise
+ KeyError if the symbol does not exist."""
+ return self.syms.get(name)
+
+ def get_top_level_items(self):
+ """Returns a list containing the items (symbols, menus, choice
+ statements and comments) at the top level of the configuration -- that
+ is, all items that do not appear within a menu or choice. The items
+ appear in the same order as within the configuration."""
+ return self.top_block.get_items()
+
+ def get_symbols(self, all_symbols = True):
+ """Returns a list of symbols from the configuration. An alternative for
+ iterating over all defined symbols (in the order of definition) is
+
+ for sym in config:
+ ...
+
+ which relies on Config implementing __iter__() and is equivalent to
+
+ for sym in config.get_symbols(False):
+ ...
+
+ all_symbols (default: True) -- If True, all symbols - including special
+ and undefined symbols - will be included in the result, in
+ an undefined order. If False, only symbols actually defined
+ and not merely referred to in the configuration will be
+ included in the result, and will appear in the order that
+ they are defined within the Kconfig configuration files."""
+ return self.syms.values() if all_symbols else self.kconfig_syms
+
+ def get_choices(self):
+ """Returns a list containing all choice statements in the
+ configuration, in the order they appear in the Kconfig files."""
+ return self.choices
+
+ def get_menus(self):
+ """Returns a list containing all menus in the configuration, in the
+ order they appear in the Kconfig files."""
+ return self.menus
+
+ def get_comments(self):
+ """Returns a list containing all comments in the configuration, in the
+ order they appear in the Kconfig files."""
+ return self.comments
+
+ def eval(self, s, transform_m = False):
+ """Returns the value of the expression 's' -- where 's' is represented
+ as a string -- in the context of the configuration. Raises
+ Kconfig_Syntax_Error if syntax errors are detected in 's'.
+
+ For example, if FOO and BAR are tristate symbols at least
+ one of which has the value "y", then
+ config.eval("y && (FOO || BAR)") => "y"
+
+ Note that this functions always yields a tristate value. To get the
+ value of non-bool, non-tristate symbols, use calc_value().
+
+ transform_m (default: False) --
+ Within conditional expressions (those following e.g. 'if' and
+ 'depends on') "m" is rewritten as "m" && MODULES internally by the C
+ implementation and by kconfiglib, so that MODULES needs to be enabled
+ for the expression to be true. Pass True here if you want that to
+ happen; otherwise, pass False."""
+ return self._eval_expr(self._parse_expr(self._tokenize(s, False),
+ None,
+ s,
+ transform_m = transform_m))
+
+ def get_config_header(self):
+ """Returns the (uncommented) textual header of the .config file most
+ recently loaded with load_config(). Returns None if no .config file has
+ been loaded or if the most recently loaded .config file has no header.
+ The header comprises all lines up to but not including the first line
+ that either
+
+ 1. Does not start with "#"
+ 2. Has the form "# CONFIG_FOO is not set."
+ """
+ return self.config_header
+
+ def get_base_dir(self):
+ """Returns the base directory for 'source' statements, passed as an
+ argument to Config.__init__()."""
+ return self.base_dir
+
+ def set_print_warnings(self, print_warnings):
+ """Determines whether warnings related to this configuration (for
+ things like attempting to assign illegal values to symbols) should be
+ printed to stderr.
+
+ print_warnings -- True if warnings should be
+ printed, otherwise False."""
+ self.print_warnings = print_warnings
+
+ def set_print_undef_assign(self, print_undef_assign):
+ """Determines whether informational messages related to assignments to
+ undefined symbols should be printed to stderr for this configuration.
+
+ print_undef_assign -- If True, such messages will be printed."""
+ self.print_undef_assign = print_undef_assign
+
+ def __getitem__(self, key):
+ """Returns the symbol with name 'name'. Raises KeyError if the symbol
+ does not appear in the configuration."""
+ return self.syms[key]
+
+ def __iter__(self):
+ """Convenience function for iterating over the set of all defined
+ symbols in the configuration, used like
+
+ for sym in conf:
+ ...
+
+ The iteration happens in the order of definition within the Kconfig
+ configuration files. Symbols only referred to but not defined will not
+ be included, nor will the special symbols n, m, and y. If you want to
+ include such symbols as well, see config.get_symbols()."""
+ return iter(self.kconfig_syms)
+
+ def reset(self):
+ """Resets the values of all symbols, as if Config.load_config() or
+ Symbol.set_value() had never been called."""
+ for sym in self.syms.itervalues():
+ sym._reset_no_recursive_invalidate()
+
+ def __str__(self):
+ """Returns a string containing various information about the Config."""
+ return _sep_lines("Configuration",
+ "File : " + self.filename,
+ "Base directory : " + self.base_dir,
+ "Arch (value of ARCH at time of creation) : " + self.arch,
+ "Most recently loaded .config : " +
+ ("(no .config loaded)" if self.config_filename is None else
+ self.config_filename),
+ "Print warnings : " +
+ bool_str[self.print_warnings],
+ "Print assignments to undefined symbols : " +
+ bool_str[self.print_undef_assign])
+
+
+ #
+ # Private methods
+ #
+
+ def _invalidate_all(self):
+ for sym in self.syms.itervalues():
+ sym._invalidate()
+
+ def _tokenize(self,
+ s,
+ add_sym_if_not_exists = True,
+ filename = None,
+ linenr = None):
+ """Returns a _Feed instance containing tokens derived from the string
+ 's'. Registers any new symbols encountered (via _sym_lookup()).
+
+ (I experimented with a regular expression implementation, but it came
+ out 5% _slower_ and wouldn't have been as flexible).
+
+ add_sym_if_not_exists -- False when we do not want to register new
+ symbols, such as when called from
+ Config.eval()."""
+ tokens = []
+ previous = None
+
+ strlen = len(s)
+ append = tokens.append
+
+ # The current index in the string being tokenized
+ i = 0
+
+ while i < strlen:
+ c = s[i]
+
+ if c.isspace():
+ i += 1
+ continue
+
+ elif c == "=":
+ append(T_EQUAL)
+ i += 1
+
+ elif c == "!":
+ if i + 1 >= strlen:
+ _tokenization_error(s, strlen, filename, linenr)
+ if s[i + 1] == "=":
+ append(T_UNEQUAL)
+ i += 2
+ else:
+ append(T_NOT)
+ i += 1
+
+ elif c == "(":
+ append(T_OPEN_PAREN)
+ i += 1
+
+ elif c == ")":
+ append(T_CLOSE_PAREN)
+ i += 1
+
+ elif c == "&":
+ if i + 1 >= strlen:
+ _tokenization_error(s, strlen, filename, linenr)
+ if s[i + 1] != "&":
+ _tokenization_error(s, i + 1, filename, linenr)
+ append(T_AND)
+ i += 2
+
+ elif c == "|":
+ if i + 1 >= strlen:
+ _tokenization_error(s, strlen, filename, linenr)
+ if s[i + 1] != "|":
+ _tokenization_error(s, i + 1, filename, linenr)
+ append(T_OR)
+ i += 2
+
+ elif c == '"' or c == "'":
+ quote = c
+ value = ""
+ i += 1
+ while True:
+ if i >= strlen:
+ _tokenization_error(s, strlen, filename, linenr)
+ c = s[i]
+ if c == quote:
+ break
+ elif c == "\\":
+ if i + 1 >= strlen:
+ _tokenization_error(s, strlen, filename, linenr)
+ value += s[i + 1]
+ i += 2
+ else:
+ value += c
+ i += 1
+ i += 1
+ append(value)
+
+ elif c == "#":
+ break
+
+ else: # Symbol or keyword
+ name_start = i
+
+ # Locate the end of the symbol/keyword
+ i += 1
+ while i < strlen and s[i] in sym_chars:
+ i += 1
+
+ name = s[name_start:i]
+
+ keyword = keywords.get(name)
+
+ if keyword is not None:
+ append(keyword)
+ # What would ordinarily be considered a name is treated as a
+ # string after certain tokens.
+ elif previous in string_lex:
+ append(name)
+ else:
+ # We're dealing with a symbol. _sym_lookup() will take care
+ # of allocating a new Symbol instance if it's the first
+ # time we see it.
+ sym = self._sym_lookup(name, add_sym_if_not_exists)
+
+ if previous == T_CONFIG:
+ # If the previous token is T_CONFIG ("config"), we're
+ # tokenizing the first line of a symbol definition, and
+ # should remember this as a location where the symbol
+ # is defined.
+ sym.def_locations.append((filename, linenr))
+ else:
+ # Otherwise, it's a reference to the symbol
+ sym.ref_locations.append((filename, linenr))
+
+ append(sym)
+
+ previous = tokens[-1]
+
+ return _Feed(tokens)
+
+ #
+ # Parsing
+ #
+
+ # Expression grammar:
+ #
+ # <expr> -> <symbol>
+ # <symbol> '=' <symbol>
+ # <symbol> '!=' <symbol>
+ # '(' <expr> ')'
+ # '!' <expr>
+ # <expr> '&&' <expr>
+ # <expr> '||' <expr>
+
+ def _parse_expr(self,
+ feed,
+ cur_sym_or_choice,
+ line,
+ filename = None,
+ linenr = None,
+ transform_m = True):
+ """Parse an expression from the tokens in 'feed' using a simple
+ top-down approach. The result has the form (<operator>, <list
+ containing parsed operands>).
+
+ feed -- _Feed instance containing the tokens for the expression.
+
+ cur_sym_or_choice -- The symbol or choice currently being parsed, or
+ None if we're not parsing a symbol or choice.
+ Used for recording references to symbols.
+
+ line -- The line containing the expression being parsed.
+
+ filename (default: None) -- The file containing the expression.
+
+ linenr (default: None) -- The line number containing the expression.
+
+ transform_m (default: False) -- Determines if 'm' should be rewritten to
+ 'm && MODULES' -- see
+ parse_val_and_cond()."""
+
+ # Use instance variables to avoid having to pass these as arguments
+ # through the top-down parser in _parse_expr_2(), which is tedious and
+ # obfuscates the code. A profiler run shows no noticeable performance
+ # difference.
+ self.parse_expr_cur_sym_or_choice = cur_sym_or_choice
+ self.parse_expr_line = line
+ self.parse_expr_filename = filename
+ self.parse_expr_linenr = linenr
+ self.parse_expr_transform_m = transform_m
+
+ return self._parse_expr_2(feed)
+
+ def _parse_expr_2(self, feed):
+ or_terms = [self._parse_or_term(feed)]
+ # Keep parsing additional terms while the lookahead is '||'
+ while feed.check(T_OR):
+ or_terms.append(self._parse_or_term(feed))
+
+ if len(or_terms) == 1:
+ return or_terms[0]
+ else:
+ return (OR, or_terms)
+
+ def _parse_or_term(self, feed):
+ and_terms = [self._parse_factor(feed)]
+ # Keep parsing additional terms while the lookahead is '&&'
+ while feed.check(T_AND):
+ and_terms.append(self._parse_factor(feed))
+
+ if len(and_terms) == 1:
+ return and_terms[0]
+ else:
+ return (AND, and_terms)
+
+ def _parse_factor(self, feed):
+ if feed.check(T_OPEN_PAREN):
+ expr_parse = self._parse_expr_2(feed)
+
+ if not feed.check(T_CLOSE_PAREN):
+ _parse_error(self.parse_expr_line,
+ "missing end parenthesis.",
+ self.parse_expr_filename,
+ self.parse_expr_linenr)
+
+ return expr_parse
+
+ if feed.check(T_NOT):
+ return (NOT, self._parse_factor(feed))
+
+ sym_or_string = feed.get_next()
+
+ if not isinstance(sym_or_string, (Symbol, str)):
+ _parse_error(self.parse_expr_line,
+ "malformed expression.",
+ self.parse_expr_filename,
+ self.parse_expr_linenr)
+
+ if self.parse_expr_cur_sym_or_choice is not None and \
+ isinstance(sym_or_string, Symbol):
+ self.parse_expr_cur_sym_or_choice.referenced_syms.add(sym_or_string)
+
+ if feed.peek_next() not in (T_EQUAL, T_UNEQUAL):
+ if self.parse_expr_transform_m and (sym_or_string is self.m or
+ sym_or_string == "m"):
+ return (AND, ["m", self._sym_lookup("MODULES")])
+ return sym_or_string
+
+ relation = EQUAL if (feed.get_next() == T_EQUAL) else UNEQUAL
+ sym_or_string_2 = feed.get_next()
+
+ if sym_or_string is self.m:
+ sym_or_string = "m"
+
+ if sym_or_string_2 is self.m:
+ sym_or_string_2 = "m"
+
+ return (relation, sym_or_string, sym_or_string_2)
+
+ def _parse_file(self, filename, parent, deps, visible_if_deps):
+ """Parse the Kconfig file 'filename'. The result is
+ a _Block with all items from the file."""
+ line_feeder = _FileFeed(_get_lines(filename), filename)
+ return self._parse_block(line_feeder, None, parent, deps, visible_if_deps)
+
+ def _parse_block(self, line_feeder, end_marker, parent, deps, visible_if_deps = None):
+ """Parses a block, which is the contents of either a file or an if,
+ menu, or choice statement.
+
+ end_marker -- The token that ends the block, e.g. T_ENDIF ("endif") for
+ if's. None for files.
+
+ parent -- The enclosing menu, choice or if, or None if we're at the top
+ level.
+
+ deps -- Dependencies from enclosing menus, choices and if's.
+
+ visible_if_deps (default: None) -- 'visible if' dependencies from
+ enclosing menus."""
+
+ block = _Block()
+
+ filename = line_feeder.get_filename()
+
+ while True:
+
+ # Do we already have a tokenized line that we determined wasn't
+ # part of whatever we were parsing earlier? See comment in
+ # Config.__init__().
+ if self.end_line is not None:
+ assert self.end_line_tokens is not None
+ tokens = self.end_line_tokens
+ tokens.go_to_start()
+
+ line = self.end_line
+ linenr = line_feeder.get_linenr()
+
+ self.end_line = None
+ self.end_line_tokens = None
+
+ else:
+ line = line_feeder.get_next()
+ if line is None:
+ if end_marker is not None:
+ raise Kconfig_Syntax_Error, (
+ "Unexpected end of file {0}."
+ .format(line_feeder.get_filename()))
+ return block
+
+ linenr = line_feeder.get_linenr()
+
+ tokens = self._tokenize(line, True, filename, linenr)
+
+ if tokens.is_empty():
+ continue
+
+ t0 = tokens.get_next()
+
+ # Have we reached the end of the block?
+ if t0 == end_marker:
+ return block
+
+ elif t0 in (T_CONFIG, T_MENUCONFIG):
+ # The tokenizer will automatically allocate a new Symbol object
+ # for any new names it encounters, so we don't need to worry
+ # about that here.
+ sym = tokens.get_next()
+
+ # Symbols defined in multiple places get the parent of their
+ # first definition. However, for symbols whose parents are choice
+ # statements, the choice statement takes precedence.
+ if not sym.is_defined_ or isinstance(parent, Choice):
+ sym.parent = parent
+
+ sym.is_defined_ = True
+
+ self.kconfig_syms.append(sym)
+ block.add_item(sym)
+
+ self._parse_properties(line_feeder, sym, deps, visible_if_deps)
+
+ elif t0 == T_MENU:
+ menu = Menu()
+ menu.config = self
+ menu.parent = parent
+ menu.title = tokens.get_next()
+
+ menu.filename = filename
+ menu.linenr = linenr
+
+ # Parse properties and contents
+ self._parse_properties(line_feeder, menu, deps, visible_if_deps)
+ menu.block = self._parse_block(line_feeder,
+ T_ENDMENU,
+ menu,
+ menu.dep_expr,
+ self._make_and(visible_if_deps,
+ menu.visible_if_expr))
+
+ block.add_item(menu)
+ self.menus.append(menu)
+
+ elif t0 == T_IF:
+ # If statements are treated as syntactic sugar for adding
+ # dependencies to enclosed items and do not have an explicit
+ # object representation.
+
+ dep_expr = self._parse_expr(tokens, None, line, filename, linenr)
+ if_block = self._parse_block(line_feeder,
+ T_ENDIF,
+ parent,
+ self._make_and(dep_expr, deps),
+ visible_if_deps)
+
+ block.add_items_from_block(if_block)
+
+ elif t0 == T_CHOICE:
+ # We support named choices
+ already_defined = False
+ name = None
+ if len(tokens) > 1 and isinstance(tokens[1], str):
+ name = tokens[1]
+ already_defined = name in self.named_choices
+
+ if already_defined:
+ choice = self.named_choices[name]
+ else:
+ choice = Choice()
+ if name is not None:
+ choice.name = name
+ self.named_choices[name] = choice
+
+ choice.config = self
+
+ choice.def_locations.append((filename, linenr))
+
+ # Parse properties and contents
+ self._parse_properties(line_feeder, choice, deps, visible_if_deps)
+ choice.block = self._parse_block(line_feeder,
+ T_ENDCHOICE,
+ choice,
+ None,
+ visible_if_deps)
+
+ choice._determine_actual_items()
+
+ # If no type is set for the choice, its type is that of the first
+ # choice item
+ if choice.type == UNKNOWN:
+ for item in choice.get_actual_items():
+ if item.get_type() != UNKNOWN:
+ choice.type = item.get_type()
+ break
+
+ # Each choice item of UNKNOWN type gets the type of the choice
+ for item in choice.get_actual_items():
+ if item.type == UNKNOWN:
+ item.type = choice.type
+
+ # For named choices defined in multiple locations, only record
+ # at the first definition
+ if not already_defined:
+ block.add_item(choice)
+ self.choices.append(choice)
+
+ elif t0 == T_COMMENT:
+ comment = Comment()
+ comment.config = self
+ comment.parent = parent
+
+ comment.filename = filename
+ comment.linenr = linenr
+
+ comment.text = tokens.get_next()
+ self._parse_properties(line_feeder, comment, deps, visible_if_deps)
+
+ block.add_item(comment)
+ self.comments.append(comment)
+
+ elif t0 == T_SOURCE:
+ kconfig_file = tokens.get_next()
+ f = os.path.join(self.base_dir, os.path.expandvars(kconfig_file))
+
+ if not os.path.exists(f):
+ raise IOError, ('{0}:{1}: sourced file "{2}" not found. Perhaps '
+ 'base_dir (argument to Config.__init__(), currently '
+ '"{3}") is set to the wrong value.'
+ .format(filename,
+ linenr,
+ kconfig_file,
+ self.base_dir))
+
+ file_block = self._parse_file(f, parent, deps, visible_if_deps)
+ block.add_items_from_block(file_block)
+
+ elif t0 == T_MAINMENU:
+ text = tokens.get_next()
+
+ if self.mainmenu_text is not None:
+ self._warn("overriding 'mainmenu' text. "
+ 'Old value: "{0}", new value: "{1}".'
+ .format(self.mainmenu_text, text),
+ filename,
+ linenr)
+
+ self.mainmenu_text = os.path.expandvars(text)
+
+ else:
+ _parse_error(line, "unrecognized construct.", filename, linenr)
+
+ def _parse_properties(self, line_feeder, stmt, deps, visible_if_deps):
+ """Parsing of properties for symbols, menus, choices, and comments."""
+
+ def parse_val_and_cond(tokens, line, filename, linenr):
+ """Parses '<expr1> if <expr2>' constructs, where the 'if' part is
+ optional. Returns a tuple containing the parsed expressions, with
+ None as the second element if the 'if' part is missing."""
+ val = self._parse_expr(tokens, stmt, line, filename, linenr, False)
+
+ if tokens.check(T_IF):
+ return (val, self._parse_expr(tokens, stmt, line, filename, linenr))
+
+ return (val, None)
+
+ # In case the symbol is defined in multiple locations, we need to
+ # remember what prompts, defaults, and selects are new for this
+ # definition, as "depends on" should only apply to the local
+ # definition.
+ new_prompt = None
+ new_def_exprs = []
+ new_selects = []
+
+ # Dependencies from 'depends on' statements
+ depends_on_expr = None
+
+ while True:
+ line = line_feeder.get_next()
+ if line is None:
+ break
+
+ filename = line_feeder.get_filename()
+ linenr = line_feeder.get_linenr()
+
+ tokens = self._tokenize(line, True, filename, linenr)
+
+ if tokens.is_empty():
+ continue
+
+ t0 = tokens.get_next()
+
+ if t0 == T_HELP:
+ # Find first non-empty line and get its indentation
+
+ line_feeder.remove_while(str.isspace)
+ line = line_feeder.get_next()
+
+ if line is None:
+ stmt.help = ""
+ break
+
+ indent = _indentation(line)
+
+ # If the first non-empty lines has zero indent, there is no
+ # help text
+ if indent == 0:
+ stmt.help = ""
+ line_feeder.go_back()
+ break
+
+ help_lines = [_deindent(line, indent)]
+
+ # The help text goes on till the first non-empty line with less
+ # indent
+ while True:
+ line = line_feeder.get_next()
+ if (line is None) or \
+ (not line.isspace() and _indentation(line) < indent):
+ stmt.help = "".join(help_lines)
+ break
+
+ help_lines.append(_deindent(line, indent))
+
+ if line is None:
+ break
+
+ line_feeder.go_back()
+
+ elif t0 == T_PROMPT:
+ # 'prompt' properties override each other within a single
+ # definition of a symbol, but additional prompts can be added
+ # by defining the symbol multiple times; hence 'new_prompt'
+ # instead of 'prompt'.
+ new_prompt = parse_val_and_cond(tokens, line, filename, linenr)
+
+ elif t0 == T_DEFAULT:
+ new_def_exprs.append(parse_val_and_cond(tokens, line, filename, linenr))
+
+ elif t0 == T_DEPENDS:
+ if not tokens.check(T_ON):
+ _parse_error(line, 'expected "on" after "depends".', filename, linenr)
+
+ parsed_deps = self._parse_expr(tokens, stmt, line, filename, linenr)
+
+ if isinstance(stmt, (Menu, Comment)):
+ stmt.dep_expr = self._make_and(stmt.dep_expr, parsed_deps)
+ else:
+ depends_on_expr = self._make_and(depends_on_expr, parsed_deps)
+
+ elif t0 == T_VISIBLE:
+ if not tokens.check(T_IF):
+ _parse_error(line, 'expected "if" after "visible".', filename, linenr)
+ if not isinstance(stmt, Menu):
+ _parse_error(line,
+ "'visible if' is only valid for menus.",
+ filename,
+ linenr)
+
+ parsed_deps = self._parse_expr(tokens, stmt, line, filename, linenr)
+ stmt.visible_if_expr = self._make_and(stmt.visible_if_expr, parsed_deps)
+
+ elif t0 == T_SELECT:
+ target = tokens.get_next()
+
+ stmt.referenced_syms.add(target)
+ stmt.selected_syms.add(target)
+
+ if tokens.check(T_IF):
+ new_selects.append((target,
+ self._parse_expr(tokens, stmt, line, filename, linenr)))
+ else:
+ new_selects.append((target, None))
+
+ elif t0 in (T_BOOL, T_TRISTATE, T_INT, T_HEX, T_STRING):
+ stmt.type = token_to_type[t0]
+
+ if len(tokens) > 1:
+ new_prompt = parse_val_and_cond(tokens, line, filename, linenr)
+
+ elif t0 == T_RANGE:
+ lower = tokens.get_next()
+ upper = tokens.get_next()
+
+ if tokens.check(T_IF):
+ stmt.ranges.append((lower, upper,
+ self._parse_expr(tokens, stmt, line, filename, linenr)))
+ else:
+ stmt.ranges.append((lower, upper, None))
+
+ elif t0 == T_DEF_BOOL:
+ stmt.type = BOOL
+
+ if len(tokens) > 1:
+ new_def_exprs.append(parse_val_and_cond(tokens, line, filename, linenr))
+
+ elif t0 == T_DEF_TRISTATE:
+ stmt.type = TRISTATE
+
+ if len(tokens) > 1:
+ new_def_exprs.append(parse_val_and_cond(tokens, line, filename, linenr))
+
+ elif t0 == T_OPTIONAL:
+ if not isinstance(stmt, Choice):
+ _parse_error(line,
+ '"optional" is only valid for choices.',
+ filename,
+ linenr)
+ stmt.optional = True
+
+ elif t0 == T_OPTION:
+ if tokens.check(T_ENV) and tokens.check(T_EQUAL):
+ env_var = tokens.get_next()
+
+ stmt.is_special_ = True
+ stmt.is_from_env = True
+
+ if env_var not in os.environ:
+ self._warn("""
+The symbol {0} references the non-existent environment variable {1} and will
+get the empty string as its value. You could set it in os.environ before
+calling Config() or make sure it is exported before invoking the script by
+running e.g.
+
+$ export {1}=<value>
+$ python foo.py
+
+or
+
+$ {1}=<value> python foo.py
+
+If you're using kconfiglib via 'make scriptconfig' it should have set up
+the environment correctly for you. If you still got this message, that
+might be an error, and you should e-mail [email protected].
+.""" .format(stmt.get_name(), env_var),
+ filename,
+ linenr)
+
+ stmt.cached_value = ""
+ else:
+ stmt.cached_value = os.environ[env_var]
+
+ elif tokens.check(T_DEFCONFIG_LIST):
+ self.defconfig_sym = stmt
+
+ elif tokens.check(T_MODULES):
+ self._warn("the 'modules' option is not supported. "
+ "Let me know if this is a problem for you; "
+ "it shouldn't be that hard to implement.",
+ filename,
+ linenr)
+
+ else:
+ _parse_error(line, "unrecognized option.", filename, linenr)
+
+ else:
+ # See comment in Config.__init__()
+ self.end_line = line
+ self.end_line_tokens = tokens
+ break
+
+ # Propagate dependencies from enclosing menus and if's.
+
+ # For menus and comments..
+ if isinstance(stmt, (Menu, Comment)):
+ stmt.orig_deps = stmt.dep_expr
+ stmt.deps_from_containing = deps
+ stmt.dep_expr = self._make_and(stmt.dep_expr, deps)
+
+ stmt.all_referenced_syms = \
+ stmt.referenced_syms.union(self._get_expr_syms(deps))
+
+ # For symbols and choices..
+ else:
+
+ # See comment for 'menu_dep'
+ stmt.menu_dep = depends_on_expr
+
+ # Propagate dependencies specified with 'depends on' to any new
+ # default expressions, prompts, and selections ("new" since a
+ # symbol might be defined in multiple places and the dependencies
+ # should only apply to the local definition).
+
+ new_def_exprs = [(val_expr, self._make_and(cond_expr, depends_on_expr))
+ for (val_expr, cond_expr) in new_def_exprs]
+
+ new_selects = [(target, self._make_and(cond_expr, depends_on_expr))
+ for (target, cond_expr) in new_selects]
+
+ if new_prompt is not None:
+ prompt, cond_expr = new_prompt
+
+ # 'visible if' dependencies from enclosing menus get propagated
+ # to prompts
+ if visible_if_deps is not None:
+ cond_expr = self._make_and(cond_expr, visible_if_deps)
+
+ new_prompt = (prompt, self._make_and(cond_expr, depends_on_expr))
+
+ # We save the original expressions -- before any menu and if
+ # conditions have been propagated -- so these can be retrieved
+ # later.
+
+ stmt.orig_def_exprs.extend(new_def_exprs)
+ if new_prompt is not None:
+ stmt.orig_prompts.append(new_prompt)
+
+ # Only symbols can select
+ if isinstance(stmt, Symbol):
+ stmt.orig_selects.extend(new_selects)
+
+ # Save dependencies from enclosing menus and if's
+ stmt.deps_from_containing = deps
+
+ # The set of symbols referenced directly by the symbol/choice plus
+ # all symbols referenced by enclosing menus and if's.
+ stmt.all_referenced_syms \
+ = stmt.referenced_syms.union(self._get_expr_syms(deps))
+
+ # Propagate dependencies from enclosing menus and if's
+
+ stmt.def_exprs.extend([(val_expr, self._make_and(cond_expr, deps))
+ for (val_expr, cond_expr) in new_def_exprs])
+
+ for (target, cond) in new_selects:
+ target.rev_dep = self._make_or(target.rev_dep,
+ self._make_and(stmt,
+ self._make_and(cond, deps)))
+
+ if new_prompt is not None:
+ prompt, cond_expr = new_prompt
+ stmt.prompts.append((prompt, self._make_and(cond_expr, deps)))
+
+ #
+ # Symbol table manipulation
+ #
+
+ def _sym_lookup(self, name, add_sym_if_not_exists = True):
+ """Fetches the symbol 'name' from the symbol table, optionally adding
+ it if it does not exist (this is usually what we want)."""
+ if name in self.syms:
+ return self.syms[name]
+
+ new_sym = Symbol()
+ new_sym.config = self
+ new_sym.name = name
+
+ if add_sym_if_not_exists:
+ self.syms[name] = new_sym
+ else:
+ # This warning is generated while evaluating an expression
+ # containing undefined symbols using Config.eval()
+ self._warn("no symbol {0} in configuration".format(name))
+
+ return new_sym
+
+ #
+ # Evaluation of symbols and expressions
+ #
+
+ def _eval_expr(self, expr):
+ """Evaluates an expression and returns one of the tristate values "n",
+ "m" or "y"."""
+ res = self._eval_expr_2(expr)
+
+ # Promote "m" to "y" if we're running without modules. Internally, "m"
+ # is often rewritten to "m" && MODULES by both the C implementation and
+ # kconfiglib, which takes care of cases where "m" should be false if
+ # we're running without modules.
+ if res == "m" and not self._has_modules():
+ return "y"
+
+ return res
+
+ def _eval_expr_2(self, expr):
+ if expr is None:
+ return "y"
+
+ if isinstance(expr, (str, Symbol)):
+ val = self._get_str_value(expr)
+ # Expressions always have a tristate value
+ return val if val in ("y", "m") else "n"
+
+ first_expr = expr[0]
+
+ if first_expr == OR:
+ res = "n"
+
+ for subexpr in expr[1]:
+ ev = self._eval_expr_2(subexpr)
+
+ # Return immediately upon discovering a "y" term
+ if ev == "y":
+ return "y"
+
+ if ev == "m":
+ res = "m"
+
+ # 'res' is either "n" or "m" here; we already handled the
+ # short-circuiting "y" case in the loop.
+ return res
+
+ if first_expr == AND:
+ res = "y"
+
+ for subexpr in expr[1]:
+ ev = self._eval_expr_2(subexpr)
+
+ # Return immediately upon discovering an "n" term
+ if ev == "n":
+ return "n"
+
+ if ev == "m":
+ res = "m"
+
+ # 'res' is either "m" or "y" here; we already handled the
+ # short-circuiting "n" case in the loop.
+ return res
+
+ if first_expr == NOT:
+ ev = self._eval_expr_2(expr[1])
+
+ if ev == "y":
+ return "n"
+
+ return "y" if (ev == "n") else "m"
+
+ if first_expr in (EQUAL, UNEQUAL):
+ _, sym_or_str_1, sym_or_str_2 = expr
+
+ val_1 = self._get_str_value(sym_or_str_1)
+ val_2 = self._get_str_value(sym_or_str_2)
+
+ if (first_expr == EQUAL and val_1 == val_2) or \
+ (first_expr == UNEQUAL and val_1 != val_2):
+
+ return "y"
+
+ return "n"
+
+ _internal_error("Internal error while evaluating expression with token stream {0}: "
+ "unknown type {0}."
+ .format(expr))
+
+ def _eval_to_int(self, expr):
+ return values[self._eval_expr(expr)]
+
+ def _get_str_value(self, obj):
+ if isinstance(obj, str):
+ return obj
+
+ # obj is a Symbol
+ return obj.calc_value()
+
+ def _eval_min(self, e1, e2):
+ e1_eval = self._eval_expr(e1)
+ e2_eval = self._eval_expr(e2)
+
+ return e1_eval if tri_less(e1_eval, e2_eval) else e2_eval
+
+ def _eval_max(self, e1, e2):
+ e1_eval = self._eval_expr(e1)
+ e2_eval = self._eval_expr(e2)
+
+ return e1_eval if tri_greater(e1_eval, e2_eval) else e2_eval
+
+ #
+ # Construction of expressions
+ #
+
+ # These functions as well as the _eval_min/max() functions above equate
+ # None with "y", which is usually what we want, but needs to be kept in
+ # mind.
+
+ def _make_or(self, e1, e2):
+ # Perform trivial simplification and avoid None's (which
+ # correspond to y's)
+ if "y" in (e1, e2) or None in (e1, e2):
+ return "y"
+
+ if e1 == "n":
+ return e2
+
+ if e2 == "n":
+ return e1
+
+ # Prefer to merge/update argument list if possible instead of creating
+ # a new OR node
+
+ if isinstance(e1, tuple) and e1[0] == OR:
+ if isinstance(e2, tuple) and e2[0] == OR:
+ return (OR, e1[1] + e2[1])
+ else:
+ return (OR, e1[1] + [e2])
+
+ if isinstance(e2, tuple) and e2[0] == OR:
+ return (OR, e2[1] + [e1])
+
+ return (OR, [e1, e2])
+
+ # Note: returns None if e1 == e2 == None
+
+ def _make_and(self, e1, e2):
+ if "n" in (e1, e2):
+ return "n"
+
+ if e1 in (None, "y"):
+ return e2
+
+ if e2 in (None, "y"):
+ return e1
+
+ # Prefer to merge/update argument list if possible instead of creating
+ # a new AND node
+
+ if isinstance(e1, tuple) and e1[0] == AND:
+ if isinstance(e2, tuple) and e2[0] == AND:
+ return (AND, e1[1] + e2[1])
+ else:
+ return (AND, e1[1] + [e2])
+
+ if isinstance(e2, tuple) and e2[0] == AND:
+ return (AND, e2[1] + [e1])
+
+ return (AND, [e1, e2])
+
+ #
+ # Methods related to the MODULES symbol
+ #
+
+ def _has_modules(self):
+ modules_sym = self.syms.get("MODULES")
+ return (modules_sym is not None) and (modules_sym.calc_value() == "y")
+
+ #
+ # Dependency tracking
+ #
+
+ def _build_dep(self):
+ """Populates the dep dictionary, linking a symbol to the symbols that
+ immediately depend on it."""
+ for sym in self.syms.itervalues():
+ self.dep[sym] = set()
+
+ def add_expr_deps(e, sym):
+ for s in self._get_expr_syms(e):
+ self.dep[s].add(sym)
+
+ # The directly dependent symbols of a symbol are:
+ # - Any symbols whose prompt or default value depends on the symbol
+ # - Any symbols whose rev_dep (select condition) depends on the symbol
+ # - Any symbols that belong to the same choice statement as the symbol
+ # (these won't be included in 'dep' as that makes the dependency
+ # graph unwieldy, but Symbol._get_dependent() will include them)
+ # - Any symbols in a choice statement that depends on the symbol
+ for sym in self.syms.itervalues():
+ for (_, e) in sym.prompts:
+ add_expr_deps(e, sym)
+
+ for (v, e) in sym.def_exprs:
+ add_expr_deps(v, sym)
+ add_expr_deps(e, sym)
+
+ add_expr_deps(sym.rev_dep, sym)
+
+ if sym.is_choice_item():
+ choice = sym.parent
+
+ for (_, e) in choice.prompts:
+ add_expr_deps(e, sym)
+
+ for (_, e) in choice.def_exprs:
+ add_expr_deps(e, sym)
+
+ def _get_expr_syms(self, expr):
+ """Returns the set() of symbols appearing in expr."""
+ if expr is None or isinstance(expr, str):
+ return set()
+
+ if isinstance(expr, Symbol):
+ return set((expr,))
+
+ elif expr[0] in (OR, AND):
+ res = set()
+ for subres in [self._get_expr_syms(e) for e in expr[1]]:
+ res.update(subres)
+ return res
+
+ elif expr[0] == NOT:
+ return self._get_expr_syms(expr[1])
+
+ elif expr[0] in (EQUAL, UNEQUAL):
+ res = set()
+
+ (_, v1, v2) = expr
+
+ if isinstance(v1, Symbol):
+ res.add(v1)
+
+ if isinstance(v2, Symbol):
+ res.add(v2)
+
+ return res
+
+ else:
+ _internal_error("Internal error while fetching symbols from an expression with "
+ "token stream {0}.".format(expr))
+
+ def _expr_val_str(self, expr, no_value_str = "(none)", get_val_instead_of_eval = False):
+ # Since values are valid expressions, _expr_to_str() will get a nice
+ # string representation for those as well.
+
+ if expr is None:
+ return no_value_str
+
+ if get_val_instead_of_eval:
+ if isinstance(expr, str):
+ return _expr_to_str(expr)
+ val = expr.calc_value()
+ else:
+ val = self._eval_expr(expr)
+
+ return "{0} (value: {1})".format(_expr_to_str(expr), _expr_to_str(val))
+
+ def _get_sym_or_choice_str(self, sc):
+ """Symbols and choices have many properties in common, so we factor out
+ common __str__() stuff here. "sc" is short for "symbol or choice"."""
+
+ # As we deal a lot with string representations here, use some
+ # convenient shorthand:
+ s = _expr_to_str
+
+ #
+ # Common symbol/choice properties
+ #
+
+ user_value_str = "(no user value)" if sc.user_val is None else s(sc.user_val)
+
+ visibility_str = s(sc.get_visibility())
+
+ # Build prompts string
+ if sc.prompts == []:
+ prompts_str = " (no prompts)"
+ else:
+ prompts_str_rows = []
+
+ for (prompt, cond_expr) in sc.orig_prompts:
+ if cond_expr is None:
+ prompts_str_rows.append(' "{0}"'.format(prompt))
+ else:
+ prompts_str_rows.append(' "{0}" if '.format(prompt) +
+ self._expr_val_str(cond_expr))
+
+ prompts_str = "\n".join(prompts_str_rows)
+
+ # Build locations string
+ if sc.def_locations == []:
+ locations_str = "(no locations)"
+ else:
+ locations_str = " ".join(["{0}:{1}".format(filename, linenr) for
+ (filename, linenr) in sc.def_locations])
+
+ # Build additional-dependencies-from-menus-and-if's string
+ additional_deps_str = " " + self._expr_val_str(sc.deps_from_containing,
+ "(no additional dependencies)")
+
+ #
+ # Symbol-specific stuff
+ #
+
+ if isinstance(sc, Symbol):
+
+ # Build value string
+ value_str = s(sc.calc_value())
+
+ # Build ranges string
+ if isinstance(sc, Symbol):
+ if sc.ranges == []:
+ ranges_str = " (no ranges)"
+ else:
+ ranges_str_rows = []
+
+ for (l, u, cond_expr) in sc.ranges:
+ if cond_expr is None:
+ ranges_str_rows.append(" [{0}, {1}]".format(s(l), s(u)))
+ else:
+ ranges_str_rows.append(" [{0}, {1}] if {2}"
+ .format(s(l), s(u), self._expr_val_str(cond_expr)))
+
+ ranges_str = "\n".join(ranges_str_rows)
+
+ # Build default values string
+ if sc.def_exprs == []:
+ defaults_str = " (no default values)"
+ else:
+ defaults_str_rows = []
+
+ for (val_expr, cond_expr) in sc.orig_def_exprs:
+ row_str = " " + self._expr_val_str(val_expr, "(none)", sc.get_type() == STRING)
+ defaults_str_rows.append(row_str)
+ defaults_str_rows.append(" Condition: " + self._expr_val_str(cond_expr))
+
+ defaults_str = "\n".join(defaults_str_rows)
+
+ # Build selects string
+ if sc.orig_selects == []:
+ selects_str = " (no selects)"
+ else:
+ selects_str_rows = []
+
+ for (target, cond_expr) in sc.orig_selects:
+ if cond_expr is None:
+ selects_str_rows.append(" {0}".format(target.name))
+ else:
+ selects_str_rows.append(" {0} if ".format(target.name) +
+ self._expr_val_str(cond_expr))
+
+ selects_str = "\n".join(selects_str_rows)
+
+ # Build reverse dependencies string
+ if sc.rev_dep == "n":
+ rev_dep_str = " (no reverse dependencies)"
+ else:
+ rev_dep_str = " " + self._expr_val_str(sc.rev_dep)
+
+ res = _sep_lines("Symbol " + (sc.name if sc.name is not None else "(no name)"),
+ "Type : " + typename[sc.type],
+ "Value : " + value_str,
+ "User value : " + user_value_str,
+ "Visibility : " + visibility_str,
+ "Is choice item : " + bool_str[sc.is_choice_item()],
+ "Is defined : " + bool_str[sc.is_defined_],
+ "Is from env. : " + bool_str[sc.is_from_environment()],
+ "Is special : " + bool_str[sc.is_special_] + "\n")
+
+ if sc.has_ranges():
+ res += _sep_lines("Ranges:",
+ ranges_str + "\n")
+
+ res += _sep_lines("Prompts:",
+ prompts_str,
+ "Default values:",
+ defaults_str,
+ "Selects:",
+ selects_str,
+ "Reverse dependencies:",
+ rev_dep_str,
+ "Additional dependencies from enclosing menus and if's:",
+ additional_deps_str,
+ "Locations: " + locations_str)
+
+ return res
+
+ #
+ # Choice-specific stuff
+ #
+
+ # Build name string (for named choices)
+ if sc.get_name() is None:
+ name_str = "(no name)"
+ else:
+ name_str = sc.get_name()
+
+ # Build selected symbol string
+ sel = sc.get_selection()
+ if sel is None:
+ sel_str = "(no selection)"
+ else:
+ sel_str = sel.get_name()
+
+ # Build mode string
+ mode_str = s(sc.calc_mode())
+
+ # Build default values string
+ if sc.def_exprs == []:
+ defaults_str = " (no default values)"
+ else:
+ defaults_str_rows = []
+
+ for (sym, cond_expr) in sc.orig_def_exprs:
+ if cond_expr is None:
+ defaults_str_rows.append(" {0}".format(sym.get_name()))
+ else:
+ defaults_str_rows.append(" {0} if ".format(sym.get_name()) +
+ self._expr_val_str(cond_expr))
+
+ defaults_str = "\n".join(defaults_str_rows)
+
+ # Build contained symbols string
+ names = [sym.get_name() for sym in sc.get_actual_items()]
+
+ if names == []:
+ syms_string = "(empty)"
+ else:
+ syms_string = " ".join(names)
+
+ return _sep_lines("Choice",
+ "Name (for named choices): " + name_str,
+ "Type : " + typename[sc.type],
+ "Selected symbol : " + sel_str,
+ "User value : " + user_value_str,
+ "Mode : " + mode_str,
+ "Visibility : " + visibility_str,
+ "Optional : " + bool_str[sc.optional],
+ "Prompts:",
+ prompts_str,
+ "Defaults:",
+ defaults_str,
+ "Choice symbols:",
+ " " + syms_string,
+ "Additional dependencies from enclosing menus and if's:",
+ additional_deps_str,
+ "Locations: " + locations_str)
+
+ def _expr_depends_on(self, expr, sym):
+ """Reimplementation of expr_depends_symbol() from mconf.c. Used to
+ determine if a submenu should be implicitly created, which influences what
+ items inside choice statements are considered choice items."""
+ if expr is None:
+ return False
+
+ def rec(expr):
+ if isinstance(expr, str):
+ return False
+
+ if isinstance(expr, Symbol):
+ return expr is sym
+
+ if expr[0] in (EQUAL, UNEQUAL):
+ return self._eq_to_sym(expr) is sym
+
+ if expr[0] == AND:
+ for and_expr in expr[1]:
+ if rec(and_expr):
+ return True
+
+ return False
+
+ return False
+
+ return rec(expr)
+
+ def _eq_to_sym(self, eq):
+ """_expr_depends_on() helper. For (in)equalities of the form sym = y/m
+ or sym != n, returns sym. For other (in)equalities, returns None."""
+ (relation, left, right) = eq
+
+ left = self._transform_n_m_y(left)
+ right = self._transform_n_m_y(right)
+
+ # Make sure the symbol (if any) appears to the left
+ if not isinstance(left, Symbol):
+ left, right = right, left
+
+ if not isinstance(left, Symbol):
+ return None
+
+ if (relation == EQUAL and right in ("m", "y")) or \
+ (relation == UNEQUAL and right == "n"):
+ return left
+
+ return None
+
+ def _transform_n_m_y(self, item):
+ """_eq_to_sym() helper. Translates the symbols n, m, and y to their
+ string equivalents."""
+ if item is self.syms["n"]:
+ return "n"
+ if item is self.syms["m"]:
+ return "m"
+ if item is self.syms["y"]:
+ return "y"
+ return item
+
+ def _warn(self, msg, filename = None, linenr = None):
+ """For printing warnings to stderr."""
+ if self.print_warnings:
+ self._warn_or_undef_assign(msg, WARNING, filename, linenr)
+
+ def _undef_assign(self, msg, filename = None, linenr = None):
+ """For printing informational messages related to assignments
+ to undefined variables to stderr."""
+ if self.print_undef_assign:
+ self._warn_or_undef_assign(msg, UNDEF_ASSIGN, filename, linenr)
+
+ def _warn_or_undef_assign(self, msg, msg_type, filename, linenr):
+ if filename is not None:
+ sys.stderr.write("{0}:".format(_clean_up_path(filename)))
+ if linenr is not None:
+ sys.stderr.write("{0}:".format(linenr))
+
+ if msg_type == WARNING:
+ sys.stderr.write("warning: ")
+ elif msg_type == UNDEF_ASSIGN:
+ sys.stderr.write("info: ")
+ else:
+ _internal_error('Internal error while printing warning: unknown warning type "{0}".'
+ .format(msg_type))
+
+ sys.stderr.write(msg + "\n")
+
+#
+# Constants and functions related to types, parsing, evaluation and printing,
+# put globally to unclutter the Config class a bit.
+#
+
+# Tokens
+(T_OR, T_AND, T_NOT,
+ T_OPEN_PAREN, T_CLOSE_PAREN,
+ T_EQUAL, T_UNEQUAL,
+ T_MAINMENU, T_MENU, T_ENDMENU,
+ T_SOURCE, T_CHOICE, T_ENDCHOICE,
+ T_COMMENT, T_CONFIG, T_MENUCONFIG,
+ T_HELP, T_IF, T_ENDIF, T_DEPENDS, T_ON,
+ T_OPTIONAL, T_PROMPT, T_DEFAULT,
+ T_BOOL, T_TRISTATE, T_HEX, T_INT, T_STRING,
+ T_DEF_BOOL, T_DEF_TRISTATE,
+ T_SELECT, T_RANGE, T_OPTION, T_ENV,
+ T_DEFCONFIG_LIST, T_MODULES, T_VISIBLE) = range(0, 38)
+
+# Keyword to token map
+keywords = {
+ "mainmenu" : T_MAINMENU,
+ "menu" : T_MENU,
+ "endmenu" : T_ENDMENU,
+ "endif" : T_ENDIF,
+ "endchoice" : T_ENDCHOICE,
+ "source" : T_SOURCE,
+ "choice" : T_CHOICE,
+ "config" : T_CONFIG,
+ "comment" : T_COMMENT,
+ "menuconfig" : T_MENUCONFIG,
+ "help" : T_HELP,
+ "---help---" : T_HELP,
+ "---" : T_HELP, # Hack to handle CONFIG_W1_CON
+ "if" : T_IF,
+ "depends" : T_DEPENDS,
+ "on" : T_ON,
+ "optional" : T_OPTIONAL,
+ "prompt" : T_PROMPT,
+ "default" : T_DEFAULT,
+ "bool" : T_BOOL,
+ "boolean" : T_BOOL,
+ "tristate" : T_TRISTATE,
+ "int" : T_INT,
+ "hex" : T_HEX,
+ "def_bool" : T_DEF_BOOL,
+ "def_tristate" : T_DEF_TRISTATE,
+ "string" : T_STRING,
+ "select" : T_SELECT,
+ "range" : T_RANGE,
+ "option" : T_OPTION,
+ "env" : T_ENV,
+ "defconfig_list" : T_DEFCONFIG_LIST,
+ "modules" : T_MODULES,
+ "visible" : T_VISIBLE }
+
+# Strings to use for True and False
+bool_str = { False : "false", True : "true" }
+
+# Tokens after which identifier-like lexemes are treated as strings. T_CHOICE
+# is included to avoid symbols being registered for named choices.
+string_lex = (T_BOOL, T_TRISTATE, T_INT, T_HEX, T_STRING,
+ T_CHOICE,
+ T_PROMPT,
+ T_MENU,
+ T_COMMENT,
+ T_SOURCE,
+ T_MAINMENU)
+
+# Characters that may appear in symbol names
+sym_chars = frozenset(string.ascii_letters + string.digits + "._/-")
+
+# Regular expressions for parsing .config files
+set_re = r"CONFIG_(\w+)=(.*)"
+unset_re = r"# CONFIG_(\w+) is not set"
+
+# Integers representing symbol types
+UNKNOWN, BOOL, TRISTATE, STRING, HEX, INT = range(0, 6)
+
+# Strings to use for types
+typename = {
+ UNKNOWN : "unknown",
+ BOOL : "bool",
+ TRISTATE : "tristate",
+ STRING : "string",
+ HEX : "hex",
+ INT : "int" }
+
+# Token to type mapping
+token_to_type = { T_BOOL : BOOL,
+ T_TRISTATE : TRISTATE,
+ T_STRING : STRING,
+ T_INT : INT,
+ T_HEX : HEX }
+
+# Default values for symbols of different types (the value the symbol gets if
+# it is not assigned a user value and none of its 'default' clauses kick in)
+default_value = { BOOL : "n",
+ TRISTATE : "n",
+ STRING : "",
+ INT : "",
+ HEX : "" }
+
+# Indicates that no item is selected in a choice statement
+NO_SELECTION = 0
+
+# Integers representing expression types
+OR, AND, NOT, EQUAL, UNEQUAL = range(0, 5)
+
+# Map from tristate values to integers
+values = {"n" : 0, "m" : 1, "y" : 2}
+
+# Printing-related stuff
+
+op_to_str = { AND : " && ",
+ OR : " || ",
+ EQUAL : " = ",
+ UNEQUAL : " != " }
+
+precedence = {OR : 0, AND : 1, NOT : 2}
+
+# Types of informational messages
+WARNING = 0
+UNDEF_ASSIGN = 1
+
+def _intersperse(lst, op):
+ """_expr_to_str() helper. Gets the string representation of each expression in lst
+ and produces a list where op has been inserted between the elements."""
+ if lst == []:
+ return ""
+
+ res = []
+
+ def handle_sub_expr(expr):
+ no_parens = isinstance(expr, (str, Symbol)) or \
+ expr[0] in (EQUAL, UNEQUAL) or \
+ precedence[op] <= precedence[expr[0]]
+ if not no_parens:
+ res.append("(")
+ res.extend(_expr_to_str_rec(expr))
+ if not no_parens:
+ res.append(")")
+
+ op_str = op_to_str[op]
+
+ handle_sub_expr(lst[0])
+ for expr in lst[1:]:
+ res.append(op_str)
+ handle_sub_expr(expr)
+
+ return res
+
+def _expr_to_str(expr):
+ s = "".join(_expr_to_str_rec(expr))
+ return s
+
+def _sym_str_string(sym_or_str):
+ if isinstance(sym_or_str, str):
+ return '"{0}"'.format(sym_or_str)
+ return sym_or_str.name
+
+def _expr_to_str_rec(expr):
+ if expr is None:
+ return [""]
+
+ if isinstance(expr, (Symbol, str)):
+ return [_sym_str_string(expr)]
+
+ if expr[0] in (OR, AND):
+ return _intersperse(expr[1], expr[0])
+
+ if expr[0] == NOT:
+ need_parens = not isinstance(expr[1], (str, Symbol))
+
+ res = ["!"]
+ if need_parens:
+ res.append("(")
+ res.extend(_expr_to_str_rec(expr[1]))
+ if need_parens:
+ res.append(")")
+ return res
+
+ if expr[0] in (EQUAL, UNEQUAL):
+ return [_sym_str_string(expr[1]),
+ op_to_str[expr[0]],
+ _sym_str_string(expr[2])]
+
+class _Block:
+
+ """Represents a list of items (symbols, menus, choice
+ statements and comments) appearing at the top-level of a
+ file or witin a menu, choice or if statement."""
+
+ def __init__(self):
+ self.items = []
+
+ def get_items(self):
+ return self.items
+
+ def add_item(self, item):
+ self.items.append(item)
+
+ def add_items_from_block(self, block):
+ self.items.extend(block.items)
+
+ def _make_conf(self):
+ # Collect the substrings in a list and later use join() instead of +=
+ # to build the final .config contents. With older Python versions, this
+ # yields linear instead of quadratic complexity.
+ strings = []
+ for item in self.items:
+ strings.extend(item._make_conf())
+
+ return strings
+
+ def add_depend_expr(self, expr):
+ for item in self.items:
+ item.add_depend_expr(expr)
+
+class Item():
+
+ """Base class for symbols and other Kconfig constructs. Subclasses are
+ Symbol, Choice, Menu, and Comment."""
+
+ def is_symbol(self):
+ """Returns True if the item is a symbol, otherwise False. Short for
+ isinstance(item, kconfiglib.Symbol)."""
+ return isinstance(self, Symbol)
+
+ def is_choice(self):
+ """Returns True if the item is a choice, otherwise False. Short for
+ isinstance(item, kconfiglib.Choice)."""
+ return isinstance(self, Choice)
+
+ def is_menu(self):
+ """Returns True if the item is a menu, otherwise False. Short for
+ isinstance(item, kconfiglib.Menu)."""
+ return isinstance(self, Menu)
+
+ def is_comment(self):
+ """Returns True if the item is a comment, otherwise False. Short for
+ isinstance(item, kconfiglib.Comment)."""
+ return isinstance(self, Comment)
+
+class _HasVisibility():
+
+ """Base class for elements that have a "visibility" that acts as an upper
+ limit on the values a user can set for them. Subclasses are Symbol and
+ Choice."""
+
+ def __init__(self):
+ self.cached_visibility = None
+ self.prompts = []
+
+ def _invalidate(self):
+ self.cached_visibility = None
+
+ def _calc_visibility(self):
+ if self.cached_visibility is None:
+ vis = "n"
+ for (prompt, cond_expr) in self.prompts:
+ vis = self.config._eval_max(vis, cond_expr)
+
+ if isinstance(self, Symbol) and self.is_choice_item():
+ vis = self.config._eval_min(vis, self.parent._calc_visibility())
+
+ # Promote "m" to "y" if we're dealing with a non-tristate
+ if vis == "m" and self.type != TRISTATE:
+ vis = "y"
+
+ self.cached_visibility = vis
+
+ return self.cached_visibility
+
+class Symbol(Item, _HasVisibility):
+
+ """Represents a configuration symbol."""
+
+ #
+ # Public interface
+ #
+
+ def calc_value(self):
+ """Calculate and return the value of the symbol."""
+
+ if self.cached_value is not None:
+ return self.cached_value
+
+ self.write_to_conf = False
+
+ # The following means that what to the user looks like a reference to
+ # an undefined symbol will actually get treated like a string whose
+ # value is the symbol name, which can be a bit unexpected.
+ if self.type == UNKNOWN:
+ self.cached_value = self.name
+ return self.name
+
+ new_val = default_value[self.type]
+
+ vis = self._calc_visibility()
+
+ if self.type in (BOOL, TRISTATE):
+ # The visibility and mode (modules-only or single-selection) of
+ # choice items will be taken into account in self._calc_visibility()
+
+ if self.is_choice_item():
+ if vis != "n":
+ choice = self.parent
+ mode = choice.calc_mode()
+
+ self.write_to_conf = (mode != "n")
+
+ if mode == "y":
+ new_val = "y" if (choice.get_selection() is self) else "n"
+ elif mode == "m":
+ if self.user_val in ("m", "y"):
+ new_val = "m"
+
+ else:
+ use_defaults = True
+
+ if vis != "n":
+ # If the symbol is visible and has a user value, use that.
+ # Otherwise, look at defaults.
+ self.write_to_conf = True
+
+ if self.user_val is not None:
+ new_val = self.config._eval_min(self.user_val, vis)
+ use_defaults = False
+
+ if use_defaults:
+ for (val_expr, cond_expr) in self.def_exprs:
+ cond_eval = self.config._eval_expr(cond_expr)
+
+ if cond_eval != "n":
+ self.write_to_conf = True
+ new_val = self.config._eval_min(val_expr, cond_eval)
+ break
+
+ # Reverse dependencies take precedence
+ rev_dep_val = self.config._eval_expr(self.rev_dep)
+
+ if rev_dep_val != "n":
+ self.write_to_conf = True
+ new_val = self.config._eval_max(new_val, rev_dep_val)
+
+ # Promote "m" to "y" for booleans
+ if new_val == "m" and self.type == BOOL:
+ new_val = "y"
+
+ elif self.type == STRING:
+ use_defaults = True
+
+ if vis != "n":
+ self.write_to_conf = True
+ if self.user_val is not None:
+ new_val = self.user_val
+ use_defaults = False
+
+ if use_defaults:
+ for (val_expr, cond_expr) in self.def_exprs:
+ if self.config._eval_expr(cond_expr) != "n":
+ self.write_to_conf = True
+ new_val = self.config._get_str_value(val_expr)
+ break
+
+ elif self.type in (HEX, INT):
+ active_range = None
+ use_defaults = True
+
+ # TODO: HEX case should only accept hex and dec numbers
+
+ for(l, u, cond_expr) in self.ranges:
+ if self.config._eval_expr(cond_expr) != "n":
+ l_str = self.config._get_str_value(l)
+ u_str = self.config._get_str_value(u)
+
+ if self.type == INT:
+ if not _is_dec(l_str):
+ l_str = "0"
+
+ if not _is_dec(u_str):
+ u_str = "0"
+
+ active_range = (int(l_str), int(u_str))
+
+ else: # self.type == HEX
+ if not _is_hex(l_str):
+ l_str = "0x0"
+
+ if not _is_hex(u_str):
+ u_str = "0x0"
+
+ active_range = (int(l_str, 16), int(u_str, 16))
+
+ break
+
+ if vis != "n":
+ self.write_to_conf = True
+
+ if self.user_val is not None:
+ if self.type == INT:
+ if _is_dec(self.user_val):
+ num = int(self.user_val)
+ if active_range is None or \
+ active_range[0] <= num <= active_range[1]:
+
+ use_defaults = False
+ new_val = self.user_val
+
+ else: # HEX
+ if _is_hex(self.user_val):
+ num = int(self.user_val, 16)
+ if active_range is None or \
+ active_range[0] <= num <= active_range[1]:
+
+ use_defaults = False
+ new_val = self.user_val
+
+ if use_defaults:
+ for (val_expr, cond_expr) in self.def_exprs:
+ if self.config._eval_expr(cond_expr) != "n":
+ self.write_to_conf = True
+
+ # Kconfig allows arbitrary string values for hex/int
+ # (possibly a bug, but we still emulate it).
+
+ new_val = self.config._get_str_value(val_expr)
+
+ if self.type == INT:
+ if _is_dec(new_val):
+ new_val_num = int(new_val)
+ elif self.type == HEX:
+ if _is_hex(new_val):
+ new_val_num = int(new_val, 16)
+
+ if new_val_num is not None:
+ if active_range is not None:
+ l_num, u_num = active_range
+
+ was_clamped = False
+
+ if new_val_num < l_num:
+ new_val_num = l_num
+ was_clamped = True
+ elif new_val_num > u_num:
+ new_val_num = u_num
+ was_clamped = True
+
+ if was_clamped:
+ new_val = (str(new_val_num) if self.type == INT else
+ hex(new_val_num))
+
+ break
+
+ self.cached_value = new_val
+ return new_val
+
+ def calc_default_value(self):
+ """Calculates the value the symbol would get purely from defaults,
+ ignoring visibility (assumed to be "y"), reverse dependencies
+ (selects), user values and dependencies from enclosing menus and if's.
+ Returns None if no default would kick in."""
+ for (val_expr, cond_expr) in self.orig_def_exprs:
+ cond_eval = self.config._eval_expr(cond_expr)
+
+ if cond_eval != "n":
+ return self.config._eval_expr(val_expr)
+
+ return None
+
+ def set_value(self, v):
+ """Sets the (user) value of the symbol. Equal in effect to assigning
+ the value to the symbol within a .config file. Use
+ get_lower/upper_bound() to find the range of valid values for bool and
+ tristate symbols; setting values outside this range will cause the user
+ value to differ from the result of Symbol.calc_value(). Any value that
+ is valid for the type (bool, tristate, etc.) will end up being
+ reflected in Symbol.get_user_value() though.
+
+ Any symbols dependent on the symbol are (recursively) invalidated, so
+ things should just work with regards to dependencies.
+
+ v -- The value to give to the symbol."""
+ self._set_value_no_invalidate(v, False)
+
+ # There might be something more efficient you could do here, but play
+ # it safe.
+ if self.name == "MODULES":
+ self.config._invalidate_all()
+ return
+
+ self._invalidate()
+ self._invalidate_dependent()
+
+ def reset(self):
+ """Resets the value of the symbol, as if the symbol had never gotten a
+ (user) value via Config.load_config() or Symbol.set_value(). Dependent
+ symbols are recursively invalidated."""
+ self._reset_no_recursive_invalidate()
+ self._invalidate_dependent()
+
+ def get_user_value(self):
+ """Returns the value assigned to the symbol in a .config or via
+ Symbol.set_value() (provided the value was valid for the type of the
+ symbol). Returns None in case of no user value."""
+ return self.user_val
+
+ def get_name(self):
+ """Returns the name of the symbol."""
+ return self.name
+
+ def get_upper_bound(self):
+ """Returns the highest value the symbol can be given via
+ Symbol.set_value() (that will not be truncated): one of "m" or "y",
+ arranged from lowest to highest. This corresponds to the highest value
+ the symbol could be given in the 'make menuconfig' interface. Returns
+ None if the symbol is not visible (would not appear in the 'make
+ menuconfig' interface), or if the symbol's value cannot be changed (if
+ it is selected to "y", or to "m" if its visibility also happens to be
+ "m"). Also returns None for non-bool, non-tristate symbols and special
+ symbols.
+
+ See also the tri_less*() and tri_greater*() functions, which could
+ come in handy here."""
+ if not self._is_assignable_bool_or_tristate():
+ return None
+
+ return self._calc_visibility()
+
+ def get_lower_bound(self):
+ """Returns the lowest value the symbol can be given via
+ Symbol.set_value() (that will not be truncated): one of "n" or "m",
+ arranged from lowest to highest. This corresponds to the lowest value
+ the symbol could be given in the 'make menuconfig' interface. Returns
+ None if the symbol is not visible (would not appear in the 'make
+ menuconfig' interface), or if the symbol's value cannot be changed (if
+ it is selected to "y", or to "m" if its visibility also happens to be
+ "m"). Also returns None for non-bool, non-tristate symbols and special
+ symbols.
+
+ See also the tri_less*() and tri_greater*() functions, which could
+ come in handy here."""
+ if not self._is_assignable_bool_or_tristate():
+ return None
+
+ return self.config._eval_expr(self.rev_dep)
+
+ def get_assignable_values(self):
+ """For bool and tristate symbols, returns a list containing the values
+ the symbol can be given via Symbol.set_value() (see get_lower_bound()/
+ get_upper_bound()). Returns the empty list for symbol that cannot be
+ given a new value (that cannot be assigned a value that won't be
+ truncated/ignored that is), as well as for non-bool, non-tristate and
+ special symbols. Usage example:
+
+ if "m" in sym.get_assignable_values():
+ sym.set_value("m")"""
+ if not self._is_assignable_bool_or_tristate():
+ return []
+
+ return ["n", "m", "y"][values[self.config._eval_expr(self.rev_dep)] :
+ values[self._calc_visibility()] + 1]
+
+ def get_type(self):
+ """Returns the type of the symbol: one of UNKNOWN, BOOL, TRISTATE,
+ STRING, HEX, or INT. These are defined at the top level of the module,
+ so you'd do something like
+
+ if sym.get_type() == kconfiglib.STRING:
+ ..."""
+ return self.type
+
+ def get_visibility(self):
+ """Returns the visibility of the symbol: one of "n", "m" or "y". For
+ bool and tristate symbols, this is an upper bound on the value users
+ can set for the symbol. For other types of symbols, a visibility of "n"
+ means the user value will be ignored. A visibility of "n" corresponds
+ to not being visible in the 'make *config' interfaces."""
+ return self._calc_visibility()
+
+ def get_parent(self):
+ """Returns the menu or choice statement that contains the symbol, or
+ None if the symbol is at the top level. Note that if statements are
+ treated as syntactic sugar and do not have an explicit class
+ representation."""
+ return self.parent
+
+ def get_sibling_symbols(self, include_self = False):
+ """Returns a list containing all symbols that are in the same menu or
+ choice statement as the symbol, or that are also at the top level in
+ case the symbol is at the top level.
+
+ include_self (default: False) -- True if the symbol itself should be
+ included in the result, otherwise
+ False."""
+
+ return [item for item in self.get_sibling_items(include_self)
+ if isinstance(item, Symbol)]
+
+ def get_sibling_items(self, include_self = False):
+ """Returns a list containing all items (symbols, menus, choice
+ statements and comments) that are in the same menu or choice statement
+ as the symbol, or that are also at the top level in case the symbol is
+ at the top level. The items appear in the same order as within the
+ configuration.
+
+ include_self (default: False) -- True if the symbol itself should be
+ included in the result, otherwise
+ False."""
+
+ if self.parent is None:
+ items = self.config.get_top_level_items()
+ else:
+ items = self.parent.get_items()
+
+ if include_self:
+ return items
+
+ return [item for item in items if item is not self]
+
+ def get_referenced_symbols(self, refs_from_enclosing = False):
+ """Returns the set() of all symbols referenced by this symbol. For
+ example, the symbol defined by
+
+ config FOO
+ bool
+ prompt "foo" if A && B
+ default C if D
+ depends on E
+ select F if G
+
+ references the symbols A through G.
+
+ refs_from_enclosing (default: False) -- If True, the symbols
+ referenced by enclosing menus and if's will be
+ included in the result."""
+ return self.all_referenced_syms if refs_from_enclosing else self.referenced_syms
+
+ def get_selected_symbols(self):
+ """Returns the set() of all symbols X for which this symbol has a
+ 'select X' or 'select X if Y' (regardless of whether Y is satisfied or
+ not). This is a subset of the symbols returned by
+ get_referenced_symbols()."""
+ return self.selected_syms
+
+ def get_help(self):
+ """Returns the help text of the symbol, or None if the symbol has no
+ help text."""
+ return self.help
+
+ def get_config(self):
+ """Returns the Config instance that represents the configuration this
+ symbol is from."""
+ return self.config
+
+ def get_def_locations(self):
+ """Returns a list of (filename, linenr) tuples, where filename (string)
+ and linenr (int) represent a location where the symbol is defined. For
+ the vast majority of symbols this list will only contain one element.
+ For the following Kconfig, FOO would get two entries: the lines marked
+ with *.
+
+ config FOO *
+ bool "foo prompt 1"
+
+ config FOO *
+ bool "foo prompt 2"
+ """
+ return self.def_locations
+
+ def get_ref_locations(self):
+ """Returns a list of (filename, linenr) tuples, where filename (string)
+ and linenr (int) represent a location where the symbol is referenced in
+ the configuration. For example, the lines marked by * would be included
+ for FOO below:
+
+ config A
+ bool
+ default BAR || FOO *
+
+ config B
+ tristate
+ depends on FOO *
+ default m if FOO *
+
+ if FOO *
+ config A
+ bool "A"
+ endif
+
+ config FOO (definition not included)
+ bool
+ """
+ return self.ref_locations
+
+ def is_modifiable(self):
+ """Returns True or False depending on if the value of symbol could be
+ modified by setting a user value with Symbol.set_value(). This
+ corresponds to symbols that would appear in the 'make menuconfig'
+ interface and not already be pinned to a specific value by being
+ selected. Returns False for special symbols (e.g. n, m and y)."""
+ if self.is_special_:
+ return False
+
+ allowed = range(self.config._eval_to_int(self.config._eval_expr(self.rev_dep)),
+ self.config._eval_to_int(self._calc_visibility()) + 1)
+
+ return len(allowed) > 1
+
+ def is_defined(self):
+ """Returns False if the symbol is referred to in the Kconfig but never
+ actually defined, otherwise True."""
+ return self.is_defined_
+
+ def is_special(self):
+ """Returns True if the symbol is one of the special symbols n, m or y,
+ or gets its value from the environment. Otherwise, returns False."""
+ return self.is_special_
+
+ def is_from_environment(self):
+ """Returns True if the symbol gets its value from the environment.
+ Otherwise, returns False."""
+ return self.is_from_env
+
+ def has_ranges(self):
+ """Returns True if the symbol is of type INT or HEX and has ranges that
+ limits what values it can take on, otherwise False."""
+ return self.ranges != []
+
+ def is_choice_item(self):
+ """Returns True if the symbol is in a choice statement and is an actual
+ choice item (see Choice.get_actual_items()); otherwise, returns
+ False."""
+ return self.is_choice_item_
+
+ def is_choice_selection(self):
+ """Returns True if the symbol is contained in a choice statement and is
+ the selected item, otherwise False. Equivalent to 'sym.is_choice_item()
+ and sym.get_parent().get_selection() is sym'."""
+ return self.is_choice_item_ and \
+ self.parent.get_selection() is self
+
+ def __str__(self):
+ """Returns a string containing various information about the symbol."""
+ return self.config._get_sym_or_choice_str(self)
+
+ #
+ # Private methods
+ #
+
+ def __init__(self):
+ """Symbol constructor -- not intended to be called directly by
+ kconfiglib clients."""
+
+ # Set default values
+ _HasVisibility.__init__(self)
+
+ self.config = None
+
+ self.parent = None
+ self.name = None
+ self.type = UNKNOWN
+
+ self.def_exprs = []
+ self.ranges = []
+ self.rev_dep = "n"
+
+ # The prompt, default value and select conditions without any
+ # dependencies from menus or if's propagated to them
+
+ self.orig_prompts = []
+ self.orig_def_exprs = []
+ self.orig_selects = []
+
+ # Dependencies inherited from containing menus and if's
+ self.deps_from_containing = None
+
+ self.help = None
+
+ # The set of symbols referenced by this symbol (see
+ # get_referenced_symbols())
+ self.referenced_syms = set()
+
+ # The set of symbols selected by this symbol (see
+ # get_selected_symbols())
+ self.selected_syms = set()
+
+ # Like 'referenced_syms', but includes symbols from
+ # dependencies inherited from enclosing menus and if's
+ self.all_referenced_syms = set()
+
+ # This is set to True for "actual" choice items. See
+ # Choice._determine_actual_items(). The trailing underscore avoids a
+ # collision with is_choice_item().
+ self.is_choice_item_ = False
+
+ # This records only dependencies specified with 'depends on'. Needed
+ # when determining actual choice items (hrrrr...). See also
+ # Choice._determine_actual_items().
+ self.menu_dep = None
+
+ # See Symbol.get_ref/def_locations().
+ self.def_locations = []
+ self.ref_locations = []
+
+ self.user_val = None
+
+ # Flags
+
+ # Should the symbol get an entry in .config?
+ self.write_to_conf = False
+
+ # Stores the calculated value to avoid unnecessary recalculation
+ self.cached_value = None
+
+ # Does the symbol have an entry in the Kconfig file? The Trailing
+ # underscore avoids a collision with is_defined().
+ self.is_defined_ = False
+
+ # Does the symbol get its value in some special way, e.g. from the
+ # environment or by being one of the special symbols n, m, and y? If
+ # so, the value is stored in self.cached_value, which is never
+ # invalidated. The trailing underscore avoids a collision with
+ # is_special().
+ self.is_special_ = False
+
+ # Does the symbol get its value from the environment?
+ self.is_from_env = False
+
+ # See Choice._make_conf()
+ self.already_written = False
+
+ def _invalidate(self):
+ if self.is_special_:
+ return
+
+ if self.is_choice_item():
+ self.parent._invalidate()
+
+ _HasVisibility._invalidate(self)
+
+ self.write_to_conf = False
+ self.cached_value = None
+
+ def _invalidate_dependent(self):
+ for sym in self._get_dependent():
+ sym._invalidate()
+
+ def _set_value_no_invalidate(self, v, suppress_load_warnings):
+ """Like set_value(), but does not invalidate any symbols.
+
+ suppress_load_warnings --
+ some warnings don't make sense when loading a .config that do make
+ sense when manually invoking set_value(). This flag is set to True to
+ suppress such warnings."""
+
+ if self.is_special_:
+ if self.is_from_env:
+ self.config._warn('attempt to assign the value "{0}" to the '
+ 'symbol {1}, which gets its value from the '
+ 'environment. Assignment ignored.'
+ .format(v, self.name))
+ else:
+ self.config._warn('attempt to assign the value "{0}" to the '
+ 'special symbol {1}. Assignment ignored.'
+ .format(v, self.name))
+
+ return
+
+
+ if not self.is_defined_:
+ filename, linenr = self.ref_locations[0]
+
+ self.config._undef_assign('attempt to assign the value "{0}" to {1}, '
+ "which is referenced at {2}:{3} but never "
+ "defined. Assignment ignored."
+ .format(v, self.name, filename, linenr))
+ return
+
+ # Check if the value is valid for our type
+
+ valid = ( self.type == BOOL and v in ("n", "y") ) or \
+ ( self.type == TRISTATE and v in ("n", "m", "y") ) or \
+ ( self.type == STRING ) or \
+ ( self.type == INT and _is_dec(v) ) or \
+ ( self.type == HEX and _is_hex(v) )
+
+ if not valid:
+ self.config._warn('the value "{0}" is invalid for {1}, which has type {2}. '
+ "Assignment ignored."
+ .format(v, self.name, typename[self.type]))
+ return
+
+ if self.prompts == [] and not suppress_load_warnings:
+ self.config._warn('assigning "{0}" to the symbol {1} which lacks '
+ 'prompts and thus has visibility "n". The assignment '
+ 'will have no effect.'
+ .format(v, self.name))
+
+ self.user_val = v
+
+ if self.is_choice_item() and self.type in (BOOL, TRISTATE):
+ choice = self.parent
+ if v == "y":
+ choice.user_val = self
+ choice.user_mode = "y"
+ elif v == "m":
+ choice.user_val = None
+ choice.user_mode = "m"
+
+ def _reset_no_recursive_invalidate(self):
+ self._invalidate()
+ self.user_val = None
+
+ if self.is_choice_item():
+ self.parent._reset()
+
+ def _should_write(self):
+ # This check shouldn't be necessary as write_to_conf is never modified
+ # in calc_value() for special symbols, but just to be on the safe side:
+ if self.is_special_:
+ return False
+
+ # Symbols defined in multiple locations only get one entry in the
+ # .config.
+ if self.already_written:
+ return False
+
+ # write_to_conf is determined in calc_value(), so we need to call that
+ # first
+ self.calc_value()
+
+ return self.write_to_conf
+
+ def _make_conf(self):
+ if not self._should_write():
+ return []
+
+ self.already_written = True
+
+ if self.type in (BOOL, TRISTATE):
+ if self.calc_value() in ("m", "y"):
+ return ["CONFIG_{0}={1}".format(self.name, self.calc_value())]
+ return ["# CONFIG_{0} is not set".format(self.name)]
+
+ elif self.type == STRING:
+ return ['CONFIG_{0}="{1}"'.format(self.name, self.calc_value())]
+
+ elif self.type in (INT, HEX):
+ return ["CONFIG_{0}={1}".format(self.name, self.calc_value())]
+
+ else:
+ _internal_error('Internal error while creating .config: unknown type "{0}".'
+ .format(self.type))
+
+ def _get_dependent(self):
+ """Returns the list of symbols that should be invalidated if the value
+ of the symbol changes."""
+ res = set()
+
+ def rec(sym, ignore_choice = False):
+ for s in self.config.dep[sym]:
+ if s not in res:
+ res.add(s)
+ rec(s)
+
+ # Handle choices specially to avoid lots of hopping around between
+ # choice items (which all depend on each other) while calculating
+ # dependencies
+ if sym.is_choice_item() and not ignore_choice:
+ choice = sym.get_parent()
+
+ for s in choice.get_actual_items():
+ if s not in res and s is not self:
+ res.add(s)
+ rec(s, True)
+
+ rec(self)
+ return res
+
+ def _has_auto_menu_dep_on(self, on):
+ """See Choice._determine_actual_items()."""
+ if not isinstance(self.parent, Choice):
+ _internal_error("Attempt to determine auto menu dependency for symbol ouside of choice.")
+
+ if self.prompts == []:
+ # If we have no prompt, use the menu dependencies instead (what was
+ # specified with 'depends on')
+ return self.menu_dep is not None and \
+ self.config._expr_depends_on(self.menu_dep, on)
+
+ for (_, cond_expr) in self.prompts:
+ if self.config._expr_depends_on(cond_expr, on):
+ return True
+
+ return False
+
+ def _is_assignable_bool_or_tristate(self):
+ """Returns True if the symbol is a bool or tristate whose value can be
+ changed by the user."""
+ return self.type in (BOOL, TRISTATE) and \
+ not self.is_special_ and \
+ (self.config._eval_to_int(self._calc_visibility()) -
+ self.config._eval_to_int(self.config._eval_expr(self.rev_dep))) >= 1
+
+class Menu(Item):
+
+ """Represents a menu statement."""
+
+ #
+ # Public interface
+ #
+
+ def get_depends_on_visibility(self):
+ """Returns the visibility the menu gets from 'depends on' conditions.
+ This is propagated to subitems."""
+ return self.config._eval_expr(self.dep_expr)
+
+ def get_visible_if_visibility(self):
+ """Returns the visibility the menu gets from its 'visible if'
+ condition. "y" is the menu has no 'visible if' condition."""
+ return self.config._eval_expr(self.visible_if_expr)
+
+ def get_items(self, recursive = False):
+ """Returns a list containing the items (symbols, menus, choice
+ statements and comments) in in the menu, in the same order that the
+ items appear within the menu.
+
+ recursive (default: False) -- True if items contained in items within
+ the menu should be included
+ recursively (preorder)."""
+
+ if not recursive:
+ return self.block.get_items()
+
+ res = []
+ for item in self.block.get_items():
+ res.append(item)
+ if isinstance(item, Menu):
+ res.extend(item.get_items(True))
+ elif isinstance(item, Choice):
+ res.extend(item.get_items())
+ return res
+
+ def get_symbols(self, recursive = False):
+ """Returns a list containing the symbols in the menu, in the same order
+ that they appear within the menu.
+
+ recursive (default: False) -- True if symbols contained in items within
+ the menu should be included
+ recursively."""
+
+ return [item for item in self.get_items(recursive) if isinstance(item, Symbol)]
+
+ def get_title(self):
+ """Returns the title text of the menu."""
+ return self.title
+
+ def get_parent(self):
+ """Returns the menu or choice statement that contains the menu, or
+ None if the menu is at the top level. Note that if statements are
+ treated as syntactic sugar and do not have an explicit class
+ representation."""
+ return self.parent
+
+ def get_referenced_symbols(self, refs_from_enclosing = False):
+ """See Symbol.get_referenced_symbols()."""
+ return self.all_referenced_syms if refs_from_enclosing else self.referenced_syms
+
+ def get_location(self):
+ """Returns the location of the menu as a (filename, linenr) tuple,
+ where filename is a string and linenr an int."""
+ return (self.filename, self.linenr)
+
+ def __str__(self):
+ """Returns a string containing various information about the menu."""
+ depends_on_str = self.config._expr_val_str(self.orig_deps,
+ "(no dependencies)")
+ visible_if_str = self.config._expr_val_str(self.visible_if_expr,
+ "(no dependencies)")
+
+ additional_deps_str = " " + self.config._expr_val_str(self.deps_from_containing,
+ "(no additional dependencies)")
+
+ return _sep_lines("Menu",
+ "Title : " + self.title,
+ "'depends on' dependencies : " + depends_on_str,
+ "'visible if' dependencies : " + visible_if_str,
+ "Additional dependencies from enclosing menus and if's:",
+ additional_deps_str,
+ "Location: {0}:{1}".format(self.filename, self.linenr))
+
+ #
+ # Private methods
+ #
+
+ def __init__(self):
+ """Menu constructor -- not intended to be called directly by
+ kconfiglib clients."""
+
+ self.config = None
+
+ self.parent = None
+ self.title = None
+ self.block = None
+ self.dep_expr = None
+
+ # Dependency expression without dependencies from enclosing menus and
+ # if's propagated
+ self.orig_deps = None
+
+ # Dependencies inherited from containing menus and if's
+ self.deps_from_containing = None
+
+ # The 'visible if' expression
+ self.visible_if_expr = None
+
+ # The set of symbols referenced by this menu (see
+ # get_referenced_symbols())
+ self.referenced_syms = set()
+
+ # Like 'referenced_syms', but includes symbols from
+ # dependencies inherited from enclosing menus and if's
+ self.all_referenced_syms = None
+
+ self.filename = None
+ self.linenr = None
+
+ def _make_conf(self):
+ item_conf = self.block._make_conf()
+
+ if self.config._eval_expr(self.dep_expr) != "n" and \
+ self.config._eval_expr(self.visible_if_expr) != "n":
+ return ["\n#\n# {0}\n#".format(self.title)] + item_conf
+ else:
+ return item_conf
+
+class Choice(Item, _HasVisibility):
+
+ """Represents a choice statement. A choice can be in one of three modes:
+ "n", "m" and "y". "n" mode is for non-visible choices and optional choices
+ with no symbol selected; "m" means any number of symbols can be set to "m"
+ while the rest will be "n" (only tristate choices can be in this mode); and
+ "y" means one symbol will be "y" while the rest will be "n" (the most
+ common case). The visibility is an upper bound on the mode, and the mode
+ changes automatically as values are assigned to symbols within the
+ choice."""
+
+ #
+ # Public interface
+ #
+
+ def get_selection(self):
+ """Returns the symbol selected (either by the user or through
+ defaults), or None if either no symbol is selected or the mode is not
+ "y"."""
+ if self.cached_selection is not None:
+ if self.cached_selection == NO_SELECTION:
+ return None
+ return self.cached_selection
+
+ if self.calc_mode() != "y":
+ return self._cache_ret(None)
+
+ # User choice available?
+ if self.user_val is not None and \
+ self.user_val._calc_visibility() == "y":
+ return self._cache_ret(self.user_val)
+
+ if self.optional:
+ return self._cache_ret(None)
+
+ return self._cache_ret(self.get_selection_from_defaults())
+
+ def get_selection_from_defaults(self):
+ """Like Choice.get_selection(), but acts as if no symbol has been
+ selected by the user and no 'optional' flag is in effect."""
+
+ if self.actual_items == []:
+ return None
+
+ for (symbol, cond_expr) in self.def_exprs:
+ if self.config._eval_expr(cond_expr) != "n":
+ chosen_symbol = symbol
+ break
+ else:
+ chosen_symbol = self.actual_items[0]
+
+ # Is the chosen symbol visible?
+ if chosen_symbol._calc_visibility() != "n":
+ return chosen_symbol
+ else:
+ # Otherwise, pick the first visible symbol
+ for sym in self.actual_items:
+ if sym._calc_visibility() != "n":
+ return sym
+
+ return None
+
+ def get_user_selection(self):
+ """If the choice is in "y" mode and has a user-selected
+ symbol, returns that symbol. Otherwise, returns None."""
+ return self.user_val
+
+ def get_name(self):
+ """For named choices, returns the name. Returns None for unnamed
+ choices. No named choices appear anywhere in the kernel Kconfig
+ files as of Linux 2.6.38-rc3."""
+ return self.name
+
+ def get_type(self):
+ """Returns the type of the choice. See Symbol.get_type()."""
+ return self.type
+
+ def get_items(self):
+ """Gets all items contained in the choice in the same order as within
+ the configuration ("items" instead of "symbols" since choices and
+ comments might appear within choices. This only happens in one place as
+ of Linux 2.6.38-rc3, in drivers/usb/gadget/Kconfig)."""
+ return self.block.get_items()
+
+ def get_actual_items(self):
+ """A quirk (perhaps better described as a bug -- at least for symbols)
+ of kconfig is that you can put items within a choice that will not be
+ considered members of the choice insofar as selection is concerned.
+ This happens for example if one symbol within a choice 'depends on' the
+ symbol preceding it, or if you put non-symbol items within choices.
+
+ This function gets a list of the "proper" elements of the choice,
+ excluding such items."""
+ return self.actual_items
+
+ def get_parent(self):
+ """Returns the menu or choice statement that contains the choice, or
+ None if the choice is at the top level. Note that if statements are
+ treated as syntactic sugar and do not have an explicit class
+ representation."""
+ return self.parent
+
+ def get_referenced_symbols(self, refs_from_enclosing = False):
+ """See Symbol.get_referenced_symbols()."""
+ return self.all_referenced_syms if refs_from_enclosing else self.referenced_syms
+
+ def get_def_locations(self):
+ """Returns a list of (filename, linenr) tuples, where filename (string)
+ and linenr (int) represent a location where the choice is defined. For
+ the vast majority of choices this list will only contain one element,
+ but its possible for named choices to be defined in multiple
+ locations."""
+ return self.def_locations
+
+ def get_visibility(self):
+ """Returns the visibility of the choice statement: one of "n", "m" or
+ "y". This acts as an upper limit on the mode of the choice (though bool
+ choices can only have the mode "y"). See the class documentation for an
+ explanation of modes."""
+ return self._calc_visibility()
+
+ def calc_mode(self):
+ """Returns the mode of the choice. See the class documentation for
+ an explanation of modes."""
+ minimum_mode = "n" if self.optional else "m"
+
+ mode = self.user_mode if self.user_mode is not None else minimum_mode
+ mode = self.config._eval_min(mode, self._calc_visibility())
+
+ # Promote "m" to "y" for boolean choices
+ if mode == "m" and self.type == BOOL:
+ mode = "y"
+
+ return mode
+
+ def is_optional(self):
+ """Returns True if the symbol has the optional flag set (and so will default
+ to "n" mode). Otherwise, returns False."""
+ return self.optional
+
+ def __str__(self):
+ """Returns a string containing various information about the choice
+ statement."""
+ return self.config._get_sym_or_choice_str(self)
+
+ #
+ # Private methods
+ #
+
+ def __init__(self):
+ """Choice constructor -- not intended to be called directly by
+ kconfiglib clients."""
+
+ _HasVisibility.__init__(self)
+
+ self.config = None
+
+ self.parent = None
+ self.name = None # Yes, choices can be named
+ self.type = UNKNOWN
+ self.def_exprs = []
+ self.help = None
+ self.optional = False
+ self.block = None
+
+ # The prompts and default values without any dependencies from
+ # enclosing menus or if's propagated
+
+ self.orig_prompts = []
+ self.orig_def_exprs = []
+
+ # Dependencies inherited from containing menus and if's
+ self.deps_from_containing = None
+
+ # We need to filter out symbols that appear within the choice block but
+ # are not considered choice items (see
+ # Choice._determine_actual_items()) This list holds the "actual" choice
+ # items.
+ self.actual_items = []
+
+ # The set of symbols referenced by this choice (see
+ # get_referenced_symbols())
+ self.referenced_syms = set()
+
+ # Like 'referenced_syms', but includes symbols from
+ # dependencies inherited from enclosing menus and if's
+ self.all_referenced_syms = set()
+
+ # See Choice.get_def_locations()
+ self.def_locations = []
+
+ self.user_val = None
+ self.user_mode = None
+
+ self.cached_selection = None
+
+ def _determine_actual_items(self):
+ """If a symbol's visibility depends on the preceding symbol within a
+ choice, it is no longer viewed as a choice item (quite possibly a bug,
+ but some things consciously use it.. ugh. It stems from automatic
+ submenu creation). In addition, it's possible to have choices and
+ comments within choices, and those shouldn't be considered as choice
+ items either. Only drivers/usb/gadget/Kconfig seems to depend on any of
+ this. This method computes the "actual" items in the choice and sets
+ the is_choice_item_ flag on them (retrieved via is_choice_item()).
+
+ Don't let this scare you: an earlier version simply checked for a
+ sequence of symbols where all symbols after the first appeared in the
+ 'depends on' expression of the first, and that worked fine. The added
+ complexity is to be future-proof in the event that
+ drivers/usb/gadget/Kconfig turns even more sinister. It might very well
+ be overkilling things (especially if that file is refactored ;)."""
+
+ items = self.block.get_items()
+
+ # Items might depend on each other in a tree structure, so we need a
+ # stack to keep track of the current tentative parent
+ stack = []
+
+ for item in items:
+ if not isinstance(item, Symbol):
+ stack = []
+ continue
+
+ while stack != []:
+ if item._has_auto_menu_dep_on(stack[-1]):
+ # The item should not be viewed as a choice item, so don't
+ # set item.is_choice_item_.
+ stack.append(item)
+ break
+ else:
+ stack.pop()
+ else:
+ item.is_choice_item_ = True
+ self.actual_items.append(item)
+ stack.append(item)
+
+ def _cache_ret(self, selection):
+ # As None is used to indicate the lack of a cached value we can't use
+ # that to cache the fact that the choice has no selection. Instead, we
+ # use the symbolic constant NO_SELECTION.
+ if selection is None:
+ self.cached_selection = NO_SELECTION
+ else:
+ self.cached_selection = selection
+
+ return selection
+
+ def _get_user_mode(self):
+ return self.user_mode
+
+ def _invalidate(self):
+ _HasVisibility._invalidate(self)
+ self.cached_selection = None
+
+ def _reset(self):
+ self._invalidate()
+ self.user_val = None
+ self.user_mode = None
+
+ def _make_conf(self):
+ return self.block._make_conf()
+
+class Comment(Item):
+
+ """Represents a comment statement."""
+
+ #
+ # Public interface
+ #
+
+ def get_text(self):
+ """Returns the text of the comment."""
+ return self.text
+
+ def get_parent(self):
+ """Returns the menu or choice statement that contains the comment, or
+ None if the comment is at the top level. Note that if statements are
+ treated as syntactic sugar and do not have an explicit class
+ representation."""
+ return self.parent
+
+ def get_referenced_symbols(self, refs_from_enclosing = False):
+ """See Symbol.get_referenced_symbols()."""
+ return self.all_referenced_syms if refs_from_enclosing else self.referenced_syms
+
+ def get_location(self):
+ """Returns the location of the comment as a (filename, linenr) tuple,
+ where filename is a string and linenr an int."""
+ return (self.filename, self.linenr)
+
+ def __str__(self):
+ """Returns a string containing various information about the comment."""
+ dep_str = self.config._expr_val_str(self.orig_deps, "(no dependencies)")
+
+ additional_deps_str = " " + self.config._expr_val_str(self.deps_from_containing,
+ "(no additional dependencies)")
+
+ return _sep_lines("Comment",
+ "Text: " + str(self.text),
+ "Dependencies: " + dep_str,
+ "Additional dependencies from enclosing menus and if's:",
+ additional_deps_str,
+ "Location: {0}:{1}".format(self.filename, self.linenr))
+
+ #
+ # Private methods
+ #
+
+ def __init__(self):
+ """Comment constructor -- not intended to be called directly by
+ kconfiglib clients."""
+
+ self.config = None
+
+ self.parent = None
+ self.text = None
+ self.dep_expr = None
+
+ # Dependency expression without dependencies from enclosing menus and
+ # if's propagated
+ self.orig_deps = None
+
+ # Dependencies inherited from containing menus and if's
+ self.deps_from_containing = None
+
+ # The set of symbols referenced by this comment (see
+ # get_referenced_symbols())
+ self.referenced_syms = set()
+
+ # Like 'referenced_syms', but includes symbols from
+ # dependencies inherited from enclosing menus and if's
+ self.all_referenced_syms = None
+
+ self.filename = None
+ self.linenr = None
+
+ def _make_conf(self):
+ if self.config._eval_expr(self.dep_expr) != "n":
+ return ["\n#\n# {0}\n#".format(self.text)]
+ else:
+ return []
+
+class _Feed:
+
+ """Class for working with sequences in a stream-like fashion; handy for tokens."""
+
+ def __init__(self, items):
+ self.items = items
+ self.length = len(self.items)
+ self.i = 0
+
+ def get_next(self):
+ if self.i >= self.length:
+ return None
+
+ item = self.items[self.i]
+ self.i += 1
+ return item
+
+ def peek_next(self):
+ return None if self.i >= self.length else self.items[self.i]
+
+ def go_to_start(self):
+ self.i = 0
+
+ def __getitem__(self, index):
+ return self.items[index]
+
+ def __len__(self):
+ return len(self.items)
+
+ def is_empty(self):
+ return self.items == []
+
+ def is_at_end(self):
+ return self.i >= self.length
+
+ def check(self, token):
+ """Check if the next token is 'token'. If so, remove it from the token
+ feed and return True. Otherwise, leave it in and return False."""
+ if self.i >= self.length:
+ return None
+
+ if self.items[self.i] == token:
+ self.i += 1
+ return True
+
+ return False
+
+ def remove_while(self, pred):
+ while self.i < self.length and pred(self.items[self.i]):
+ self.i += 1
+
+ def go_back(self):
+ if self.i <= 0:
+ _internal_error("Attempt to move back in Feed while already at the beginning.")
+ self.i -= 1
+
+class _FileFeed(_Feed):
+
+ """Feed subclass that keeps track of the current filename and line
+ number."""
+
+ def __init__(self, lines, filename):
+ self.filename = _clean_up_path(filename)
+ _Feed.__init__(self, lines)
+
+ def get_filename(self):
+ return self.filename
+
+ def get_linenr(self):
+ return self.i
+
+#
+# Misc. public global utility functions
+#
+
+def tri_less(v1, v2):
+ """Returns True if the tristate v1 is less than the tristate v2, where "n",
+ "m" and "y" are ordered from lowest to highest. Otherwise, returns
+ False."""
+ return values[v1] < values[v2]
+
+def tri_less_eq(v1, v2):
+ """Returns True if the tristate v1 is less than or equal to the tristate
+ v2, where "n", "m" and "y" are ordered from lowest to highest. Otherwise,
+ returns False."""
+ return values[v1] <= values[v2]
+
+def tri_greater(v1, v2):
+ """Returns True if the tristate v1 is greater than the tristate v2, where
+ "n", "m" and "y" are ordered from lowest to highest. Otherwise, returns
+ False."""
+ return values[v1] > values[v2]
+
+def tri_greater_eq(v1, v2):
+ """Returns True if the tristate v1 is greater than or equal to the tristate
+ v2, where "n", "m" and "y" are ordered from lowest to highest. Otherwise,
+ returns False."""
+ return values[v1] >= values[v2]
+
+#
+# Helper functions, mostly related to text processing
+#
+
+def _strip_quotes(s, line, filename, linenr):
+ """Removes any quotes surrounding 's' if it has them; otherwise returns 's'
+ unmodified."""
+ s = s.strip()
+ if s[0] == '"' or s[0] == "'":
+ if len(s) < 2 or s[-1] != s[0]:
+ _parse_error(line,
+ "malformed string literal",
+ filename,
+ linenr)
+ return s[1:-1]
+ else:
+ return s
+
+def _indentation(line):
+ """Returns the indentation of the line, treating tab stops as being spaced
+ 8 characters apart."""
+ if line.isspace():
+ _internal_error("Attempt to take indentation of blank line.")
+ indent = 0
+ for c in line:
+ if c == " ":
+ indent += 1
+ elif c == "\t":
+ # Go to the next tab stop
+ indent = (indent + 8) & ~7
+ else:
+ return indent
+
+def _deindent(line, indent):
+ """Deindent 'line' by 'indent' spaces."""
+ line = line.expandtabs()
+ if len(line) <= indent:
+ return line
+ return line[indent:]
+
+def _is_hex(s):
+ return _is_base_n(s, 16)
+
+def _is_dec(s):
+ return _is_base_n(s, 10)
+
+def _is_base_n(s, n):
+ try:
+ int(s, n)
+ return True
+ except ValueError:
+ return False
+
+def _sep_lines(*args):
+ """Returns a string comprised of all arguments, with newlines inserted
+ between them."""
+ return "\n".join(args)
+
+def _comment(s):
+ """Returns a new string with "# " inserted before each line in 's'."""
+ return "\n".join(["# " + line for line in s.splitlines()])
+
+def _get_lines(filename):
+ """Returns a list of lines from 'filename', joining any line ending in \\
+ with the following line."""
+ with open(filename, "r") as f:
+ lines = []
+ accum = ""
+ while True:
+ line = f.readline()
+
+ if line == "":
+ return lines
+
+ if line.endswith("\\\n"):
+ accum += line[:-2]
+ else:
+ accum += line
+ lines.append(accum)
+ accum = ""
+
+def _strip_trailing_slash(path):
+ """Removes any trailing slash from 'path'."""
+ return path[:-1] if path.endswith("/") else path
+
+def _clean_up_path(path):
+ """Strips any initial "./" and trailing slash from 'path'."""
+ if path.startswith("./"):
+ path = path[2:]
+ return _strip_trailing_slash(path)
+
+#
+# Error handling
+#
+
+class Kconfig_Syntax_Error(Exception):
+ """Exception raised for syntax errors."""
+ pass
+
+class Internal_Error(Exception):
+ """Exception raised for internal errors."""
+ pass
+
+def _tokenization_error(s, index, filename, linenr):
+ if filename is not None:
+ assert linenr is not None
+ sys.stderr.write("{0}:{1}:\n".format(filename, linenr))
+
+ if s.endswith("\n"):
+ s = s[:-1]
+
+ # Calculate the visual offset corresponding to index 'index' in 's'
+ # assuming tabstops are spaced 8 characters apart
+ vis_index = 0
+ for c in s[:index]:
+ if c == "\t":
+ vis_index = (vis_index + 8) & ~7
+ else:
+ vis_index += 1
+
+ # Don't output actual tabs to be independent of how the terminal renders
+ # them
+ s = s.expandtabs()
+
+ raise Kconfig_Syntax_Error, (
+ _sep_lines("Error during tokenization at location indicated by caret.\n",
+ s,
+ " " * vis_index + "^\n"))
+
+def _parse_error(s, msg, filename, linenr):
+ error_str = ""
+
+ if filename is not None:
+ assert linenr is not None
+ error_str += "{0}:{1}: ".format(filename, linenr)
+
+ if s.endswith("\n"):
+ s = s[:-1]
+
+ error_str += 'Error while parsing "{0}"'.format(s)
+
+ if msg is None:
+ error_str += "."
+ else:
+ error_str += ": " + msg
+
+ raise Kconfig_Syntax_Error, error_str
+
+def _internal_error(msg):
+ msg += "\nSorry! You may want to send an email to [email protected] " \
+ "to tell me about this. Include the message above and the stack " \
+ "trace and describe what you were doing."
+
+ raise Internal_Error, msg
+
+if use_psyco:
+ import psyco
+
+ Config._tokenize = psyco.proxy(Config._tokenize)
+ Config._eval_expr = psyco.proxy(Config._eval_expr)
+
+ _indentation = psyco.proxy(_indentation)
+ _get_lines = psyco.proxy(_get_lines)
diff --git a/scripts/kconfig/kconfigtest.py b/scripts/kconfig/kconfigtest.py
new file mode 100644
index 0000000..9d27dca
--- /dev/null
+++ b/scripts/kconfig/kconfigtest.py
@@ -0,0 +1,396 @@
+# This is a test suite for kconfiglib, primarily testing compatibility with the
+# C kconfig implementation by comparing outputs. It should be run from the
+# top-level kernel directory with
+#
+# $ PYTHONPATH=scripts/kconfig python scripts/kconfig/kconfigtest.py
+#
+# Note that running these could take a long time: running all tests on a Core
+# [email protected] GHz takes ~2 hours. The tests have been arranged in order of time
+# needed.
+#
+# All tests should pass. Report regressions to [email protected]
+
+import kconfiglib
+import os
+import re
+import subprocess
+import sys
+
+# Assume that the value of KERNELVERSION does not affect the configuration
+# (true as of Linux 2.6.38-rc3). Here we could fetch the correct version
+# instead.
+os.environ["KERNELVERSION"] = "2"
+
+# Number of arch/defconfig pairs tested so far
+nconfigs = 0
+
+def run_tests():
+ # The set of tests that want to run for all architectures in the kernel
+ # tree -- currently, all tests. The boolean flag indicates whether .config
+ # (generated by the C implementation) should be compared to ._config
+ # (generated by us) after each invocation.
+ all_arch_tests = [(test_all_no, True),
+ (test_config_absent, True),
+ (test_all_yes, True),
+ (test_call_all, False),
+ # Needs to report success/failure for each arch/defconfig
+ # combo, hence False.
+ (test_defconfig, False)]
+
+ print "Loading Config instances for all architectures..."
+ arch_configs = get_arch_configs()
+
+ for (test_fn, compare_configs) in all_arch_tests:
+ print "Resetting all architecture Config instances prior to next test..."
+ for arch in arch_configs:
+ arch.reset()
+
+ # The test description is taken from the docstring of the corresponding
+ # function
+ print test_fn.__doc__
+
+ for conf in arch_configs:
+ rm_configs()
+
+ # This should be set correctly for any 'make *config' commands the
+ # test might run
+ os.environ["ARCH"] = conf.get_arch()
+
+ test_fn(conf)
+
+ if compare_configs:
+ sys.stdout.write(" {0:<14}".format(conf.get_arch()))
+
+ if equal_confs():
+ print "OK"
+ else:
+ print "FAIL"
+ fail()
+
+ print ""
+
+ if all_ok():
+ print "All OK"
+ print nconfigs, "arch/defconfig pairs tested"
+ else:
+ print "Some tests failed"
+
+def get_arch_configs():
+ """Returns a list with Config instances corresponding to all arch Kconfigs."""
+
+ def add_arch(ARCH, res):
+ os.environ["SRCARCH"] = archdir
+ os.environ["ARCH"] = ARCH
+ res.append(kconfiglib.Config())
+
+ res = []
+
+ # Nothing looks at this as of Linux 2.6.38-rc3
+ os.environ["KERNELVERSION"] = "2"
+
+ for archdir in os.listdir("arch"):
+ if archdir == "h8300":
+ # Broken Kconfig as of Linux 2.6.38-rc3
+ continue
+
+ if os.path.exists(os.path.join("arch", archdir, "Kconfig")):
+ if archdir == "x86":
+ add_arch("i386", res)
+ add_arch("x86_64", res)
+ elif archdir == "sparc":
+ add_arch("sparc32", res)
+ add_arch("sparc64", res)
+ elif archdir == "sh":
+ add_arch("sh64", res)
+ else:
+ # For most architectures, ARCH = SRCARCH
+ add_arch(archdir, res)
+
+ # Don't want subsequent 'make *config' commands in tests to see this
+ del os.environ["ARCH"]
+ del os.environ["SRCARCH"]
+
+ return res
+
+def test_all_no(conf):
+ """Test if our allnoconfig implementation generates the same .config as 'make allnoconfig', for all architectures"""
+
+ while True:
+ done = True
+
+ for sym in conf:
+ # Choices take care of themselves for allnoconf, so we only need to
+ # worry about non-choice symbols
+ if not sym.is_choice_item():
+ lower_bound = sym.get_lower_bound()
+
+ # If we can assign a lower value to the symbol (where "n", "m" and
+ # "y" are ordered from lowest to highest), then do so.
+ # lower_bound() returns None for symbols whose values cannot
+ # (currently) be changed, as well as for non-bool, non-tristate
+ # symbols.
+ if lower_bound is not None and \
+ kconfiglib.tri_less(lower_bound, sym.calc_value()):
+
+ sym.set_value(lower_bound)
+
+ # We just changed the value of some symbol. As this may effect
+ # other symbols, we need to keep looping.
+ done = False
+
+ if done:
+ break
+
+ conf.write_config("._config")
+
+ shell("make allnoconfig")
+
+def test_all_yes(conf):
+ """Test if our allyesconfig implementation generates the same .config as 'make allyesconfig', for all architectures"""
+
+ # Get a list of all symbols that are not choice items
+ non_choice_syms = [sym for sym in conf.get_symbols() if
+ not sym.is_choice_item()]
+
+ while True:
+ done = True
+
+ # Handle symbols outside of choices
+
+ for sym in non_choice_syms:
+ upper_bound = sym.get_upper_bound()
+
+ # See corresponding comment for allnoconf implementation
+ if upper_bound is not None and \
+ kconfiglib.tri_less(sym.calc_value(), upper_bound):
+ sym.set_value(upper_bound)
+ done = False
+
+ # Handle symbols within choices
+
+ for choice in conf.get_choices():
+
+ # Handle choices whose visibility allow them to be in "y" mode
+
+ if choice.get_visibility() == "y":
+ selection = choice.get_selection_from_defaults()
+ if selection is not None and \
+ selection is not choice.get_user_selection():
+ selection.set_value("y")
+ done = False
+
+ # Handle choices whose visibility only allow them to be in "m" mode
+
+ elif choice.get_visibility() == "m":
+ for sym in choice.get_items():
+ if sym.calc_value() != "m" and \
+ sym.get_upper_bound() != "n":
+ sym.set_value("m")
+ done = False
+
+
+ if done:
+ break
+
+ conf.write_config("._config")
+
+ shell("make allyesconfig")
+
+def test_call_all(conf):
+ """Call all public methods on all symbols, menus, choices and comments (nearly all public methods: some are hard to test like this, but are exercised by other tests) for all architectures to make sure we never crash or hang"""
+ print " For {0}...".format(conf.get_arch())
+
+ conf.get_arch()
+ conf.get_defconfig_filename()
+ conf.get_top_level_items()
+ conf.eval("y && ARCH")
+
+ # Syntax error
+ caught_exception = False
+ try:
+ conf.eval("y & y")
+ except kconfiglib.Kconfig_Syntax_Error:
+ caught_exception = True
+
+ if not caught_exception:
+ print "Fail: no exception generated for expression with syntax error"
+ fail()
+
+ conf.get_config_header()
+ conf.get_base_dir()
+ conf.reset()
+ conf.get_symbols(False)
+ conf.get_mainmenu_text()
+
+ for s in conf.get_symbols():
+ s.reset()
+ s.calc_value()
+ s.calc_default_value()
+ s.get_user_value()
+ s.get_name()
+ s.get_upper_bound()
+ s.get_lower_bound()
+ s.get_assignable_values()
+ s.get_type()
+ s.get_visibility()
+ s.get_parent()
+ s.get_sibling_symbols()
+ s.get_sibling_items()
+ s.get_referenced_symbols()
+ s.get_referenced_symbols(True)
+ s.get_selected_symbols()
+ s.get_help()
+ s.get_config()
+ s.get_def_locations()
+ s.get_ref_locations()
+ s.is_modifiable()
+ s.is_defined()
+ s.is_special()
+ s.is_from_environment()
+ s.has_ranges()
+ s.is_choice_item()
+ s.is_choice_selection()
+ s.__str__()
+
+ for c in conf.get_choices():
+ if c.get_name() is not None:
+ print "LOOOOOOOOL", c.get_name()
+ c.get_selection()
+ c.get_selection_from_defaults()
+ c.get_user_selection()
+ c.get_type()
+ c.get_name()
+ c.get_items()
+ c.get_actual_items()
+ c.get_parent()
+ c.get_referenced_symbols()
+ c.get_referenced_symbols(True)
+ c.get_def_locations()
+ c.get_visibility()
+ c.calc_mode()
+ c.is_optional()
+ c.__str__()
+
+ for m in conf.get_menus():
+ m.get_items()
+ m.get_symbols(False)
+ m.get_symbols(True)
+ m.get_depends_on_visibility()
+ m.get_visible_if_visibility()
+ m.get_title()
+ m.get_parent()
+ m.get_referenced_symbols()
+ m.get_referenced_symbols(True)
+ m.get_location()
+ m.__str__()
+
+ for c in conf.get_comments():
+ c.get_text()
+ c.get_parent()
+ c.get_referenced_symbols()
+ c.get_referenced_symbols(True)
+ c.get_location()
+ c.__str__()
+
+def test_config_absent(conf):
+ """Test if kconfiglib generates the same configuration as 'conf' without a .config, for each architecture"""
+ conf.write_config("._config")
+
+ # Use an empty .config
+ shell("> .config")
+ shell("make kconfiglibtestconfig")
+
+def test_defconfig(conf):
+ """Test if kconfiglib generates the same .config as conf for each architecture/defconfig pair (this takes two hours on a Core [email protected] GHz system)"""
+ # Collect defconfigs. This could be done once instead, but it's a speedy
+ # operation comparatively.
+
+ global nconfigs
+
+ defconfigs = []
+
+ for arch in os.listdir("arch"):
+ arch_dir = os.path.join("arch", arch)
+
+ # Some arches have a "defconfig" in their
+ # arch directory.
+ defconfig = os.path.join(arch_dir, "defconfig")
+ if os.path.exists(defconfig):
+ defconfigs.append(defconfig)
+
+ defconfigs_dir = os.path.join(arch_dir, "configs")
+ if os.path.isdir(defconfigs_dir):
+ for c in os.listdir(defconfigs_dir):
+ defconfig = os.path.join(defconfigs_dir, c)
+ if os.path.isfile(defconfig):
+ defconfigs.append(defconfig)
+
+ # Test architecture for each defconfig
+
+ for defconfig in defconfigs:
+ rm_configs()
+
+ nconfigs += 1
+
+ conf.load_config(defconfig)
+ conf.write_config("._config")
+ shell("cp {0} .config".format(defconfig))
+ shell("make kconfiglibtestconfig")
+
+ sys.stdout.write(" {0:<14}with {1:<60} ".format(conf.get_arch(), defconfig))
+
+ if equal_confs():
+ print "OK"
+ else:
+ print "FAIL"
+ fail()
+
+#
+# Helper functions
+#
+
+def shell(cmd):
+ subprocess.Popen(cmd,
+ shell = True,
+ stdout = subprocess.PIPE,
+ stderr = subprocess.PIPE).wait()
+
+def rm_configs():
+ """Delete any old ".config" (generated by the C implementation) and
+ "._config" (generated by us), if present."""
+ def rm_if_exists(f):
+ if os.path.exists(f):
+ os.remove(f)
+
+ rm_if_exists(".config")
+ rm_if_exists("._config")
+
+def equal_confs():
+ with open(".config") as menu_conf:
+ l1 = menu_conf.readlines()
+
+ with open("._config") as my_conf:
+ l2 = my_conf.readlines()
+
+ # Skip the header generated by 'conf'
+ unset_re = r"# CONFIG_(\w+) is not set"
+ i = 0
+ for line in l1:
+ if not line.startswith("#") or \
+ re.match(unset_re, line):
+ break
+ i += 1
+
+ return (l1[i:] == l2)
+
+_all_ok = True
+
+def fail():
+ global _all_ok
+ _all_ok = False
+
+def all_ok():
+ return _all_ok
+
+if __name__ == "__main__":
+ run_tests()
--
1.7.0.4

2011-02-02 00:47:20

by Arnaud Lacombe

Subject: Re: [PATCH] [ANNOUNCE] kconfig: Kconfiglib: a flexible Python Kconfig parser

Hi,

On Tue, Feb 1, 2011 at 6:27 PM, Ulf Magnusson <[email protected]> wrote:
> Hi,
>
> This is the initial release of Kconfiglib: a Python library for
> scripting, debugging, and extracting information from Kconfig-based
> configuration systems. ?It can be used to programmatically generate a
> .config when the '*conf' tools are too inflexible, to quickly find out
> interesting information about a Kconfig configuration such as dependency
> relations between symbols and where undefined symbols are referenced,
> and in applications that need to parse and extract information from
> Kconfig files.
>
Does that mean that when a kconfig-language change will happen, one
will not only must have to be Lex/YaCC and C fluent, but also, perl
and python ? That seem to be a lot of duplication to me.

I'd rather see the backend be changed so that it could be used with
SWIG to generate language bindings. I had WIP in this area, but
changes needed are rather intrusive. That said, the testsuite might be
interesting, I've been thinking about that for quite some time.

- Arnaud

> For a much longer introduction including multiple examples, see
> arch/kconfig/kconfiglib.py.
>
> Have fun!
>
> Signed-off-by: Ulf Magnusson <[email protected]>
> ---
> Convenience links:
>
> Documentation, generated from kconfiglib.py with pydoc -w:
> http://dl.dropbox.com/u/10406197/kconfiglib.html
>
> Examples as separate files:
> http://dl.dropbox.com/u/10406197/kconfiglib-examples.tar.gz
>
>
> The patch should be preferably be applied to a recent kernel, i.e. Linus's
> (2.6.38-rc3 at the time of writing). ?Due to recent Kconfig changes, the
> kconfigtest.py test suite - which compares output character-for-character -
> will indicate failure on older (a few months old) kernels versions even though
> the outputs are functionally equivalent.
>
> ?Documentation/kbuild/kconfig-language.txt | ? ?5 +
> ?Documentation/kbuild/kconfig.txt ? ? ? ? ?| ? ?8 +
> ?README ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| ? 13 +
> ?scripts/kconfig/Makefile ? ? ? ? ? ? ? ? ?| ? 26 +-
> ?scripts/kconfig/kconfiglib.py ? ? ? ? ? ? | 3918 +++++++++++++++++++++++++++++
> ?scripts/kconfig/kconfigtest.py ? ? ? ? ? ?| ?396 +++
> ?6 files changed, 4365 insertions(+), 1 deletions(-)
> ?create mode 100644 scripts/kconfig/kconfiglib.py
> ?create mode 100644 scripts/kconfig/kconfigtest.py
> [...]

2011-02-02 02:02:40

by Ulf Magnusson

Subject: Re: [PATCH] [ANNOUNCE] kconfig: Kconfiglib: a flexible Python Kconfig parser

On Tue, Feb 01, 2011 at 07:47:14PM -0500, Arnaud Lacombe wrote:
> Hi,
>
> On Tue, Feb 1, 2011 at 6:27 PM, Ulf Magnusson <[email protected]> wrote:
> > Hi,
> >
> > This is the initial release of Kconfiglib: a Python library for
> > scripting, debugging, and extracting information from Kconfig-based
> > configuration systems.  It can be used to programmatically generate a
> > .config when the '*conf' tools are too inflexible, to quickly find out
> > interesting information about a Kconfig configuration such as dependency
> > relations between symbols and where undefined symbols are referenced,
> > and in applications that need to parse and extract information from
> > Kconfig files.
> >
> Does that mean that when a kconfig-language change will happen, one
> will not only must have to be Lex/YaCC and C fluent, but also, perl
> and python ? That seem to be a lot of duplication to me.
>
> I'd rather see the backend be changed so that it could be used with
> SWIG to generate language bindings. I had WIP in this area, but
> changes needed are rather intrusive. That said, the testsuite might be
> interesting, I've been thinking about that for quite some time.
>
> - Arnaud
>
I originally experimented with patching the back end, but as you say the
changes would probably have to be very invasive/obfuscating in order to
extract all the information you can get out of Kconfiglib. So I instead
went the minimally invasive route with a completely stand-alone library
together with a test suite.

I'm willing to update Kconfiglib when future additions/modifications are
made to the Kconfig language so that all tests in the compatibility test
suite keep passing. I've added lots of comments and documentation to
internal methods and tried to keep the code clean and straightforward,
so it should be pretty easy for others to get into the code as well.

If by Perl you're referring to streamline_config.pl, I suspect it could
be rewritten as a smallish Kconfiglib script, though I haven't looked at
it closely. That version might also be much better at respecting symbol
dependencies.
> > For a much longer introduction including multiple examples, see
> > arch/kconfig/kconfiglib.py.
> >
> > Have fun!
> >
> > Signed-off-by: Ulf Magnusson <[email protected]>
> > ---
> > Convenience links:
> >
> > Documentation, generated from kconfiglib.py with pydoc -w:
> > http://dl.dropbox.com/u/10406197/kconfiglib.html
> >
> > Examples as separate files:
> > http://dl.dropbox.com/u/10406197/kconfiglib-examples.tar.gz
> >
> >
> > The patch should be preferably be applied to a recent kernel, i.e. Linus's
> > (2.6.38-rc3 at the time of writing).  Due to recent Kconfig changes, the
> > kconfigtest.py test suite - which compares output character-for-character -
> > will indicate failure on older (a few months old) kernels versions even though
> > the outputs are functionally equivalent.
> >
> >  Documentation/kbuild/kconfig-language.txt |    5 +
> >  Documentation/kbuild/kconfig.txt          |    8 +
> >  README                                    |   13 +
> >  scripts/kconfig/Makefile                  |   26 +-
> >  scripts/kconfig/kconfiglib.py             | 3918 +++++++++++++++++++++++++++++
> >  scripts/kconfig/kconfigtest.py            |  396 +++
> >  6 files changed, 4365 insertions(+), 1 deletions(-)
> >  create mode 100644 scripts/kconfig/kconfiglib.py
> >  create mode 100644 scripts/kconfig/kconfigtest.py
> > [...]

2011-02-03 21:59:07

by Ulf Magnusson

Subject: Re: [PATCH] [ANNOUNCE] kconfig: Kconfiglib: a flexible Python Kconfig parser

On Wed, Feb 02, 2011 at 12:27:52AM +0100, Ulf Magnusson wrote:
> Hi,
>
> This is the initial release of Kconfiglib: a Python library for
> scripting, debugging, and extracting information from Kconfig-based
> configuration systems. It can be used to programmatically generate a
> .config when the '*conf' tools are too inflexible, to quickly find out
> interesting information about a Kconfig configuration such as dependency
> relations between symbols and where undefined symbols are referenced,
> and in applications that need to parse and extract information from
> Kconfig files.
>
> For a much longer introduction including multiple examples, see
> arch/kconfig/kconfiglib.py.
>
> Have fun!
>
> Signed-off-by: Ulf Magnusson <[email protected]>
> ---
> Convenience links:
>
> Documentation, generated from kconfiglib.py with pydoc -w:
> http://dl.dropbox.com/u/10406197/kconfiglib.html
>
> Examples as separate files:
> http://dl.dropbox.com/u/10406197/kconfiglib-examples.tar.gz
>
>
> The patch should be preferably be applied to a recent kernel, i.e. Linus's
> (2.6.38-rc3 at the time of writing). Due to recent Kconfig changes, the
> kconfigtest.py test suite - which compares output character-for-character -
> will indicate failure on older (a few months old) kernels versions even though
> the outputs are functionally equivalent.
>
> [...]

Seems linux-kbuild and linux-doc won't accept the patch (too large?), so
here's a link to the message on linux-kernel:
https://lkml.org/lkml/2011/2/1/439

/Ulf Magnusson

2011-02-03 22:17:00

by Ulf Magnusson

Subject: Re: [PATCH] [ANNOUNCE] kconfig: Kconfiglib: a flexible Python Kconfig parser

On Thu, Feb 03, 2011 at 10:58:55PM +0100, Ulf Magnusson wrote:
> On Wed, Feb 02, 2011 at 12:27:52AM +0100, Ulf Magnusson wrote:
> > Hi,
> >
> > This is the initial release of Kconfiglib: a Python library for
> > scripting, debugging, and extracting information from Kconfig-based
> > configuration systems. It can be used to programmatically generate a
> > .config when the '*conf' tools are too inflexible, to quickly find out
> > interesting information about a Kconfig configuration such as dependency
> > relations between symbols and where undefined symbols are referenced,
> > and in applications that need to parse and extract information from
> > Kconfig files.
> >
> > For a much longer introduction including multiple examples, see
> > arch/kconfig/kconfiglib.py.
> >
> > Have fun!
> >
> > Signed-off-by: Ulf Magnusson <[email protected]>
> > ---
> > Convenience links:
> >
> > Documentation, generated from kconfiglib.py with pydoc -w:
> > http://dl.dropbox.com/u/10406197/kconfiglib.html
> >
> > Examples as separate files:
> > http://dl.dropbox.com/u/10406197/kconfiglib-examples.tar.gz
> >
> >
> > The patch should be preferably be applied to a recent kernel, i.e. Linus's
> > (2.6.38-rc3 at the time of writing). Due to recent Kconfig changes, the
> > kconfigtest.py test suite - which compares output character-for-character -
> > will indicate failure on older (a few months old) kernels versions even though
> > the outputs are functionally equivalent.
> >
> > [...]
>
> Seems linux-kbuild and linux-doc won't accept the patch (too large?), so
> here's a link to the message on linux-kernel:
> https://lkml.org/lkml/2011/2/1/439
>
> /Ulf Magnusson

Oh, and here's the patch message. Apply with 'git am'.
http://dl.dropbox.com/u/10406197/kconfiglib

2011-02-04 22:35:42

by Randy Dunlap

Subject: Re: [PATCH] [ANNOUNCE] kconfig: Kconfiglib: a flexible Python Kconfig parser

On Wed, 2 Feb 2011 00:27:55 +0100 Ulf Magnusson wrote:

> Hi,
>
> This is the initial release of Kconfiglib: a Python library for
> scripting, debugging, and extracting information from Kconfig-based
> configuration systems. It can be used to programmatically generate a
> .config when the '*conf' tools are too inflexible, to quickly find out
> interesting information about a Kconfig configuration such as dependency
> relations between symbols and where undefined symbols are referenced,
> and in applications that need to parse and extract information from
> Kconfig files.
>
> For a much longer introduction including multiple examples, see
> arch/kconfig/kconfiglib.py.
>
> Have fun!
>
> Signed-off-by: Ulf Magnusson <[email protected]>
> ---
> Convenience links:
>
> Documentation, generated from kconfiglib.py with pydoc -w:
> http://dl.dropbox.com/u/10406197/kconfiglib.html
>
> Examples as separate files:
> http://dl.dropbox.com/u/10406197/kconfiglib-examples.tar.gz
>
>
> The patch should be preferably be applied to a recent kernel, i.e. Linus's
> (2.6.38-rc3 at the time of writing). Due to recent Kconfig changes, the
> kconfigtest.py test suite - which compares output character-for-character -
> will indicate failure on older (a few months old) kernels versions even though
> the outputs are functionally equivalent.
>
> Documentation/kbuild/kconfig-language.txt | 5 +
> Documentation/kbuild/kconfig.txt | 8 +
> README | 13 +
> scripts/kconfig/Makefile | 26 +-
> scripts/kconfig/kconfiglib.py | 3918 +++++++++++++++++++++++++++++
> scripts/kconfig/kconfigtest.py | 396 +++
> 6 files changed, 4365 insertions(+), 1 deletions(-)
> create mode 100644 scripts/kconfig/kconfiglib.py
> create mode 100644 scripts/kconfig/kconfigtest.py
>

> diff --git a/README b/README
> index 1b81d28..bb5e68f 100644
> --- a/README
> +++ b/README
> @@ -196,6 +196,19 @@ CONFIGURING the kernel:
> values to 'n' as much as possible.
> "make randconfig" Create a ./.config file by setting symbol
> values to random values.
> + "make scriptconfig SCRIPT=<path to script>" Run a Kconfiglib
> + script (see scripts/kconfig/kconfiglib.py). This
> + can be used to programatically generate a
> + ./.config, and for applications that need to
> + extract information from Kconfig files.
> + "make iscriptconfig" Launch an interactive Python shell
> + for running Kconfiglib on the architecture's
> + Kconfig configuration. The kconfiglib and sys
> + (for sys.argv[1] - the base Kconfig file) modules
> + will be imported automatically, and a Config
> + instance 'c' will be created for the architecture
> + (using c = kconfiglib.Config(sys.argv[1])).
> +
>
> You can find more information on using the Linux kernel config tools
> in Documentation/kbuild/kconfig.txt.

Hi Ulf,

This is interesting. I just wish I could read it. ;)
I'll get over it.


1. It would be really Good to have "make scriptconfig SCRIPT=<path to script>"
and "make iscriptconfig" (similar to above, but shortened) in "make help" output.

2. My first test (using your ex1.py script) failed because I used O=xx64 (build
directory):

rddunlap@chimera:lnx-2638-rc3> make O=xx64 scriptconfig SCRIPT=~/pkg/kconfiglib/ex1.py
GEN /lnx/src/lnx-2638-rc3/xx64/Makefile
Traceback (most recent call last):
File "/home/rddunlap/pkg/kconfiglib/ex1.py", line 1, in <module>
import kconfiglib
ImportError: No module named kconfiglib
make[2]: *** [scriptconfig] Error 1
make[1]: *** [scriptconfig] Error 2
make: *** [sub-make] Error 2

Does kconfiglib support O=builddir generally? I can't tell that it does (yet)
since all ex[1-7].py fail in this manner.
It needs to support/allow O=builddir.

Thanks. I'll keep looking...

---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***

2011-02-04 23:42:59

by Rob Landley

Subject: Re: [PATCH] [ANNOUNCE] kconfig: Kconfiglib: a flexible Python Kconfig parser

On 02/04/2011 04:35 PM, Randy Dunlap wrote:
>> The patch should be preferably be applied to a recent kernel, i.e. Linus's
>> (2.6.38-rc3 at the time of writing). Due to recent Kconfig changes, the
>> kconfigtest.py test suite - which compares output character-for-character -
>> will indicate failure on older (a few months old) kernels versions even though
>> the outputs are functionally equivalent.
>>
>> Documentation/kbuild/kconfig-language.txt | 5 +
>> Documentation/kbuild/kconfig.txt | 8 +
>> README | 13 +
>> scripts/kconfig/Makefile | 26 +-
>> scripts/kconfig/kconfiglib.py | 3918 +++++++++++++++++++++++++++++
>> scripts/kconfig/kconfigtest.py | 396 +++
>> 6 files changed, 4365 insertions(+), 1 deletions(-)
>> create mode 100644 scripts/kconfig/kconfiglib.py
>> create mode 100644 scripts/kconfig/kconfigtest.py

Wait, I thought this was an extra standalone library. Are you saying
you want to make it so Linux will no longer compile on a build machine
that doesn't have Python installed?

If this is merely an extra developer tool ala bloat-o-meter and
checkpatch.pl then it's merely uninteresting to me. (I myself wrote a
quick and dirty http://kernel.org/doc/make/menuconfig2html.py to
generate http://kernel.org/doc/menuconfig/x86.html and friends years
ago, and I still run it to update that once in a while. There's not
much to it.)

But adding new prerequities to a build machine would be really annoying
for my use cases.

Rob

2011-02-05 00:28:55

by Ulf Magnusson

Subject: Re: [PATCH] [ANNOUNCE] kconfig: Kconfiglib: a flexible Python Kconfig parser

On Fri, Feb 04, 2011 at 02:35:29PM -0800, Randy Dunlap wrote:
> On Wed, 2 Feb 2011 00:27:55 +0100 Ulf Magnusson wrote:
>
> > Hi,
> >
> > This is the initial release of Kconfiglib: a Python library for
> > scripting, debugging, and extracting information from Kconfig-based
> > configuration systems. It can be used to programmatically generate a
> > .config when the '*conf' tools are too inflexible, to quickly find out
> > interesting information about a Kconfig configuration such as dependency
> > relations between symbols and where undefined symbols are referenced,
> > and in applications that need to parse and extract information from
> > Kconfig files.
> >
> > For a much longer introduction including multiple examples, see
> > arch/kconfig/kconfiglib.py.
> >
> > Have fun!
> >
> > Signed-off-by: Ulf Magnusson <[email protected]>
> > ---
> > Convenience links:
> >
> > Documentation, generated from kconfiglib.py with pydoc -w:
> > http://dl.dropbox.com/u/10406197/kconfiglib.html
> >
> > Examples as separate files:
> > http://dl.dropbox.com/u/10406197/kconfiglib-examples.tar.gz
> >
> >
> > The patch should be preferably be applied to a recent kernel, i.e. Linus's
> > (2.6.38-rc3 at the time of writing). Due to recent Kconfig changes, the
> > kconfigtest.py test suite - which compares output character-for-character -
> > will indicate failure on older (a few months old) kernels versions even though
> > the outputs are functionally equivalent.
> >
> > Documentation/kbuild/kconfig-language.txt | 5 +
> > Documentation/kbuild/kconfig.txt | 8 +
> > README | 13 +
> > scripts/kconfig/Makefile | 26 +-
> > scripts/kconfig/kconfiglib.py | 3918 +++++++++++++++++++++++++++++
> > scripts/kconfig/kconfigtest.py | 396 +++
> > 6 files changed, 4365 insertions(+), 1 deletions(-)
> > create mode 100644 scripts/kconfig/kconfiglib.py
> > create mode 100644 scripts/kconfig/kconfigtest.py
> >
>
> > diff --git a/README b/README
> > index 1b81d28..bb5e68f 100644
> > --- a/README
> > +++ b/README
> > @@ -196,6 +196,19 @@ CONFIGURING the kernel:
> > values to 'n' as much as possible.
> > "make randconfig" Create a ./.config file by setting symbol
> > values to random values.
> > + "make scriptconfig SCRIPT=<path to script>" Run a Kconfiglib
> > + script (see scripts/kconfig/kconfiglib.py). This
> > + can be used to programatically generate a
> > + ./.config, and for applications that need to
> > + extract information from Kconfig files.
> > + "make iscriptconfig" Launch an interactive Python shell
> > + for running Kconfiglib on the architecture's
> > + Kconfig configuration. The kconfiglib and sys
> > + (for sys.argv[1] - the base Kconfig file) modules
> > + will be imported automatically, and a Config
> > + instance 'c' will be created for the architecture
> > + (using c = kconfiglib.Config(sys.argv[1])).
> > +
> >
> > You can find more information on using the Linux kernel config tools
> > in Documentation/kbuild/kconfig.txt.
>
> Hi Ulf,
>
> This is interesting. I just wish I could read it. ;)
> I'll get over it.
>
>
> 1. It would be really Good to have "make scriptconfig SCRIPT=<path to script>"
> and "make iscriptconfig" (similar to above, but shortened) in "make help" output.

I have added short descriptions to "make help", mostly referring people
to the script itself. Do you think I should explain it more fully? I
felt bad about adding big blobs of text when the "make help"
descriptions for all the other *config targets are short oneliners :)

>
> 2. My first test (using your ex1.py script) failed because I used O=xx64 (build
> directory):
>
> rddunlap@chimera:lnx-2638-rc3> make O=xx64 scriptconfig SCRIPT=~/pkg/kconfiglib/ex1.py
> GEN /lnx/src/lnx-2638-rc3/xx64/Makefile
> Traceback (most recent call last):
> File "/home/rddunlap/pkg/kconfiglib/ex1.py", line 1, in <module>
> import kconfiglib
> ImportError: No module named kconfiglib
> make[2]: *** [scriptconfig] Error 1
> make[1]: *** [scriptconfig] Error 2
> make: *** [sub-make] Error 2
>
> Does kconfiglib support O=builddir generally? I can't tell that it does (yet)
> since all ex[1-7].py fail in this manner.
> It needs to support/allow O=builddir.

I hadn't considered O. I'll see if I can cook up a patch.

/Ulf

2011-02-05 00:28:56

by Filip Honckiewicz

Subject: Re: [PATCH] [ANNOUNCE] kconfig: Kconfiglib: a flexible Python Kconfig parser

Hello Ulf!

I'm working on pretty same thing also in python... Now I'm sad,
because you made it and I don't, and also my code looks like dung in
comparison to yours. How long did you make this?

BTW. "SLOW_WORK" symbol used in drivers/gpu/drm/Kconfig in "menucofnig
DRM" as one of selects isn't an unnecessary symbol? I can't find any
SLOW_WORK entry.

Filip

2011-02-05 00:38:22

by Ulf Magnusson

Subject: Re: [PATCH] [ANNOUNCE] kconfig: Kconfiglib: a flexible Python Kconfig parser

On Fri, Feb 04, 2011 at 05:42:50PM -0600, Rob Landley wrote:
> On 02/04/2011 04:35 PM, Randy Dunlap wrote:
> >> The patch should be preferably be applied to a recent kernel, i.e. Linus's
> >> (2.6.38-rc3 at the time of writing). Due to recent Kconfig changes, the
> >> kconfigtest.py test suite - which compares output character-for-character -
> >> will indicate failure on older (a few months old) kernels versions even though
> >> the outputs are functionally equivalent.
> >>
> >> Documentation/kbuild/kconfig-language.txt | 5 +
> >> Documentation/kbuild/kconfig.txt | 8 +
> >> README | 13 +
> >> scripts/kconfig/Makefile | 26 +-
> >> scripts/kconfig/kconfiglib.py | 3918 +++++++++++++++++++++++++++++
> >> scripts/kconfig/kconfigtest.py | 396 +++
> >> 6 files changed, 4365 insertions(+), 1 deletions(-)
> >> create mode 100644 scripts/kconfig/kconfiglib.py
> >> create mode 100644 scripts/kconfig/kconfigtest.py
>
> Wait, I thought this was an extra standalone library. Are you saying
> you want to make it so Linux will no longer compile on a build machine
> that doesn't have Python installed?

No - it's completely standalone, and should have no effect on things
that do not use Kconfiglib. The only changes besides adding the script
itself is to add two (well, three - there's one that's only used by
kconfigtest.py) targets to scripts/kconfig/Makefile that make it easier
to use the library by ensuring the environment is set up correctly.

>
> If this is merely an extra developer tool ala bloat-o-meter and
> checkpatch.pl then it's merely uninteresting to me. (I myself wrote a
> quick and dirty http://kernel.org/doc/make/menuconfig2html.py to
> generate http://kernel.org/doc/menuconfig/x86.html and friends years
> ago, and I still run it to update that once in a while. There's not
> much to it.)
>
> But adding new prerequities to a build machine would be really annoying
> for my use cases.
>
> Rob

You would only need to have Python installed if you are going to use
Kconfiglib directly (hard to get around with a Python library).

/Ulf

2011-02-05 01:01:03

by Ulf Magnusson

Subject: Re: [PATCH] [ANNOUNCE] kconfig: Kconfiglib: a flexible Python Kconfig parser

On Sat, Feb 05, 2011 at 01:28:50AM +0100, Filip Honckiewicz wrote:
> Hello Ulf!
>
> I'm working on pretty same thing also in python... Now I'm sad,
> because you made it and I don't, and also my code looks like dung in
> comparison to yours. How long did you make this?

You didn't see the early versions ;)

I was a bit afraid that someone would come out with something similar
before it was done. Now you've given me a bad conscience :(

I've been working on it on and off in my spare time for six months or so.
Originally it was a project for automatically generating a minimal kernel for a
given system by automatically turning off options one by one and testing the
resulting kernel in an emulator (yeah, the minimal kernel you get out of that
process is broken in many ways, but you get some idea of what's needed and
what's not at least). That then grew into a general-purpose library.

>
> BTW. "SLOW_WORK" symbol used in drivers/gpu/drm/Kconfig in "menucofnig
> DRM" as one of selects isn't an unnecessary symbol? I can't find any
> SLOW_WORK entry.

iscriptconfig says

A Config instance 'c' for the architecture (i386) has been created.
>>> c["SLOW_WORK"].is_defined()
False

so yes, it it's undefined on i386 at least. Grepping through the kernel shows
it only appears in drivers/gpu/drm/Kconfig, so it's in fact undefined on all
arches.

/Ulf

2011-02-10 23:14:50

by Michal Marek

Subject: Re: [PATCH] [ANNOUNCE] kconfig: Kconfiglib: a flexible Python Kconfig parser

On 2.2.2011 00:27, Ulf Magnusson wrote:
> Hi,
>
> This is the initial release of Kconfiglib: a Python library for
> scripting, debugging, and extracting information from Kconfig-based
> configuration systems. It can be used to programmatically generate a
> .config when the '*conf' tools are too inflexible, to quickly find out
> interesting information about a Kconfig configuration such as dependency
> relations between symbols and where undefined symbols are referenced,
> and in applications that need to parse and extract information from
> Kconfig files.
>
> For a much longer introduction including multiple examples, see
> arch/kconfig/kconfiglib.py.

Hi,

this looks like a very powerful tool, but I have a similar concern like
Arnaud had - being completely standalone, it reimplements most of the C
kconfig code. One option to reduce this duplication would be a swig
wrapper, another one would be to let the C code parse the Kconfig files
and write the required information in some digested form, that would be
easier to parse by scripts. Something like:
$ scripts/kconfig/conf_inspect --kconfig=Kconfig --eval='FOO || BAR'
y
$ scripts/kconfig/conf_inspect ... --dump-symbols
config FOO
type: bool
valule: m
visible: y
prompt: "zzz"
depends: X & Y
select: Z
...
$ scripts/kconfig/conf_inspect ... --dump-symbols \
--fields='depends,select,value'
config FOO
depends: X & Y
select: Z
value: m

config BAR
...

etc. The idea is that for instance instead of parsing the Kconfig files,
the Python code could fill it's data structures by reading the flat dump
provided by the C kconfig. There would be still lot to do in Python,
e.g. parsing and evaluating expressions, but it would be a small step
forward already. And people wanting to write quick&dirty scripts in
bash/awk/perl would make use of the C code as well.

Michal

2011-02-16 04:48:44

by Ulf Magnusson

Subject: Re: [PATCH] [ANNOUNCE] kconfig: Kconfiglib: a flexible Python Kconfig parser

On Fri, Feb 11, 2011 at 12:14:36AM +0100, Michal Marek wrote:
> On 2.2.2011 00:27, Ulf Magnusson wrote:
> > Hi,
> >
> > This is the initial release of Kconfiglib: a Python library for
> > scripting, debugging, and extracting information from Kconfig-based
> > configuration systems. It can be used to programmatically generate a
> > .config when the '*conf' tools are too inflexible, to quickly find out
> > interesting information about a Kconfig configuration such as dependency
> > relations between symbols and where undefined symbols are referenced,
> > and in applications that need to parse and extract information from
> > Kconfig files.
> >
> > For a much longer introduction including multiple examples, see
> > arch/kconfig/kconfiglib.py.
>
> Hi,
>
> this looks like a very powerful tool, but I have a similar concern like
> Arnaud had - being completely standalone, it reimplements most of the C
> kconfig code. One option to reduce this duplication would be a swig
> wrapper, another one would be to let the C code parse the Kconfig files
> and write the required information in some digested form, that would be
> easier to parse by scripts. Something like:
> $ scripts/kconfig/conf_inspect --kconfig=Kconfig --eval='FOO || BAR'
> y
> $ scripts/kconfig/conf_inspect ... --dump-symbols
> config FOO
> type: bool
> valule: m
> visible: y
> prompt: "zzz"
> depends: X & Y
> select: Z
> ...
> $ scripts/kconfig/conf_inspect ... --dump-symbols \
> --fields='depends,select,value'
> config FOO
> depends: X & Y
> select: Z
> value: m
>
> config BAR
> ...
>
> etc. The idea is that for instance instead of parsing the Kconfig files,
> the Python code could fill it's data structures by reading the flat dump
> provided by the C kconfig. There would be still lot to do in Python,
> e.g. parsing and evaluating expressions, but it would be a small step
> forward already. And people wanting to write quick&dirty scripts in
> bash/awk/perl would make use of the C code as well.
>
Offloading some of the parsing to a new tool would probably be doable,
though some Kconfig subleties might complicate things. For example,
symbols that appear within choices aren't seen as real selectable choice
items if they depend on previous symbols in specific ways (see
_determine_actual_items() in kconfiglib.py), and capturing this
information in the output while at the same preserving the ordering of
the symbols could be messy. I guess you could sort this out within
Kconfiglib instead, though other tools might then have to reimplement
the same (messy) rules.

I'm not so sure about offloading anything else this way. For dynamically
updating symbol values in a script you would need to somehow preserve
state between invocations (or pass huge lists of assignments back and
forth), which could lead to an unwieldy protocol. You might need to go
the SWIG route instead.

/Ulf

2011-02-16 15:20:34

by Ulf Magnusson

Subject: Re: [PATCH] [ANNOUNCE] kconfig: Kconfiglib: a flexible Python Kconfig parser

On Wed, Feb 16, 2011 at 05:48:32AM +0100, Ulf Magnusson wrote:
> On Fri, Feb 11, 2011 at 12:14:36AM +0100, Michal Marek wrote:
> > On 2.2.2011 00:27, Ulf Magnusson wrote:
> > > Hi,
> > >
> > > This is the initial release of Kconfiglib: a Python library for
> > > scripting, debugging, and extracting information from Kconfig-based
> > > configuration systems. It can be used to programmatically generate a
> > > .config when the '*conf' tools are too inflexible, to quickly find out
> > > interesting information about a Kconfig configuration such as dependency
> > > relations between symbols and where undefined symbols are referenced,
> > > and in applications that need to parse and extract information from
> > > Kconfig files.
> > >
> > > For a much longer introduction including multiple examples, see
> > > arch/kconfig/kconfiglib.py.
> >
> > Hi,
> >
> > this looks like a very powerful tool, but I have a similar concern like
> > Arnaud had - being completely standalone, it reimplements most of the C
> > kconfig code. One option to reduce this duplication would be a swig
> > wrapper, another one would be to let the C code parse the Kconfig files
> > and write the required information in some digested form, that would be
> > easier to parse by scripts. Something like:
> > $ scripts/kconfig/conf_inspect --kconfig=Kconfig --eval='FOO || BAR'
> > y
> > $ scripts/kconfig/conf_inspect ... --dump-symbols
> > config FOO
> > type: bool
> > valule: m
> > visible: y
> > prompt: "zzz"
> > depends: X & Y
> > select: Z
> > ...
> > $ scripts/kconfig/conf_inspect ... --dump-symbols \
> > --fields='depends,select,value'
> > config FOO
> > depends: X & Y
> > select: Z
> > value: m
> >
> > config BAR
> > ...
> >
> > etc. The idea is that for instance instead of parsing the Kconfig files,
> > the Python code could fill it's data structures by reading the flat dump
> > provided by the C kconfig. There would be still lot to do in Python,
> > e.g. parsing and evaluating expressions, but it would be a small step
> > forward already. And people wanting to write quick&dirty scripts in
> > bash/awk/perl would make use of the C code as well.
> >
> Offloading some of the parsing to a new tool would probably be doable,
> though some Kconfig subleties might complicate things. For example,
> symbols that appear within choices aren't seen as real selectable choice
> items if they depend on previous symbols in specific ways (see
> _determine_actual_items() in kconfiglib.py), and capturing this
> information in the output while at the same preserving the ordering of
> the symbols could be messy. I guess you could sort this out within
> Kconfiglib instead, though other tools might then have to reimplement
> the same (messy) rules.
>
Then again, ordering information is retained (for example, assignments
in .config files are written out in the order symbols appear within the
Kconfig files), so you could probably just iterate over all items in
Kconfig order after parsing + implicit submenu creation (which causes
the weird choice behavior) to capture such stuff as well.

> I'm not so sure about offloading anything else this way. For dynamically
> updating symbol values in a script you would need to somehow preserve
> state between invocations (or pass huge lists of assignments back and
> forth), which could lead to an unwieldy protocol. You might need to go
> the SWIG route instead.

/Ulf