Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751913Ab1BAX2P (ORCPT ); Tue, 1 Feb 2011 18:28:15 -0500 Received: from mail-bw0-f46.google.com ([209.85.214.46]:47718 "EHLO mail-bw0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751536Ab1BAX2L (ORCPT ); Tue, 1 Feb 2011 18:28:11 -0500 DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:mail-followup-to:mime-version :content-type:content-disposition:user-agent; b=xvcJRZUITuJIzK8C+w9COMkI4y0sBtAkDpjfMHduYUf93byS+mThFfvoeyJe8b8yMg Mk1RXt6/PyMQiPPB2ODmgx2KFQ1ZQtSs56oKMDM5T3sIJJlt9bHGWXnEekPxr30kLTR5 o3Og5vVUhqd4z3L3Y538yXozOqRGU6k1nc0ss= Date: Wed, 2 Feb 2011 00:27:55 +0100 From: Ulf Magnusson To: linux-kbuild@vger.kernel.org Cc: zippel@linux-m68k.org, mmarek@suse.cz, rdunlap@xenotime.net, akpm@linux-foundation.org, andrea.gelmini@gelma.net, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org Subject: [PATCH] [ANNOUNCE] kconfig: Kconfiglib: a flexible Python Kconfig parser Message-ID: <20110201232750.GA30800@ulf> Mail-Followup-To: Ulf Magnusson , linux-kbuild@vger.kernel.org, zippel@linux-m68k.org, mmarek@suse.cz, rdunlap@xenotime.net, akpm@linux-foundation.org, andrea.gelmini@gelma.net, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-06-14) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 161629 Lines: 4482 Hi, This is the initial release of Kconfiglib: a Python library for scripting, debugging, and extracting information from Kconfig-based configuration systems. It can be used to programmatically generate a .config when the '*conf' tools are too inflexible, to quickly find out interesting information about a Kconfig configuration such as dependency relations between symbols and where undefined symbols are referenced, and in applications that need to parse and extract information from Kconfig files. For a much longer introduction including multiple examples, see arch/kconfig/kconfiglib.py. Have fun! Signed-off-by: Ulf Magnusson --- Convenience links: Documentation, generated from kconfiglib.py with pydoc -w: http://dl.dropbox.com/u/10406197/kconfiglib.html Examples as separate files: http://dl.dropbox.com/u/10406197/kconfiglib-examples.tar.gz The patch should be preferably be applied to a recent kernel, i.e. Linus's (2.6.38-rc3 at the time of writing). Due to recent Kconfig changes, the kconfigtest.py test suite - which compares output character-for-character - will indicate failure on older (a few months old) kernels versions even though the outputs are functionally equivalent. Documentation/kbuild/kconfig-language.txt | 5 + Documentation/kbuild/kconfig.txt | 8 + README | 13 + scripts/kconfig/Makefile | 26 +- scripts/kconfig/kconfiglib.py | 3918 +++++++++++++++++++++++++++++ scripts/kconfig/kconfigtest.py | 396 +++ 6 files changed, 4365 insertions(+), 1 deletions(-) create mode 100644 scripts/kconfig/kconfiglib.py create mode 100644 scripts/kconfig/kconfigtest.py diff --git a/Documentation/kbuild/kconfig-language.txt b/Documentation/kbuild/kconfig-language.txt index b507d61..63d63ba 100644 --- a/Documentation/kbuild/kconfig-language.txt +++ b/Documentation/kbuild/kconfig-language.txt @@ -381,3 +381,8 @@ config FOO limits FOO to module (=m) or disabled (=n). +Kconfiglib +~~~~~~~~~~ +If you need to programmatically generate a .config or parse Kconfig files, +Kconfiglib might come handy. See 'scriptconfig' and 'iscriptconfig' in +'make help' and scripts/kconfig/kconfiglib.py. diff --git a/Documentation/kbuild/kconfig.txt b/Documentation/kbuild/kconfig.txt index cca46b1..73cf58f 100644 --- a/Documentation/kbuild/kconfig.txt +++ b/Documentation/kbuild/kconfig.txt @@ -195,4 +195,12 @@ Searching in gconfig: however, gconfig does have a few more viewing choices than xconfig does. +====================================================================== +scriptconfig +-------------------------------------------------- + +If you need to programmatically generate a .config or parse Kconfig files, +Kconfiglib might come handy. See 'scriptconfig' and 'iscriptconfig' in +'make help' and scripts/kconfig/kconfiglib.py. + ### diff --git a/README b/README index 1b81d28..bb5e68f 100644 --- a/README +++ b/README @@ -196,6 +196,19 @@ CONFIGURING the kernel: values to 'n' as much as possible. "make randconfig" Create a ./.config file by setting symbol values to random values. + "make scriptconfig SCRIPT=" Run a Kconfiglib + script (see scripts/kconfig/kconfiglib.py). This + can be used to programatically generate a + ./.config, and for applications that need to + extract information from Kconfig files. + "make iscriptconfig" Launch an interactive Python shell + for running Kconfiglib on the architecture's + Kconfig configuration. The kconfiglib and sys + (for sys.argv[1] - the base Kconfig file) modules + will be imported automatically, and a Config + instance 'c' will be created for the architecture + (using c = kconfiglib.Config(sys.argv[1])). + You can find more information on using the Linux kernel config tools in Documentation/kbuild/kconfig.txt. diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile index 368ae30..cfffe87 100644 --- a/scripts/kconfig/Makefile +++ b/scripts/kconfig/Makefile @@ -3,7 +3,7 @@ # These targets are used from top-level makefile PHONY += oldconfig xconfig gconfig menuconfig config silentoldconfig update-po-config \ - localmodconfig localyesconfig + localmodconfig localyesconfig scriptconfig iscriptconfig kconfiglibtestconfig ifdef KBUILD_KCONFIG Kconfig := $(KBUILD_KCONFIG) @@ -33,6 +33,28 @@ silentoldconfig: $(obj)/conf $(Q)mkdir -p include/generated $< --$@ $(Kconfig) +ifneq ($(filter scriptconfig iscriptconfig,$(MAKECMDGOALS)),) +PYTHONCMD ?= python +endif + +scriptconfig: + $(Q)if [ ! -n "$(SCRIPT)" ]; then \ + echo 'No script argument provided; use "make scriptconfig SCRIPT=".'; \ + else \ + PYTHONPATH="$(obj):$$PYTHONPATH" "$(PYTHONCMD)" "$(SCRIPT)" "$(Kconfig)"; \ + fi + +iscriptconfig: + $(Q)PYTHONPATH="$(obj):$$PYTHONPATH" "$(PYTHONCMD)" -i -c \ + "import kconfiglib; \ + import sys; \ + c = kconfiglib.Config(sys.argv[1]); \ + print \"A Config instance 'c' for the architecture ({0}) has been created.\".format(c.get_arch())" "$(Kconfig)" + +# Used by kconfigtest.py to prevent an 'option defconfig' .config from being loaded +kconfiglibtestconfig: $(obj)/conf + $(Q)$< --defconfig=.config $(Kconfig) + # if no path is given, then use src directory to find file ifdef LSMOD LSMOD_F := $(LSMOD) @@ -139,6 +161,8 @@ help: @echo ' randconfig - New config with random answer to all options' @echo ' listnewconfig - List new options' @echo ' oldnoconfig - Same as silentoldconfig but set new symbols to n (unset)' + @echo ' scriptconfig - Run a kconfiglib script (see scripts/kconfig/kconfiglib.py)' + @echo ' iscriptconfig - Launch interactive Python shell for using kconfiglib' # lxdialog stuff check-lxdialog := $(srctree)/$(src)/lxdialog/check-lxdialog.sh diff --git a/scripts/kconfig/kconfiglib.py b/scripts/kconfig/kconfiglib.py new file mode 100644 index 0000000..84e70c3 --- /dev/null +++ b/scripts/kconfig/kconfiglib.py @@ -0,0 +1,3918 @@ +# This is Kconfiglib, a Python library for scripting, debugging, and extracing +# information from Kconfig-based configuration systems. To view the +# documentation, run +# +# $ pydoc kconfiglib +# +# or, if you prefer HTML, +# +# $ pydoc -w kconfiglib +# +# By Ulf "Ulfalizer" Magnusson. + +""" +Kconfiglib is a Python library for scripting, debugging, and extracting +information from Kconfig-based configuration systems. Features include the +following: + + - Symbol values and properties can be looked up and values assigned + programmatically. + - .config files can be read and written. + - Expressions can be evaluated in the context of a Kconfig configuration. + - Relations between symbols can be quickly determined, such as finding all + symbols that reference a particular symbol. + - Symbol values are automatically cached and reevaluated only when needed. + - Highly compatible with the C Kconfig implementation: + * Generates a .config file that is character-for-character identical (modulo + header) to the one generated by the C implementation (mconf) when fed each + architecture/defconfig pair in the kernel and asked to generate a + configuration. This includes nonsensical pairings such as powerpc with + arch/x86/configs/x86_64_defconfig, for a total of 9096 combinations as of + Linux 2.6.38-rc3. + * Also generates a .config that is character-for-character identical to the + one generated by mconf for all architectures when no .config is supplied. + * The 'make allyesconfig' and 'make allnoconfig' implementations below + generate output character-for-character identical to the C implementation + for all architectures. + + See scripts/kconfig/kconfigtest.py for the Kconfiglib test suite. + +For the Linux kernel, scripts should be run with + +$ make scriptconfig SCRIPT= + +to ensure that the environment (SRCARCH, ARCH, and KERNELVERSION) is set up +correctly. Alternative architectures can be specified like for other 'make +*config' targets: + +$ make scriptconfig ARCH=mips SCRIPT= + +The script will receive the name of the Kconfig file to load in sys.argv[1] (as +of Linux 2.6.38-rc3 this is always "Kconfig" from the kernel top-level +directory). + +To get an interactive Python prompt with Kconfiglib preloaded, use + +$ make iscriptconfig ARCH= + +Kconfiglib requires Python 2. For (i)scriptconfig the command to run the Python +interpreter can be passed in the environment variable PYTHONCMD (defaults to +'python'). + +Learning to use the library is probably easiest by looking at a few examples, +reading the documentation, and experimenting. The API is designed to be +intuitive and easy to use. + +=============================================================================== +Example 1: loading a configuration and a .config and printing information about +a symbol. + +import kconfiglib +import sys + +# Create a Config object representing a Kconfig configuration (any number of +# these can be created -- the library has no global state). +conf = kconfiglib.Config(sys.argv[1]) + +# Load values from a .config file. +conf.load_config("arch/x86/configs/i386_defconfig") + +# Print some information about a symbol (the Config class implements +# __getitem__() to provide a handy syntax for getting symbols). +print conf["SERIAL_UARTLITE_CONSOLE"] + +Output for ARCH=i386: +Symbol SERIAL_UARTLITE_CONSOLE +Type : bool +Value : "n" +User value : (no user value) +Visibility : "n" +Is choice item : false +Is defined : true +Is from env. : false +Is special : false +Prompts: + "Support for console on Xilinx uartlite serial port" if SERIAL_UARTLITE = y (value: "n") +Default values: + (no default values) +Selects: + SERIAL_CORE_CONSOLE if SERIAL_UARTLITE = y (value: "n") +Reverse dependencies: + (no reverse dependencies) +Additional dependencies from enclosing menus and if's: + HAS_IOMEM (value: "y") +Locations: drivers/serial/Kconfig:913 + +=============================================================================== +Example 2: evaluate an expression in the context of a configuration (here we + could load a .config as well). + +import kconfiglib +import sys + +conf = kconfiglib.Config(sys.argv[1]) +print conf.eval("(TRACE_IRQFLAGS_SUPPORT || PPC32) && STACKTRACE_SUPPORT") + +Output for ARCH=mips: +y + +=============================================================================== +Example 3: print the names of all symbols that are referenced but never defined + in the current configuration together with the locations where they + are referenced. Integers being included in the list is not a bug, as + these need to be treated as symbols per the design of Kconfig. + +import kconfiglib +import sys + +conf = kconfiglib.Config(sys.argv[1]) + +for sym in conf.get_symbols(): + if not sym.is_defined(): + print sym.get_name() + for (filename, linenr) in sym.get_ref_locations(): + print " {0}:{1}".format(filename, linenr) + +Output for ARCH=i386: +MACH_NET2BIG_V2 + drivers/leds/Kconfig:365 +MACH_OMAP3517EVM + sound/soc/omap/Kconfig:84 +ARCH_OMAP16XX + drivers/pcmcia/Kconfig:271 + drivers/char/hw_random/Kconfig:117 + drivers/watchdog/Kconfig:210 + drivers/rtc/Kconfig:694 +SOC_JZ4740 + sound/soc/codecs/Kconfig:30 +MACH_AT91SAM9261EK + drivers/video/Kconfig:1055 +PPC_83xx + drivers/mtd/nand/Kconfig:463 + drivers/watchdog/Kconfig:951 + drivers/usb/Kconfig:58 + drivers/edac/Kconfig:221 +PPC_PSERIES + init/Kconfig:999 + drivers/pci/hotplug/Kconfig:146 + drivers/scsi/Kconfig:964 + drivers/scsi/Kconfig:975 + drivers/scsi/Kconfig:989 + drivers/net/Kconfig:1353 + drivers/serial/Kconfig:1261 + drivers/char/Kconfig:628 + drivers/char/Kconfig:712 + drivers/char/Kconfig:729 +... (lots more) + +=============================================================================== +Example 4: print the names of all symbols that reference a particular symbol + (there's also a method get_selected_symbols() for determining + just selection relations). + +import kconfiglib +import sys + +conf = kconfiglib.Config(sys.argv[1]) + +x86 = conf["X86"] +for sym in conf: + if x86 in sym.get_referenced_symbols(): + print sym.get_name() + +Output for ARCH=i386: +AUDITSYSCALL +PCSPKR_PLATFORM +OPROFILE_EVENT_MULTIPLEX +GCOV_PROFILE_ALL +SCHED_OMIT_FRAME_POINTER +MEMORY_HOTPLUG +PM_TRACE_RTC +ACPI +ACPI_AC +ACPI_BATTERY +ACPI_VIDEO +ACPI_PROCESSOR_AGGREGATOR +ACPI_NUMA +X86_PM_TIMER +ACPI_SBS +ACPI_APEI +ACPI_APEI_GHES +INTEL_IDLE +XEN_PCIDEV_FRONTEND +EISA_VLB_PRIMING +EISA_VIRTUAL_ROOT +HOTPLUG_PCI_COMPAQ +HOTPLUG_PCI_IBM +HOTPLUG_PCI_CPCI_ZT5550 +HOTPLUG_PCI_CPCI_GENERIC +MTD_SC520CDP +... (lots more) + +=============================================================================== +Example 5: print a tree of all items in the configuration. + +import kconfiglib +import sys + +def print_with_indent(s, indent): + print (" " * indent) + s + +def print_items(items, indent): + for item in items: + if item.is_symbol(): + print_with_indent("config {0}".format(item.get_name()), indent) + elif item.is_menu(): + print_with_indent('menu "{0}"'.format(item.get_title()), indent) + print_items(item.get_items(), indent + 2) + elif item.is_choice(): + print_with_indent('choice', indent) + print_items(item.get_items(), indent + 2) + elif item.is_comment(): + print_with_indent('comment "{0}"'.format(item.get_text()), indent) + +conf = kconfiglib.Config(sys.argv[1]) +print_items(conf.get_top_level_items(), 0) + +Output for ARCH=i386: +... +config ARCH +config KERNELVERSION +config CONSTRUCTORS +config HAVE_IRQ_WORK +config IRQ_WORK +menu "General setup" + config EXPERIMENTAL + config BROKEN + config BROKEN_ON_SMP + config LOCK_KERNEL + config INIT_ENV_ARG_LIMIT + config CROSS_COMPILE + config LOCALVERSION + config LOCALVERSION_AUTO + config HAVE_KERNEL_GZIP + config HAVE_KERNEL_BZIP2 + config HAVE_KERNEL_LZMA + config HAVE_KERNEL_XZ + config HAVE_KERNEL_LZO + choice + config KERNEL_GZIP + config KERNEL_BZIP2 + config KERNEL_LZMA + config KERNEL_XZ + config KERNEL_LZO + config SWAP + config SYSVIPC + config SYSVIPC_SYSCTL + config POSIX_MQUEUE + config POSIX_MQUEUE_SYSCTL + config BSD_PROCESS_ACCT + config BSD_PROCESS_ACCT_V3 +... + +=============================================================================== +Example 6: this does the same thing as entering "make menuconfig" and + immediately saving and exiting. + +import kconfiglib +import os +import sys + +conf = kconfiglib.Config(sys.argv[1]) + +if os.path.exists(".config"): + conf.load_config(".config") +else: + defconfig = conf.get_defconfig_filename() + if defconfig is not None: + print "Using " + defconfig + conf.load_config(defconfig) + +conf.write_config(".config") + +=============================================================================== +Example 7: as a more complex example, this is a reimplementation of allnoconfig + (verified to produce identical output to 'make allnoconfig' for all + ARCHes). The looping is done in case setting one symbol to "n" + allows other symbols to be set to "n" (due to dependencies). + +import kconfiglib +import sys + +conf = kconfiglib.Config(sys.argv[1]) + +while True: + done = True + + for sym in conf: + # Choices take care of themselves for allnoconfig, so we only need to + # worry about non-choice symbols + if not sym.is_choice_item(): + lower_bound = sym.get_lower_bound() + + # If we can assign a lower value to the symbol (where "n", "m" and + # "y" are ordered from lowest to highest), then do so. + # lower_bound() returns None for symbols whose values cannot + # (currently) be changed, as well as for non-bool, non-tristate + # symbols. + if lower_bound is not None and \\ + kconfiglib.tri_less(lower_bound, sym.calc_value()): + + sym.set_value(lower_bound) + + # We just changed the value of some symbol. As this may effect + # other symbols, keep going. + done = False + + if done: + break + +conf.write_config(".config") + +=============================================================================== +Example 8: here's allyesconfig (also verified), which is a bit more involved + as we need to handle choices in two different modes ("y", i.e. + one-is-y-rest-is-n, and "m", i.e. + any-number-of-symbols-are-m-rest-are-n). The looping is in case + setting one symbol to "y" (or "m") allows the value of other symbols + to be raised. + +import kconfiglib +import sys + +conf = kconfiglib.Config(sys.argv[1]) + +# Get a list of all symbols that are not in choices +non_choice_syms = [sym for sym in conf.get_symbols() if + not sym.is_choice_item()] + +while True: + done = True + + # Handle symbols outside of choices + + for sym in non_choice_syms: + upper_bound = sym.get_upper_bound() + + # See corresponding comment for allnoconfig implementation + if upper_bound is not None and \\ + kconfiglib.tri_less(sym.calc_value(), upper_bound): + sym.set_value(upper_bound) + done = False + + # Handle symbols within choices + + for choice in conf.get_choices(): + + # Handle choices whose visibility allow them to be in "y" mode + + if choice.get_visibility() == "y": + selection = choice.get_selection_from_defaults() + if selection is not None and \\ + selection is not choice.get_user_selection(): + selection.set_value("y") + done = False + + # Handle choices whose visibility only allow them to be in "m" mode + + elif choice.get_visibility() == "m": + for sym in choice.get_items(): + if sym.calc_value() != "m" and \\ + sym.get_upper_bound() != "n": + sym.set_value("m") + done = False + + + if done: + break + +conf.write_config(".config") + + +Credits: written by Ulf "Ulfalizer" Magnusson + +Send bug reports, suggestions (any missing APIs that would make your life +easier?) and other feedback to kconfiglib@gmail.com .""" + +# If you have Psyco installed (32-bit installations, Python <= 2.6 only), +# setting this to True (right here, not at runtime) might give a nice speedup +# (22% faster for parsing arch/x86/Kconfig and 58% faster for evaluating all +# symbols in it without a .config on my Core Duo). +use_psyco = False + +import os +import re +import string +import sys + +class Config(): + + """Represents a Kconfig configuration.""" + + # + # Public interface + # + + def __init__(self, + filename = "Kconfig", + base_dir = ".", + print_warnings = True, + print_undef_assign = False): + """Creates a new Config object, representing a Kconfig configuration. + Raises Kconfig_Syntax_Error on syntax errors. + + filename (default: "Kconfig") -- The base Kconfig file of the + configuration. For the Linux kernel, this should usually be be + "Kconfig" from the top-level directory, as environment + variables will make sure the right Kconfig is included from + there (usually arch/ "y" + + Note that this functions always yields a tristate value. To get the + value of non-bool, non-tristate symbols, use calc_value(). + + transform_m (default: False) -- + Within conditional expressions (those following e.g. 'if' and + 'depends on') "m" is rewritten as "m" && MODULES internally by the C + implementation and by kconfiglib, so that MODULES needs to be enabled + for the expression to be true. Pass True here if you want that to + happen; otherwise, pass False.""" + return self._eval_expr(self._parse_expr(self._tokenize(s, False), + None, + s, + transform_m = transform_m)) + + def get_config_header(self): + """Returns the (uncommented) textual header of the .config file most + recently loaded with load_config(). Returns None if no .config file has + been loaded or if the most recently loaded .config file has no header. + The header comprises all lines up to but not including the first line + that either + + 1. Does not start with "#" + 2. Has the form "# CONFIG_FOO is not set." + """ + return self.config_header + + def get_base_dir(self): + """Returns the base directory for 'source' statements, passed as an + argument to Config.__init__().""" + return self.base_dir + + def set_print_warnings(self, print_warnings): + """Determines whether warnings related to this configuration (for + things like attempting to assign illegal values to symbols) should be + printed to stderr. + + print_warnings -- True if warnings should be + printed, otherwise False.""" + self.print_warnings = print_warnings + + def set_print_undef_assign(self, print_undef_assign): + """Determines whether informational messages related to assignments to + undefined symbols should be printed to stderr for this configuration. + + print_undef_assign -- If True, such messages will be printed.""" + self.print_undef_assign = print_undef_assign + + def __getitem__(self, key): + """Returns the symbol with name 'name'. Raises KeyError if the symbol + does not appear in the configuration.""" + return self.syms[key] + + def __iter__(self): + """Convenience function for iterating over the set of all defined + symbols in the configuration, used like + + for sym in conf: + ... + + The iteration happens in the order of definition within the Kconfig + configuration files. Symbols only referred to but not defined will not + be included, nor will the special symbols n, m, and y. If you want to + include such symbols as well, see config.get_symbols().""" + return iter(self.kconfig_syms) + + def reset(self): + """Resets the values of all symbols, as if Config.load_config() or + Symbol.set_value() had never been called.""" + for sym in self.syms.itervalues(): + sym._reset_no_recursive_invalidate() + + def __str__(self): + """Returns a string containing various information about the Config.""" + return _sep_lines("Configuration", + "File : " + self.filename, + "Base directory : " + self.base_dir, + "Arch (value of ARCH at time of creation) : " + self.arch, + "Most recently loaded .config : " + + ("(no .config loaded)" if self.config_filename is None else + self.config_filename), + "Print warnings : " + + bool_str[self.print_warnings], + "Print assignments to undefined symbols : " + + bool_str[self.print_undef_assign]) + + + # + # Private methods + # + + def _invalidate_all(self): + for sym in self.syms.itervalues(): + sym._invalidate() + + def _tokenize(self, + s, + add_sym_if_not_exists = True, + filename = None, + linenr = None): + """Returns a _Feed instance containing tokens derived from the string + 's'. Registers any new symbols encountered (via _sym_lookup()). + + (I experimented with a regular expression implementation, but it came + out 5% _slower_ and wouldn't have been as flexible). + + add_sym_if_not_exists -- False when we do not want to register new + symbols, such as when called from + Config.eval().""" + tokens = [] + previous = None + + strlen = len(s) + append = tokens.append + + # The current index in the string being tokenized + i = 0 + + while i < strlen: + c = s[i] + + if c.isspace(): + i += 1 + continue + + elif c == "=": + append(T_EQUAL) + i += 1 + + elif c == "!": + if i + 1 >= strlen: + _tokenization_error(s, strlen, filename, linenr) + if s[i + 1] == "=": + append(T_UNEQUAL) + i += 2 + else: + append(T_NOT) + i += 1 + + elif c == "(": + append(T_OPEN_PAREN) + i += 1 + + elif c == ")": + append(T_CLOSE_PAREN) + i += 1 + + elif c == "&": + if i + 1 >= strlen: + _tokenization_error(s, strlen, filename, linenr) + if s[i + 1] != "&": + _tokenization_error(s, i + 1, filename, linenr) + append(T_AND) + i += 2 + + elif c == "|": + if i + 1 >= strlen: + _tokenization_error(s, strlen, filename, linenr) + if s[i + 1] != "|": + _tokenization_error(s, i + 1, filename, linenr) + append(T_OR) + i += 2 + + elif c == '"' or c == "'": + quote = c + value = "" + i += 1 + while True: + if i >= strlen: + _tokenization_error(s, strlen, filename, linenr) + c = s[i] + if c == quote: + break + elif c == "\\": + if i + 1 >= strlen: + _tokenization_error(s, strlen, filename, linenr) + value += s[i + 1] + i += 2 + else: + value += c + i += 1 + i += 1 + append(value) + + elif c == "#": + break + + else: # Symbol or keyword + name_start = i + + # Locate the end of the symbol/keyword + i += 1 + while i < strlen and s[i] in sym_chars: + i += 1 + + name = s[name_start:i] + + keyword = keywords.get(name) + + if keyword is not None: + append(keyword) + # What would ordinarily be considered a name is treated as a + # string after certain tokens. + elif previous in string_lex: + append(name) + else: + # We're dealing with a symbol. _sym_lookup() will take care + # of allocating a new Symbol instance if it's the first + # time we see it. + sym = self._sym_lookup(name, add_sym_if_not_exists) + + if previous == T_CONFIG: + # If the previous token is T_CONFIG ("config"), we're + # tokenizing the first line of a symbol definition, and + # should remember this as a location where the symbol + # is defined. + sym.def_locations.append((filename, linenr)) + else: + # Otherwise, it's a reference to the symbol + sym.ref_locations.append((filename, linenr)) + + append(sym) + + previous = tokens[-1] + + return _Feed(tokens) + + # + # Parsing + # + + # Expression grammar: + # + # -> + # '=' + # '!=' + # '(' ')' + # '!' + # '&&' + # '||' + + def _parse_expr(self, + feed, + cur_sym_or_choice, + line, + filename = None, + linenr = None, + transform_m = True): + """Parse an expression from the tokens in 'feed' using a simple + top-down approach. The result has the form (, ). + + feed -- _Feed instance containing the tokens for the expression. + + cur_sym_or_choice -- The symbol or choice currently being parsed, or + None if we're not parsing a symbol or choice. + Used for recording references to symbols. + + line -- The line containing the expression being parsed. + + filename (default: None) -- The file containing the expression. + + linenr (default: None) -- The line number containing the expression. + + transform_m (default: False) -- Determines if 'm' should be rewritten to + 'm && MODULES' -- see + parse_val_and_cond().""" + + # Use instance variables to avoid having to pass these as arguments + # through the top-down parser in _parse_expr_2(), which is tedious and + # obfuscates the code. A profiler run shows no noticeable performance + # difference. + self.parse_expr_cur_sym_or_choice = cur_sym_or_choice + self.parse_expr_line = line + self.parse_expr_filename = filename + self.parse_expr_linenr = linenr + self.parse_expr_transform_m = transform_m + + return self._parse_expr_2(feed) + + def _parse_expr_2(self, feed): + or_terms = [self._parse_or_term(feed)] + # Keep parsing additional terms while the lookahead is '||' + while feed.check(T_OR): + or_terms.append(self._parse_or_term(feed)) + + if len(or_terms) == 1: + return or_terms[0] + else: + return (OR, or_terms) + + def _parse_or_term(self, feed): + and_terms = [self._parse_factor(feed)] + # Keep parsing additional terms while the lookahead is '&&' + while feed.check(T_AND): + and_terms.append(self._parse_factor(feed)) + + if len(and_terms) == 1: + return and_terms[0] + else: + return (AND, and_terms) + + def _parse_factor(self, feed): + if feed.check(T_OPEN_PAREN): + expr_parse = self._parse_expr_2(feed) + + if not feed.check(T_CLOSE_PAREN): + _parse_error(self.parse_expr_line, + "missing end parenthesis.", + self.parse_expr_filename, + self.parse_expr_linenr) + + return expr_parse + + if feed.check(T_NOT): + return (NOT, self._parse_factor(feed)) + + sym_or_string = feed.get_next() + + if not isinstance(sym_or_string, (Symbol, str)): + _parse_error(self.parse_expr_line, + "malformed expression.", + self.parse_expr_filename, + self.parse_expr_linenr) + + if self.parse_expr_cur_sym_or_choice is not None and \ + isinstance(sym_or_string, Symbol): + self.parse_expr_cur_sym_or_choice.referenced_syms.add(sym_or_string) + + if feed.peek_next() not in (T_EQUAL, T_UNEQUAL): + if self.parse_expr_transform_m and (sym_or_string is self.m or + sym_or_string == "m"): + return (AND, ["m", self._sym_lookup("MODULES")]) + return sym_or_string + + relation = EQUAL if (feed.get_next() == T_EQUAL) else UNEQUAL + sym_or_string_2 = feed.get_next() + + if sym_or_string is self.m: + sym_or_string = "m" + + if sym_or_string_2 is self.m: + sym_or_string_2 = "m" + + return (relation, sym_or_string, sym_or_string_2) + + def _parse_file(self, filename, parent, deps, visible_if_deps): + """Parse the Kconfig file 'filename'. The result is + a _Block with all items from the file.""" + line_feeder = _FileFeed(_get_lines(filename), filename) + return self._parse_block(line_feeder, None, parent, deps, visible_if_deps) + + def _parse_block(self, line_feeder, end_marker, parent, deps, visible_if_deps = None): + """Parses a block, which is the contents of either a file or an if, + menu, or choice statement. + + end_marker -- The token that ends the block, e.g. T_ENDIF ("endif") for + if's. None for files. + + parent -- The enclosing menu, choice or if, or None if we're at the top + level. + + deps -- Dependencies from enclosing menus, choices and if's. + + visible_if_deps (default: None) -- 'visible if' dependencies from + enclosing menus.""" + + block = _Block() + + filename = line_feeder.get_filename() + + while True: + + # Do we already have a tokenized line that we determined wasn't + # part of whatever we were parsing earlier? See comment in + # Config.__init__(). + if self.end_line is not None: + assert self.end_line_tokens is not None + tokens = self.end_line_tokens + tokens.go_to_start() + + line = self.end_line + linenr = line_feeder.get_linenr() + + self.end_line = None + self.end_line_tokens = None + + else: + line = line_feeder.get_next() + if line is None: + if end_marker is not None: + raise Kconfig_Syntax_Error, ( + "Unexpected end of file {0}." + .format(line_feeder.get_filename())) + return block + + linenr = line_feeder.get_linenr() + + tokens = self._tokenize(line, True, filename, linenr) + + if tokens.is_empty(): + continue + + t0 = tokens.get_next() + + # Have we reached the end of the block? + if t0 == end_marker: + return block + + elif t0 in (T_CONFIG, T_MENUCONFIG): + # The tokenizer will automatically allocate a new Symbol object + # for any new names it encounters, so we don't need to worry + # about that here. + sym = tokens.get_next() + + # Symbols defined in multiple places get the parent of their + # first definition. However, for symbols whose parents are choice + # statements, the choice statement takes precedence. + if not sym.is_defined_ or isinstance(parent, Choice): + sym.parent = parent + + sym.is_defined_ = True + + self.kconfig_syms.append(sym) + block.add_item(sym) + + self._parse_properties(line_feeder, sym, deps, visible_if_deps) + + elif t0 == T_MENU: + menu = Menu() + menu.config = self + menu.parent = parent + menu.title = tokens.get_next() + + menu.filename = filename + menu.linenr = linenr + + # Parse properties and contents + self._parse_properties(line_feeder, menu, deps, visible_if_deps) + menu.block = self._parse_block(line_feeder, + T_ENDMENU, + menu, + menu.dep_expr, + self._make_and(visible_if_deps, + menu.visible_if_expr)) + + block.add_item(menu) + self.menus.append(menu) + + elif t0 == T_IF: + # If statements are treated as syntactic sugar for adding + # dependencies to enclosed items and do not have an explicit + # object representation. + + dep_expr = self._parse_expr(tokens, None, line, filename, linenr) + if_block = self._parse_block(line_feeder, + T_ENDIF, + parent, + self._make_and(dep_expr, deps), + visible_if_deps) + + block.add_items_from_block(if_block) + + elif t0 == T_CHOICE: + # We support named choices + already_defined = False + name = None + if len(tokens) > 1 and isinstance(tokens[1], str): + name = tokens[1] + already_defined = name in self.named_choices + + if already_defined: + choice = self.named_choices[name] + else: + choice = Choice() + if name is not None: + choice.name = name + self.named_choices[name] = choice + + choice.config = self + + choice.def_locations.append((filename, linenr)) + + # Parse properties and contents + self._parse_properties(line_feeder, choice, deps, visible_if_deps) + choice.block = self._parse_block(line_feeder, + T_ENDCHOICE, + choice, + None, + visible_if_deps) + + choice._determine_actual_items() + + # If no type is set for the choice, its type is that of the first + # choice item + if choice.type == UNKNOWN: + for item in choice.get_actual_items(): + if item.get_type() != UNKNOWN: + choice.type = item.get_type() + break + + # Each choice item of UNKNOWN type gets the type of the choice + for item in choice.get_actual_items(): + if item.type == UNKNOWN: + item.type = choice.type + + # For named choices defined in multiple locations, only record + # at the first definition + if not already_defined: + block.add_item(choice) + self.choices.append(choice) + + elif t0 == T_COMMENT: + comment = Comment() + comment.config = self + comment.parent = parent + + comment.filename = filename + comment.linenr = linenr + + comment.text = tokens.get_next() + self._parse_properties(line_feeder, comment, deps, visible_if_deps) + + block.add_item(comment) + self.comments.append(comment) + + elif t0 == T_SOURCE: + kconfig_file = tokens.get_next() + f = os.path.join(self.base_dir, os.path.expandvars(kconfig_file)) + + if not os.path.exists(f): + raise IOError, ('{0}:{1}: sourced file "{2}" not found. Perhaps ' + 'base_dir (argument to Config.__init__(), currently ' + '"{3}") is set to the wrong value.' + .format(filename, + linenr, + kconfig_file, + self.base_dir)) + + file_block = self._parse_file(f, parent, deps, visible_if_deps) + block.add_items_from_block(file_block) + + elif t0 == T_MAINMENU: + text = tokens.get_next() + + if self.mainmenu_text is not None: + self._warn("overriding 'mainmenu' text. " + 'Old value: "{0}", new value: "{1}".' + .format(self.mainmenu_text, text), + filename, + linenr) + + self.mainmenu_text = os.path.expandvars(text) + + else: + _parse_error(line, "unrecognized construct.", filename, linenr) + + def _parse_properties(self, line_feeder, stmt, deps, visible_if_deps): + """Parsing of properties for symbols, menus, choices, and comments.""" + + def parse_val_and_cond(tokens, line, filename, linenr): + """Parses ' if ' constructs, where the 'if' part is + optional. Returns a tuple containing the parsed expressions, with + None as the second element if the 'if' part is missing.""" + val = self._parse_expr(tokens, stmt, line, filename, linenr, False) + + if tokens.check(T_IF): + return (val, self._parse_expr(tokens, stmt, line, filename, linenr)) + + return (val, None) + + # In case the symbol is defined in multiple locations, we need to + # remember what prompts, defaults, and selects are new for this + # definition, as "depends on" should only apply to the local + # definition. + new_prompt = None + new_def_exprs = [] + new_selects = [] + + # Dependencies from 'depends on' statements + depends_on_expr = None + + while True: + line = line_feeder.get_next() + if line is None: + break + + filename = line_feeder.get_filename() + linenr = line_feeder.get_linenr() + + tokens = self._tokenize(line, True, filename, linenr) + + if tokens.is_empty(): + continue + + t0 = tokens.get_next() + + if t0 == T_HELP: + # Find first non-empty line and get its indentation + + line_feeder.remove_while(str.isspace) + line = line_feeder.get_next() + + if line is None: + stmt.help = "" + break + + indent = _indentation(line) + + # If the first non-empty lines has zero indent, there is no + # help text + if indent == 0: + stmt.help = "" + line_feeder.go_back() + break + + help_lines = [_deindent(line, indent)] + + # The help text goes on till the first non-empty line with less + # indent + while True: + line = line_feeder.get_next() + if (line is None) or \ + (not line.isspace() and _indentation(line) < indent): + stmt.help = "".join(help_lines) + break + + help_lines.append(_deindent(line, indent)) + + if line is None: + break + + line_feeder.go_back() + + elif t0 == T_PROMPT: + # 'prompt' properties override each other within a single + # definition of a symbol, but additional prompts can be added + # by defining the symbol multiple times; hence 'new_prompt' + # instead of 'prompt'. + new_prompt = parse_val_and_cond(tokens, line, filename, linenr) + + elif t0 == T_DEFAULT: + new_def_exprs.append(parse_val_and_cond(tokens, line, filename, linenr)) + + elif t0 == T_DEPENDS: + if not tokens.check(T_ON): + _parse_error(line, 'expected "on" after "depends".', filename, linenr) + + parsed_deps = self._parse_expr(tokens, stmt, line, filename, linenr) + + if isinstance(stmt, (Menu, Comment)): + stmt.dep_expr = self._make_and(stmt.dep_expr, parsed_deps) + else: + depends_on_expr = self._make_and(depends_on_expr, parsed_deps) + + elif t0 == T_VISIBLE: + if not tokens.check(T_IF): + _parse_error(line, 'expected "if" after "visible".', filename, linenr) + if not isinstance(stmt, Menu): + _parse_error(line, + "'visible if' is only valid for menus.", + filename, + linenr) + + parsed_deps = self._parse_expr(tokens, stmt, line, filename, linenr) + stmt.visible_if_expr = self._make_and(stmt.visible_if_expr, parsed_deps) + + elif t0 == T_SELECT: + target = tokens.get_next() + + stmt.referenced_syms.add(target) + stmt.selected_syms.add(target) + + if tokens.check(T_IF): + new_selects.append((target, + self._parse_expr(tokens, stmt, line, filename, linenr))) + else: + new_selects.append((target, None)) + + elif t0 in (T_BOOL, T_TRISTATE, T_INT, T_HEX, T_STRING): + stmt.type = token_to_type[t0] + + if len(tokens) > 1: + new_prompt = parse_val_and_cond(tokens, line, filename, linenr) + + elif t0 == T_RANGE: + lower = tokens.get_next() + upper = tokens.get_next() + + if tokens.check(T_IF): + stmt.ranges.append((lower, upper, + self._parse_expr(tokens, stmt, line, filename, linenr))) + else: + stmt.ranges.append((lower, upper, None)) + + elif t0 == T_DEF_BOOL: + stmt.type = BOOL + + if len(tokens) > 1: + new_def_exprs.append(parse_val_and_cond(tokens, line, filename, linenr)) + + elif t0 == T_DEF_TRISTATE: + stmt.type = TRISTATE + + if len(tokens) > 1: + new_def_exprs.append(parse_val_and_cond(tokens, line, filename, linenr)) + + elif t0 == T_OPTIONAL: + if not isinstance(stmt, Choice): + _parse_error(line, + '"optional" is only valid for choices.', + filename, + linenr) + stmt.optional = True + + elif t0 == T_OPTION: + if tokens.check(T_ENV) and tokens.check(T_EQUAL): + env_var = tokens.get_next() + + stmt.is_special_ = True + stmt.is_from_env = True + + if env_var not in os.environ: + self._warn(""" +The symbol {0} references the non-existent environment variable {1} and will +get the empty string as its value. You could set it in os.environ before +calling Config() or make sure it is exported before invoking the script by +running e.g. + +$ export {1}= +$ python foo.py + +or + +$ {1}= python foo.py + +If you're using kconfiglib via 'make scriptconfig' it should have set up +the environment correctly for you. If you still got this message, that +might be an error, and you should e-mail kconfiglib@gmail.com. +.""" .format(stmt.get_name(), env_var), + filename, + linenr) + + stmt.cached_value = "" + else: + stmt.cached_value = os.environ[env_var] + + elif tokens.check(T_DEFCONFIG_LIST): + self.defconfig_sym = stmt + + elif tokens.check(T_MODULES): + self._warn("the 'modules' option is not supported. " + "Let me know if this is a problem for you; " + "it shouldn't be that hard to implement.", + filename, + linenr) + + else: + _parse_error(line, "unrecognized option.", filename, linenr) + + else: + # See comment in Config.__init__() + self.end_line = line + self.end_line_tokens = tokens + break + + # Propagate dependencies from enclosing menus and if's. + + # For menus and comments.. + if isinstance(stmt, (Menu, Comment)): + stmt.orig_deps = stmt.dep_expr + stmt.deps_from_containing = deps + stmt.dep_expr = self._make_and(stmt.dep_expr, deps) + + stmt.all_referenced_syms = \ + stmt.referenced_syms.union(self._get_expr_syms(deps)) + + # For symbols and choices.. + else: + + # See comment for 'menu_dep' + stmt.menu_dep = depends_on_expr + + # Propagate dependencies specified with 'depends on' to any new + # default expressions, prompts, and selections ("new" since a + # symbol might be defined in multiple places and the dependencies + # should only apply to the local definition). + + new_def_exprs = [(val_expr, self._make_and(cond_expr, depends_on_expr)) + for (val_expr, cond_expr) in new_def_exprs] + + new_selects = [(target, self._make_and(cond_expr, depends_on_expr)) + for (target, cond_expr) in new_selects] + + if new_prompt is not None: + prompt, cond_expr = new_prompt + + # 'visible if' dependencies from enclosing menus get propagated + # to prompts + if visible_if_deps is not None: + cond_expr = self._make_and(cond_expr, visible_if_deps) + + new_prompt = (prompt, self._make_and(cond_expr, depends_on_expr)) + + # We save the original expressions -- before any menu and if + # conditions have been propagated -- so these can be retrieved + # later. + + stmt.orig_def_exprs.extend(new_def_exprs) + if new_prompt is not None: + stmt.orig_prompts.append(new_prompt) + + # Only symbols can select + if isinstance(stmt, Symbol): + stmt.orig_selects.extend(new_selects) + + # Save dependencies from enclosing menus and if's + stmt.deps_from_containing = deps + + # The set of symbols referenced directly by the symbol/choice plus + # all symbols referenced by enclosing menus and if's. + stmt.all_referenced_syms \ + = stmt.referenced_syms.union(self._get_expr_syms(deps)) + + # Propagate dependencies from enclosing menus and if's + + stmt.def_exprs.extend([(val_expr, self._make_and(cond_expr, deps)) + for (val_expr, cond_expr) in new_def_exprs]) + + for (target, cond) in new_selects: + target.rev_dep = self._make_or(target.rev_dep, + self._make_and(stmt, + self._make_and(cond, deps))) + + if new_prompt is not None: + prompt, cond_expr = new_prompt + stmt.prompts.append((prompt, self._make_and(cond_expr, deps))) + + # + # Symbol table manipulation + # + + def _sym_lookup(self, name, add_sym_if_not_exists = True): + """Fetches the symbol 'name' from the symbol table, optionally adding + it if it does not exist (this is usually what we want).""" + if name in self.syms: + return self.syms[name] + + new_sym = Symbol() + new_sym.config = self + new_sym.name = name + + if add_sym_if_not_exists: + self.syms[name] = new_sym + else: + # This warning is generated while evaluating an expression + # containing undefined symbols using Config.eval() + self._warn("no symbol {0} in configuration".format(name)) + + return new_sym + + # + # Evaluation of symbols and expressions + # + + def _eval_expr(self, expr): + """Evaluates an expression and returns one of the tristate values "n", + "m" or "y".""" + res = self._eval_expr_2(expr) + + # Promote "m" to "y" if we're running without modules. Internally, "m" + # is often rewritten to "m" && MODULES by both the C implementation and + # kconfiglib, which takes care of cases where "m" should be false if + # we're running without modules. + if res == "m" and not self._has_modules(): + return "y" + + return res + + def _eval_expr_2(self, expr): + if expr is None: + return "y" + + if isinstance(expr, (str, Symbol)): + val = self._get_str_value(expr) + # Expressions always have a tristate value + return val if val in ("y", "m") else "n" + + first_expr = expr[0] + + if first_expr == OR: + res = "n" + + for subexpr in expr[1]: + ev = self._eval_expr_2(subexpr) + + # Return immediately upon discovering a "y" term + if ev == "y": + return "y" + + if ev == "m": + res = "m" + + # 'res' is either "n" or "m" here; we already handled the + # short-circuiting "y" case in the loop. + return res + + if first_expr == AND: + res = "y" + + for subexpr in expr[1]: + ev = self._eval_expr_2(subexpr) + + # Return immediately upon discovering an "n" term + if ev == "n": + return "n" + + if ev == "m": + res = "m" + + # 'res' is either "m" or "y" here; we already handled the + # short-circuiting "n" case in the loop. + return res + + if first_expr == NOT: + ev = self._eval_expr_2(expr[1]) + + if ev == "y": + return "n" + + return "y" if (ev == "n") else "m" + + if first_expr in (EQUAL, UNEQUAL): + _, sym_or_str_1, sym_or_str_2 = expr + + val_1 = self._get_str_value(sym_or_str_1) + val_2 = self._get_str_value(sym_or_str_2) + + if (first_expr == EQUAL and val_1 == val_2) or \ + (first_expr == UNEQUAL and val_1 != val_2): + + return "y" + + return "n" + + _internal_error("Internal error while evaluating expression with token stream {0}: " + "unknown type {0}." + .format(expr)) + + def _eval_to_int(self, expr): + return values[self._eval_expr(expr)] + + def _get_str_value(self, obj): + if isinstance(obj, str): + return obj + + # obj is a Symbol + return obj.calc_value() + + def _eval_min(self, e1, e2): + e1_eval = self._eval_expr(e1) + e2_eval = self._eval_expr(e2) + + return e1_eval if tri_less(e1_eval, e2_eval) else e2_eval + + def _eval_max(self, e1, e2): + e1_eval = self._eval_expr(e1) + e2_eval = self._eval_expr(e2) + + return e1_eval if tri_greater(e1_eval, e2_eval) else e2_eval + + # + # Construction of expressions + # + + # These functions as well as the _eval_min/max() functions above equate + # None with "y", which is usually what we want, but needs to be kept in + # mind. + + def _make_or(self, e1, e2): + # Perform trivial simplification and avoid None's (which + # correspond to y's) + if "y" in (e1, e2) or None in (e1, e2): + return "y" + + if e1 == "n": + return e2 + + if e2 == "n": + return e1 + + # Prefer to merge/update argument list if possible instead of creating + # a new OR node + + if isinstance(e1, tuple) and e1[0] == OR: + if isinstance(e2, tuple) and e2[0] == OR: + return (OR, e1[1] + e2[1]) + else: + return (OR, e1[1] + [e2]) + + if isinstance(e2, tuple) and e2[0] == OR: + return (OR, e2[1] + [e1]) + + return (OR, [e1, e2]) + + # Note: returns None if e1 == e2 == None + + def _make_and(self, e1, e2): + if "n" in (e1, e2): + return "n" + + if e1 in (None, "y"): + return e2 + + if e2 in (None, "y"): + return e1 + + # Prefer to merge/update argument list if possible instead of creating + # a new AND node + + if isinstance(e1, tuple) and e1[0] == AND: + if isinstance(e2, tuple) and e2[0] == AND: + return (AND, e1[1] + e2[1]) + else: + return (AND, e1[1] + [e2]) + + if isinstance(e2, tuple) and e2[0] == AND: + return (AND, e2[1] + [e1]) + + return (AND, [e1, e2]) + + # + # Methods related to the MODULES symbol + # + + def _has_modules(self): + modules_sym = self.syms.get("MODULES") + return (modules_sym is not None) and (modules_sym.calc_value() == "y") + + # + # Dependency tracking + # + + def _build_dep(self): + """Populates the dep dictionary, linking a symbol to the symbols that + immediately depend on it.""" + for sym in self.syms.itervalues(): + self.dep[sym] = set() + + def add_expr_deps(e, sym): + for s in self._get_expr_syms(e): + self.dep[s].add(sym) + + # The directly dependent symbols of a symbol are: + # - Any symbols whose prompt or default value depends on the symbol + # - Any symbols whose rev_dep (select condition) depends on the symbol + # - Any symbols that belong to the same choice statement as the symbol + # (these won't be included in 'dep' as that makes the dependency + # graph unwieldy, but Symbol._get_dependent() will include them) + # - Any symbols in a choice statement that depends on the symbol + for sym in self.syms.itervalues(): + for (_, e) in sym.prompts: + add_expr_deps(e, sym) + + for (v, e) in sym.def_exprs: + add_expr_deps(v, sym) + add_expr_deps(e, sym) + + add_expr_deps(sym.rev_dep, sym) + + if sym.is_choice_item(): + choice = sym.parent + + for (_, e) in choice.prompts: + add_expr_deps(e, sym) + + for (_, e) in choice.def_exprs: + add_expr_deps(e, sym) + + def _get_expr_syms(self, expr): + """Returns the set() of symbols appearing in expr.""" + if expr is None or isinstance(expr, str): + return set() + + if isinstance(expr, Symbol): + return set((expr,)) + + elif expr[0] in (OR, AND): + res = set() + for subres in [self._get_expr_syms(e) for e in expr[1]]: + res.update(subres) + return res + + elif expr[0] == NOT: + return self._get_expr_syms(expr[1]) + + elif expr[0] in (EQUAL, UNEQUAL): + res = set() + + (_, v1, v2) = expr + + if isinstance(v1, Symbol): + res.add(v1) + + if isinstance(v2, Symbol): + res.add(v2) + + return res + + else: + _internal_error("Internal error while fetching symbols from an expression with " + "token stream {0}.".format(expr)) + + def _expr_val_str(self, expr, no_value_str = "(none)", get_val_instead_of_eval = False): + # Since values are valid expressions, _expr_to_str() will get a nice + # string representation for those as well. + + if expr is None: + return no_value_str + + if get_val_instead_of_eval: + if isinstance(expr, str): + return _expr_to_str(expr) + val = expr.calc_value() + else: + val = self._eval_expr(expr) + + return "{0} (value: {1})".format(_expr_to_str(expr), _expr_to_str(val)) + + def _get_sym_or_choice_str(self, sc): + """Symbols and choices have many properties in common, so we factor out + common __str__() stuff here. "sc" is short for "symbol or choice".""" + + # As we deal a lot with string representations here, use some + # convenient shorthand: + s = _expr_to_str + + # + # Common symbol/choice properties + # + + user_value_str = "(no user value)" if sc.user_val is None else s(sc.user_val) + + visibility_str = s(sc.get_visibility()) + + # Build prompts string + if sc.prompts == []: + prompts_str = " (no prompts)" + else: + prompts_str_rows = [] + + for (prompt, cond_expr) in sc.orig_prompts: + if cond_expr is None: + prompts_str_rows.append(' "{0}"'.format(prompt)) + else: + prompts_str_rows.append(' "{0}" if '.format(prompt) + + self._expr_val_str(cond_expr)) + + prompts_str = "\n".join(prompts_str_rows) + + # Build locations string + if sc.def_locations == []: + locations_str = "(no locations)" + else: + locations_str = " ".join(["{0}:{1}".format(filename, linenr) for + (filename, linenr) in sc.def_locations]) + + # Build additional-dependencies-from-menus-and-if's string + additional_deps_str = " " + self._expr_val_str(sc.deps_from_containing, + "(no additional dependencies)") + + # + # Symbol-specific stuff + # + + if isinstance(sc, Symbol): + + # Build value string + value_str = s(sc.calc_value()) + + # Build ranges string + if isinstance(sc, Symbol): + if sc.ranges == []: + ranges_str = " (no ranges)" + else: + ranges_str_rows = [] + + for (l, u, cond_expr) in sc.ranges: + if cond_expr is None: + ranges_str_rows.append(" [{0}, {1}]".format(s(l), s(u))) + else: + ranges_str_rows.append(" [{0}, {1}] if {2}" + .format(s(l), s(u), self._expr_val_str(cond_expr))) + + ranges_str = "\n".join(ranges_str_rows) + + # Build default values string + if sc.def_exprs == []: + defaults_str = " (no default values)" + else: + defaults_str_rows = [] + + for (val_expr, cond_expr) in sc.orig_def_exprs: + row_str = " " + self._expr_val_str(val_expr, "(none)", sc.get_type() == STRING) + defaults_str_rows.append(row_str) + defaults_str_rows.append(" Condition: " + self._expr_val_str(cond_expr)) + + defaults_str = "\n".join(defaults_str_rows) + + # Build selects string + if sc.orig_selects == []: + selects_str = " (no selects)" + else: + selects_str_rows = [] + + for (target, cond_expr) in sc.orig_selects: + if cond_expr is None: + selects_str_rows.append(" {0}".format(target.name)) + else: + selects_str_rows.append(" {0} if ".format(target.name) + + self._expr_val_str(cond_expr)) + + selects_str = "\n".join(selects_str_rows) + + # Build reverse dependencies string + if sc.rev_dep == "n": + rev_dep_str = " (no reverse dependencies)" + else: + rev_dep_str = " " + self._expr_val_str(sc.rev_dep) + + res = _sep_lines("Symbol " + (sc.name if sc.name is not None else "(no name)"), + "Type : " + typename[sc.type], + "Value : " + value_str, + "User value : " + user_value_str, + "Visibility : " + visibility_str, + "Is choice item : " + bool_str[sc.is_choice_item()], + "Is defined : " + bool_str[sc.is_defined_], + "Is from env. : " + bool_str[sc.is_from_environment()], + "Is special : " + bool_str[sc.is_special_] + "\n") + + if sc.has_ranges(): + res += _sep_lines("Ranges:", + ranges_str + "\n") + + res += _sep_lines("Prompts:", + prompts_str, + "Default values:", + defaults_str, + "Selects:", + selects_str, + "Reverse dependencies:", + rev_dep_str, + "Additional dependencies from enclosing menus and if's:", + additional_deps_str, + "Locations: " + locations_str) + + return res + + # + # Choice-specific stuff + # + + # Build name string (for named choices) + if sc.get_name() is None: + name_str = "(no name)" + else: + name_str = sc.get_name() + + # Build selected symbol string + sel = sc.get_selection() + if sel is None: + sel_str = "(no selection)" + else: + sel_str = sel.get_name() + + # Build mode string + mode_str = s(sc.calc_mode()) + + # Build default values string + if sc.def_exprs == []: + defaults_str = " (no default values)" + else: + defaults_str_rows = [] + + for (sym, cond_expr) in sc.orig_def_exprs: + if cond_expr is None: + defaults_str_rows.append(" {0}".format(sym.get_name())) + else: + defaults_str_rows.append(" {0} if ".format(sym.get_name()) + + self._expr_val_str(cond_expr)) + + defaults_str = "\n".join(defaults_str_rows) + + # Build contained symbols string + names = [sym.get_name() for sym in sc.get_actual_items()] + + if names == []: + syms_string = "(empty)" + else: + syms_string = " ".join(names) + + return _sep_lines("Choice", + "Name (for named choices): " + name_str, + "Type : " + typename[sc.type], + "Selected symbol : " + sel_str, + "User value : " + user_value_str, + "Mode : " + mode_str, + "Visibility : " + visibility_str, + "Optional : " + bool_str[sc.optional], + "Prompts:", + prompts_str, + "Defaults:", + defaults_str, + "Choice symbols:", + " " + syms_string, + "Additional dependencies from enclosing menus and if's:", + additional_deps_str, + "Locations: " + locations_str) + + def _expr_depends_on(self, expr, sym): + """Reimplementation of expr_depends_symbol() from mconf.c. Used to + determine if a submenu should be implicitly created, which influences what + items inside choice statements are considered choice items.""" + if expr is None: + return False + + def rec(expr): + if isinstance(expr, str): + return False + + if isinstance(expr, Symbol): + return expr is sym + + if expr[0] in (EQUAL, UNEQUAL): + return self._eq_to_sym(expr) is sym + + if expr[0] == AND: + for and_expr in expr[1]: + if rec(and_expr): + return True + + return False + + return False + + return rec(expr) + + def _eq_to_sym(self, eq): + """_expr_depends_on() helper. For (in)equalities of the form sym = y/m + or sym != n, returns sym. For other (in)equalities, returns None.""" + (relation, left, right) = eq + + left = self._transform_n_m_y(left) + right = self._transform_n_m_y(right) + + # Make sure the symbol (if any) appears to the left + if not isinstance(left, Symbol): + left, right = right, left + + if not isinstance(left, Symbol): + return None + + if (relation == EQUAL and right in ("m", "y")) or \ + (relation == UNEQUAL and right == "n"): + return left + + return None + + def _transform_n_m_y(self, item): + """_eq_to_sym() helper. Translates the symbols n, m, and y to their + string equivalents.""" + if item is self.syms["n"]: + return "n" + if item is self.syms["m"]: + return "m" + if item is self.syms["y"]: + return "y" + return item + + def _warn(self, msg, filename = None, linenr = None): + """For printing warnings to stderr.""" + if self.print_warnings: + self._warn_or_undef_assign(msg, WARNING, filename, linenr) + + def _undef_assign(self, msg, filename = None, linenr = None): + """For printing informational messages related to assignments + to undefined variables to stderr.""" + if self.print_undef_assign: + self._warn_or_undef_assign(msg, UNDEF_ASSIGN, filename, linenr) + + def _warn_or_undef_assign(self, msg, msg_type, filename, linenr): + if filename is not None: + sys.stderr.write("{0}:".format(_clean_up_path(filename))) + if linenr is not None: + sys.stderr.write("{0}:".format(linenr)) + + if msg_type == WARNING: + sys.stderr.write("warning: ") + elif msg_type == UNDEF_ASSIGN: + sys.stderr.write("info: ") + else: + _internal_error('Internal error while printing warning: unknown warning type "{0}".' + .format(msg_type)) + + sys.stderr.write(msg + "\n") + +# +# Constants and functions related to types, parsing, evaluation and printing, +# put globally to unclutter the Config class a bit. +# + +# Tokens +(T_OR, T_AND, T_NOT, + T_OPEN_PAREN, T_CLOSE_PAREN, + T_EQUAL, T_UNEQUAL, + T_MAINMENU, T_MENU, T_ENDMENU, + T_SOURCE, T_CHOICE, T_ENDCHOICE, + T_COMMENT, T_CONFIG, T_MENUCONFIG, + T_HELP, T_IF, T_ENDIF, T_DEPENDS, T_ON, + T_OPTIONAL, T_PROMPT, T_DEFAULT, + T_BOOL, T_TRISTATE, T_HEX, T_INT, T_STRING, + T_DEF_BOOL, T_DEF_TRISTATE, + T_SELECT, T_RANGE, T_OPTION, T_ENV, + T_DEFCONFIG_LIST, T_MODULES, T_VISIBLE) = range(0, 38) + +# Keyword to token map +keywords = { + "mainmenu" : T_MAINMENU, + "menu" : T_MENU, + "endmenu" : T_ENDMENU, + "endif" : T_ENDIF, + "endchoice" : T_ENDCHOICE, + "source" : T_SOURCE, + "choice" : T_CHOICE, + "config" : T_CONFIG, + "comment" : T_COMMENT, + "menuconfig" : T_MENUCONFIG, + "help" : T_HELP, + "---help---" : T_HELP, + "---" : T_HELP, # Hack to handle CONFIG_W1_CON + "if" : T_IF, + "depends" : T_DEPENDS, + "on" : T_ON, + "optional" : T_OPTIONAL, + "prompt" : T_PROMPT, + "default" : T_DEFAULT, + "bool" : T_BOOL, + "boolean" : T_BOOL, + "tristate" : T_TRISTATE, + "int" : T_INT, + "hex" : T_HEX, + "def_bool" : T_DEF_BOOL, + "def_tristate" : T_DEF_TRISTATE, + "string" : T_STRING, + "select" : T_SELECT, + "range" : T_RANGE, + "option" : T_OPTION, + "env" : T_ENV, + "defconfig_list" : T_DEFCONFIG_LIST, + "modules" : T_MODULES, + "visible" : T_VISIBLE } + +# Strings to use for True and False +bool_str = { False : "false", True : "true" } + +# Tokens after which identifier-like lexemes are treated as strings. T_CHOICE +# is included to avoid symbols being registered for named choices. +string_lex = (T_BOOL, T_TRISTATE, T_INT, T_HEX, T_STRING, + T_CHOICE, + T_PROMPT, + T_MENU, + T_COMMENT, + T_SOURCE, + T_MAINMENU) + +# Characters that may appear in symbol names +sym_chars = frozenset(string.ascii_letters + string.digits + "._/-") + +# Regular expressions for parsing .config files +set_re = r"CONFIG_(\w+)=(.*)" +unset_re = r"# CONFIG_(\w+) is not set" + +# Integers representing symbol types +UNKNOWN, BOOL, TRISTATE, STRING, HEX, INT = range(0, 6) + +# Strings to use for types +typename = { + UNKNOWN : "unknown", + BOOL : "bool", + TRISTATE : "tristate", + STRING : "string", + HEX : "hex", + INT : "int" } + +# Token to type mapping +token_to_type = { T_BOOL : BOOL, + T_TRISTATE : TRISTATE, + T_STRING : STRING, + T_INT : INT, + T_HEX : HEX } + +# Default values for symbols of different types (the value the symbol gets if +# it is not assigned a user value and none of its 'default' clauses kick in) +default_value = { BOOL : "n", + TRISTATE : "n", + STRING : "", + INT : "", + HEX : "" } + +# Indicates that no item is selected in a choice statement +NO_SELECTION = 0 + +# Integers representing expression types +OR, AND, NOT, EQUAL, UNEQUAL = range(0, 5) + +# Map from tristate values to integers +values = {"n" : 0, "m" : 1, "y" : 2} + +# Printing-related stuff + +op_to_str = { AND : " && ", + OR : " || ", + EQUAL : " = ", + UNEQUAL : " != " } + +precedence = {OR : 0, AND : 1, NOT : 2} + +# Types of informational messages +WARNING = 0 +UNDEF_ASSIGN = 1 + +def _intersperse(lst, op): + """_expr_to_str() helper. Gets the string representation of each expression in lst + and produces a list where op has been inserted between the elements.""" + if lst == []: + return "" + + res = [] + + def handle_sub_expr(expr): + no_parens = isinstance(expr, (str, Symbol)) or \ + expr[0] in (EQUAL, UNEQUAL) or \ + precedence[op] <= precedence[expr[0]] + if not no_parens: + res.append("(") + res.extend(_expr_to_str_rec(expr)) + if not no_parens: + res.append(")") + + op_str = op_to_str[op] + + handle_sub_expr(lst[0]) + for expr in lst[1:]: + res.append(op_str) + handle_sub_expr(expr) + + return res + +def _expr_to_str(expr): + s = "".join(_expr_to_str_rec(expr)) + return s + +def _sym_str_string(sym_or_str): + if isinstance(sym_or_str, str): + return '"{0}"'.format(sym_or_str) + return sym_or_str.name + +def _expr_to_str_rec(expr): + if expr is None: + return [""] + + if isinstance(expr, (Symbol, str)): + return [_sym_str_string(expr)] + + if expr[0] in (OR, AND): + return _intersperse(expr[1], expr[0]) + + if expr[0] == NOT: + need_parens = not isinstance(expr[1], (str, Symbol)) + + res = ["!"] + if need_parens: + res.append("(") + res.extend(_expr_to_str_rec(expr[1])) + if need_parens: + res.append(")") + return res + + if expr[0] in (EQUAL, UNEQUAL): + return [_sym_str_string(expr[1]), + op_to_str[expr[0]], + _sym_str_string(expr[2])] + +class _Block: + + """Represents a list of items (symbols, menus, choice + statements and comments) appearing at the top-level of a + file or witin a menu, choice or if statement.""" + + def __init__(self): + self.items = [] + + def get_items(self): + return self.items + + def add_item(self, item): + self.items.append(item) + + def add_items_from_block(self, block): + self.items.extend(block.items) + + def _make_conf(self): + # Collect the substrings in a list and later use join() instead of += + # to build the final .config contents. With older Python versions, this + # yields linear instead of quadratic complexity. + strings = [] + for item in self.items: + strings.extend(item._make_conf()) + + return strings + + def add_depend_expr(self, expr): + for item in self.items: + item.add_depend_expr(expr) + +class Item(): + + """Base class for symbols and other Kconfig constructs. Subclasses are + Symbol, Choice, Menu, and Comment.""" + + def is_symbol(self): + """Returns True if the item is a symbol, otherwise False. Short for + isinstance(item, kconfiglib.Symbol).""" + return isinstance(self, Symbol) + + def is_choice(self): + """Returns True if the item is a choice, otherwise False. Short for + isinstance(item, kconfiglib.Choice).""" + return isinstance(self, Choice) + + def is_menu(self): + """Returns True if the item is a menu, otherwise False. Short for + isinstance(item, kconfiglib.Menu).""" + return isinstance(self, Menu) + + def is_comment(self): + """Returns True if the item is a comment, otherwise False. Short for + isinstance(item, kconfiglib.Comment).""" + return isinstance(self, Comment) + +class _HasVisibility(): + + """Base class for elements that have a "visibility" that acts as an upper + limit on the values a user can set for them. Subclasses are Symbol and + Choice.""" + + def __init__(self): + self.cached_visibility = None + self.prompts = [] + + def _invalidate(self): + self.cached_visibility = None + + def _calc_visibility(self): + if self.cached_visibility is None: + vis = "n" + for (prompt, cond_expr) in self.prompts: + vis = self.config._eval_max(vis, cond_expr) + + if isinstance(self, Symbol) and self.is_choice_item(): + vis = self.config._eval_min(vis, self.parent._calc_visibility()) + + # Promote "m" to "y" if we're dealing with a non-tristate + if vis == "m" and self.type != TRISTATE: + vis = "y" + + self.cached_visibility = vis + + return self.cached_visibility + +class Symbol(Item, _HasVisibility): + + """Represents a configuration symbol.""" + + # + # Public interface + # + + def calc_value(self): + """Calculate and return the value of the symbol.""" + + if self.cached_value is not None: + return self.cached_value + + self.write_to_conf = False + + # The following means that what to the user looks like a reference to + # an undefined symbol will actually get treated like a string whose + # value is the symbol name, which can be a bit unexpected. + if self.type == UNKNOWN: + self.cached_value = self.name + return self.name + + new_val = default_value[self.type] + + vis = self._calc_visibility() + + if self.type in (BOOL, TRISTATE): + # The visibility and mode (modules-only or single-selection) of + # choice items will be taken into account in self._calc_visibility() + + if self.is_choice_item(): + if vis != "n": + choice = self.parent + mode = choice.calc_mode() + + self.write_to_conf = (mode != "n") + + if mode == "y": + new_val = "y" if (choice.get_selection() is self) else "n" + elif mode == "m": + if self.user_val in ("m", "y"): + new_val = "m" + + else: + use_defaults = True + + if vis != "n": + # If the symbol is visible and has a user value, use that. + # Otherwise, look at defaults. + self.write_to_conf = True + + if self.user_val is not None: + new_val = self.config._eval_min(self.user_val, vis) + use_defaults = False + + if use_defaults: + for (val_expr, cond_expr) in self.def_exprs: + cond_eval = self.config._eval_expr(cond_expr) + + if cond_eval != "n": + self.write_to_conf = True + new_val = self.config._eval_min(val_expr, cond_eval) + break + + # Reverse dependencies take precedence + rev_dep_val = self.config._eval_expr(self.rev_dep) + + if rev_dep_val != "n": + self.write_to_conf = True + new_val = self.config._eval_max(new_val, rev_dep_val) + + # Promote "m" to "y" for booleans + if new_val == "m" and self.type == BOOL: + new_val = "y" + + elif self.type == STRING: + use_defaults = True + + if vis != "n": + self.write_to_conf = True + if self.user_val is not None: + new_val = self.user_val + use_defaults = False + + if use_defaults: + for (val_expr, cond_expr) in self.def_exprs: + if self.config._eval_expr(cond_expr) != "n": + self.write_to_conf = True + new_val = self.config._get_str_value(val_expr) + break + + elif self.type in (HEX, INT): + active_range = None + use_defaults = True + + # TODO: HEX case should only accept hex and dec numbers + + for(l, u, cond_expr) in self.ranges: + if self.config._eval_expr(cond_expr) != "n": + l_str = self.config._get_str_value(l) + u_str = self.config._get_str_value(u) + + if self.type == INT: + if not _is_dec(l_str): + l_str = "0" + + if not _is_dec(u_str): + u_str = "0" + + active_range = (int(l_str), int(u_str)) + + else: # self.type == HEX + if not _is_hex(l_str): + l_str = "0x0" + + if not _is_hex(u_str): + u_str = "0x0" + + active_range = (int(l_str, 16), int(u_str, 16)) + + break + + if vis != "n": + self.write_to_conf = True + + if self.user_val is not None: + if self.type == INT: + if _is_dec(self.user_val): + num = int(self.user_val) + if active_range is None or \ + active_range[0] <= num <= active_range[1]: + + use_defaults = False + new_val = self.user_val + + else: # HEX + if _is_hex(self.user_val): + num = int(self.user_val, 16) + if active_range is None or \ + active_range[0] <= num <= active_range[1]: + + use_defaults = False + new_val = self.user_val + + if use_defaults: + for (val_expr, cond_expr) in self.def_exprs: + if self.config._eval_expr(cond_expr) != "n": + self.write_to_conf = True + + # Kconfig allows arbitrary string values for hex/int + # (possibly a bug, but we still emulate it). + + new_val = self.config._get_str_value(val_expr) + + if self.type == INT: + if _is_dec(new_val): + new_val_num = int(new_val) + elif self.type == HEX: + if _is_hex(new_val): + new_val_num = int(new_val, 16) + + if new_val_num is not None: + if active_range is not None: + l_num, u_num = active_range + + was_clamped = False + + if new_val_num < l_num: + new_val_num = l_num + was_clamped = True + elif new_val_num > u_num: + new_val_num = u_num + was_clamped = True + + if was_clamped: + new_val = (str(new_val_num) if self.type == INT else + hex(new_val_num)) + + break + + self.cached_value = new_val + return new_val + + def calc_default_value(self): + """Calculates the value the symbol would get purely from defaults, + ignoring visibility (assumed to be "y"), reverse dependencies + (selects), user values and dependencies from enclosing menus and if's. + Returns None if no default would kick in.""" + for (val_expr, cond_expr) in self.orig_def_exprs: + cond_eval = self.config._eval_expr(cond_expr) + + if cond_eval != "n": + return self.config._eval_expr(val_expr) + + return None + + def set_value(self, v): + """Sets the (user) value of the symbol. Equal in effect to assigning + the value to the symbol within a .config file. Use + get_lower/upper_bound() to find the range of valid values for bool and + tristate symbols; setting values outside this range will cause the user + value to differ from the result of Symbol.calc_value(). Any value that + is valid for the type (bool, tristate, etc.) will end up being + reflected in Symbol.get_user_value() though. + + Any symbols dependent on the symbol are (recursively) invalidated, so + things should just work with regards to dependencies. + + v -- The value to give to the symbol.""" + self._set_value_no_invalidate(v, False) + + # There might be something more efficient you could do here, but play + # it safe. + if self.name == "MODULES": + self.config._invalidate_all() + return + + self._invalidate() + self._invalidate_dependent() + + def reset(self): + """Resets the value of the symbol, as if the symbol had never gotten a + (user) value via Config.load_config() or Symbol.set_value(). Dependent + symbols are recursively invalidated.""" + self._reset_no_recursive_invalidate() + self._invalidate_dependent() + + def get_user_value(self): + """Returns the value assigned to the symbol in a .config or via + Symbol.set_value() (provided the value was valid for the type of the + symbol). Returns None in case of no user value.""" + return self.user_val + + def get_name(self): + """Returns the name of the symbol.""" + return self.name + + def get_upper_bound(self): + """Returns the highest value the symbol can be given via + Symbol.set_value() (that will not be truncated): one of "m" or "y", + arranged from lowest to highest. This corresponds to the highest value + the symbol could be given in the 'make menuconfig' interface. Returns + None if the symbol is not visible (would not appear in the 'make + menuconfig' interface), or if the symbol's value cannot be changed (if + it is selected to "y", or to "m" if its visibility also happens to be + "m"). Also returns None for non-bool, non-tristate symbols and special + symbols. + + See also the tri_less*() and tri_greater*() functions, which could + come in handy here.""" + if not self._is_assignable_bool_or_tristate(): + return None + + return self._calc_visibility() + + def get_lower_bound(self): + """Returns the lowest value the symbol can be given via + Symbol.set_value() (that will not be truncated): one of "n" or "m", + arranged from lowest to highest. This corresponds to the lowest value + the symbol could be given in the 'make menuconfig' interface. Returns + None if the symbol is not visible (would not appear in the 'make + menuconfig' interface), or if the symbol's value cannot be changed (if + it is selected to "y", or to "m" if its visibility also happens to be + "m"). Also returns None for non-bool, non-tristate symbols and special + symbols. + + See also the tri_less*() and tri_greater*() functions, which could + come in handy here.""" + if not self._is_assignable_bool_or_tristate(): + return None + + return self.config._eval_expr(self.rev_dep) + + def get_assignable_values(self): + """For bool and tristate symbols, returns a list containing the values + the symbol can be given via Symbol.set_value() (see get_lower_bound()/ + get_upper_bound()). Returns the empty list for symbol that cannot be + given a new value (that cannot be assigned a value that won't be + truncated/ignored that is), as well as for non-bool, non-tristate and + special symbols. Usage example: + + if "m" in sym.get_assignable_values(): + sym.set_value("m")""" + if not self._is_assignable_bool_or_tristate(): + return [] + + return ["n", "m", "y"][values[self.config._eval_expr(self.rev_dep)] : + values[self._calc_visibility()] + 1] + + def get_type(self): + """Returns the type of the symbol: one of UNKNOWN, BOOL, TRISTATE, + STRING, HEX, or INT. These are defined at the top level of the module, + so you'd do something like + + if sym.get_type() == kconfiglib.STRING: + ...""" + return self.type + + def get_visibility(self): + """Returns the visibility of the symbol: one of "n", "m" or "y". For + bool and tristate symbols, this is an upper bound on the value users + can set for the symbol. For other types of symbols, a visibility of "n" + means the user value will be ignored. A visibility of "n" corresponds + to not being visible in the 'make *config' interfaces.""" + return self._calc_visibility() + + def get_parent(self): + """Returns the menu or choice statement that contains the symbol, or + None if the symbol is at the top level. Note that if statements are + treated as syntactic sugar and do not have an explicit class + representation.""" + return self.parent + + def get_sibling_symbols(self, include_self = False): + """Returns a list containing all symbols that are in the same menu or + choice statement as the symbol, or that are also at the top level in + case the symbol is at the top level. + + include_self (default: False) -- True if the symbol itself should be + included in the result, otherwise + False.""" + + return [item for item in self.get_sibling_items(include_self) + if isinstance(item, Symbol)] + + def get_sibling_items(self, include_self = False): + """Returns a list containing all items (symbols, menus, choice + statements and comments) that are in the same menu or choice statement + as the symbol, or that are also at the top level in case the symbol is + at the top level. The items appear in the same order as within the + configuration. + + include_self (default: False) -- True if the symbol itself should be + included in the result, otherwise + False.""" + + if self.parent is None: + items = self.config.get_top_level_items() + else: + items = self.parent.get_items() + + if include_self: + return items + + return [item for item in items if item is not self] + + def get_referenced_symbols(self, refs_from_enclosing = False): + """Returns the set() of all symbols referenced by this symbol. For + example, the symbol defined by + + config FOO + bool + prompt "foo" if A && B + default C if D + depends on E + select F if G + + references the symbols A through G. + + refs_from_enclosing (default: False) -- If True, the symbols + referenced by enclosing menus and if's will be + included in the result.""" + return self.all_referenced_syms if refs_from_enclosing else self.referenced_syms + + def get_selected_symbols(self): + """Returns the set() of all symbols X for which this symbol has a + 'select X' or 'select X if Y' (regardless of whether Y is satisfied or + not). This is a subset of the symbols returned by + get_referenced_symbols().""" + return self.selected_syms + + def get_help(self): + """Returns the help text of the symbol, or None if the symbol has no + help text.""" + return self.help + + def get_config(self): + """Returns the Config instance that represents the configuration this + symbol is from.""" + return self.config + + def get_def_locations(self): + """Returns a list of (filename, linenr) tuples, where filename (string) + and linenr (int) represent a location where the symbol is defined. For + the vast majority of symbols this list will only contain one element. + For the following Kconfig, FOO would get two entries: the lines marked + with *. + + config FOO * + bool "foo prompt 1" + + config FOO * + bool "foo prompt 2" + """ + return self.def_locations + + def get_ref_locations(self): + """Returns a list of (filename, linenr) tuples, where filename (string) + and linenr (int) represent a location where the symbol is referenced in + the configuration. For example, the lines marked by * would be included + for FOO below: + + config A + bool + default BAR || FOO * + + config B + tristate + depends on FOO * + default m if FOO * + + if FOO * + config A + bool "A" + endif + + config FOO (definition not included) + bool + """ + return self.ref_locations + + def is_modifiable(self): + """Returns True or False depending on if the value of symbol could be + modified by setting a user value with Symbol.set_value(). This + corresponds to symbols that would appear in the 'make menuconfig' + interface and not already be pinned to a specific value by being + selected. Returns False for special symbols (e.g. n, m and y).""" + if self.is_special_: + return False + + allowed = range(self.config._eval_to_int(self.config._eval_expr(self.rev_dep)), + self.config._eval_to_int(self._calc_visibility()) + 1) + + return len(allowed) > 1 + + def is_defined(self): + """Returns False if the symbol is referred to in the Kconfig but never + actually defined, otherwise True.""" + return self.is_defined_ + + def is_special(self): + """Returns True if the symbol is one of the special symbols n, m or y, + or gets its value from the environment. Otherwise, returns False.""" + return self.is_special_ + + def is_from_environment(self): + """Returns True if the symbol gets its value from the environment. + Otherwise, returns False.""" + return self.is_from_env + + def has_ranges(self): + """Returns True if the symbol is of type INT or HEX and has ranges that + limits what values it can take on, otherwise False.""" + return self.ranges != [] + + def is_choice_item(self): + """Returns True if the symbol is in a choice statement and is an actual + choice item (see Choice.get_actual_items()); otherwise, returns + False.""" + return self.is_choice_item_ + + def is_choice_selection(self): + """Returns True if the symbol is contained in a choice statement and is + the selected item, otherwise False. Equivalent to 'sym.is_choice_item() + and sym.get_parent().get_selection() is sym'.""" + return self.is_choice_item_ and \ + self.parent.get_selection() is self + + def __str__(self): + """Returns a string containing various information about the symbol.""" + return self.config._get_sym_or_choice_str(self) + + # + # Private methods + # + + def __init__(self): + """Symbol constructor -- not intended to be called directly by + kconfiglib clients.""" + + # Set default values + _HasVisibility.__init__(self) + + self.config = None + + self.parent = None + self.name = None + self.type = UNKNOWN + + self.def_exprs = [] + self.ranges = [] + self.rev_dep = "n" + + # The prompt, default value and select conditions without any + # dependencies from menus or if's propagated to them + + self.orig_prompts = [] + self.orig_def_exprs = [] + self.orig_selects = [] + + # Dependencies inherited from containing menus and if's + self.deps_from_containing = None + + self.help = None + + # The set of symbols referenced by this symbol (see + # get_referenced_symbols()) + self.referenced_syms = set() + + # The set of symbols selected by this symbol (see + # get_selected_symbols()) + self.selected_syms = set() + + # Like 'referenced_syms', but includes symbols from + # dependencies inherited from enclosing menus and if's + self.all_referenced_syms = set() + + # This is set to True for "actual" choice items. See + # Choice._determine_actual_items(). The trailing underscore avoids a + # collision with is_choice_item(). + self.is_choice_item_ = False + + # This records only dependencies specified with 'depends on'. Needed + # when determining actual choice items (hrrrr...). See also + # Choice._determine_actual_items(). + self.menu_dep = None + + # See Symbol.get_ref/def_locations(). + self.def_locations = [] + self.ref_locations = [] + + self.user_val = None + + # Flags + + # Should the symbol get an entry in .config? + self.write_to_conf = False + + # Stores the calculated value to avoid unnecessary recalculation + self.cached_value = None + + # Does the symbol have an entry in the Kconfig file? The Trailing + # underscore avoids a collision with is_defined(). + self.is_defined_ = False + + # Does the symbol get its value in some special way, e.g. from the + # environment or by being one of the special symbols n, m, and y? If + # so, the value is stored in self.cached_value, which is never + # invalidated. The trailing underscore avoids a collision with + # is_special(). + self.is_special_ = False + + # Does the symbol get its value from the environment? + self.is_from_env = False + + # See Choice._make_conf() + self.already_written = False + + def _invalidate(self): + if self.is_special_: + return + + if self.is_choice_item(): + self.parent._invalidate() + + _HasVisibility._invalidate(self) + + self.write_to_conf = False + self.cached_value = None + + def _invalidate_dependent(self): + for sym in self._get_dependent(): + sym._invalidate() + + def _set_value_no_invalidate(self, v, suppress_load_warnings): + """Like set_value(), but does not invalidate any symbols. + + suppress_load_warnings -- + some warnings don't make sense when loading a .config that do make + sense when manually invoking set_value(). This flag is set to True to + suppress such warnings.""" + + if self.is_special_: + if self.is_from_env: + self.config._warn('attempt to assign the value "{0}" to the ' + 'symbol {1}, which gets its value from the ' + 'environment. Assignment ignored.' + .format(v, self.name)) + else: + self.config._warn('attempt to assign the value "{0}" to the ' + 'special symbol {1}. Assignment ignored.' + .format(v, self.name)) + + return + + + if not self.is_defined_: + filename, linenr = self.ref_locations[0] + + self.config._undef_assign('attempt to assign the value "{0}" to {1}, ' + "which is referenced at {2}:{3} but never " + "defined. Assignment ignored." + .format(v, self.name, filename, linenr)) + return + + # Check if the value is valid for our type + + valid = ( self.type == BOOL and v in ("n", "y") ) or \ + ( self.type == TRISTATE and v in ("n", "m", "y") ) or \ + ( self.type == STRING ) or \ + ( self.type == INT and _is_dec(v) ) or \ + ( self.type == HEX and _is_hex(v) ) + + if not valid: + self.config._warn('the value "{0}" is invalid for {1}, which has type {2}. ' + "Assignment ignored." + .format(v, self.name, typename[self.type])) + return + + if self.prompts == [] and not suppress_load_warnings: + self.config._warn('assigning "{0}" to the symbol {1} which lacks ' + 'prompts and thus has visibility "n". The assignment ' + 'will have no effect.' + .format(v, self.name)) + + self.user_val = v + + if self.is_choice_item() and self.type in (BOOL, TRISTATE): + choice = self.parent + if v == "y": + choice.user_val = self + choice.user_mode = "y" + elif v == "m": + choice.user_val = None + choice.user_mode = "m" + + def _reset_no_recursive_invalidate(self): + self._invalidate() + self.user_val = None + + if self.is_choice_item(): + self.parent._reset() + + def _should_write(self): + # This check shouldn't be necessary as write_to_conf is never modified + # in calc_value() for special symbols, but just to be on the safe side: + if self.is_special_: + return False + + # Symbols defined in multiple locations only get one entry in the + # .config. + if self.already_written: + return False + + # write_to_conf is determined in calc_value(), so we need to call that + # first + self.calc_value() + + return self.write_to_conf + + def _make_conf(self): + if not self._should_write(): + return [] + + self.already_written = True + + if self.type in (BOOL, TRISTATE): + if self.calc_value() in ("m", "y"): + return ["CONFIG_{0}={1}".format(self.name, self.calc_value())] + return ["# CONFIG_{0} is not set".format(self.name)] + + elif self.type == STRING: + return ['CONFIG_{0}="{1}"'.format(self.name, self.calc_value())] + + elif self.type in (INT, HEX): + return ["CONFIG_{0}={1}".format(self.name, self.calc_value())] + + else: + _internal_error('Internal error while creating .config: unknown type "{0}".' + .format(self.type)) + + def _get_dependent(self): + """Returns the list of symbols that should be invalidated if the value + of the symbol changes.""" + res = set() + + def rec(sym, ignore_choice = False): + for s in self.config.dep[sym]: + if s not in res: + res.add(s) + rec(s) + + # Handle choices specially to avoid lots of hopping around between + # choice items (which all depend on each other) while calculating + # dependencies + if sym.is_choice_item() and not ignore_choice: + choice = sym.get_parent() + + for s in choice.get_actual_items(): + if s not in res and s is not self: + res.add(s) + rec(s, True) + + rec(self) + return res + + def _has_auto_menu_dep_on(self, on): + """See Choice._determine_actual_items().""" + if not isinstance(self.parent, Choice): + _internal_error("Attempt to determine auto menu dependency for symbol ouside of choice.") + + if self.prompts == []: + # If we have no prompt, use the menu dependencies instead (what was + # specified with 'depends on') + return self.menu_dep is not None and \ + self.config._expr_depends_on(self.menu_dep, on) + + for (_, cond_expr) in self.prompts: + if self.config._expr_depends_on(cond_expr, on): + return True + + return False + + def _is_assignable_bool_or_tristate(self): + """Returns True if the symbol is a bool or tristate whose value can be + changed by the user.""" + return self.type in (BOOL, TRISTATE) and \ + not self.is_special_ and \ + (self.config._eval_to_int(self._calc_visibility()) - + self.config._eval_to_int(self.config._eval_expr(self.rev_dep))) >= 1 + +class Menu(Item): + + """Represents a menu statement.""" + + # + # Public interface + # + + def get_depends_on_visibility(self): + """Returns the visibility the menu gets from 'depends on' conditions. + This is propagated to subitems.""" + return self.config._eval_expr(self.dep_expr) + + def get_visible_if_visibility(self): + """Returns the visibility the menu gets from its 'visible if' + condition. "y" is the menu has no 'visible if' condition.""" + return self.config._eval_expr(self.visible_if_expr) + + def get_items(self, recursive = False): + """Returns a list containing the items (symbols, menus, choice + statements and comments) in in the menu, in the same order that the + items appear within the menu. + + recursive (default: False) -- True if items contained in items within + the menu should be included + recursively (preorder).""" + + if not recursive: + return self.block.get_items() + + res = [] + for item in self.block.get_items(): + res.append(item) + if isinstance(item, Menu): + res.extend(item.get_items(True)) + elif isinstance(item, Choice): + res.extend(item.get_items()) + return res + + def get_symbols(self, recursive = False): + """Returns a list containing the symbols in the menu, in the same order + that they appear within the menu. + + recursive (default: False) -- True if symbols contained in items within + the menu should be included + recursively.""" + + return [item for item in self.get_items(recursive) if isinstance(item, Symbol)] + + def get_title(self): + """Returns the title text of the menu.""" + return self.title + + def get_parent(self): + """Returns the menu or choice statement that contains the menu, or + None if the menu is at the top level. Note that if statements are + treated as syntactic sugar and do not have an explicit class + representation.""" + return self.parent + + def get_referenced_symbols(self, refs_from_enclosing = False): + """See Symbol.get_referenced_symbols().""" + return self.all_referenced_syms if refs_from_enclosing else self.referenced_syms + + def get_location(self): + """Returns the location of the menu as a (filename, linenr) tuple, + where filename is a string and linenr an int.""" + return (self.filename, self.linenr) + + def __str__(self): + """Returns a string containing various information about the menu.""" + depends_on_str = self.config._expr_val_str(self.orig_deps, + "(no dependencies)") + visible_if_str = self.config._expr_val_str(self.visible_if_expr, + "(no dependencies)") + + additional_deps_str = " " + self.config._expr_val_str(self.deps_from_containing, + "(no additional dependencies)") + + return _sep_lines("Menu", + "Title : " + self.title, + "'depends on' dependencies : " + depends_on_str, + "'visible if' dependencies : " + visible_if_str, + "Additional dependencies from enclosing menus and if's:", + additional_deps_str, + "Location: {0}:{1}".format(self.filename, self.linenr)) + + # + # Private methods + # + + def __init__(self): + """Menu constructor -- not intended to be called directly by + kconfiglib clients.""" + + self.config = None + + self.parent = None + self.title = None + self.block = None + self.dep_expr = None + + # Dependency expression without dependencies from enclosing menus and + # if's propagated + self.orig_deps = None + + # Dependencies inherited from containing menus and if's + self.deps_from_containing = None + + # The 'visible if' expression + self.visible_if_expr = None + + # The set of symbols referenced by this menu (see + # get_referenced_symbols()) + self.referenced_syms = set() + + # Like 'referenced_syms', but includes symbols from + # dependencies inherited from enclosing menus and if's + self.all_referenced_syms = None + + self.filename = None + self.linenr = None + + def _make_conf(self): + item_conf = self.block._make_conf() + + if self.config._eval_expr(self.dep_expr) != "n" and \ + self.config._eval_expr(self.visible_if_expr) != "n": + return ["\n#\n# {0}\n#".format(self.title)] + item_conf + else: + return item_conf + +class Choice(Item, _HasVisibility): + + """Represents a choice statement. A choice can be in one of three modes: + "n", "m" and "y". "n" mode is for non-visible choices and optional choices + with no symbol selected; "m" means any number of symbols can be set to "m" + while the rest will be "n" (only tristate choices can be in this mode); and + "y" means one symbol will be "y" while the rest will be "n" (the most + common case). The visibility is an upper bound on the mode, and the mode + changes automatically as values are assigned to symbols within the + choice.""" + + # + # Public interface + # + + def get_selection(self): + """Returns the symbol selected (either by the user or through + defaults), or None if either no symbol is selected or the mode is not + "y".""" + if self.cached_selection is not None: + if self.cached_selection == NO_SELECTION: + return None + return self.cached_selection + + if self.calc_mode() != "y": + return self._cache_ret(None) + + # User choice available? + if self.user_val is not None and \ + self.user_val._calc_visibility() == "y": + return self._cache_ret(self.user_val) + + if self.optional: + return self._cache_ret(None) + + return self._cache_ret(self.get_selection_from_defaults()) + + def get_selection_from_defaults(self): + """Like Choice.get_selection(), but acts as if no symbol has been + selected by the user and no 'optional' flag is in effect.""" + + if self.actual_items == []: + return None + + for (symbol, cond_expr) in self.def_exprs: + if self.config._eval_expr(cond_expr) != "n": + chosen_symbol = symbol + break + else: + chosen_symbol = self.actual_items[0] + + # Is the chosen symbol visible? + if chosen_symbol._calc_visibility() != "n": + return chosen_symbol + else: + # Otherwise, pick the first visible symbol + for sym in self.actual_items: + if sym._calc_visibility() != "n": + return sym + + return None + + def get_user_selection(self): + """If the choice is in "y" mode and has a user-selected + symbol, returns that symbol. Otherwise, returns None.""" + return self.user_val + + def get_name(self): + """For named choices, returns the name. Returns None for unnamed + choices. No named choices appear anywhere in the kernel Kconfig + files as of Linux 2.6.38-rc3.""" + return self.name + + def get_type(self): + """Returns the type of the choice. See Symbol.get_type().""" + return self.type + + def get_items(self): + """Gets all items contained in the choice in the same order as within + the configuration ("items" instead of "symbols" since choices and + comments might appear within choices. This only happens in one place as + of Linux 2.6.38-rc3, in drivers/usb/gadget/Kconfig).""" + return self.block.get_items() + + def get_actual_items(self): + """A quirk (perhaps better described as a bug -- at least for symbols) + of kconfig is that you can put items within a choice that will not be + considered members of the choice insofar as selection is concerned. + This happens for example if one symbol within a choice 'depends on' the + symbol preceding it, or if you put non-symbol items within choices. + + This function gets a list of the "proper" elements of the choice, + excluding such items.""" + return self.actual_items + + def get_parent(self): + """Returns the menu or choice statement that contains the choice, or + None if the choice is at the top level. Note that if statements are + treated as syntactic sugar and do not have an explicit class + representation.""" + return self.parent + + def get_referenced_symbols(self, refs_from_enclosing = False): + """See Symbol.get_referenced_symbols().""" + return self.all_referenced_syms if refs_from_enclosing else self.referenced_syms + + def get_def_locations(self): + """Returns a list of (filename, linenr) tuples, where filename (string) + and linenr (int) represent a location where the choice is defined. For + the vast majority of choices this list will only contain one element, + but its possible for named choices to be defined in multiple + locations.""" + return self.def_locations + + def get_visibility(self): + """Returns the visibility of the choice statement: one of "n", "m" or + "y". This acts as an upper limit on the mode of the choice (though bool + choices can only have the mode "y"). See the class documentation for an + explanation of modes.""" + return self._calc_visibility() + + def calc_mode(self): + """Returns the mode of the choice. See the class documentation for + an explanation of modes.""" + minimum_mode = "n" if self.optional else "m" + + mode = self.user_mode if self.user_mode is not None else minimum_mode + mode = self.config._eval_min(mode, self._calc_visibility()) + + # Promote "m" to "y" for boolean choices + if mode == "m" and self.type == BOOL: + mode = "y" + + return mode + + def is_optional(self): + """Returns True if the symbol has the optional flag set (and so will default + to "n" mode). Otherwise, returns False.""" + return self.optional + + def __str__(self): + """Returns a string containing various information about the choice + statement.""" + return self.config._get_sym_or_choice_str(self) + + # + # Private methods + # + + def __init__(self): + """Choice constructor -- not intended to be called directly by + kconfiglib clients.""" + + _HasVisibility.__init__(self) + + self.config = None + + self.parent = None + self.name = None # Yes, choices can be named + self.type = UNKNOWN + self.def_exprs = [] + self.help = None + self.optional = False + self.block = None + + # The prompts and default values without any dependencies from + # enclosing menus or if's propagated + + self.orig_prompts = [] + self.orig_def_exprs = [] + + # Dependencies inherited from containing menus and if's + self.deps_from_containing = None + + # We need to filter out symbols that appear within the choice block but + # are not considered choice items (see + # Choice._determine_actual_items()) This list holds the "actual" choice + # items. + self.actual_items = [] + + # The set of symbols referenced by this choice (see + # get_referenced_symbols()) + self.referenced_syms = set() + + # Like 'referenced_syms', but includes symbols from + # dependencies inherited from enclosing menus and if's + self.all_referenced_syms = set() + + # See Choice.get_def_locations() + self.def_locations = [] + + self.user_val = None + self.user_mode = None + + self.cached_selection = None + + def _determine_actual_items(self): + """If a symbol's visibility depends on the preceding symbol within a + choice, it is no longer viewed as a choice item (quite possibly a bug, + but some things consciously use it.. ugh. It stems from automatic + submenu creation). In addition, it's possible to have choices and + comments within choices, and those shouldn't be considered as choice + items either. Only drivers/usb/gadget/Kconfig seems to depend on any of + this. This method computes the "actual" items in the choice and sets + the is_choice_item_ flag on them (retrieved via is_choice_item()). + + Don't let this scare you: an earlier version simply checked for a + sequence of symbols where all symbols after the first appeared in the + 'depends on' expression of the first, and that worked fine. The added + complexity is to be future-proof in the event that + drivers/usb/gadget/Kconfig turns even more sinister. It might very well + be overkilling things (especially if that file is refactored ;).""" + + items = self.block.get_items() + + # Items might depend on each other in a tree structure, so we need a + # stack to keep track of the current tentative parent + stack = [] + + for item in items: + if not isinstance(item, Symbol): + stack = [] + continue + + while stack != []: + if item._has_auto_menu_dep_on(stack[-1]): + # The item should not be viewed as a choice item, so don't + # set item.is_choice_item_. + stack.append(item) + break + else: + stack.pop() + else: + item.is_choice_item_ = True + self.actual_items.append(item) + stack.append(item) + + def _cache_ret(self, selection): + # As None is used to indicate the lack of a cached value we can't use + # that to cache the fact that the choice has no selection. Instead, we + # use the symbolic constant NO_SELECTION. + if selection is None: + self.cached_selection = NO_SELECTION + else: + self.cached_selection = selection + + return selection + + def _get_user_mode(self): + return self.user_mode + + def _invalidate(self): + _HasVisibility._invalidate(self) + self.cached_selection = None + + def _reset(self): + self._invalidate() + self.user_val = None + self.user_mode = None + + def _make_conf(self): + return self.block._make_conf() + +class Comment(Item): + + """Represents a comment statement.""" + + # + # Public interface + # + + def get_text(self): + """Returns the text of the comment.""" + return self.text + + def get_parent(self): + """Returns the menu or choice statement that contains the comment, or + None if the comment is at the top level. Note that if statements are + treated as syntactic sugar and do not have an explicit class + representation.""" + return self.parent + + def get_referenced_symbols(self, refs_from_enclosing = False): + """See Symbol.get_referenced_symbols().""" + return self.all_referenced_syms if refs_from_enclosing else self.referenced_syms + + def get_location(self): + """Returns the location of the comment as a (filename, linenr) tuple, + where filename is a string and linenr an int.""" + return (self.filename, self.linenr) + + def __str__(self): + """Returns a string containing various information about the comment.""" + dep_str = self.config._expr_val_str(self.orig_deps, "(no dependencies)") + + additional_deps_str = " " + self.config._expr_val_str(self.deps_from_containing, + "(no additional dependencies)") + + return _sep_lines("Comment", + "Text: " + str(self.text), + "Dependencies: " + dep_str, + "Additional dependencies from enclosing menus and if's:", + additional_deps_str, + "Location: {0}:{1}".format(self.filename, self.linenr)) + + # + # Private methods + # + + def __init__(self): + """Comment constructor -- not intended to be called directly by + kconfiglib clients.""" + + self.config = None + + self.parent = None + self.text = None + self.dep_expr = None + + # Dependency expression without dependencies from enclosing menus and + # if's propagated + self.orig_deps = None + + # Dependencies inherited from containing menus and if's + self.deps_from_containing = None + + # The set of symbols referenced by this comment (see + # get_referenced_symbols()) + self.referenced_syms = set() + + # Like 'referenced_syms', but includes symbols from + # dependencies inherited from enclosing menus and if's + self.all_referenced_syms = None + + self.filename = None + self.linenr = None + + def _make_conf(self): + if self.config._eval_expr(self.dep_expr) != "n": + return ["\n#\n# {0}\n#".format(self.text)] + else: + return [] + +class _Feed: + + """Class for working with sequences in a stream-like fashion; handy for tokens.""" + + def __init__(self, items): + self.items = items + self.length = len(self.items) + self.i = 0 + + def get_next(self): + if self.i >= self.length: + return None + + item = self.items[self.i] + self.i += 1 + return item + + def peek_next(self): + return None if self.i >= self.length else self.items[self.i] + + def go_to_start(self): + self.i = 0 + + def __getitem__(self, index): + return self.items[index] + + def __len__(self): + return len(self.items) + + def is_empty(self): + return self.items == [] + + def is_at_end(self): + return self.i >= self.length + + def check(self, token): + """Check if the next token is 'token'. If so, remove it from the token + feed and return True. Otherwise, leave it in and return False.""" + if self.i >= self.length: + return None + + if self.items[self.i] == token: + self.i += 1 + return True + + return False + + def remove_while(self, pred): + while self.i < self.length and pred(self.items[self.i]): + self.i += 1 + + def go_back(self): + if self.i <= 0: + _internal_error("Attempt to move back in Feed while already at the beginning.") + self.i -= 1 + +class _FileFeed(_Feed): + + """Feed subclass that keeps track of the current filename and line + number.""" + + def __init__(self, lines, filename): + self.filename = _clean_up_path(filename) + _Feed.__init__(self, lines) + + def get_filename(self): + return self.filename + + def get_linenr(self): + return self.i + +# +# Misc. public global utility functions +# + +def tri_less(v1, v2): + """Returns True if the tristate v1 is less than the tristate v2, where "n", + "m" and "y" are ordered from lowest to highest. Otherwise, returns + False.""" + return values[v1] < values[v2] + +def tri_less_eq(v1, v2): + """Returns True if the tristate v1 is less than or equal to the tristate + v2, where "n", "m" and "y" are ordered from lowest to highest. Otherwise, + returns False.""" + return values[v1] <= values[v2] + +def tri_greater(v1, v2): + """Returns True if the tristate v1 is greater than the tristate v2, where + "n", "m" and "y" are ordered from lowest to highest. Otherwise, returns + False.""" + return values[v1] > values[v2] + +def tri_greater_eq(v1, v2): + """Returns True if the tristate v1 is greater than or equal to the tristate + v2, where "n", "m" and "y" are ordered from lowest to highest. Otherwise, + returns False.""" + return values[v1] >= values[v2] + +# +# Helper functions, mostly related to text processing +# + +def _strip_quotes(s, line, filename, linenr): + """Removes any quotes surrounding 's' if it has them; otherwise returns 's' + unmodified.""" + s = s.strip() + if s[0] == '"' or s[0] == "'": + if len(s) < 2 or s[-1] != s[0]: + _parse_error(line, + "malformed string literal", + filename, + linenr) + return s[1:-1] + else: + return s + +def _indentation(line): + """Returns the indentation of the line, treating tab stops as being spaced + 8 characters apart.""" + if line.isspace(): + _internal_error("Attempt to take indentation of blank line.") + indent = 0 + for c in line: + if c == " ": + indent += 1 + elif c == "\t": + # Go to the next tab stop + indent = (indent + 8) & ~7 + else: + return indent + +def _deindent(line, indent): + """Deindent 'line' by 'indent' spaces.""" + line = line.expandtabs() + if len(line) <= indent: + return line + return line[indent:] + +def _is_hex(s): + return _is_base_n(s, 16) + +def _is_dec(s): + return _is_base_n(s, 10) + +def _is_base_n(s, n): + try: + int(s, n) + return True + except ValueError: + return False + +def _sep_lines(*args): + """Returns a string comprised of all arguments, with newlines inserted + between them.""" + return "\n".join(args) + +def _comment(s): + """Returns a new string with "# " inserted before each line in 's'.""" + return "\n".join(["# " + line for line in s.splitlines()]) + +def _get_lines(filename): + """Returns a list of lines from 'filename', joining any line ending in \\ + with the following line.""" + with open(filename, "r") as f: + lines = [] + accum = "" + while True: + line = f.readline() + + if line == "": + return lines + + if line.endswith("\\\n"): + accum += line[:-2] + else: + accum += line + lines.append(accum) + accum = "" + +def _strip_trailing_slash(path): + """Removes any trailing slash from 'path'.""" + return path[:-1] if path.endswith("/") else path + +def _clean_up_path(path): + """Strips any initial "./" and trailing slash from 'path'.""" + if path.startswith("./"): + path = path[2:] + return _strip_trailing_slash(path) + +# +# Error handling +# + +class Kconfig_Syntax_Error(Exception): + """Exception raised for syntax errors.""" + pass + +class Internal_Error(Exception): + """Exception raised for internal errors.""" + pass + +def _tokenization_error(s, index, filename, linenr): + if filename is not None: + assert linenr is not None + sys.stderr.write("{0}:{1}:\n".format(filename, linenr)) + + if s.endswith("\n"): + s = s[:-1] + + # Calculate the visual offset corresponding to index 'index' in 's' + # assuming tabstops are spaced 8 characters apart + vis_index = 0 + for c in s[:index]: + if c == "\t": + vis_index = (vis_index + 8) & ~7 + else: + vis_index += 1 + + # Don't output actual tabs to be independent of how the terminal renders + # them + s = s.expandtabs() + + raise Kconfig_Syntax_Error, ( + _sep_lines("Error during tokenization at location indicated by caret.\n", + s, + " " * vis_index + "^\n")) + +def _parse_error(s, msg, filename, linenr): + error_str = "" + + if filename is not None: + assert linenr is not None + error_str += "{0}:{1}: ".format(filename, linenr) + + if s.endswith("\n"): + s = s[:-1] + + error_str += 'Error while parsing "{0}"'.format(s) + + if msg is None: + error_str += "." + else: + error_str += ": " + msg + + raise Kconfig_Syntax_Error, error_str + +def _internal_error(msg): + msg += "\nSorry! You may want to send an email to kconfiglib@gmail.com " \ + "to tell me about this. Include the message above and the stack " \ + "trace and describe what you were doing." + + raise Internal_Error, msg + +if use_psyco: + import psyco + + Config._tokenize = psyco.proxy(Config._tokenize) + Config._eval_expr = psyco.proxy(Config._eval_expr) + + _indentation = psyco.proxy(_indentation) + _get_lines = psyco.proxy(_get_lines) diff --git a/scripts/kconfig/kconfigtest.py b/scripts/kconfig/kconfigtest.py new file mode 100644 index 0000000..9d27dca --- /dev/null +++ b/scripts/kconfig/kconfigtest.py @@ -0,0 +1,396 @@ +# This is a test suite for kconfiglib, primarily testing compatibility with the +# C kconfig implementation by comparing outputs. It should be run from the +# top-level kernel directory with +# +# $ PYTHONPATH=scripts/kconfig python scripts/kconfig/kconfigtest.py +# +# Note that running these could take a long time: running all tests on a Core +# i7@2.67 GHz takes ~2 hours. The tests have been arranged in order of time +# needed. +# +# All tests should pass. Report regressions to kconfiglib@gmail.com + +import kconfiglib +import os +import re +import subprocess +import sys + +# Assume that the value of KERNELVERSION does not affect the configuration +# (true as of Linux 2.6.38-rc3). Here we could fetch the correct version +# instead. +os.environ["KERNELVERSION"] = "2" + +# Number of arch/defconfig pairs tested so far +nconfigs = 0 + +def run_tests(): + # The set of tests that want to run for all architectures in the kernel + # tree -- currently, all tests. The boolean flag indicates whether .config + # (generated by the C implementation) should be compared to ._config + # (generated by us) after each invocation. + all_arch_tests = [(test_all_no, True), + (test_config_absent, True), + (test_all_yes, True), + (test_call_all, False), + # Needs to report success/failure for each arch/defconfig + # combo, hence False. + (test_defconfig, False)] + + print "Loading Config instances for all architectures..." + arch_configs = get_arch_configs() + + for (test_fn, compare_configs) in all_arch_tests: + print "Resetting all architecture Config instances prior to next test..." + for arch in arch_configs: + arch.reset() + + # The test description is taken from the docstring of the corresponding + # function + print test_fn.__doc__ + + for conf in arch_configs: + rm_configs() + + # This should be set correctly for any 'make *config' commands the + # test might run + os.environ["ARCH"] = conf.get_arch() + + test_fn(conf) + + if compare_configs: + sys.stdout.write(" {0:<14}".format(conf.get_arch())) + + if equal_confs(): + print "OK" + else: + print "FAIL" + fail() + + print "" + + if all_ok(): + print "All OK" + print nconfigs, "arch/defconfig pairs tested" + else: + print "Some tests failed" + +def get_arch_configs(): + """Returns a list with Config instances corresponding to all arch Kconfigs.""" + + def add_arch(ARCH, res): + os.environ["SRCARCH"] = archdir + os.environ["ARCH"] = ARCH + res.append(kconfiglib.Config()) + + res = [] + + # Nothing looks at this as of Linux 2.6.38-rc3 + os.environ["KERNELVERSION"] = "2" + + for archdir in os.listdir("arch"): + if archdir == "h8300": + # Broken Kconfig as of Linux 2.6.38-rc3 + continue + + if os.path.exists(os.path.join("arch", archdir, "Kconfig")): + if archdir == "x86": + add_arch("i386", res) + add_arch("x86_64", res) + elif archdir == "sparc": + add_arch("sparc32", res) + add_arch("sparc64", res) + elif archdir == "sh": + add_arch("sh64", res) + else: + # For most architectures, ARCH = SRCARCH + add_arch(archdir, res) + + # Don't want subsequent 'make *config' commands in tests to see this + del os.environ["ARCH"] + del os.environ["SRCARCH"] + + return res + +def test_all_no(conf): + """Test if our allnoconfig implementation generates the same .config as 'make allnoconfig', for all architectures""" + + while True: + done = True + + for sym in conf: + # Choices take care of themselves for allnoconf, so we only need to + # worry about non-choice symbols + if not sym.is_choice_item(): + lower_bound = sym.get_lower_bound() + + # If we can assign a lower value to the symbol (where "n", "m" and + # "y" are ordered from lowest to highest), then do so. + # lower_bound() returns None for symbols whose values cannot + # (currently) be changed, as well as for non-bool, non-tristate + # symbols. + if lower_bound is not None and \ + kconfiglib.tri_less(lower_bound, sym.calc_value()): + + sym.set_value(lower_bound) + + # We just changed the value of some symbol. As this may effect + # other symbols, we need to keep looping. + done = False + + if done: + break + + conf.write_config("._config") + + shell("make allnoconfig") + +def test_all_yes(conf): + """Test if our allyesconfig implementation generates the same .config as 'make allyesconfig', for all architectures""" + + # Get a list of all symbols that are not choice items + non_choice_syms = [sym for sym in conf.get_symbols() if + not sym.is_choice_item()] + + while True: + done = True + + # Handle symbols outside of choices + + for sym in non_choice_syms: + upper_bound = sym.get_upper_bound() + + # See corresponding comment for allnoconf implementation + if upper_bound is not None and \ + kconfiglib.tri_less(sym.calc_value(), upper_bound): + sym.set_value(upper_bound) + done = False + + # Handle symbols within choices + + for choice in conf.get_choices(): + + # Handle choices whose visibility allow them to be in "y" mode + + if choice.get_visibility() == "y": + selection = choice.get_selection_from_defaults() + if selection is not None and \ + selection is not choice.get_user_selection(): + selection.set_value("y") + done = False + + # Handle choices whose visibility only allow them to be in "m" mode + + elif choice.get_visibility() == "m": + for sym in choice.get_items(): + if sym.calc_value() != "m" and \ + sym.get_upper_bound() != "n": + sym.set_value("m") + done = False + + + if done: + break + + conf.write_config("._config") + + shell("make allyesconfig") + +def test_call_all(conf): + """Call all public methods on all symbols, menus, choices and comments (nearly all public methods: some are hard to test like this, but are exercised by other tests) for all architectures to make sure we never crash or hang""" + print " For {0}...".format(conf.get_arch()) + + conf.get_arch() + conf.get_defconfig_filename() + conf.get_top_level_items() + conf.eval("y && ARCH") + + # Syntax error + caught_exception = False + try: + conf.eval("y & y") + except kconfiglib.Kconfig_Syntax_Error: + caught_exception = True + + if not caught_exception: + print "Fail: no exception generated for expression with syntax error" + fail() + + conf.get_config_header() + conf.get_base_dir() + conf.reset() + conf.get_symbols(False) + conf.get_mainmenu_text() + + for s in conf.get_symbols(): + s.reset() + s.calc_value() + s.calc_default_value() + s.get_user_value() + s.get_name() + s.get_upper_bound() + s.get_lower_bound() + s.get_assignable_values() + s.get_type() + s.get_visibility() + s.get_parent() + s.get_sibling_symbols() + s.get_sibling_items() + s.get_referenced_symbols() + s.get_referenced_symbols(True) + s.get_selected_symbols() + s.get_help() + s.get_config() + s.get_def_locations() + s.get_ref_locations() + s.is_modifiable() + s.is_defined() + s.is_special() + s.is_from_environment() + s.has_ranges() + s.is_choice_item() + s.is_choice_selection() + s.__str__() + + for c in conf.get_choices(): + if c.get_name() is not None: + print "LOOOOOOOOL", c.get_name() + c.get_selection() + c.get_selection_from_defaults() + c.get_user_selection() + c.get_type() + c.get_name() + c.get_items() + c.get_actual_items() + c.get_parent() + c.get_referenced_symbols() + c.get_referenced_symbols(True) + c.get_def_locations() + c.get_visibility() + c.calc_mode() + c.is_optional() + c.__str__() + + for m in conf.get_menus(): + m.get_items() + m.get_symbols(False) + m.get_symbols(True) + m.get_depends_on_visibility() + m.get_visible_if_visibility() + m.get_title() + m.get_parent() + m.get_referenced_symbols() + m.get_referenced_symbols(True) + m.get_location() + m.__str__() + + for c in conf.get_comments(): + c.get_text() + c.get_parent() + c.get_referenced_symbols() + c.get_referenced_symbols(True) + c.get_location() + c.__str__() + +def test_config_absent(conf): + """Test if kconfiglib generates the same configuration as 'conf' without a .config, for each architecture""" + conf.write_config("._config") + + # Use an empty .config + shell("> .config") + shell("make kconfiglibtestconfig") + +def test_defconfig(conf): + """Test if kconfiglib generates the same .config as conf for each architecture/defconfig pair (this takes two hours on a Core i7@2.67 GHz system)""" + # Collect defconfigs. This could be done once instead, but it's a speedy + # operation comparatively. + + global nconfigs + + defconfigs = [] + + for arch in os.listdir("arch"): + arch_dir = os.path.join("arch", arch) + + # Some arches have a "defconfig" in their + # arch directory. + defconfig = os.path.join(arch_dir, "defconfig") + if os.path.exists(defconfig): + defconfigs.append(defconfig) + + defconfigs_dir = os.path.join(arch_dir, "configs") + if os.path.isdir(defconfigs_dir): + for c in os.listdir(defconfigs_dir): + defconfig = os.path.join(defconfigs_dir, c) + if os.path.isfile(defconfig): + defconfigs.append(defconfig) + + # Test architecture for each defconfig + + for defconfig in defconfigs: + rm_configs() + + nconfigs += 1 + + conf.load_config(defconfig) + conf.write_config("._config") + shell("cp {0} .config".format(defconfig)) + shell("make kconfiglibtestconfig") + + sys.stdout.write(" {0:<14}with {1:<60} ".format(conf.get_arch(), defconfig)) + + if equal_confs(): + print "OK" + else: + print "FAIL" + fail() + +# +# Helper functions +# + +def shell(cmd): + subprocess.Popen(cmd, + shell = True, + stdout = subprocess.PIPE, + stderr = subprocess.PIPE).wait() + +def rm_configs(): + """Delete any old ".config" (generated by the C implementation) and + "._config" (generated by us), if present.""" + def rm_if_exists(f): + if os.path.exists(f): + os.remove(f) + + rm_if_exists(".config") + rm_if_exists("._config") + +def equal_confs(): + with open(".config") as menu_conf: + l1 = menu_conf.readlines() + + with open("._config") as my_conf: + l2 = my_conf.readlines() + + # Skip the header generated by 'conf' + unset_re = r"# CONFIG_(\w+) is not set" + i = 0 + for line in l1: + if not line.startswith("#") or \ + re.match(unset_re, line): + break + i += 1 + + return (l1[i:] == l2) + +_all_ok = True + +def fail(): + global _all_ok + _all_ok = False + +def all_ok(): + return _all_ok + +if __name__ == "__main__": + run_tests() -- 1.7.0.4 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/