Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751626AbWKBMzH (ORCPT ); Thu, 2 Nov 2006 07:55:07 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1752129AbWKBMzH (ORCPT ); Thu, 2 Nov 2006 07:55:07 -0500 Received: from fgwmail6.fujitsu.co.jp ([192.51.44.36]:50826 "EHLO fgwmail6.fujitsu.co.jp") by vger.kernel.org with ESMTP id S1751626AbWKBMzE (ORCPT ); Thu, 2 Nov 2006 07:55:04 -0500 Date: Thu, 02 Nov 2006 21:55:28 +0900 Message-ID: <87lkmuuhxb.wl%takeuchi_satoru@jp.fujitsu.com> From: Satoru Takeuchi To: Linux Kernel Cc: ashok.raj@intel.com Subject: [PATHC] doc: fixing cpu-hotplug document User-Agent: Wanderlust/2.11.30 (Wonderwall) SEMI/1.14.6 (Maruoka) FLIM/1.14.6 (Marutamachi) APEL/10.6 Emacs/21.4 (i386-pc-linux-gnu) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 12549 Lines: 282 Fixing cpu-hotplug document as follows: - `cpucontrol' mutex no longer exists and now `cpu_bitmask_lock' is used instead. - unifying the notation of CPU to `CPU' in the document - decolating captions to improve readability - fixing some minor typos Signed-off-by: Satoru Takeuchi Cc: Ashok Raj Index: linux-2.6.19-rc4/Documentation/cpu-hotplug.txt =================================================================== --- linux-2.6.19-rc4.orig/Documentation/cpu-hotplug.txt 2006-10-31 12:37:36.000000000 +0900 +++ linux-2.6.19-rc4/Documentation/cpu-hotplug.txt 2006-11-02 21:22:15.000000000 +0900 @@ -1,8 +1,9 @@ CPU hotplug Support in Linux(tm) Kernel + ======================================= Maintainers: CPU Hotplug Core: - Rusty Russell + Rusty Russell Srivatsa Vaddagiri i386: Zwane Mwaikambo @@ -18,7 +19,9 @@ Authors: Ashok Raj Lots of feedback: Nathan Lynch , Joel Schopp + Introduction +============ Modern advances in system architectures have introduced advanced error reporting and correction capabilities in processors. CPU architectures permit @@ -32,38 +35,39 @@ provisioning reasons, or for RAS purpose system execution path. Hence the need for CPU hotplug support in the Linux kernel. -A more novel use of CPU-hotplug support is its use today in suspend +A more novel use of CPU hotplug support is its use today in suspend resume support for SMP. Dual-core and HT support makes even a laptop run SMP kernels which didn't support these methods. SMP support for suspend/resume is a work in progress. + General Stuff about CPU Hotplug --------------------------------- +=============================== Command Line Switches --------------------- -maxcpus=n Restrict boot time cpus to n. Say if you have 4 cpus, using +maxcpus=n Restrict boot time CPUs to n. Say if you have 4 CPUs, using maxcpus=2 will only boot 2. You can choose to bring the - other cpus later online, read FAQ's for more info. + other CPUs later online, read FAQ's for more info. -additional_cpus=n (*) Use this to limit hotpluggable cpus. This option sets +additional_cpus=n (*) Use this to limit hotpluggable CPUs. This option sets cpu_possible_map = cpu_present_map + additional_cpus (*) Option valid only for following architectures - x86_64, ia64, s390 ia64 and x86_64 use the number of disabled local apics in ACPI tables MADT -to determine the number of potentially hot-pluggable cpus. The implementation -should only rely on this to count the #of cpus, but *MUST* not rely on the +to determine the number of potentially hot-pluggable CPUs. The implementation +should only rely on this to count the # of CPUs, but *MUST* not rely on the apicid values in those tables for disabled apics. In the event BIOS doesnt -mark such hot-pluggable cpus as disabled entries, one could use this -parameter "additional_cpus=x" to represent those cpus in the cpu_possible_map. +mark such hot-pluggable CPUs as disabled entries, one could use this +parameter "additional_cpus=x" to represent those CPUs in the cpu_possible_map. -s390 uses the number of cpus it detects at IPL time to also the number of bits -in cpu_possible_map. If it is desired to add additional cpus at a later time +s390 uses the number of CPUs it detects at IPL time to also the number of bits +in cpu_possible_map. If it is desired to add additional CPUs at a later time the number should be specified using this option or the possible_cpus option. -possible_cpus=n [s390 only] use this to set hotpluggable cpus. +possible_cpus=n [s390 only] use this to set hotpluggable CPUs. This option sets possible_cpus bits in cpu_possible_map. Thus keeping the numbers of bits set constant even if the machine gets rebooted. @@ -83,8 +87,8 @@ upfront can save some boot time memory. in x86_64 case to keep this under check. cpu_online_map: Bitmap of all CPUs currently online. Its set in __cpu_up() -after a cpu is available for kernel scheduling and ready to receive -interrupts from devices. Its cleared when a cpu is brought down using +after a CPU is available for kernel scheduling and ready to receive +interrupts from devices. Its cleared when a CPU is brought down using __cpu_disable(), before which all OS services including interrupts are migrated to another target CPU. @@ -95,8 +99,8 @@ from the map depending on the event is h no locking rules as of now. Typical usage is to init topology during boot, at which time hotplug is disabled. -You really dont need to manipulate any of the system cpu maps. They should -be read-only for most use. When setting up per-cpu resources almost always use +You really dont need to manipulate any of the system CPU maps. They should +be read-only for most use. When setting up per-CPU resources almost always use cpu_possible_map/for_each_possible_cpu() to iterate. Never use anything other than cpumask_t to represent bitmap of CPUs. @@ -106,24 +110,26 @@ Never use anything other than cpumask_t for_each_possible_cpu - Iterate over cpu_possible_map for_each_online_cpu - Iterate over cpu_online_map for_each_present_cpu - Iterate over cpu_present_map - for_each_cpu_mask(x,mask) - Iterate over some random collection of cpu mask. + for_each_cpu_mask(x,mask) - Iterate over some random collection of CPU mask. #include lock_cpu_hotplug() and unlock_cpu_hotplug(): -The above calls are used to inhibit cpu hotplug operations. While holding the -cpucontrol mutex, cpu_online_map will not change. If you merely need to avoid -cpus going away, you could also use preempt_disable() and preempt_enable() +The above calls are used to inhibit CPU hotplug operations. While holding the +cpu_bitmask_lock mutex, cpu_online_map will not change. If you merely need to avoid +CPUs going away, you could also use preempt_disable() and preempt_enable() for those sections. Just remember the critical section cannot call any function that can sleep or schedule this process away. The preempt_disable() -will work as long as stop_machine_run() is used to take a cpu down. +will work as long as stop_machine_run() is used to take a CPU down. + CPU Hotplug - Frequently Asked Questions. +========================================= Q: How to enable my kernel to support CPU hotplug? -A: When doing make defconfig, Enable CPU hotplug support +A: When doing make defconfig, enable CPU hotplug support - "Processor type and Features" -> Support for Hotpluggable CPUs + "Processor type and Features" -> Support for hot-pluggable CPUs Make sure that you have CONFIG_HOTPLUG, and CONFIG_SMP turned on as well. @@ -147,10 +153,10 @@ an entry as shown below in the output. If this is not mounted, do the following. - #mkdir /sysfs + #mkdir /sysfs #mount -t sysfs sys /sys -Now you should see entries for all present cpu, the following is an example +Now you should see entries for all present CPUs, the following is an example in a 8-way system. #pwd @@ -171,7 +177,7 @@ in a 8-way system. Under each directory you would find an "online" file which is the control file to logically online/offline a processor. -Q: Does hot-add/hot-remove refer to physical add/remove of cpus? +Q: Does hot-add/hot-remove refer to physical add/remove of CPUs? A: The usage of hot-add/remove may not be very consistently used in the code. CONFIG_HOTPLUG_CPU enables logical online/offline capability in the kernel. To support physical addition/removal, one would need some BIOS hooks and @@ -188,21 +194,21 @@ Once the logical offline is successful, #cat /proc/interrupts You should now not see the CPU that you removed. Also online file will report -the state as 0 when a cpu if offline and 1 when its online. +the state as 0 when a CPU is offline and 1 when its online. - #To display the current cpu state. + #To display the current CPU state. #cat /sys/devices/system/cpu/cpuX/online Q: Why cant i remove CPU0 on some systems? A: Some architectures may have some special dependency on a certain CPU. For e.g in IA64 platforms we have ability to sent platform interrupts to the -OS. a.k.a Corrected Platform Error Interrupts (CPEI). In current ACPI +OS, a.k.a Corrected Platform Error Interrupts (CPEI). In current ACPI specifications, we didn't have a way to change the target CPU. Hence if the current ACPI version doesn't support such re-direction, we disable that CPU by making it not-removable. -In such cases you will also notice that the online file is missing under cpu0. +In such cases you will also notice that the online file is missing under CPU0. Q: How do i find out if a particular CPU is not removable? A: Depending on the implementation, some architectures may show this by the @@ -218,9 +224,9 @@ A: The following happen, listed in no pa - A notification is sent to in-kernel registered modules by sending an event CPU_DOWN_PREPARE -- All process is migrated away from this outgoing CPU to a new CPU +- All processes are migrated away from this outgoing CPU to a new CPU - All interrupts targeted to this CPU is migrated to a new CPU -- timers/bottom half/task lets are also migrated to a new CPU +- timers/bottom halves/tasklets are also migrated to a new CPU - Once all services are migrated, kernel calls an arch specific routine __cpu_disable() to perform arch specific cleanup. - Once this is successful, an event for successful cleanup is sent by an event @@ -253,7 +259,7 @@ A: This is what you would need in your k static struct notifier_block __cpuinitdata foobar_cpu_notifer = { - .notifier_call = foobar_cpu_callback, + .notifier_call = foobar_cpu_callback, }; You need to call register_cpu_notifier() from your init function. @@ -284,7 +290,7 @@ A: Yes, CPU notifiers are called only wh foobar_cpu_callback(&foobar_cpu_notifier, CPU_ONLINE, i); } -Q: If i would like to develop cpu hotplug support for a new architecture, +Q: If i would like to develop CPU hotplug support for a new architecture, what do i need at a minimum? A: The following are what is required for CPU hotplug infrastructure to work correctly. @@ -303,9 +309,9 @@ A: The following are what is required fo per_cpu state to be set, to ensure the processor dead routine is called to be sure positively. -Q: I need to ensure that a particular cpu is not removed when there is some - work specific to this cpu is in progress. -A: First switch the current thread context to preferred cpu +Q: I need to ensure that a particular CPU is not removed when there is some + work specific to this CPU is in progress. +A: First switch the current thread context to preferred CPU. int my_func_on_cpu(int cpu) { @@ -334,11 +340,11 @@ A: First switch the current thread conte * Do work : But cant sleep, since get_cpu() disables preempt */ } - ret: - put_cpu(); - set_cpus_allowed(current, saved_mask); - return err; - } + ret: + put_cpu(); + set_cpus_allowed(current, saved_mask); + return err; + } Q: How do we determine how many CPUs are available for hotplug. @@ -348,13 +354,15 @@ A: There is no clear spec defined way fr CPUs in a system with disabled status. Andi implemented some simple heuristics that count the number of disabled - CPUs in MADT as hotpluggable CPUS. In the case there are no disabled CPUS + CPUs in MADT as hotpluggable CPUS. In the case there are no disabled CPUs we assume 1/2 the number of CPUs currently present can be hotplugged. Caveat: Today's ACPI MADT can only provide 256 entries since the apicid field in MADT is only 8 bits. + User Space Notification +======================= Hotplug support for devices is common in Linux today. Its being used today to support automatic configuration of network, usb and pci devices. A hotplug @@ -381,6 +389,6 @@ scripts. ;; *) debug_mesg CPU $ACTION event not supported - exit 1 - ;; + exit 1 + ;; esac - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/