Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S940698AbdDTDlZ (ORCPT ); Wed, 19 Apr 2017 23:41:25 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:39557 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S940643AbdDTDlW (ORCPT ); Wed, 19 Apr 2017 23:41:22 -0400 From: Michael Ellerman To: "Paul E. McKenney" , linux-kernel@vger.kernel.org, "paulus@samba.org" , kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, "linuxppc-dev@lists.ozlabs.org" , pbonzini@redhat.com Cc: mingo@kernel.org, jiangshanlai@gmail.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@efficios.com, josh@joshtriplett.org, tglx@linutronix.de, peterz@infradead.org, rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com, fweisbec@gmail.com, oleg@redhat.com, bobby.prani@gmail.com, "Paul E. McKenney" Subject: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling) In-Reply-To: <1492018825-25634-40-git-send-email-paulmck@linux.vnet.ibm.com> References: <20170412174003.GA23207@linux.vnet.ibm.com> <1492018825-25634-40-git-send-email-paulmck@linux.vnet.ibm.com> User-Agent: Notmuch/0.21 (https://notmuchmail.org) Date: Thu, 20 Apr 2017 13:40:13 +1000 MIME-Version: 1.0 Content-Type: text/plain X-TM-AS-MML: disable x-cbid: 17042003-1617-0000-0000-000001C18578 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17042003-1618-0000-0000-00004801389E Message-Id: <871ssn692a.fsf@concordia.ellerman.id.au> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-04-19_16:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1704200028 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2695 Lines: 73 "Paul E. McKenney" writes: > diff --git a/include/linux/srcutree.h b/include/linux/srcutree.h > index f2b3bd6c6bc2..0400e211aa44 100644 > --- a/include/linux/srcutree.h > +++ b/include/linux/srcutree.h > @@ -24,25 +24,75 @@ ... > +/* > + * Per-SRCU-domain structure, similar in function to rcu_state. > + */ > struct srcu_struct { > - unsigned long completed; > - unsigned long srcu_gp_seq; > - atomic_t srcu_exp_cnt; > - struct srcu_array __percpu *per_cpu_ref; > - spinlock_t queue_lock; /* protect ->srcu_cblist */ > - struct rcu_segcblist srcu_cblist; > + struct srcu_node node[NUM_RCU_NODES]; /* Combining tree. */ > + struct srcu_node *level[RCU_NUM_LVLS + 1]; > + /* First node at each level. */ > + struct mutex srcu_cb_mutex; /* Serialize CB preparation. */ > + spinlock_t gp_lock; /* protect ->srcu_cblist */ > + struct mutex srcu_gp_mutex; /* Serialize GP work. */ > + unsigned int srcu_idx; /* Current rdr array element. */ > + unsigned long srcu_gp_seq; /* Grace-period seq #. */ > + unsigned long srcu_gp_seq_needed; /* Latest gp_seq needed. */ > + atomic_t srcu_exp_cnt; /* # ongoing expedited GPs. */ > + struct srcu_data __percpu *sda; /* Per-CPU srcu_data array. */ > + unsigned long srcu_barrier_seq; /* srcu_barrier seq #. */ > + struct mutex srcu_barrier_mutex; /* Serialize barrier ops. */ > + struct completion srcu_barrier_completion; > + /* Awaken barrier rq at end. */ > + atomic_t srcu_barrier_cpu_cnt; /* # CPUs not yet posting a */ > + /* callback for the barrier */ > + /* operation. */ This change seems to have had the non-obvious effect of breaking the powerpc KVM build. Because struct kvm contains two srcu_structs which are located before the kvm_arch struct, the increase in size of srcu_struct has caused the offset from the start of struct kvm to kvm_arch to be too big for some of our asm. struct kvm { spinlock_t mmu_lock; struct mutex slots_lock; struct mm_struct *mm; /* userspace tied to this vm */ struct kvm_memslots *memslots[KVM_ADDRESS_SPACE_NUM]; struct srcu_struct srcu; struct srcu_struct irq_srcu; ... struct kvm_arch arch; Example error: arch/powerpc/kvm/book3s_hv_rmhandlers.S:617: Error: operand out of range (0x000000000000b328 is not between 0xffffffffffff8000 and 0x0000000000007fff) Where line 617 is: lwz r7,KVM_LPID(r9) And the KVM_LPID constant comes from asm-offsets.s. The diff of old vs new is: -->KVM_LPID 17752 offsetof(struct kvm, arch.lpid) # +->KVM_LPID 45864 offsetof(struct kvm, arch.lpid) # We can probably just fix it by changing the asm to keep the address of kvm_arch in a register, and then offset from that. cheers