Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758162Ab3GZONi (ORCPT ); Fri, 26 Jul 2013 10:13:38 -0400 Received: from mx4-phx2.redhat.com ([209.132.183.25]:47434 "EHLO mx4-phx2.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755628Ab3GZONh (ORCPT ); Fri, 26 Jul 2013 10:13:37 -0400 Date: Fri, 26 Jul 2013 10:12:26 -0400 (EDT) From: John Kacur To: scameron@beardog.cce.hp.com Cc: James Bottomley , linux-kernel@vger.kernel.org, "Luis Claudio R. Goncalves" , Clark Williams , linux-rt-users@vger.kernel.org, linux-scsi@vger.kernel.org Message-ID: <94298596.6633840.1374847946705.JavaMail.root@redhat.com> In-Reply-To: <20130726133644.GM24664@beardog.cce.hp.com> References: <1374796568.1952.43.camel@dabdike> <766237766.6462824.1374834482391.JavaMail.root@redhat.com> <20130726133644.GM24664@beardog.cce.hp.com> Subject: Re: hpsa - BUG: using smp_processor_id() in preemptible [00000000 00000000] code: kworker/u:0/6 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.3_GA_5664 (ZimbraWebClient - FF21 (Linux)/8.0.3_GA_5664) Thread-Topic: hpsa - BUG: using smp_processor_id() in preemptible [00000000 00000000] code: kworker/u:0/6 Thread-Index: xmheeLLc9gVnT8LluF7e+1oSQIHgZQ== Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 5485 Lines: 126 ----- Original Message ----- > On Fri, Jul 26, 2013 at 06:28:02AM -0400, John Kacur wrote: > > > > > > ----- Original Message ----- > > > [Adding missing cc to linux-scsi] > > > On Thu, 2013-07-25 at 23:33 +0200, John Kacur wrote: > > > > Hi > > > > > > > > We're seeing this on a 3.6 kernel with the real-time patch applied, but > > > > it > > > > looks like it is relevant with the real-time patch in the latest kernel > > > > This should read, "it looks like it is relevant WITHOUT the real-time patch > > in the latest kernel". > > > > > > > > too. > > > > > > > > [ 49.688847] hpsa 0000:03:00.0: hpsa0: <0x323a> at IRQ 67 using DAC > > > > [ 49.749928] scsi0 : hpsa > > > > [ 49.784437] BUG: using smp_processor_id() in preemptible [00000000 > > > > 00000000] code: kworker/u:0/6 > > > > [ 49.784465] caller is enqueue_cmd_and_start_io+0x5a/0x100 [hpsa] > > > > [ 49.784468] Pid: 6, comm: kworker/u:0 Not tainted > > > > 3.6.11.5-rt37.52.el6rt.x86_64.debug #1 > > > > [ 49.784471] Call Trace: > > > > [ 49.784512] [] debug_smp_processor_id+0x123/0x150 > > > > [ 49.784520] [] > > > > enqueue_cmd_and_start_io+0x5a/0x100 > > > > [hpsa] > > > > [ 49.784529] [] > > > > hpsa_scsi_do_simple_cmd_core+0xeb/0x110 [hpsa] > > > > [ 49.784537] [] ? > > > > swiotlb_dma_mapping_error+0x18/0x30 > > > > [ 49.784544] [] ? > > > > swiotlb_dma_mapping_error+0x18/0x30 > > > > [ 49.784553] [] > > > > hpsa_scsi_do_simple_cmd_with_retry+0x91/0x280 [hpsa] > > > > [ 49.784562] [] > > > > hpsa_scsi_do_report_luns.clone.2+0xd8/0x130 [hpsa] > > > > [ 49.784571] [] > > > > hpsa_gather_lun_info.clone.3+0x3a/0x1a0 [hpsa] > > > > [ 49.784580] [] > > > > hpsa_update_scsi_devices+0x11f/0x4f0 > > > > [hpsa] > > > > [ 49.784592] [] ? sub_preempt_count+0xa9/0xe0 > > > > [ 49.784601] [] hpsa_scan_start+0xfd/0x150 [hpsa] > > > > [ 49.784613] [] ? > > > > rt_spin_lock_slowunlock+0x78/0x90 > > > > [ 49.784626] [] do_scsi_scan_host+0x37/0xa0 > > > > [ 49.784632] [] do_scan_async+0x1a/0x30 > > > > [ 49.784643] [] async_run_entry_fn+0x9b/0x1d0 > > > > [ 49.784655] [] process_one_work+0x1f2/0x620 > > > > [ 49.784661] [] ? process_one_work+0x180/0x620 > > > > [ 49.784668] [] ? worker_thread+0x5e/0x3a0 > > > > [ 49.784674] [] ? async_schedule+0x20/0x20 > > > > [ 49.784681] [] worker_thread+0x133/0x3a0 > > > > [ 49.784688] [] ? manage_workers+0x190/0x190 > > > > [ 49.784696] [] kthread+0xa6/0xb0 > > > > [ 49.784707] [] kernel_thread_helper+0x4/0x10 > > > > [ 49.784715] [] ? finish_task_switch+0x8c/0x110 > > > > [ 49.784721] [] ? _raw_spin_unlock_irq+0x3b/0x70 > > > > [ 49.784727] [] ? retint_restore_args+0xe/0xe > > > > [ 49.784734] [] ? kthreadd+0x1e0/0x1e0 > > > > [ 49.784739] [] ? gs_change+0xb/0xb > > > > > > > > ------- > > > > > > > > When I look at the code I see this call chain > > > > enqueue_cmd_and_start_io()-> > > > > set_performant_mode()-> > > > > smp_processor_id() > > > > Which if you have debugging enabled calls debug_processor_id() and > > > > triggers the warning. > > > > > > > > I'm not very familiar with the hpsa code, so I'm not entirely sure what > > > > the purpose of this line is > > > > > > > > c->Header.ReplyQueue = smp_processor_id() % h->nreply_queues; > > > > > > > > Is the purpose to simply try to get a range of ReplyQueue numbers, but > > > > somewhat arbitrary? Or is it necessary that the current processor_id > > > > is used? If it is the former, and you're not accessing per cpu > > > > structures, > > > > or pinning a cpu, or anything like that then I would think it is safe > > > > to > > > > change this to a raw_smp_processor_id() to get rid of a false positive > > > > warning. > > It's not critical that they match (will work if they don't) but for certain > workloads you can get more performance if you pin processes to cpus and > arrange msix interrupt vectors so that commands are likely to complete on > the same cpu they originated from. > > In any case, I think your analysis is correct. Thanks. > > > > > > > > > diff --git a/drivers/scsi/hpsa.c b/drivers/scsi/hpsa.c > > > > index 7f4f790..4e19267 100644 > > > > --- a/drivers/scsi/hpsa.c > > > > +++ b/drivers/scsi/hpsa.c > > > > @@ -583,7 +583,7 @@ static void set_performant_mode(struct ctlr_info > > > > *h, > > > > struct CommandList *c) > > > > c->busaddr |= 1 | (h->blockFetchTable[c->Header.SGList] << 1); > > > > if (likely(h->msix_vector)) > > > > c->Header.ReplyQueue = > > > > - smp_processor_id() % h->nreply_queues; > > > > + raw_smp_processor_id() % h->nreply_queues; > > > > } > > > > } > > Ack. > > -- steve > Ok, thanks, I'll put the patch in another mail with my sign-off. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/