From: "Kabir Ahsan-r9aahw" Subject: [Openswan dev] Openswan OCF : ----- USE_BATCH mode + ERESTART Date: Mon, 4 Dec 2006 10:26:41 -0700 Message-ID: <351763BE49AC8743AFCCF925821FF6BE015036AE@az33exm22.fsl.freescale.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0303934571==" Cc: linux-crypto@vger.kernel.org Return-path: Content-class: urn:content-classes:message To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dev-bounces@openswan.org Errors-To: dev-bounces@openswan.org List-Id: linux-crypto.vger.kernel.org This is a multi-part message in MIME format. --===============0303934571== Content-class: urn:content-classes:message Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C717C9.5D0D73E7" This is a multi-part message in MIME format. ------_=_NextPart_001_01C717C9.5D0D73E7 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi All I am trying to get the USE_BATCH (=3D1) mode to work for my crypto = driver. I guess I have found some problem, that is, crypto driver signals ERESTART as a function call return (line 1317 below), then the crypto thread sets the cc_qblocked to 1 (line 1333 below). =20 1303 if (submit !=3D NULL) { 1304 /* AK added */ 1305 //if (!( gKeepTrackOfCryptoThread % 100)) { 1306 // printk("6," ); 1307 //} 1308 list_del(&submit->crp_list); 1309 CRYPTO_Q_UNLOCK(); 1310 1311 //trace_set_L1(c,30); /* 30 is the id for OCF code */ 1312 //trace_set_L1(f,7); /* trace point */ 1313 //trace_log_L1(trace_var(c), trace_var(f), 200); /* log */ 1314 1315 result =3D crypto_invoke(submit, hint); 1316 CRYPTO_Q_LOCK(); 1317 if (result =3D=3D ERESTART) { 1318 1319 /* AK added */ 1320 //if (!( gKeepTrackOfCryptoThread % 100)) { 1321 // printk("7," ); 1322 //} 1323 /* 1324 * The driver ran out of resources, mark the 1325 * driver ``blocked'' for cryptop's and put 1326 * the request back in the queue. It would 1327 * best to put the request back where we got 1328 * it but that's hard so for now we put it 1329 * at the front. This should be ok; putting 1330 * it at the end does not work. 1331 */ 1332 /* XXX validate sid again? */ 1333 crypto_drivers[CRYPTO_SESID2HID(submit->crp_sid)].cc_qblocked =3D 1 ; 1334 list_add(&submit->crp_list, &crp_q); 1335 cryptostats.cs_blocks++; 1336 } 1337 } =20 Because the crypto hardware is not able to cope up with the request now it makes sense for the crypto thread to go to sleep and later be awakened by the crypto driver. But I guess this thread doesn't go to sleep because the wait_on_event condition argument is not turning out to be false. Here is the code segment. Line 1414 should put the thread to sleep but it does not because the submit queue is not empty, when crypto driver returned ERESTART.=20 =20 1375 if (submit =3D=3D NULL && krp =3D=3D NULL) { 1376 /* AK added */ 1377 //if (!( gKeepTrackOfCryptoThread % 100)) { 1378 // printk("8," ); 1379 //} 1380 /* 1381 * Nothing more to be processed. Sleep until we're 1382 * woken because there are more ops to process. 1383 * This happens either by submission or by a driver 1384 * becoming unblocked and notifying us through 1385 * crypto_unblock. Note that when we wakeup we 1386 * start processing each queue again from the 1387 * front. It's not clear that it's important to 1388 * preserve this ordering since ops may finish 1389 * out of order if dispatched to different devices 1390 * and some become blocked while others do not. 1391 */ 1392 //printk("1 : sleeping \n"); 1393 dprintk("%s - sleeping\n", __FUNCTION__); 1394 CRYPTO_Q_UNLOCK(); 1395 1414 wait_event_interruptible(cryptoproc_wait, 1415 cryptoproc =3D=3D = (pid_t) -1 || 1416 !list_empty(&crp_q) || 1417 !list_empty(&crp_kq)); 1418 ... =20 =20 >From the crypto driver routine I call crypto_unblock and believe the purpose of this is to wake up the crypto thread. In the first place this thread didn't go to sleep. Am I missing something? How this scheme is supposed to work? Didn't anyone run into ERESTART condition and was able to get out this condition gracefully? This ERESTART problem will show up more when the injected traffic to the VPN testbed is very high. For instance, in the lab I was blasting 100% of line rate (gigabit) with packet size of 1024B.=20 =20 Please shed some light. =20 Ahsan.=20 ------_=_NextPart_001_01C717C9.5D0D73E7 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable
Hi=20 All
I am = trying to get=20 the USE_BATCH (=3D1) mode to work for my crypto driver.  I = guess I have=20 found some problem, that is, crypto driver signals ERESTART as a = function call=20 return (line 1317 below), then the crypto thread sets the = cc_qblocked to 1=20 (line 1333 below).
 
  = 1303          &n= bsp;     =20 if (submit !=3D NULL) {
  =20 1304           &nb= sp;           &nbs= p;=20 /* AK added */
  =20 1305           &nb= sp;           &nbs= p;=20 //if (!( gKeepTrackOfCryptoThread % 100)) {
  =20 1306           &nb= sp;           &nbs= p;=20 //      printk("6," );
  =20 1307           &nb= sp;           &nbs= p;=20 //}
  =20 1308           &nb= sp;           &nbs= p;=20 list_del(&submit->crp_list);
  =20 1309           &nb= sp;           &nbs= p;=20 CRYPTO_Q_UNLOCK();
   1310
  =20 1311           &nb= sp;           &nbs= p;=20 //trace_set_L1(c,30); /* 30 is the id for OCF code */
  =20 1312           &nb= sp;           &nbs= p;=20 //trace_set_L1(f,7); /* trace point */
  =20 1313           &nb= sp;           &nbs= p;=20 //trace_log_L1(trace_var(c), trace_var(f), 200); /* log = */
  =20 1314
  =20 1315           &nb= sp;           &nbs= p;=20 result =3D crypto_invoke(submit, hint);
  =20 1316           &nb= sp;           &nbs= p;=20 CRYPTO_Q_LOCK();
  =20 1317           &nb= sp;           &nbs= p;=20 if (result =3D=3D ERESTART) {
   1318
  =20 1319           &nb= sp;           &nbs= p;        =20 /* AK added */
  =20 1320           &nb= sp;           &nbs= p;        =20 //if (!( gKeepTrackOfCryptoThread % 100)) {
  =20 1321           &nb= sp;           &nbs= p;        =20 //      printk("7," );
  =20 1322           &nb= sp;           &nbs= p;        =20 //}
  =20 1323           &nb= sp;           &nbs= p;        =20 /*
  =20 1324           &nb= sp;           &nbs= p;         =20 * The driver ran out of resources, mark the
  =20 1325           &nb= sp;           &nbs= p;         =20 * driver ``blocked'' for cryptop's and put
  =20 1326           &nb= sp;           &nbs= p;         =20 * the request back in the queue.  It would
  =20 1327           &nb= sp;           &nbs= p;         =20 * best to put the request back where we got
  =20 1328           &nb= sp;           &nbs= p;         =20 * it but that's hard so for now we put it
  =20 1329           &nb= sp;           &nbs= p;         =20 * at the front.  This should be ok; putting
  =20 1330           &nb= sp;           &nbs= p;         =20 * it at the end does not work.
  =20 1331           &nb= sp;           &nbs= p;         =20 */
  =20 1332           &nb= sp;           &nbs= p;        =20 /* XXX validate sid again? */
  =20 1333           &nb= sp;           &nbs= p;        =20 crypto_drivers[CRYPTO_SESID2HID(submit->crp_sid)].cc_qblocked =3D=20 1        ;
  =20 1334           &nb= sp;           &nbs= p;        =20 list_add(&submit->crp_list, &crp_q);
  =20 1335           &nb= sp;           &nbs= p;        =20 cryptostats.cs_blocks++;
  =20 1336           &nb= sp;           &nbs= p;=20 }
  =20 1337           &nb= sp;    =20 }
 
Because the=20 crypto hardware is not able to cope up with the request now it makes = sense for=20 the crypto thread to go to sleep and later be awakened by the crypto = driver. But=20 I guess this thread doesn't go to sleep because the wait_on_event = condition=20 argument is not turning out to be false. Here is the code segment. Line = 1414=20 should put the thread to sleep but it does not because the submit queue = is not=20 empty, when crypto driver returned ERESTART.
 
 =20  1375          &nb= sp;     =20 if (submit =3D=3D NULL && krp =3D=3D NULL) {
  =20 1376           &nb= sp;           &nbs= p;=20 /* AK added */
  =20 1377           &nb= sp;           &nbs= p;=20 //if (!( gKeepTrackOfCryptoThread % 100)) {
  =20 1378           &nb= sp;           &nbs= p;=20 //      printk("8," );
  =20 1379           &nb= sp;           &nbs= p;=20 //}
  =20 1380           &nb= sp;           &nbs= p;=20 /*
  =20 1381           &nb= sp;           &nbs= p; =20 * Nothing more to be processed.  Sleep until we're
  =20 1382           &nb= sp;           &nbs= p; =20 * woken because there are more ops to process.
  =20 1383           &nb= sp;           &nbs= p; =20 * This happens either by submission or by a driver
  =20 1384           &nb= sp;           &nbs= p; =20 * becoming unblocked and notifying us through
  =20 1385           &nb= sp;           &nbs= p; =20 * crypto_unblock.  Note that when we wakeup we
  =20 1386           &nb= sp;           &nbs= p; =20 * start processing each queue again from the
  =20 1387           &nb= sp;           &nbs= p; =20 * front. It's not clear that it's important to
  =20 1388           &nb= sp;           &nbs= p; =20 * preserve this ordering since ops may finish
  =20 1389           &nb= sp;           &nbs= p; =20 * out of order if dispatched to different devices
  =20 1390           &nb= sp;           &nbs= p; =20 * and some become blocked while others do not.
  =20 1391           &nb= sp;           &nbs= p; =20 */
  =20 1392           &nb= sp;           &nbs= p;=20 //printk("1 : sleeping \n");
  =20 1393           &nb= sp;           &nbs= p;=20 dprintk("%s - sleeping\n", __FUNCTION__);
  =20 1394           &nb= sp;           &nbs= p;=20 CRYPTO_Q_UNLOCK();
  =20 1395
   1414       &= nbsp;           &n= bsp;    =20 wait_event_interruptible(cryptoproc_wait,
  =20 1415           &nb= sp;           &nbs= p;            = ;    =20 cryptoproc =3D=3D (pid_t) -1 ||
  =20 1416           &nb= sp;           &nbs= p;            = ;    =20 !list_empty(&crp_q) ||
  =20 1417           &nb= sp;           &nbs= p;            = ;    =20 !list_empty(&crp_kq));
  =20 1418           &nb= sp;           &nbs= p;=20 ...

 
 
From the=20 crypto driver routine I call crypto_unblock and believe the purpose of = this is=20 to wake up the crypto thread. In the first place this thread didn't go = to sleep.=20 Am I missing something? How this scheme is supposed to work? Didn't = anyone run=20 into ERESTART condition and was able to get out this condition = gracefully? This=20 ERESTART problem will show up more when the injected traffic to the=20 VPN testbed is very high. For instance, in the lab I was blasting = 100% of=20 line rate (gigabit) with packet size of 1024B. =
 
Please shed=20 some light.
 
Ahsan.=20
------_=_NextPart_001_01C717C9.5D0D73E7-- --===============0303934571== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Dev mailing list Dev@openswan.org http://lists.openswan.org/mailman/listinfo/dev --===============0303934571==--