Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932375AbcCHPie (ORCPT ); Tue, 8 Mar 2016 10:38:34 -0500 Received: from mail-wm0-f48.google.com ([74.125.82.48]:34298 "EHLO mail-wm0-f48.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751531AbcCHPiZ (ORCPT ); Tue, 8 Mar 2016 10:38:25 -0500 Subject: Re: [PATCH 2/5] usb: gadget: f_midi: added spinlock on transmit function To: Felipe Balbi , linux-usb@vger.kernel.org References: <1456947640-20673-1-git-send-email-eu@felipetonello.com> <1456947640-20673-3-git-send-email-eu@felipetonello.com> <87twkm676d.fsf@ti.com> <87h9gikak2.fsf@intel.com> <56DD49AC.8040201@felipetonello.com> <87k2ldifo4.fsf@intel.com> <56DED7C2.2090603@felipetonello.com> <87r3flgjb1.fsf@intel.com> Cc: linux-kernel@vger.kernel.org, Michal Nazarewicz , Clemens Ladisch From: Felipe Ferreri Tonello Message-ID: <56DEF267.5090803@felipetonello.com> Date: Tue, 8 Mar 2016 15:40:23 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0 MIME-Version: 1.0 In-Reply-To: <87r3flgjb1.fsf@intel.com> Content-Type: multipart/mixed; boundary="------------090300070707020800050508" Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 16127 Lines: 358 This is a multi-part message in MIME format. --------------090300070707020800050508 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Hi Balbi, On 08/03/16 14:01, Felipe Balbi wrote: > > Hi, > > Felipe Ferreri Tonello writes: >>>>>>>> Since f_midi_transmit is called by both ALSA and USB frameworks, it >>>>>>> can >>>>>>>> potentially cause a race condition between both calls. This is bad >>>>>>> because the >>>>>>>> way f_midi_transmit is implemented can't handle concurrent calls. >>>>>>> This is due >>>>>>>> to the fact that the usb request fifo looks for the next element and >>>>>>> only if >>>>>>>> it has data to process it enqueues the request, otherwise re-uses it. >>>>>>> If both >>>>>>>> (ALSA and USB) frameworks calls this function at the same time, the >>>>>>>> kfifo_seek() will return the same usb_request, which will cause a >>>>>>> race >>>>>>>> condition. >>>>>>>> >>>>>>>> To solve this problem a syncronization mechanism is necessary. In >>>>>>> this case it >>>>>>>> is used a spinlock since f_midi_transmit is also called by >>>>>>> usb_request->complete >>>>>>>> callback in interrupt context. >>>>>>>> >>>>>>>> On benchmarks realized by me, spinlocks were more efficient then >>>>>>> scheduling >>>>>>>> the f_midi_transmit tasklet in process context and using a mutex >>>>>>>> to synchronize. Also it performs better then previous >>>>>>>> implementation >>>>>>> that >>>>>>>> allocated a usb_request for every new transmit made. >>>>>>> >>>>>>> behaves better in what way ? Also, previous implementation would not >>>>>>> suffer from this concurrency problem, right ? >>>>>> >>>>>> The spin lock is faster than allocating usb requests all the time, >>>>>> even if the udc uses da for it. >>>>> >>>>> did you measure ? Is the extra speed really necessary ? How did you >>>>> benchmark this ? >>>> >>>> Yes I did measure and it was not that significant. This is not about >>>> speed. There was a bug in that approach that I already explained on >>> >>> you have very confusing statements. When I mentioned that previous code >>> wouldn't have the need for the spinlock you replied that spinlock was >>> faster. >>> >>> When I asked you about benchmarks you reply saying it's not about the >>> speed. >>> >>> Make up your mind dude. What are you trying to achieve ? >>> >>>> that patch, which was approved and applied BTW. >>> >>> patches can be reverted if we realise we're better off without >>> them. Don't get cocky, please. >> >> Yes am I aware of that, but I honestly think that is the wrong way of >> dealing with this. >> >> ?? I don't get why am I giving this impression. > > re-read your emails. The gist goes like this: > > . Send patch > . Got comments > . Well, whatever, you can just ignore if you don't agree This is one of the problems with email. It can give the wrong impression and feelings. :) That was not what I meant at all. I mean that for real, not in a childish manner. I'm sorry if I gave you that impression. > >>>> Any way, this spinlock should've been there since that patch but I >>>> couldn't really trigger this problem without a stress test. >>> >>> which tells me you sent me patches without properly testing. How much >>> time did it take to trigger this ? How did you trigger this situation ? >> >> No, that is no true. The implementation I sent is working properly for >> any real world usage. >> >> The stress test I made to break the current implementation is *not* a >> real use-case. I made it in order to push as far as possible how fast >> the driver can *reliably* handle while sending and reading data. Then I >> noticed the bug. >> >> So, to answer your question. To trigger this bug is not a matter of >> time. The following needs to happen: >> 1. Device send MIDI message that is *bigger* than the usb request >> length. (just this by itself is really unlikely to happen in real world >> usage) > > I wouldn't say it's unlikely. You just cannot trust the other side of > the wire. We've seen e.g. Xbox 360's SCSI layer sending messages of the > wrong size and we worked around them in g_mass_storage. > > Broken implementations are a real thing ;-) Fair enough. And that's why I am pushing this fix. :) > >> 2. Host send a MIDI message back *exactly* at the same time as the >> device is processing the second part of the usb request from the same >> message. > > also not that unlikely to happen ;-) You can't assume the host will only > shift tokens on the wire at the time you're expecting it to. > >> I couldn't trigger this in all the tests we've made. I just triggered >> when I was sending huge messages back and forth (device <-> host) as >> mentioned. > > fair enough. > >> In fact, we have thousands of devices out there without this patch (but >> with my previous patch that introduced this bug). > > that's thousands of devices waiting to have a problem, right ? :-) :X > >> I am not trying to say it wasn't a mistake. That patch unfortunately >> introduces this bug, but it has real improvements over the previous >> implementation. AFAIR the improvements are: >> * Fixes a bug that was causing the DMA buffer to fill it up causing a >> kernel panic. > > this is a good point. Had forgotten about that detail. Thanks > >> * Pre allocate IN usb requests so there is no allocation overhead while >> sending data (same behavior already existed for the OUT endpoint). This >> ensure that the DMA memory is not misused affecting the rest of the >> system. > > also, arguably, a good idea. Recycling requests is a lot nicer and it's > what most gadget drivers do. > >> * It doesn't crash if the host doesn't send an ACK after IN data >> packets and we have reached the limit of available memory. Also, this is >> useful because it causes the ALSA layer to timeout, which is the correct >> userspace behavior. > > right > >> * Continuous to send data to the correct Jack (associated to each ALSA >> substream) if that was interrupted somehow, for instance by the size >> limit of a usb request. > > ok. > >>>> So, this patch fixes a bug in the current implementation. >>> >>> fixes a regression introduced by you, true. I'm trying to figure out if >>> we're better off without the original patch; to make a good decision I >>> need to know if the extra "speed" we get from not allocating requests on >>> demand are really that important. >>> >>> So, how much faster did you get and is that extra "speed" really >>> important ? >> >> The speed is not relevant at all in this case. It was not the goal of >> the patch, but I mentioned because it is obvious that with no memory >> allocation there will be an increase of speed that the code is executed. >> >> I did measure the speed improvements at that time, it was real but not >> relevant. I don't think we should be discussing this anyway. > > fair enough. This was probably the first email from you which gave me > some peace of mind that you know what you're doing with this fix. Keep > in mind that we all receive hundreds of emails a day and it's difficult > to track things over time. True. I will try to keep this always in mind. > > It's also a big PITA when someone sends fixes and cleanups on the same > series and/or with dependencies between them. The correct way is to send > *only* fixes first. They should be minimal patches that *only* fix the > problem. If the code looks messy or doesn't follow the coding style, > that's something you do on a completely separate fix and, usually, from > a clean topic branch starting at a tag from Linus (exceptions may arise, > of course). Got it. > > So anyway, to finally finish this up. Can you send JUST the bare minimum > fix necessary to avoid the regression ? Also, add a proper Fixes: foobar > line on commit log (see commit e18b7975c885bc3a938b9a76daf32957ea0235fa > for an example). > > Then we can get that merged. Keep in mind that you might have to Cc > stable (see same commit listed above). Ok. I will send the state-machine refactor as another patch in another topic then. > > After this is sorted out, then let's see how we can help you move your > product to libusbgx and check if there's anything missing in configfs > to cope with your use-case. That will be great, thanks! I will keep the list posted. > > ps: can you point me to your devices shipping with f_midi ? Which > architecture are they using ? Which USB Peripheral Controller ? This > might be a good addition to my test farm depending on your answers above > :-p Seaboard GRAND[1]. Freescale's i.MX 6 running an ARM A9. The controller is Chip Idea. [1] https://www.roli.com/products/seaboard-grand -- Felipe --------------090300070707020800050508 Content-Type: application/pgp-keys; name="0x92698E6A.asc" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="0x92698E6A.asc" -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v2 mQINBFYedIcBEACVKGKoEjb3zlvAz5SUvBej7Sx13BPd8hVulQD+mqjfuRFPmZA5 LBXPX1zTRWwGEbbZegP3tLfKP+XekzO6BQhDihMmKuRusdgDsdMtldwhjHuUwKn7 kxB2k79jSG802lAjIv2l5hijOfKIGTATKwiMijuXho54DGltIgNyN/Onwk9HnM6d jsV5uubaI468JRH6j8HXXievo24BDvsimIE75ImiM53ruiwPwEry1hi1CnE5OpqG oe/lt27+nLXijfNZOpBZ3Q+RPVBdqPTkMlBJAa4sg+qwZoSMvQJFAGROiJ7+ICCW O4GPMrAn8CRcCI9ENKBj2dQ4bBEP1a+f16GNMUUU37wocqtyNHo0Pa/DnFh91kcu /2dvUX4XPeEimEoSKroRLOXC9RGSFYB/r9UXqFgbmyQ4TZLx0mAWIjoUQtbIJNRz Pt46UeznCVLJTg7CzIvtv8vwmMFvaepr8ONoZn+tpX8VW4dgzsMZDrVspE0Vg3oo K9JRi1nN3GcJJK4zG2ShvEkPffRHuBuyX5wR8MPRYTShKnJR5qd1cCSK73fCv4DP bjywmGjucqcLiyYbByjmHaqzRaKJclmT/jhs6qZHs+pVLkmHkHdf/WLP6Xgcvmo8 c+SATrJwRsyW9riyMB7uNg3T84umbQrl00nAhcq73rem/602H+Qrh8rEfQARAQAB tC1GZWxpcGUgRmVycmVyaSBUb25lbGxvIDxldUBmZWxpcGV0b25lbGxvLmNvbT6J AjwEEwEIACYCGyMHCwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAIZAQUCVlXsdwAK CRDMRLTQkmmOapJFD/9XZZdX5XdXrQ2Qr7znQCOMPfOFiTqK8A1vVJPfLcPdkDyW iDfV2w4jyxKjgkcO8hgjZWAGdXhKxEXt8bQIZ+Po3eOqKo/O69+WyYS33uAKfNbr t4BcJOM2Vh2MwTymzh5EKjsKi/vmqFqvcpa81Jc4exDaHhb2mqMW8DZc96nMPwij +Z4vVooOVt5DVGeG4o2rztoti+KaXys6nycm8ErMmqWmL0viEnXHDRTOUHpnhEQY Vlg+hfxYrk00DphAePrpRp/HaxNncxR+ID0SHmYlLCaEy7s5ksOORjlKSk+7Nqat X69ymfXeEI84ror11IR7kuAA+rkhHMacWXANPkXtuEBi9Nl3V5Rvud4f/RL3Sh2x lwPfxca9rIR/7JYwoCzQ8iORok3VqKOtpufLKMsIl9sdj7ZxI6f6tZDm5uAaCZAt f15EcGiVWZgGdKlXzIWFkUsdRhtGYEQBF1Li/qLrnoS5T32eSCt8cgJlkDyKqNbX fxvV0KON8tJ9jBnMmoADetF/9V3A6EIBtTdxsz+UKjk6PV0bx398AbdvNB22r6Kw n1E2ZqzfNIaAOv5lEHh7VP7+s3vdXcKfnfobW0qd56NpAGTPhAN00wl7T5oMN3xS MNTPIC4316E8KChA5CQ3h6RRlZYvAJx7QWfJKX5zb8EVcJb0ul67V8hn1h/s54kC PwQTAQgAKQUCVh50hwIbIwUJCWYBgAcLCQgHAwIBBhUIAgkKCwQWAgMBAh4BAheA AAoJEMxEtNCSaY5qHxAP/0LmkXtaZvFelixfYibDHwpz+62mWydYN8I13ikV7uK0 DxpN6W3SeoAY6mF6mwHKyhsNhlOYPR8Rb+CpJxjIez6sPHAA02m4HqfTiNRuAeLM hSpiMIVesGgR9Lt/2gnCcF/R/pNJd5RC7wWzeeiS9/b2XtFImRSFvctyHwGXLBYM nlRx/o34kWscTCqixBF7lm3umAErDUd0fvL4oV2sC3W8Ncjfk/WQWfschV50lNtO h1Imvt+5ud3AJEcwXew0pbHKHvtrVc7toIO3jhZypD9YXP3aR9ZEkHS/bnd5HRHV t3iEwXH+bNu/evEB5dQcSf6M6Bwt7Ty2fqFx2PUmnZ22W4HGelPmE5aCqkBQqAkR kBmPEPlUIz6GLoj8jUL1ey+T+oCyD5wZoVb841bJDKIPxAa+P+MjFtQUpSXgyi8x G0Ic10pK9u3xsu5xzqLoAwt+pK+oxeWrZhJXg9/MFmqQQTX2dH8lIrgIOsHis5uK ySohiPwYQ0HUbbSFB9wM6e5dq621RvOkMA/myFtdjJX+Ynr7FmIpIZQiWZvOv0nM 0CeF7MotMIVa1ioh9K6/Z5WLjTBQh32DrGL07H0h+SMOvcL1IJuw9aG5qDKH4VW9 VSbHAogVSntamVrCg8jcT8uxEtwZNZH3aBqoYv4pa4MJsIs6vI0/9Y/mVyYVhQPN tDFGZWxpcGUgRmVycmVyaSBUb25lbGxvIDxmZWxpcGUudG9uZWxsb0BnbWFpbC5j b20+iQI5BBMBCAAjAhsjBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AFAlZV7HoA CgkQzES00JJpjmqexA//UHC0P+5NMN0HS0EC3vqdf+9AFfX/Gx+q6BjlaYh8gkGI 5SNGGb25dooGBRhXzFvp5UmxQjngvZO0Jl071kbqOs7IylZWvUB0tIB/9kcvIgJM U13CvdD2XwHDgKoCCS07ymFd9j31FhfYK6eRpr72oGHtjhGOSIj1u8T/4mJDstoc zK39gZjTQjzT/2sDbr2sd41KLJ9Ly3H67EgoxYIAQQwwT1Z7x88a8BORIXS8iZPw z3D8A1r2BT55qukwEP7GktNArKlhrieFRHCxk8PAtZRj8TrOEyG9nnuZWZQWcb6M suIY6s4cyL5KRTUgGfnpc+VMcNKnm0WmKg5jZXRFwg0DECZi0wJqsS8GPrq4La+K laxPcXj3ShJePj0YuTcJ/fPYoqqelcbmlqg6m+S9s8PRKPzlESt2jYPbgqMzE0Lu DLRGT7SBYgHgDVFwIuFyHQaqEhnU/nLWmyMSuTOE5L4s/CR1xY4qQUtOTnDFbwqL 125kbOCiRSMBBMOB4Q+UW3+dJ7uqo6LSqIKa/LWvzRmn5po4iSHAD18ivOh1wTe6 0d/ngjzpTm80uFrwfgkQTexxESYjlBbceBC8kpb1zPFrip41hIi3iPP9UQnnpOO2 hDrQu5z0N8HMIFg3kciR22wUMzQnyVN4iKfkrhERjjuYWOXJmC5I76DBqlEKLCWJ Aj8EEwEIACkFAlYyWW0CGyMFCQlmAYAHCwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIX gAAKCRDMRLTQkmmOakxGD/4xoYnRMuy8m6IwT1Y6mWzXZMkcClxwisUkwgF37jMR Xk7aitPqlD7AOn/jzKZBmKqOokyH3Y65u/AV6O7jXWWM3Ru0fDYYt0RZR0CkE0c2 Rio64ol6GWnF4dIiyTTSyxBNadVJXRYBpp5G78ie495ZOp35WUoz2RhExeIQGkBb ++UIyCh3eK70PPJ+1/dDV9sSRncIBZLSv+4hc4gS6YoprtSgiS8sdf8uBVtjT3r0 nmrKfTpktADqOOACd/K3LIdf6rlvAYsuWdQ/q32MF7Clrlpt9e5oqi+ASJcLNWUz cT8nFyvjJzM5kQEBblT6uzyWE5N6+K/t23Uw13vSx3Z2uhhfRQjcSOcZgHdvc8w1 3UXhE+XRCMmtA7jonJuFLYOXbjeeNEbJj5ManrOQMSnVy+kRlf3pLoc7VjJGd2k9 w1U+62b1IVypQCoDNhIy3YTjw7D151z/i5tK5yBvntswji10InowqPRuvt3C3fpW kv2MnB6qI+u6M5bP/1CvkLSC9vX+gavUFYZ/wCESVrl1FYQPfOoEgvEId9Ajzx0B qyR/5jXY3ZzWhnXNtYpHXy5/mJdvZ+v5ra5JLWKVA3pa+QbTd3v6ELo9NKavylcD FX9X2l2s7kp/FRVUZg3mr/3Pf0R1EFZAwM2Fgb5iKPLPjGi4b4sVtMxeXfbXRMBf 5rQwRmVsaXBlIEZlcnJlcmkgVG9uZWxsbyA8ZmVsaXBlLnRvbmVsbG9Acm9saS5j b20+iQI5BBMBCAAjAhsjBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AFAlZV7HoA CgkQzES00JJpjmrkTA//YiyBcGn+hjsKsVxFHseUw0ygN7TDLGeG7z0sP7AybVXR h4AwVcqyNkJjJ1UPXxPm5ra6/TbJf3XEalhIJtqh0MI0p+NQTFVOcGG58e6oMWj1 yBHAPmCyMJCi3GrB1/weP3qMiCdKWOWjxt+NoWss3pqqpSKHGW2rgmM3VP60iheR uZDDDI9DtoDz1ePT2AQZnopez2k75syZix1zkAI/VM/T3Y5lcnbaO4C+akDusfN0 galjwveYPIv6bBY6gd3B/7leSe2l4M3VIahj/1e4R2aLB78/pyy5BaI47zprkzPF zOAtZTurrwnE/1s3qWMP3OFeKiD1NLCm8kkvVakk/FjG8avFTAB3idNjpRT3wyPp nGv3G5Bp2jII35l7L5e2/zGcIigDNAUtdL3IgszfqtTVwGgISZDoxHIyBucqpYqV +ZesP1CpvyXaG4ME1QNTYZcVj1VTtz97IwntHw51r0d4IXjTbbrwJYR/g15XqNRB HwQyumPr3kMXLKTHCroK0z1OoT+pwNAO/XEWqQ5NQwyjalkFa1PzSPgcbTH46gs+ lilkrSgDuj/Amt0pkBSadgviJhsq2nQU49E8OOrrgDab0Z+6ur2lTENPWD4U9PS1 PisVd3ZAm8WmLx1ZktMgv2IPS9Tm+HlIdTPcw8OY12iPP1tRcE7pu23HLP3Un++0 KEZlbGlwZSBGZXJyZXJpIFRvbmVsbG8gPGZlbGlwZUByb2xpLmNvbT6JAjkEEwEI ACMCGyMHCwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAUCVlXsegAKCRDMRLTQkmmO aoymD/94mkk+RCXUQI6i3Z5qfenWHaJF86SH1tekpJl3eFHEUji0JWICOWS9225E 3oBbLDAA3szFWFxXNA2JdOv0ljwqWHT/aRXJlQ19LHMItguUyxfoELf818aym8bL MtazXRrYTy71SfvJwIleMYrNeodZnC4GwSOaKz5Rh/Qbnfzz/wG4dCNtGTcQ+EKP vcA06bNbYaqGA+qtrZ3st3lEmz/OPqRox7LqZTs/fvdAo49F8BlTZ6p09s2u7wVF UfYbi4+PQonGpn47QTx41OjFzzeA3NnCDak6JuSYa92QznShP0b03G5g4LdonwbU P/ivpzkSdWYVNwp/a7GfuhBAyk2VP/StF6WzzGC0j4TyBx+H5RzU32brNNsye1bL 8AMvHl9C3BEdgLnIlsTRMRCjs7f4EoFyEzmW40FUXPaJzxF6bCY/+GzDIhtMisjK fUFyakHnjITZSjo7Wqhp3W6a64esqPFbuFoX7sp5mWYzqDuKy0yD7S9vjA8wLdCt usBRw6TvVhaxmnOOa8zkyhOLqJ6le+zw8mW+Cot4LFFkeXyySNyRbrqOx3JqezRc olouAh2AJEir2sW3kdP53xRwUE5CJogC2QPuGSl5T1pcARmQJmudco2dk7C+hriF hkCHjp5ue9evv6u/Z7m+AcPJguVlq21ynYRrTeFJXCd2rgkAWbkCDQRWHnSHARAA yrLAhPO4JiqRk2sem+8bweimfnKmIm+ttTqjDni1DdBKtCZFUxPwEKzuOpU4aals 9Ohk4rQMnm6Q2XaJxIs0lijQJjVFbExtm9G2R2gkPJ5fnk4+k2mvps5F/iJjQk0k zWMITEA6cJzt9B8YC4dfsIq3lhCInOvSMBIVtDapruDGU4OskFBiKfzIq8diq/Ep qNfwCxZX2IcPhFv2+SJjph60oUC4WJ4zgINfJWdUGlIZrQp8sr/aEa77BVtLnTsu MHwqOF7P7yk4qpb3EFuyNCsJVAirYTqZeMWEv/pYiwtAgYOewdwptP8+5lbcXAor Y8Cs0GdW0r7LUzQjfhXl67EQWJuvGDBKMQ35LyUJtOm5m+qAnalz9Eyb0xRc9Arl rzH2GfCIr4ga+2RPw7fq7fZcbBcJDs770Mz0kVrLv3IAcc6fmnvuo30TFeIPB62/ c4xIe8njJ8RxbfwYAtd2KoAzwRQEqJQECyNnqENFHj2cimEueXf7NAw+z1nl1HIl 7MrHUeA0FQTw4WdGCahTuFRaJCjmODmKCFAooDWEek6jwNv18hCDJytSDc7uLAvH D2b9Sd40P1V2ochSf/DS/osfEIEeBf5mkG/MHBBbNxvSGP1h2yUM5C7g4nc6B+nz 5bIULhw0ojZx+U1i63gJkMsVMjKDAa+mGsWu0vAI3v0AEQEAAYkCHwQYAQgACQIb DAUCVlXsegAKCRDMRLTQkmmOao4vD/9GYLw5VmydK5BJ0sX12QmR7oMK4skMkcg/ gofkJ6njI4ETDG/pQfgBSQ/P0gUMXmhHgmpaRgs0tkS2NwsvURianXiQNpJwUA2g qKngqWlEvGOVkgOw7JKpdLWBvEW4vfP0Z0Q9brdeQG//9T9xyoyTC/jPM9A33NtI KlexBfBT7lXBy/y4tqf96KTOXk3STmI6nmSwmpPPtjxXXGQiMExGxAru+VX0HFxi JBS2fw/ucluVk2kffAO0y7DTL0UH3IWq4wEtHJQlEcujmUkP8PJSVIkhJpNlppR2 97DVxLcMAbnyxCW+ms1lCuZ5KB98KtowipqY9jhAPeP06W2WikENdmpLfNHhyIaM NKF7J+4zqPda7jZN1aKqs5OguJ4V82xXRJ0O6qSAd3Kvwi3MaKtNAv09JFG7UIcK ALJ+OQBzc00z54ii19ZPHjER6cYmcVutBUGOssqtyCOZymVbc7VYecdjxVrLwdDp LHHpcIjFn5ADjIAwGlt5ttkPvFTmqLeSWG9+vtoBShzQdeKJ4AN4r6/for/XsmF4 Jf7qGK8RamVX8lpWaNjUGtVSUOMTaSR1pmQJERDO2/WEZCm5hUFkYDbu+h+9m1W+ KL2ZkyGRC8FoZ46ly1J8koEeEoa57Au/uyK3KyvzlXuCI5zgGE+TohIjOr6d3TN4 SkAyCfQaSw=3D=3D =3DJwBB -----END PGP PUBLIC KEY BLOCK----- --------------090300070707020800050508--