SGVsbG8gR3JlZywKCgpPbiBXZWQsIEFwciAxMSwgMjAxMiBhdCA3OjExIFBNLCBHcmVnIEtIIDxn
cmVna2hAbGludXhmb3VuZGF0aW9uLm9yZz4gd3JvdGU6Cj4gVGhpcyBpcyB0aGUgc3RhcnQgb2Yg
dGhlIHN0YWJsZSByZXZpZXcgY3ljbGUgZm9yIHRoZSAzLjMuMiByZWxlYXNlLgo+IFRoZXJlIGFy
ZSA3OCBwYXRjaGVzIGluIHRoaXMgc2VyaWVzLCBhbGwgd2lsbCBiZSBwb3N0ZWQgYXMgYSByZXNw
b25zZQo+IHRvIHRoaXMgb25lLiCgSWYgYW55b25lIGhhcyBhbnkgaXNzdWVzIHdpdGggdGhlc2Ug
YmVpbmcgYXBwbGllZCwgcGxlYXNlCj4gbGV0IG1lIGtub3cuCj4KPiBSZXNwb25zZXMgc2hvdWxk
IGJlIG1hZGUgYnkgRnJpIEFwciAxMyAyMzoxMDoxNiBVVEMgMjAxMi4KPiBBbnl0aGluZyByZWNl
aXZlZCBhZnRlciB0aGF0IHRpbWUgbWlnaHQgYmUgdG9vIGxhdGUuCj4KCmlzIHRoZXJlIGFueSBj
aGFuY2UgZm9yIHRoaXMgb25lIHRvIGJlIGluY2x1ZGVkIGluIHRoaXMgcmV2aWV3IGN5Y2xlPwoK
aHR0cDovL3d3dy5zcGluaWNzLm5ldC9saXN0cy9saW51eC13aXJlbGVzcy9tc2c4Nzk5OS5odG1s
CgpiZXN0IHJlZ2FyZHMsCnNlcmdpbwoKPiBUaGUgd2hvbGUgcGF0Y2ggc2VyaWVzIGNhbiBiZSBm
b3VuZCBpbiBvbmUgcGF0Y2ggYXQ6Cj4goCCgIKAgoGtlcm5lbC5vcmcvcHViL2xpbnV4L2tlcm5l
bC92My4wL3N0YWJsZS1yZXZpZXcvcGF0Y2gtMy4zLjItcmMxLmd6Cj4gYW5kIHRoZSBkaWZmc3Rh
dCBjYW4gYmUgZm91bmQgYmVsb3cuCj4KPiB0aGFua3MsCj4KPiBncmVnIGstaAo+Cj4gLS0tLS0t
LS0tLS0tLQo+IKBNYWtlZmlsZSCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCg
IKAgoCB8IKAgoDQgKy0KPiCgYXJjaC9hcm0vbWFjaC1hdDkxL2F0OTFzYW05MjYzX2RldmljZXMu
YyCgIKAgoCCgIKAgfCCgIKAzICstCj4goGFyY2gvYXJtL21hY2gtYXQ5MS9hdDkxc2FtOWc0NV9k
ZXZpY2VzLmMgoCCgIKAgoCCgIHwgoCCgNiArLQo+IKBhcmNoL2FybS9tYWNoLWF0OTEvYm9hcmQt
c2FtOTI2M2VrLmMgoCCgIKAgoCCgIKAgoCB8IKAgoDEgKwo+IKBhcmNoL2FybS9tYWNoLWF0OTEv
Ym9hcmQtc2FtOW0xMGc0NWVrLmMgoCCgIKAgoCCgIKB8IKAgoDEgKwo+IKBhcmNoL202OGsvbWFj
L2NvbmZpZy5jIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCB8IKAgoDMgKwo+IKBhcmNoL3g4
Ni9pbmNsdWRlL2FzbS90aW1lci5oIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCB8IKAgoDggKy0KPiCg
YXJjaC94ODYva2VybmVsL2FwaWMvaW9fYXBpYy5jIKAgoCCgIKAgoCCgIKAgoCCgIKAgfCCgIDQw
ICstLS0tCj4goGFyY2gveDg2L2tlcm5lbC9rZ2RiLmMgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAg
oCCgIHwgoCA2MCArKysrKysrKwo+IKBhcmNoL3g4Ni9rZXJuZWwvdHNjLmMgoCCgIKAgoCCgIKAg
oCCgIKAgoCCgIKAgoCCgIKB8IKAgoDMgKy0KPiCgYXJjaC94ODYvbmV0L2JwZl9qaXRfY29tcC5j
IKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgfCCgIKAyICstCj4goGRyaXZlcnMvYWNwaS9hY3BpY2Ev
dGJmYWR0LmMgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIHwgoCCgOCArLQo+IKBkcml2ZXJzL2FjcGkv
cHJvY2Vzc29yX3RoZXJtYWwuYyCgIKAgoCCgIKAgoCCgIKAgoCB8IKAgNDUgKysrKystCj4goGRy
aXZlcnMvYmFzZS9maXJtd2FyZV9jbGFzcy5jIKAgoCCgIKAgoCCgIKAgoCCgIKAgoHwgoCA5NiAr
KysrKysrKy0tLS0KPiCgZHJpdmVycy9iYXNlL3Bvd2VyL3J1bnRpbWUuYyCgIKAgoCCgIKAgoCCg
IKAgoCCgIKAgfCCgIKAzICstCj4goGRyaXZlcnMvYmFzZS9yZWdtYXAvcmVnY2FjaGUtcmJ0cmVl
LmMgoCCgIKAgoCCgIKAgoHwgoCCgOCArLQo+IKBkcml2ZXJzL2RtYS9pb2F0L2RtYS5jIKAgoCCg
IKAgoCCgIKAgoCCgIKAgoCCgIKAgoCB8IKAgMTYgKy0KPiCgZHJpdmVycy9kbWEvaW9hdC9kbWEu
aCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgfCCgIKA2ICstCj4goGRyaXZlcnMvZG1hL2lv
YXQvZG1hX3YyLmMgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoHwgoCCgOCArLQo+IKBkcml2ZXJz
L2RtYS9pb2F0L2RtYV92My5jIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKB8IKAgoDggKy0KPiCg
ZHJpdmVycy9ncHUvZHJtL2RybV9mYl9oZWxwZXIuYyCgIKAgoCCgIKAgoCCgIKAgoCCgfCCgIKA4
ICstCj4goGRyaXZlcnMvZ3B1L2RybS9pOTE1L2k5MTVfZHJ2LmMgoCCgIKAgoCCgIKAgoCCgIKAg
oHwgoCCgMiArCj4goGRyaXZlcnMvZ3B1L2RybS9pOTE1L2k5MTVfcmVnLmggoCCgIKAgoCCgIKAg
oCCgIKAgoHwgoCCgMSArCj4goGRyaXZlcnMvZ3B1L2RybS9pOTE1L2ludGVsX2Jpb3MuYyCgIKAg
oCCgIKAgoCCgIKAgoHwgoCAyMyArKy0KPiCgZHJpdmVycy9ncHUvZHJtL2k5MTUvaW50ZWxfZGlz
cGxheS5jIKAgoCCgIKAgoCCgIKAgfCCgIKA2ICsKPiCgZHJpdmVycy9ncHUvZHJtL2k5MTUvaW50
ZWxfbHZkcy5jIKAgoCCgIKAgoCCgIKAgoCCgfCCgIKA4ICsKPiCgZHJpdmVycy9ncHUvZHJtL2k5
MTUvaW50ZWxfc3ByaXRlLmMgoCCgIKAgoCCgIKAgoCCgfCCgIKAzICsKPiCgZHJpdmVycy9ncHUv
ZHJtL3JhZGVvbi9hdG9tLmMgoCCgIKAgoCCgIKAgoCCgIKAgoCCgfCCgIDE1ICstCj4goGRyaXZl
cnMvZ3B1L2RybS9yYWRlb24vYXRvbS5oIKAgoCCgIKAgoCCgIKAgoCCgIKAgoHwgoCCgMSArCj4g
oGRyaXZlcnMvbWVkaWEvZHZiL2R2Yi1jb3JlL2R2Yl9mcm9udGVuZC5jIKAgoCCgIKAgoHwgoCAx
MiArKwo+IKBkcml2ZXJzL21lZGlhL3ZpZGVvL3V2Yy91dmNfdmlkZW8uYyCgIKAgoCCgIKAgoCCg
IKB8IKAgNTAgKysrLS0tCj4goGRyaXZlcnMvbWZkL2RhOTA1Mi1zcGkuYyCgIKAgoCCgIKAgoCCg
IKAgoCCgIKAgoCCgIHwgoCCgNCArLQo+IKBkcml2ZXJzL21mZC90d2w2MDMwLWlycS5jIKAgoCCg
IKAgoCCgIKAgoCCgIKAgoCCgIKB8IKAgMTMgKy0KPiCgZHJpdmVycy9taXNjL2tnZGJ0cy5jIKAg
oCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgfCCgMTYwICsrKysrKysrKysrKysrLS0tLS0tCj4g
oGRyaXZlcnMvbW1jL2NvcmUvc2Rpb19idXMuYyCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoHwgoCAx
MiArLQo+IKBkcml2ZXJzL21tYy9ob3N0L2F0bWVsLW1jaS5jIKAgoCCgIKAgoCCgIKAgoCCgIKAg
oCB8IKAgoDkgKy0KPiCgZHJpdmVycy9tbWMvaG9zdC9zZGhjaS1kb3ZlLmMgoCCgIKAgoCCgIKAg
oCCgIKAgoCCgfCCgIKAxICsKPiCgZHJpdmVycy9tdGQvZGV2aWNlcy9ibG9jazJtdGQuYyCgIKAg
oCCgIKAgoCCgIKAgoCCgfCCgIKAxICsKPiCgZHJpdmVycy9tdGQvZGV2aWNlcy9kb2MyMDAwLmMg
oCCgIKAgoCCgIKAgoCCgIKAgoCCgfCCgIKAyICstCj4goGRyaXZlcnMvbXRkL2RldmljZXMvZG9j
MjAwMS5jIKAgoCCgIKAgoCCgIKAgoCCgIKAgoHwgoCCgMiArLQo+IKBkcml2ZXJzL210ZC9kZXZp
Y2VzL2RvYzIwMDFwbHVzLmMgoCCgIKAgoCCgIKAgoCCgIKB8IKAgoDIgKy0KPiCgZHJpdmVycy9t
dGQvZGV2aWNlcy9kb2NnMy5jIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgfCCgIKAyICstCj4goGRy
aXZlcnMvbXRkL2RldmljZXMvbGFydC5jIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIHwgoCCgMSAr
Cj4goGRyaXZlcnMvbXRkL2RldmljZXMvbTI1cDgwLmMgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIHwg
oCCgMSArCj4goGRyaXZlcnMvbXRkL2RldmljZXMvc3N0MjVsLmMgoCCgIKAgoCCgIKAgoCCgIKAg
oCCgIHwgoCCgMSArCj4goGRyaXZlcnMvbXRkL21hcHMvaXhwNHh4LmMgoCCgIKAgoCCgIKAgoCCg
IKAgoCCgIKAgoHwgoCCgNSArLQo+IKBkcml2ZXJzL210ZC9tYXBzL2xhbnRpcS1mbGFzaC5jIKAg
oCCgIKAgoCCgIKAgoCCgIKB8IKAgoDMgKy0KPiCgZHJpdmVycy9tdGQvbmFuZC9ncG1pLW5hbmQv
Z3BtaS1uYW5kLmMgoCCgIKAgoCCgIKAgfCCgIKAyICstCj4goGRyaXZlcnMvbmV0L2V0aGVybmV0
L2Jyb2FkY29tL3RnMy5jIKAgoCCgIKAgoCCgIKAgoHwgoCCgNCArLQo+IKBkcml2ZXJzL25ldC9l
dGhlcm5ldC9mcmVlc2NhbGUvZnNsX3BxX21kaW8uYyCgIKAgoCB8IKAgMTIgKy0KPiCgZHJpdmVy
cy9uZXQvZXRoZXJuZXQvbWFydmVsbC9za3kyLmMgoCCgIKAgoCCgIKAgoCCgfCCgIKA1ICstCj4g
oGRyaXZlcnMvbmV0L2V0aGVybmV0L3ZpYS92aWEtcmhpbmUuYyCgIKAgoCCgIKAgoCCgIHwgoCAx
MiArLQo+IKBkcml2ZXJzL25ldC91c2IvY2RjX2VlbS5jIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCg
IKB8IKAgoDEgKwo+IKBkcml2ZXJzL25ldC91c2IvemF1cnVzLmMgoCCgIKAgoCCgIKAgoCCgIKAg
oCCgIKAgoCB8IKAgoDUgKwo+IKBkcml2ZXJzL25ldC93aXJlbGVzcy9hdGgvYXRoOWsvY2FsaWIu
YyCgIKAgoCCgIKAgoCB8IKAgoDUgKy0KPiCgZHJpdmVycy9uZXQvd2lyZWxlc3MvaXdsZWdhY3kv
Mzk0NS1tYWMuYyCgIKAgoCCgIKAgfCCgIKAxIC0KPiCgZHJpdmVycy9uZXQvd2lyZWxlc3MvaXds
ZWdhY3kvNDk2NS1tYWMuYyCgIKAgoCCgIKAgfCCgIKAxIC0KPiCgZHJpdmVycy9uZXQvd2lyZWxl
c3MvaXdsZWdhY3kvY29tbW9uLmMgoCCgIKAgoCCgIKAgfCCgIDE4ICsrLQo+IKBkcml2ZXJzL25l
dC93aXJlbGVzcy9ydGx3aWZpL3J0bDgxOTJjL3BoeV9jb21tb24uYyB8IKAgoDIgKy0KPiCgZHJp
dmVycy9uZXQvd2lyZWxlc3MvcnRsd2lmaS9ydGw4MTkyZGUvcGh5LmMgoCCgIKAgfCCgIKAyICst
Cj4goGRyaXZlcnMvcGxhdGZvcm0veDg2L2FjZXItd21pLmMgoCCgIKAgoCCgIKAgoCCgIKAgoHwg
oCCgMSArCj4goGRyaXZlcnMvcG5wL3BucGFjcGkvY29yZS5jIKAgoCCgIKAgoCCgIKAgoCCgIKAg
oCCgIHwgoCCgNyArLQo+IKBkcml2ZXJzL3N0YWdpbmcvYW5kcm9pZC9sb3dtZW1vcnlraWxsZXIu
YyCgIKAgoCCgIKB8IKAgNDcgKy0tLS0tCj4goGRyaXZlcnMvdGFyZ2V0L3RjbV9mYy90Y21fZmMu
aCCgIKAgoCCgIKAgoCCgIKAgoCCgIHwgoCCgMSArCj4goGRyaXZlcnMvdGFyZ2V0L3RjbV9mYy90
ZmNfY21kLmMgoCCgIKAgoCCgIKAgoCCgIKAgoHwgoCAxMCArLQo+IKBkcml2ZXJzL3RhcmdldC90
Y21fZmMvdGZjX2NvbmYuYyCgIKAgoCCgIKAgoCCgIKAgoCB8IKAgMTMgKy0KPiCgZHJpdmVycy90
YXJnZXQvdGNtX2ZjL3RmY19pby5jIKAgoCCgIKAgoCCgIKAgoCCgIKAgfCCgIKAyICsKPiCgZHJp
dmVycy91c2IvaG9zdC9vaGNpLWF0OTEuYyCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgfCCgIKA0ICst
Cj4goGZzL2NpZnMvZmlsZS5jIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIHwg
oCAxMCArLQo+IKBmcy9sb2Nrcy5jIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCg
IKAgoCB8IKAgoDMgKy0KPiCgZnMvbmZzL25mczRwcm9jLmMgoCCgIKAgoCCgIKAgoCCgIKAgoCCg
IKAgoCCgIKAgoCCgfCCgIKAyICstCj4goGluY2x1ZGUvbGludXgvZnMuaCCgIKAgoCCgIKAgoCCg
IKAgoCCgIKAgoCCgIKAgoCCgIHwgoCCgNSArCj4goGluY2x1ZGUvbGludXgva2VybmVsLmggoCCg
IKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIHwgoCAxMyArKwo+IKBpbmNsdWRlL2xpbnV4L2tnZGIu
aCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCB8IKAgoDcgKy0KPiCgaW5jbHVkZS9saW51
eC9rbW9kLmggoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgfCCgIDI3ICsrKy0KPiCga2Vy
bmVsL2NyZWQuYyCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgfCCgIKAyICsK
PiCga2VybmVsL2RlYnVnL2RlYnVnX2NvcmUuYyCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgfCCg
IDUzICsrKy0tLS0KPiCga2VybmVsL2lycS9taWdyYXRpb24uYyCgIKAgoCCgIKAgoCCgIKAgoCCg
IKAgoCCgIKAgfCCgIDEwICstCj4goGtlcm5lbC9rbW9kLmMgoCCgIKAgoCCgIKAgoCCgIKAgoCCg
IKAgoCCgIKAgoCCgIKAgoHwgoDExNyArKysrKysrKysrLS0tLQo+IKBrZXJuZWwvcG93ZXIvaGli
ZXJuYXRlLmMgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCB8IKAgMTggKy0tCj4goGtlcm5lbC9w
b3dlci9wcm9jZXNzLmMgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIHwgoCCgOCArCj4goGtl
cm5lbC9wb3dlci9zdXNwZW5kLmMgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIHwgoCCgNyAt
Cj4goGtlcm5lbC9wb3dlci91c2VyLmMgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoHwg
oCAxMCArLQo+IKBrZXJuZWwvc3lzY3RsLmMgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAg
oCCgIKB8IKAgoDggKy0KPiCga2VybmVsL3RyYWNlL3RyYWNlLmMgoCCgIKAgoCCgIKAgoCCgIKAg
oCCgIKAgoCCgIKAgfCCgIKA0ICsKPiCga2VybmVsL3RyYWNlL3RyYWNlX2VudHJpZXMuaCCgIKAg
oCCgIKAgoCCgIKAgoCCgIKAgfCCgIDE2ICstCj4goGtlcm5lbC90cmFjZS90cmFjZV9leHBvcnQu
YyCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoHwgoCCgMiArLQo+IKBuZXQvbWFjODAyMTEvYWdnLXJ4
LmMgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKB8IKAgoDMgKy0KPiCgbmV0L3Jvc2Uvcm9z
ZV9kZXYuYyCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgfCCgIKA0ICstCj4goHNjcmlw
dHMvbW9kL21vZHBvc3QuYyCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoHwgoCCgOSArLQo+
IKBzY3JpcHRzL21vZC9tb2Rwb3N0LmggoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKB8IKAg
oDEgKwo+IKBzZWN1cml0eS90b21veW8vbW91bnQuYyCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCg
IKB8IKAgMzggKystLS0KPiCgc291bmQvcGNpL2hkYS9wYXRjaF9yZWFsdGVrLmMgoCCgIKAgoCCg
IKAgoCCgIKAgoCCgfCCgIDExICstCj4goHNvdW5kL3NvYy9jb2RlY3MvYWs0NjQyLmMgoCCgIKAg
oCCgIKAgoCCgIKAgoCCgIKAgoHwgoCCgMiArLQo+IKBzb3VuZC9zb2MvY29kZWNzL3dtODk5NC5j
IKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKB8IKAgoDIgKy0KPiCgc291bmQvc29jL3RlZ3JhL3Rl
Z3JhX2kycy5jIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgfCCgIKAyICstCj4goDk2IGZpbGVzIGNo
YW5nZWQsIDgxNSBpbnNlcnRpb25zKCspLCA0MTEgZGVsZXRpb25zKC0pCj4KPiAtLQo+IFRvIHVu
c3Vic2NyaWJlIGZyb20gdGhpcyBsaXN0OiBzZW5kIHRoZSBsaW5lICJ1bnN1YnNjcmliZSBsaW51
eC1rZXJuZWwiIGluCj4gdGhlIGJvZHkgb2YgYSBtZXNzYWdlIHRvIG1ham9yZG9tb0B2Z2VyLmtl
cm5lbC5vcmcKPiBNb3JlIG1ham9yZG9tbyBpbmZvIGF0IKBodHRwOi8vdmdlci5rZXJuZWwub3Jn
L21ham9yZG9tby1pbmZvLmh0bWwKPiBQbGVhc2UgcmVhZCB0aGUgRkFRIGF0IKBodHRwOi8vd3d3
LnR1eC5vcmcvbGttbC8K
On Apr 14 Felipe Contreras wrote:
> On Sat, Apr 14, 2012 at 8:55 PM, Stefan Richter
> <[email protected]> wrote:
> > Generally, "commit + push out" makes it undroppable. In case of -stable,
> > commit/ push out/ tag are close and virtually identical.
> >
> > After a change was pushed out, the choice narrows down to add a reverting
> > change for a subsequent release or to rebase to a point before the
> > defective commit. The latter is not done to -stable, obviously. (3.3.2
> > is not being restarted from 3.3 if a regression from 3.3 to 3.3.1 was
> > discovered.)
>
> That's irrelevant; whether you rebase and drop the patch, or you
> revert it, the resulting code is *exactly* the same. It's also the
> same if Greg drops it from his quilt queue before pushing (assuming
> that's what he uses).
The result may be the same (sometimes it actually isn't), but the way to
get there is not.
> Let's avoid this semantic red herring, by undroppable I mean
> unrevertable, or undiscardable, or anything that effectively makes the
> patch not be there.
How do you "make it not be there"? By adding a reverting patch.
The reverting patch needs to come from somewhere, and the only
disagreement is basically through which channels the patch should be
allowed to come, or which qualifications this reverting patch should
fulfill.
> >> But *why*? You say you *really* need to problem to fixed in mainline,
> >> that's why it cannot be dropped, but that applies also to patches in
> >> the queue *before* the tag is made, doesn't it? If you find a patch in
> >> the stable review queue causes problems, why don't you leave it there?
> >
> > As Willy wrote, that's actually what is done sometimes. I didn't know
> > that.
>
> Don't avoid the question.
Willy answered that, hence I didn't think I have to. The patch can in
fact be kept in the stable queue as a reminder, it just should not be
applied to stable as long as there is no resolution for mainline.
> I don't mean just leave it in the queue, I mean leave it so it gets
> tagged in the release.
That would be even sillier than the discussion which we are having.
> >> You *really* need to problem fixed in mainline, don't you?
> >>
> >> So yesterday the priority is stability > 'upstream first', but today
> >> it's 'upstream first' > stability. Nobody has put forward an argument
> >> for this shift in priorities--
> >
> > Yesterday, folks cared about mainline too.
>
> Who suggested otherwise? Being priority #2 doesn't mean you don't
> care. We always care about mainline, even for patches that are not
> proposed for 'stable'.
>
> But if we found yesterday that the patch would break things, it would
> not make it to the stable release.
>
> You are again avoiding the argument.
The priorities argument is bogus. The procedure of receiving stable
patches only as backports from mainline is exactly about stabilization,
nothing else.
[...]
> > if a defective change was not dropped but released, and now requires
> > a fix (e.g. revert) "today: There are now /two/ bugs: In the mainline
> > and in a stable kernel.
>
> What?! So, if an issue has been in the kernel for the last 10 years,
> and it's just fixed, that means we suddenly have hundreds of bugs?
You would have fixed hundreds of bugs if you released fixes into hundreds
of kernel branches.
> You are playing with words; it's *one* bug that is present in multiple releases.
Count it as a single bug if you prefer. The fact is, the bugs or bug
needs fixes in each affected maintained kernel branch. So even if you
count one bug, there are still more than one bug resolutions to be done.
> e) if it gets into Greg's stable queue; it's still *one* bug
> f) and if it gets into v3.4.1; it's still *one* bug.
[...]
> By saying it's "two bugs", you are still avoiding the question of why
> they are different. *Why* are they two bugs? What is so different
> between e) and f)
e) requires a fix to be published in the mainline. f) requires a fix to
be published in the mainline and another fix to be published in stable.
> that makes bad patches undroppable?
f) makes stable require a reverting patch.
> I appreciate you are exploring this discussion, but I wonder if you
> accept the possibility that there's actually no good reason for this
> change in priorities, or if you accept that even a change in
> priorities is taking place.
Stabilization is the only priority. There is nothing else.
I stated one reason for the procedure in case of f):
Fixing mainline first and then backporting to stable is always easier and
safer than fixing stable first and only then start to think about how
mainline can be fixed. I and others also explained a bit more why it is
easier and safer.
I realise that this is not a good enough reason in your opinion, at least
as far as revert-type fixes are concerned. Some of the folks who are
unfortunate enough to be Cc'd on this discussion have quite a bit of
experience with varying kernel maintenance models though, so I find their
opinions in these matters well worth thinking about.
--
Stefan Richter
-=====-===-- -=-- -===-
http://arcgraph.de/sr/
On Sun, Apr 15, 2012 at 8:49 PM, Linus Torvalds
<[email protected]> wrote:
> On Sun, Apr 15, 2012 at 10:15 AM, Felipe Contreras
> <[email protected]> wrote:
>>
>> This is not a reason, this is just stating what happens without
>> explaining *why*.
>>
>> Q: What changes when a tag is made?
>> A: A tag is made
[...]
> The only thing that matters is "it's been made available to others".
Exactly! Now *this* is a reason. After sending that mail I tried to
think of one since nobody else came up with it. This is in fact, a
good reason, but it implies something I'll explain below.
[...]
I do agree with all what you said above.
> And the thing is, it's not just 3.3.3 that follows 3.3.2. It's 3.4.1
> too. The reason we have the "no stable fixes before it's been
> upstreamed" (and again: a "revert" is *nothing* but another name for a
> fix, that gives some additional information about *how* we fixed
> something too) is because we want to make nice plodding reliable
> forward progress. Mistakes (== bugs) will happen, but the stable rules
> are there to make sure that those mistakes get fixed, and don't have
> long-term impact. THAT is the "stable" part of the stable series.
> Things are monotonic in the big picture, even if we may have some
> non-monotonic noise in the details.
I'm not going to argue the semantics of what is a revert, but I am
going to show the difference between the two situations:
a) v3.0* (good), v3.1* (good), v3.2* (good), v3.3 (good), v3.3.1
(bad), v3.3.2 (good), v3.4 (bad)
b) v3.0* (bad), v3.1* (bad), v3.2* (bad), v3.3 (bad), v3.3.1 (good), v3.4 (bad)
Maybe they are the same from certain point of view, but you just
argued that what *others* see is what makes the patch unrevertable,
well, it's obvious that from the point of view of the users the two
situations are clearly different. I believe it was you who said that
breaking user experience is the absolute no-no any project could
make[1]--so from that point of view a) is *much* worst.
But what is done is done, and as you said, you can't change the past,
now the important thing is what to do next. And here are the two
options' worst-case scenarios:
a.1) v3.0* (good), ... v3.3 (good), v3.3.1 (bad), v3.3.2 (bad), v3.3.3
(bad), v3.3.4 (bad), v3.3.5 (bad), v3.4 (good)
a.2) v3.0* (good), ... v3.3 (good), v3.3.1 (bad), v3.3.2 (good),
v3.3.3 (good), v3.3.4 (good), v3.3.5 (good), v3.4 (bad)
These two scenarios are unlikely, but either way, in order to
guarantee that you don't end up with a.2) You are willing to risk
going into a.1)
So, *obviously* v3.4 is more important than v3.3.x. I could argue that
the users out there would prefer a.1) any day, because it's unlikely
anyway (v3.4 would be good), but I won't.
All I want now is to agree on a reason, you have finally pointed out
that the reason for this different treatment is the user's visibility,
but that still doesn't explain why the rules for b) automatically
apply to a), since it's clearly different from the users's point of
view; if you agree that v3.4 is more important than v3.3.x, I believe
we have the reason right there.
Cheers.
[1] http://www.youtube.com/watch?v=kzKzUBLEzwk
--
Felipe Contreras
On Fri, Apr 13, 2012 at 4:42 PM, Stefan Richter
<[email protected]> wrote:
> On Apr 13 Felipe Contreras wrote:
>> On Fri, Apr 13, 2012 at 11:57 AM, Stefan Richter
>> <[email protected]> wrote:
>> > On Apr 12 Felipe Contreras wrote:
>> >> But this is exactly the opposite; the patch that broke things is in
>> >> the 'release branch' (3.3.1); it's not in upstream (3.3). Sure, it's
>> >> also on a later upstream, which is also broken.
>> > ^^^^^
>> > No, upstream /earlier/ than 3.3.1 contains the defect.
>>
>> Time is not relevant for the point being made, but fine:
>
> Time of occurrence of a regression in mainline and stable is indeed
> irrelevant, as there can only be two kinds of regressions in stable:
No. First of all, all regressions will happen first on mainline, and
then on stable. Maybe it's possible that a 3.3.x stable release
happens before a 3.4-rcx release that contains the patch, but which is
tagged first is irrelevant.
My point was that the patch is in 3.3.1, and 3.4-rc1, but not on 3.3.
The timing of these releases is irrelevant.
> A) First the regression was introduced into mainline, and accidentally it
> was carried over from there into stable.
>
> B) The regression only happened in stable because a backport from
> mainline to stable went wrong, e.g. a prerequisite to the backport
> was forgotten to be backported beforehand.
>
> AFAIU we are talking about a regression of type A.
>
> It seems you are arguing that stable candidate patches which fix
> regressions of type A should be treated differently from other stable
> candidate patches.
No, I'm not arguing that.
I am arguing that *any* patch in stable should be droppable, even
after it has been tagged.
>> But this is exactly the opposite; the patch that broke things is in
>> the 'release branch' (3.3.1); it's not in the upstream release from
>> where stable began (3.3). Sure, it's also on upstream, which is also
>> broken.
>
> (To what is this the opposite?)
>
> So the defect is present in two kernel branches: Linus'es and Greg's.
> The fix needs eventually go into both branches. For reasons which have
> been enumerated many times in this thread already, Greg takes the fix from
> Linus, not the other way around.
You can enumerate the reasons as many times as you want, that doesn't
make them any more valid.
I explain below how you are providing reasons for a different situation.
> If you do not like to wait for Linus and Greg, you simply have to derive
> an own kernel which additionally contains your preferred fixes.
Yes, because clearly everybody thinks the process is perfect, and
criticizing it is heresy.
> The reasons for the Linus->Greg order of maintaining the stable series 100%
> apply to fixes for type A regressions as well.
Why? Because it does. IOW; dogma.
>> So you are saying that:
>>
>> a) v3.3.1 (bad), v3.3.2 (bad), v3.4 (good)
>> b) v3.3.1 (bad), v3.3.2 (good), v3.4 (bad)
>> c) v3.3.1 (bad), v3.3.2 (good), v3.4 (good)
> [...]
>
> Not exactly.
>
> I and others are saying that procedures must ensure that if e.g. v3.3.3
> was "good", then v3.4 and hence v3.4.y must be "good" too. ("Good" here
> meaning "contains fix xyzabc".)
That's exactly what I said: if you are saying v3.4 *must* be good, you
are saying that b) *must* be avoided at all costs. Even if we end up
in a).
> Furthermore I was saying that due to the time-based instead of
> feature-based release schedule, the procedure which gives above guarantee
> is a time-based procedure: Greg takes fixes *if* they were published by
> Linus == *after* they were published by Linus.
>
> I add: The second reason for this procedure is that v3.x.y is a successor
> of v3.x but not of v3.x-1.y. Forward-porting from v3.x-1.y to v3.x.y is
> not in scope of the stable series. The reasons why there is no
> forward-porting done in stable series, again, have been mentioned several
> times in this thread.
Again, you can repeat the same thing as much as you want, it still
doesn't answer what I have asked.
>> a) v3.3.1 (bad), v3.3.2 (bad), v3.4 (good)
>> b) v3.3.1 (bad), v3.3.2 (good), v3.4 (bad)
>> c) v3.3.1 (bad), v3.3.2 (good), v3.4 (good)
Why is b) so much worst than c)?
Where is the evidence of that happening ever?
The truth is that b) is very unlikely to happen anyway, but you still
prefer a) to c), just to make sure that b) never *ever* happens, no
matter how unlikely.
The "reasons" explained are for an entirely different situation:
a') v3.3 (bad), v3.3.1 (bad), v3.3.2 (bad), v3.4 (good)
b') v3.3 (bad), v3.3.1 (bad), v3.3.2 (good), v3.4 (bad)
c') v3.3 (bad), v3.3.1 (bad), v3.3.2 (good), v3.4 (good)
You are providing the obvious answer, but to a question nobody asked.
I'm not talking about a', b', c', I'm talking about a, b, c, they are
*very* different.
I already exemplified how they are very different, but here it goes
again. The patch "drm/i915: Add lock on drm_helper_resume_force_mode"
was just tagged in 3.3.2, if I had said yesterday "this patch breaks
things on my machine", then that patch would have been dropped for
3.3.2 even though it's still on mainline--why? Because we know it's
broken, and broken patches are not supposed to land to stable. But
today, one day later, we have to wait until it's fixed in master
first. Why?
What makes a patch droppable yesterday, but dependent on mainline today?
This of course, has *not* been explained.
--
Felipe Contreras
On Sat, Apr 14, 2012 at 01:53:22AM +0300, Felipe Contreras wrote:
> No, I don't want the latest fixes, I want the latest *stable* kernel.
You have it, it's 3.3.2.
> v3.3 is stable, v3.4-rcx are not.
v3.3 *aims to be stable*. That's the big difference. A kernel starts
unstable and converges to something more stable over the time. 2.4 was
still experiencing minor issues that we didn't have in 2.6 when I did
the last release. There were even some security issues we decided not
to fix and to document instead. Still for its users it was considered
much more stable than 2.6 or 3.x.
> v3.4 would take months to cook,
> there will be several release candidates, and it won't be released
> until the known issues decrease to a reasonable level.
>
> v3.3.x on the other hand are *not* stable.
Nobody said they *are* stable as per the definition of this word.
"stable" is the name of the branch which defines the longterm goal
which is achieved as releases are issued. It's the same with longterm
kernels. They try to maintain even less bugs and try to to introduce
new ones, eventhough this happens from time to time, development is
not an exact science.
> They contain patches
> backported from v3.4, but nobody guarantees they will work. There was
> no v3.3.1-rc1,
Few people test rc, and not all bugs or regressions can be detected by
just a build and a boot.
> so the first time the patches compromising v3.3.1 were
> generally tested together is in v3.3.1, at which point if somebody
> finds issues, it's too late; bad patches are *not* going to be removed
> in v3.3.2.
They would have been if the issue was specific to the backport, which it
was not.
> Once a tag is made, all the patches in it are dependent on
> the pace of the development of mainline (v3.4-rcx), which is
> definitely not stable, specially in the first release candidates.
>
> IOW, the "stable" branch tries to be stable up to a point, then, it
> becomes a testing ground for mainline, and a tracking device for
> certain mainline issues.
No it's the opposite, it tries to be stable past one point. It's well
known that first stable releases still have a number of bugs, and it's
the reason why Greg takes time to maintain multiple branches in parallel.
Please stop playing the fool, everybody understands this and you make it
appear like bugs are put in stable on purpose, your reasoning really does
not make sense.
Please fork the kernel and maintain your own "morestable" branch, it will
be useful, really, because it will mean that whatever you have in your
branch that is not in stable has to be fixed in mainline, so it will hold
the information Linus wants not to lose. This is a lot of work but you'll
get it done without asking anyone else to do it. I'm not kidding, I'd
probably use it if it's maintained long enough.
Willy
Felipe Contreras wrote:
> With more people using Arch Linux and thus the latest "stable"
> release, I'd say we might see an increase in these kinds of issues.
You touch on an important point. Arch has it's own support channels,
and after a few of these kinds of issues perhaps Arch will decide to
push a different kernel to their users.
None of the Arch users I know (though only a few) would have had a
critical problem without wifi.
What kernel to push to users is anyway a distribution problem, so if
you must discuss then please discuss within the Arch community the
pros and cons of the various available kernel trees. Maybe they will
prefer another one than they currently do, after your reasoning.
I personally don't care about what any distribution does; I just use
Linus' tree, and I might merge some other trees if they have
important changes which I want before Linus takes them. When I did
this the first time is btw the time where I really learned to
appreciate just how powerful git is.
//Peter
On Fri, Apr 13, 2012 at 11:57 AM, Stefan Richter
<[email protected]> wrote:
> On Apr 12 Felipe Contreras wrote:
>> But this is exactly the opposite; the patch that broke things is in
>> the 'release branch' (3.3.1); it's not in upstream (3.3). Sure, it's
>> also on a later upstream, which is also broken.
> ^^^^^
> No, upstream /earlier/ than 3.3.1 contains the defect.
Time is not relevant for the point being made, but fine:
But this is exactly the opposite; the patch that broke things is in
the 'release branch' (3.3.1); it's not in the upstream release from
where stable began (3.3). Sure, it's also on upstream, which is also
broken.
> Furthermore, consider this: You as user of the 3.3.y series are using a
> temporary, dead-end side branch. Its maintenance will stop at some point,
> and you will be left looking for a different, maintained series to migrate
> to. You will be most interested in that series /not/ containing any
> regressions that you suffered already through the 3.3.y lifetime.
Of course, I will be interested in that, although most likely I would
be switching to another stable release (v3.4.1), not the upstream
release (v3.4), and most distros would do the same. Even in the
unlikely event that v3.4 is broken, most likely v3.4.1 would contain
the fix. But I'm also interested in v3.3.2 working.
So you are saying that:
a) v3.3.1 (bad), v3.3.2 (bad), v3.4 (good)
b) v3.3.1 (bad), v3.3.2 (good), v3.4 (bad)
c) v3.3.1 (bad), v3.3.2 (good), v3.4 (good)
b) is clearly better than a). Well, I don't see how; both situations
have the same number of releases broken, plus b) is very unlikely
anyway and we would end up with c). Plus, in all situation v3.4.1
would most likely contain the fix anyway.
> The rule is there to protect you, as a user of the stable series, from
> repeated regressions.
So in order to avoid b), you would rather go into a), than c)? Sorry,
I most definitely don't *need* that "protection". I guess I should
avoid the "stable" series then.
Cheers.
--
Felipe Contreras
* Willy Tarreau <[email protected]> wrote:
> It's not the user's visibility, it's published code. Once code
> is published, you cannot magically fix it without emitting a
> new patch for this code and announcing so that users apply it.
> These patches are called stable releases. Users want a good
> reason to apply these patches, rebuild and reboot, and that's
> one reason we absolutely want to have the commit descriptions
> from upstream which detail the exact reason for the patch
> (even if it's a revert).
Let me also outline why reverts are important and why they are
not just 'a patch removed'.
A revert carries valuable information: we revert a broken commit
not just with the terse description that it's broken, but with
an exact description about *why* it's broken and how that
breakage relates to the original patch.
In the limited 'patch space' nothing happened: a patch was done,
then undone.
In the Git 'commit space' the picture is very different: we have
a commit that does a change with a justification and we have
*another* commit that undoes the code modification with
*another* justification.
We got richer by two justifications and the kernel got
*more stable* in the process even though no actual code
changed:
- We very likely won't repeat this mistake again, it's
documented for eternity to any developer touching this
code in the future. See Linus's arguments about 'monotonic
progress'. Git hashes force the causality of the real
physical world on us, which is *especially* useful when we
make mistakes and would like to use a digital time machine
that whitewashes our shades of grey history.
- In practice developers tend to be more careful with code that
sees an above average ration of reverts. It's a red flag - it
shows that the code, the hardware or the development process
maintaining that code is somehow non-obvious and fragile for
one reason or another.
- In most cases a change+revert also spurs further development:
the original justification is likely valid and people want to
achieve that advantage but via some other, non-broken means.
In that sense the revert is a persistent TODO entry as well,
for developers.
If the bug was just a report somewhere in a mailing list archive
on the web then it would bitrot quickly and people would not
have a way to find it and react to it in an organized way.
By making it a Git commit; the change, the fix and all the
justifications around it become a permanent part of Linux -
which is more than just 15 million lines of code ...
Thanks,
Ingo
On Sun, Apr 15, 2012 at 9:51 AM, Ingo Molnar <[email protected]> wrote:
>
> * Felipe Contreras <[email protected]> wrote:
>> You are avoiding the argument you replying to; yesterday a
>> patch was droppable from the stable review queue, but today,
>> after the release, now we *need* it to stay there until it's
>> fixed in the mainline. What changed?
>
> What changed: the stable kernel was released and a new cycle
> started.
This is not a reason, this is just stating what happens without
explaining *why*.
Q: What changes when a tag is made?
A: A tag is made
> If something is broken in -stable it needs to be reverted upstream. Full stop.
And now you are describing the rule, without explaining the reason.
>> What makes a patch droppable yesterday, but dependent on
>> mainline today?
>
> Time and version release engineering: once a stable kernel is
> released any temporary integration testing results are flushed -
Yeah, but *why*? *Why* flush the testing results?
> the upstream kernel is where we maintain the information of
> which patch is broken and which not.
Sure, but that has nothing to do with the difference between today and
yesterday; both patches come from upstream. In both cases upstream
needs to be fixed.
> This memory-less process, amongst other things, helps the *next*
> major stable kernel become more robust, as it removes the
> possibility for version skew between stable and upstream.
If you keep the broken patch from yesterday (which is also from
upstream) and push it into the release, wouldn't that reduce the
version skew between stable and upstream as well?
Why is it that the patch from yesterday doesn't reduce the version
skew, but the patch from today does? You are not explaining *why*.
> If you want a revert you either need to report your problem fast
> enough to be caught in -stable integration testing, or you need
> to work with upstream to push the fix through properly.
And now you are assuming your assumption is right, and go right onto
the conclusion.
> People explained this to you, again and again, you refused to
> listen, repeatedly, again and again. There does not appear to be
> any rational set of arguments that will make you admit that you
> were wrong about this.
I'm sure if I try to argue rationally why hunting is bad with hunters,
they'll say I've refused to listen to all their rational set of
arguments. It's impossible that you are not explaining the *reason*
(there is none), it's impossible that you are being affected by herd
mentality, overconfidence, and confirmation bias. No, it cannot be any
of those things.
> Anyway, in this discussion you have demonstrated it again why
> you are one of the very few commenters I had to permanently
> block on G+ ...
Please, I provided irrefutable evidence for my argument, and I said I
would stop if you wanted me to, nobody forced you to continue the
discussion. You still blocked me even though you could have asked me
not to reply.
I can stop the discussion, I told so to Linus Torvalds on G+, if I'm told so.
I guess that would make you happy, but that doesn't mean you are
right. You are tip-toeing around the question and *never* giving a
real reason for why the patch from yesterday is different from the one
today.
It's like I'm asking why the freezer area is different from the normal
refrigerator area, and you are saying; well it's colder, it's usually
smaller, and it makes all food taste better. So you are either just
describing the differences without explaining the reason (being colder
helps for some foods), or you are providing an unsubstantiated claim;
you go from a) it's colder, to b) it makes all food taste better. And
when asked *why* again, you repeat, well, because it's colder. And
then you immediately go to the conclusion; therefore you should put as
much food in the freezer as possible.
Of course you would think all your answers are obvious and reasonable,
what else would you think? You, like everybody else (including me),
has a bias blind spot.
The fact of the matter is that there's *no reason* why the patch from
yesterday can be dropped, and the patch today can't. A tag was made,
but it doesn't change the nature of the patch; either way the patch is
bad, either way the patch is still in upstream.
You assume (correctly) that by keeping it in the stable branch, it
would guarantee it will be fixed in upstream sooner, but do we *need*
this guarantee? Would you buy a total guarantee for your house, if it
costs as much as the house itself? No, you need to look at the costs,
and the likelihood of the event you are guaranteeing against
happening, not just go "Oh!, guarantee! gimme!". The fact is that
without this guarantee, the fix will happen in a reasonable time in
upstream anyway. People have explained that in the past there have
been situations where fixes are lost, but that's an entirely different
situation (..v3.3), we are talking about stable reverts
(v3.3...v3.3.1), I challenged David Miller to find evidence for this
*ever* happening, and of course I didn't receive any answer. So no,
there's no evidence for that happening, you just think it will, let
alone how often will that be. You are making these patches guilty by
association. Plus, as I argued, somehow the 10000 of patches in queue
for the next stable release don't need this guarantee, so what's so
bad about making the patch from today one more of them?
And even more, even supposing we want this guarantee for the patch
from today, that doesn't explain why we don't want it from the patch
from yesterday. Again, this difference has *never* been explained. You
keep explaining why we want the guarantee, but never explain we why
don't want/need it for the patch from yesterday.
So you say bad patch should obviously not make it into stable:
Ingo: "if a -stable commit does not even apply or does not even boot
on Greg's box or on the handful of boxes that test stable release
candidates then it obviously cannot become part of the -stable queue."
And when asking why not get the same guarantee for the bad patch from yesterday:
Stefan: "That would be even sillier than the discussion which we are having."
When pressed about why it's sillier for the yesterday patch, but not
from today, Stefan Richter didn't reply, of course.
You are backed into a corner, you can't provide a good reason why you
treat them differently, so you are either going to keep describing the
difference, or you are going to keep explaining why the guarantee is
good, but not why B needs it, and A doesn't; because you don't have a
reason.
And when backed into a corner you are going to do what everybody does,
not stop for a second, and think deeply about the difference, but you
are going to use your bias blind spot, and assume that of course you
are right, and of course you are being reasonable, and of course your
arguments are flawless, and there's no more reason to discuss.
Accepting the possibility that there's no good reason for the
different treatment is not an option, of course, that would mean you
were wrong, and you can't.
FTR. I accept I might be wrong, and there's a good reason for this
different treatment, but I haven't seen any.
Cheers.
--
Felipe Contreras
On Thu, Apr 12, 2012 at 4:13 AM, Greg KH <[email protected]> wrote:
> On Thu, Apr 12, 2012 at 04:03:59AM +0300, Felipe Contreras wrote:
>> On Thu, Apr 12, 2012 at 3:29 AM, Greg KH <[email protected]> wrote:
>> > On Wed, Apr 11, 2012 at 07:59:14PM -0400, Sergio Correia wrote:
>> >> Hello Greg,
>> >>
>> >>
>> >> On Wed, Apr 11, 2012 at 7:11 PM, Greg KH <[email protected]> wrote:
>> >> > This is the start of the stable review cycle for the 3.3.2 release.
>> >> > There are 78 patches in this series, all will be posted as a response
>> >> > to this one. If anyone has any issues with these being applied, please
>> >> > let me know.
>> >> >
>> >> > Responses should be made by Fri Apr 13 23:10:16 UTC 2012.
>> >> > Anything received after that time might be too late.
>> >> >
>> >>
>> >> is there any chance for this one to be included in this review cycle?
>> >>
>> >> http://www.spinics.net/lists/linux-wireless/msg87999.html
>>
>> I was going to ask for exactly the same thing. My system is completely
>> unusable without this patch; not only does the network doesn't work,
>> but quite often the kernel is stuck consuming 100% of the CPU.
>>
>> > Have you read Documentation/stable_kernel_rules.txt? Based on that, I
>> > don't think it can, yet, right?
>>
>> Why not? This patch makes the code go back to a previous state, it is
>> obviously more stable than the current state, and the code already
>> exists in Linus's tree (in previous releases).
>
> It does? What is the git commit id of the patch? Based in the email
> above, I assumed it had not made it to Linus's tree already.
It's a revert of c1afdaff90538ef085b756454f12b29575411214, so so just
take a look at the code in c1afdaff90538ef085b756454f12b29575411214^.
>> But hey, I guess it's OK that 3.3.x is stuck in and endless loop right
>> after booting, because rules are more important than fixing obvious
>> breakage.
>
> What rule did you think I was saying this was not acceptable for?
The fact that the patch as not been applied/reviewed/accepted upstream.
Personally I don't see what is the problem with reverts; we already
know the previous code was working. Sure, in theory it might behave
different due to other changes, but that doesn't seem to be the case
here, plus, it can't be worst than the current situation of staying in
an endless loop.
It appears in 3.4 there are more issues, so the fix there might look
completely different, but regardless, the real issue is that the
proper fix is not yet here.
So the question is do we want 3.3.2 to be completely broken for these
machines, or not?
--
Felipe Contreras
On Sat, Apr 14, 2012 at 06:29:54PM +0300, Felipe Contreras wrote:
> So, the hypothetical patch that was dropped in the stable review queue
> yesterday has to be fixed in mainline too, just like the hypothetical
> patch that made it to stable and was found problematic today has to be
> fixed in mainline.
Patches are not always dropped from the stable queue if we know they're
causing issues, sometimes they're left pending in the queue. This is how
Greg is able to ping developers from time to time.
> Again, what makes a released patch undroppable?
Being applied, in other words, having a commit ID in the branch.
(...)
> Just by going through the stable review process the patch already
> received more eyes to make the bugs more shallow. There might be a lot
> of trees out there where people pick patches from mainline, and they
> don't *need* to follow the "upstream first" rule, by reviewing the
> patch, and maybe even testing it, and then reporting the problem, they
> are helping getting it fixed for v3.4.
They do what they want with their trees. I have a number of patches in my
trees that are not worth pushing to mainline and that's fine. However the
rule is that the official stable tree has to hold patches that come from
mainline.
> You don't *need* to keep the patch in the stable review queue, you
> don't *need* to make a stable release with it in order to ensure that
> it will fixed in mainline, the information about the patch problems is
> not lost.
It's lost since nobody cares once their kernel is fixed. For instance, I
have an issue with 3.0.x on my netbook which I haven't had time to bisect
(no resume from s2r). But 3.2 works. If I find that the issue was introduced
in any stable backport, we'll have to ensure that mainline has the fix,
because the bug might simply be hidden in 3.2 but still present. If we'd
just revert the fix from 3.0, what would motivate anyone to fix mainline
if it appears to work ?
> Seems to me you are abusing the 'stable' branch as a bug tracking
> system for certain patches for the next release.
The most reliable bug tracking system are the end users. They're the only
one who remind you to get their fix merged. And you did this yourself,
quite louder than the average I'd say, but it worked perfectly again.
Willy
On Fri, Apr 13, 2012 at 12:34 AM, Linus Torvalds
<[email protected]> wrote:
> On Thu, Apr 12, 2012 at 2:20 PM, Felipe Contreras
> <[email protected]> wrote:
>>
>> I could argue in favor of exceptions, but I don't think you realize
>> the fact that this change does not affect your tree *at all*. Adding
>> and removing a patch in the stable tree is a no-op.
>
> You're a fucking moron.
>
> It's not a no-op at all, and you don't seem to understand it.
>
> It's *information*.
>
> It's "that patch didn't work". That's not a no-op. That's actual
> useful and worthwhile knowledge.
Sure, but removing that patch from the stable tree is not going the
change that information; we already know the patch is wrong.
Let's say somebody finds something wrong with one of the patches
proposed for 3.3.2 today, which is still a possibility. The patch
would be dropped, even though it's already in upstream (as all stable
patches are), and development in upstream will continue as usual, and
a proper fix will come later--there's lots of stuff broken there,
which is why not all the patches make it to 3.3.2.
But if somebody finds a problem on Saturday, after the 3.3.2 release,
well, it's too late now, the patch has been tagged and cannot be
removed for 3.3.3, now we have to wait to see what upstream does.
Wrong is wrong, before or after the 3.3.1 tag, this patch is not
'stable' material, and removing it does not affect upstream at all.
--
Felipe Contreras
On Tue, Apr 17, 2012 at 12:27 AM, Greg KH <[email protected]> wrote:
> On Tue, Apr 17, 2012 at 12:18:13AM +0300, Felipe Contreras wrote:
>> On Mon, Apr 16, 2012 at 11:58 PM, Greg KH <[email protected]> wrote:
>> > On Mon, Apr 16, 2012 at 11:11:05PM +0300, Felipe Contreras wrote:
>> >> On Mon, Apr 16, 2012 at 7:27 PM, Greg KH <[email protected]> wrote:
>> >> > Just one minor correction in this looney email thread:
>> >> >
>> >> > On Sat, Apr 14, 2012 at 01:53:22AM +0300, Felipe Contreras wrote:
>> >> >> v3.3.x on the other hand are *not* stable. They contain patches
>> >> >> backported from v3.4, but nobody guarantees they will work. There was
>> >> >> no v3.3.1-rc1, so the first time the patches compromising v3.3.1 were
>> >> >> generally tested together is in v3.3.1, at which point if somebody
>> >> >> finds issues, it's too late; bad patches are *not* going to be removed
>> >> >> in v3.3.2.
>> >> >
>> >> > Of course there was a 3.3.1-rc1, see the linux-kernel archives for the
>> >> > announcemen and the individual patches. kernel.org has the large patch
>> >> > itself if you like that format instead.
>> >>
>> >> I don't see it here:
>> >> http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=tags
>> >>
>> >> If you really want people to try it, why not tag it?
>> >
>> > That would be because I don't keep it in that tree. It is in a quilt
>> > tree you can find in the stable-queue.git repo, and I have never tagged
>> > -rc1 releases there. No one has ever asked for it before, so in the
>> > past 6 years of stable releases, I guess no one ever needed it.
>> >
>> > ketchup and tarballs seem to work well for others, perhaps you can use
>> > that as well (hint, ketchup on top of the linux-stable tree works just
>> > fine for testing this.)
>>
>> Perhaps the current process will be continue to be OK, but I do
>> believe a tagged v3.3.1-rc1 would have catched the ath9k issue.
>
> How exactly would that have helped here?
More people would have given it a try. Not that many people read the
mailing list, and the ones that do certainly might want to avoid
applying a big series of patches; even if their mail client makes it
easy (mine (Gmail) doesn't). A tag, and an announcement to give a try
would make it *much* easier.
> You point out:
>
>> I used to compile my own kernels and use your stable tree, but this a
>> new laptop and I was using Arch Linux which automatically updated to
>> v3.3.1, and with no network I had no way to revert to v3.2.x.
>> Fortunately I had the kernel sources available, but I wonder how many
>> people were completely stuck.
>
> Arch wouldn't have included a -rc in their kernel (unless they are
> crazy), so this would not have helped your situation at all.
There's *a lot* of people that got affected by the 3.3.1 release; we
don't need to break a lot of boxes to figure out there's an issue,
only a few would suffice, even one.
>> If some other 3.x.1 release get broken this way, I would seriously
>> consider tagging v3.x.1-rc1 as well. It works for Linus' tree.
>
> "this way" was for a very tiny subset of hardware, so odds are, if this
> happens again, it wouldn't be caught this way either. That subset just
> happened to show up in your machine, but, for example, not in the wide
> range of hardware I test with here, nor the machines that others test
> with.
It was certainly not just me. I have seen a lot of people mentioning
"their wifi is broken", a lot of them using Arch Linux,
coincidentally.
These issues would most likely not be caught before v3.x.1, and which
point it's too late, they cannot be reverted to v3.x.2 just like that;
they have to wait for upstream. Hopefully and probably everything
would go smooth like this time, but maybe not, we'll have to wait and
see. With more people using Arch Linux and thus the latest "stable"
release, I'd say we might see an increase in these kinds of issues.
Cheers.
--
Felipe Contreras
* Felipe Contreras <[email protected]> wrote:
> On Fri, Apr 13, 2012 at 1:07 AM, Linus Torvalds
> <[email protected]> wrote:
> > On Thu, Apr 12, 2012 at 3:04 PM, Felipe Contreras
> > <[email protected]> wrote:
> >>
> >> Sure, but removing that patch from the stable tree is not
> >> going the change that information; we already know the
> >> patch is wrong.
> >
> > .. and we wait until it has been fixed in mainline so that
> > we *know* that information doesn't get lost.
>
> So why don't we pick potentially dangerous patches that might
> benefit from some testing, put them in 'stable', and if there
> are problems, make sure they get fixed in upstream first?
>
> Or for that matter totally broken patches we want to make sure
> they get fixed in upstream.
>
> Because the priority of the 'stable' tree is *stability*. Is
> it not?
>
> But what you are saying is: *before* the final review, even a
> hint that the patch might cause problems is reason enough to
> drop it from stable, but *after* the review, if we know the
> patch is totally broken, then it's the opposite; we really
> want it in.
What you don't seem to understand is that there are good reasons
why we first fix bugs upstream, then in -stable. Greg explained
it to you, Linus explained it to you and so did many others.
Having an order of patches *necessarily* means that the
development tree will get fixes sooner than the stable tree. In
other words, this *necessarily* means that the stable tree - and
its users - will have to wait a little bit more to have the fix.
In the worst-case this 'have to wait a little bit longer' might
span the time between two minor stable kernel releases.
You seem to equate this 'have to wait a little bit longer to get
the fix' property of the maintenance model with 'we don't care
about stable tree users' - that claim is obviously idiotic and
most of your arguments in this thread are idiotic as well.
Thanks,
Ingo
On Fri, Apr 13, 2012 at 1:12 AM, David Miller <[email protected]> wrote:
> From: Felipe Contreras <[email protected]>
> Date: Fri, 13 Apr 2012 01:04:42 +0300
>
>> Wrong is wrong, before or after the 3.3.1 tag, this patch is not
>> 'stable' material, and removing it does not affect upstream at all.
>
> What you don't understand is that bug fixes will get lost if you only
> fix them in -stable, it doesn't matter HOW THEY GOT into -stable.
Let's suppose that c1afdaf was never back-ported from v3.4-rc1, how
would you have fond out there was an issue with it? There's 10000
patches in v3.4-rc2, how do you expect to find issues in them?
People found out this issue on v3.4-rc1, so the fix would not have
been lost, but lets assume it would, v3.3.1 had the issue, the patch
as reverted in v3.3.2, and v3.4 still had the issue. So what? There's
already 10000 patches that would never make it to 3.3.x, and many will
have issues, which is why there would be v3.4.x.
> In fact IT HAS FUCKING HAPPENED that we didn't fix something upstream
> that got fixed in -stable a time long ago when we didn't have the
> policy we're using now which you're going so unreasonably ape-shit
> about.
I see how a *fix* on stable could get lost, but this is not a fix.
A lost fix would be like:
v3.3 (bad), v3.3.1 (good), v3.4 (bad)
But the worst-case scenario here would be:
v3.3 (good), v3.3.1 (bad), v3.3.2 (good), v3.4 (bad)
You said this has happened in the past, I challenge you to find a
single instance of this case: a patch is applied and reverted in
'stable', and the issue triggered by the patches was not fixed in the
next version.
--
Felipe Contreras
On Apr 14 Felipe Contreras wrote:
> On Sat, Apr 14, 2012 at 10:41 AM, Stefan Richter
> <[email protected]> wrote:
> > On Apr 14 Felipe Contreras wrote:
> >> I already exemplified how they are very different, but here it goes
> >> again. The patch "drm/i915: Add lock on drm_helper_resume_force_mode"
> >> was just tagged in 3.3.2, if I had said yesterday "this patch breaks
> >> things on my machine", then that patch would have been dropped for
> >> 3.3.2 even though it's still on mainline--why? Because we know it's
> >> broken, and broken patches are not supposed to land to stable. But
> >> today, one day later, we have to wait until it's fixed in master
> >> first. Why?
> >>
> >> What makes a patch droppable yesterday, but dependent on mainline today?
> >
> > Huh?
> >
> > Because "yesterday" was a review before stable release:
> > - A buggy mainline release exists.
> > - No buggy stable release exists.
> > whereas "today" is after stable release:
> > - A buggy mainline release exists.
> > - A buggy stable release exists.
>
> IOW; a tag makes undroppable.
Generally, "commit + push out" makes it undroppable. In case of -stable,
commit/ push out/ tag are close and virtually identical.
After a change was pushed out, the choice narrows down to add a reverting
change for a subsequent release or to rebase to a point before the
defective commit. The latter is not done to -stable, obviously. (3.3.2
is not being restarted from 3.3 if a regression from 3.3 to 3.3.1 was
discovered.)
> But *why*? You say you *really* need to problem to fixed in mainline,
> that's why it cannot be dropped, but that applies also to patches in
> the queue *before* the tag is made, doesn't it? If you find a patch in
> the stable review queue causes problems, why don't you leave it there?
As Willy wrote, that's actually what is done sometimes. I didn't know
that.
> You *really* need to problem fixed in mainline, don't you?
>
> So yesterday the priority is stability > 'upstream first', but today
> it's 'upstream first' > stability. Nobody has put forward an argument
> for this shift in priorities--
Yesterday, folks cared about mainline too.
> "a tag was made" is not argument.
"A faulty commit was pushed out" means that a defective release was made
and a fix needs to be created and released. This is obviously something
different from rejecting to commit a submitted change after negative
review.
Sure, negative review of a stable submission consequently means that
mainline needs to be checked whether it requires a fix, and if yes, stable
users will be interested in getting that fix really into the mainline
rather sooner than later. So suddenly they have to track a mainline bug.
Still /one/ bug though.
So that is when a stable submission was dropped "yesterday". Now what
about if a defective change was not dropped but released, and now requires
a fix (e.g. revert) "today: There are now /two/ bugs: In the mainline
and in a stable kernel.
If you could convince stable to fix their bug before mainline does, then
you would have to track that other mainline bug too. (You don't wan the
next branch point for stable to be faulty.) Plus, either the fix from
stable needs to be forward-ported to mainline, or worse, two independent
fixes need to be developed.
What is actually done is simpler and less error prone:
- Develop a fix for mainline.
- Mark that fix Cc: stable.
- Have that fix backported into stable.
[...]
> > "Drop a stable candidate before release" is a form of "decline a
> > submission to stable", happening late in the preparations of a stable
> > release.
> >
> > The latter is when damage was done; it is now about bug fixing. This
> > involves debugging of the regression, finding a right approach to
> > fix it (e.g. revert), write + review + test + commit the fix, port the fix
> > to all relevant other kernel branches, review + test + commit those ports
> > too.
>
> Really? So if the patch doesn't make it to stable you don't need to do
> any of that?
If the bug dosn't go into stable, you don't have to track and fix two
bugs. You still have to track and fix a bug, but it is only one bug
instead of two.
> IOW; if the patch doesn't make it to stable, it's OK to
> leave it broken for v3.4? There's 10000 patches in v3.4-rc* that are
> all about bug fixing, they don't need to go into stable to be fixed,
> do they? If a non-stable patch needs to be reverted in mainline, it
> gets reverted in mainline, regardless of weather or not it made it to
> stable or not.
>
> So, the hypothetical patch that was dropped in the stable review queue
> yesterday has to be fixed in mainline too,
I count one bug.
> just like the hypothetical
> patch that made it to stable and was found problematic today has to be
> fixed in mainline.
I count two bugs.
> Again, what makes a released patch undroppable? It's not the bug
> fixing; weather or not it gets into stable, it still has to be fixed
> in mainline.
If it is released, you can't drop, only revert or rebase to an older
origin. (Well, to rebase onto older origin actually means to drop, though
not just that one commit but also all its successors.)
[...]
> Seems to me you are abusing the 'stable' branch as a bug tracking
> system for certain patches for the next release.
There is no abuse.
If a regression happens in stable, you always need to figure out whether
this is only a problem in stable or also in mainline (because mainline
will become the origin of a new stable series eventually, and the problem
should not reappear int the new stable series). If the problem is only a
stable problem, just fix it in stable. If it is also a mainline problem,
you want to have it fixed both in stable and in mainline (because their
mainline will become your stable in a new series).
Now there are three choices:
- Develop a mainline fix, backport it to stable.
- Develop a stable fix, forward-port it to mainline.
- Develop the two fixes independently.
A variation of the second and third choice: Develop a stable fix, leave
mainline unfixed for now, forward-port the stable fix from 3.M.y to 3.N.y,
until mainline is receiving the forward-port too or got an independently
developed fix.
Distributors routinely do any of the three choices. Greg does only the
first one in stable: No forward-ports, only backports. I wrote about the
downsides of forward-ports in the other post.
There is an obvious downside to backports-only too: The stable fix is
delayed until after mainline fix. This one downside seems to have
inspired this huge mailinglist thread... But you as a user of stable can
compensate for this delay in some ways: You could switch back to a stable
release before the regression, you could apply a fix locally on top of the
regressed stable, or you could find a third party kernel which contains
such local fixes.
Sure, any of these options is a nuisance. But these nuisances for end
users will still probably not change how stable is maintained. Instead,
that forward-ports are out of scope of stable is one reason why you are
getting so frequent stable releases. So you typically don't have to wait
for a regression fix in stable for unbearingly long. No forward-ports in
stable also means lower likelyhood of stable-only regressions.
--
Stefan Richter
-=====-===-- -=-- -===-
http://arcgraph.de/sr/
On Sat, Apr 14, 2012 at 1:47 PM, Ingo Molnar <[email protected]> wrote:
>
> * Felipe Contreras <[email protected]> wrote:
>
>> On Fri, Apr 13, 2012 at 1:07 AM, Linus Torvalds
>> <[email protected]> wrote:
>> > On Thu, Apr 12, 2012 at 3:04 PM, Felipe Contreras
>> > <[email protected]> wrote:
>> >>
>> >> Sure, but removing that patch from the stable tree is not
>> >> going the change that information; we already know the
>> >> patch is wrong.
>> >
>> > .. and we wait until it has been fixed in mainline so that
>> > we *know* that information doesn't get lost.
>>
>> So why don't we pick potentially dangerous patches that might
>> benefit from some testing, put them in 'stable', and if there
>> are problems, make sure they get fixed in upstream first?
>>
>> Or for that matter totally broken patches we want to make sure
>> they get fixed in upstream.
>>
>> Because the priority of the 'stable' tree is *stability*. Is
>> it not?
>>
>> But what you are saying is: *before* the final review, even a
>> hint that the patch might cause problems is reason enough to
>> drop it from stable, but *after* the review, if we know the
>> patch is totally broken, then it's the opposite; we really
>> want it in.
>
> What you don't seem to understand is that there are good reasons
> why we first fix bugs upstream, then in -stable. Greg explained
> it to you, Linus explained it to you and so did many others.
>
> Having an order of patches *necessarily* means that the
> development tree will get fixes sooner than the stable tree. In
> other words, this *necessarily* means that the stable tree - and
> its users - will have to wait a little bit more to have the fix.
> In the worst-case this 'have to wait a little bit longer' might
> span the time between two minor stable kernel releases.
>
> You seem to equate this 'have to wait a little bit longer to get
> the fix' property of the maintenance model with 'we don't care
> about stable tree users' - that claim is obviously idiotic and
> most of your arguments in this thread are idiotic as well.
This is a straw man again. Again; we are not talking about fixes in
'stable' that don't exist in mainline, we are talking about reverting
patches from 'stable' that are not part of the upstream release from
where the 'stable' branch was forked.
You are avoiding the argument you replying to; yesterday a patch was
droppable from the stable review queue, but today, after the release,
now we *need* it to stay there until it's fixed in the mainline. What
changed?
What makes a patch droppable yesterday, but dependent on mainline today?
--
Felipe Contreras
On Sat, Apr 14, 2012 at 6:57 PM, Willy Tarreau <[email protected]> wrote:
> On Sat, Apr 14, 2012 at 06:29:54PM +0300, Felipe Contreras wrote:
>> So, the hypothetical patch that was dropped in the stable review queue
>> yesterday has to be fixed in mainline too, just like the hypothetical
>> patch that made it to stable and was found problematic today has to be
>> fixed in mainline.
>
> Patches are not always dropped from the stable queue if we know they're
> causing issues, sometimes they're left pending in the queue. This is how
> Greg is able to ping developers from time to time.
That's news to me, but the important point remains; they don't make it
into the stable release, correct? Yet, we still expect them to be
fixed in mainline, correct?
>> Again, what makes a released patch undroppable?
>
> Being applied, in other words, having a commit ID in the branch.
Seriously? That's your reason?
Hey, thousands of users out there; the reason why we pushed a patch
that is known to be broken in v3.3.x is because it already has a
commit ID.
If that's your idea of a good reason then I don't see any point in
discussing with you any more. No offense intended.
Cheers.
--
Felipe Contreras
On Thu, Apr 12, 2012 at 09:43:33PM +0300, Felipe Contreras wrote:
> On Thu, Apr 12, 2012 at 8:24 PM, Adrian Chadd <[email protected]> wrote:
> > On 12 April 2012 09:49, Felipe Contreras <[email protected]> wrote:
> >
> >>>
> >>> A revert is the same as a patch. ?It needs to be in Linus's tree before
> >>> I can add it to the stable releases.
> >>
> >> Right, because otherwise people's systems would actually work.
> >>
> >> But hey, as I said, following rules is more important, regardless of
> >> what the rules are, and why they are there. The rules that actually
> >> triggered this issue in v3.3.1, as this is not in v3.3.
> >>
> >> You could just accept that the patch should have never landed in
> >> v3.3.1 in the first place, but it's much easier to arbitrarily keep
> >> stacking patches without thinking too much about them.
> >
> > Greg is doing the right thing here. We face the same deal in FreeBSD -
> > people want fixes to go into a release branch first, but if you do
> > that you break the development flow - which is "stuff goes into -HEAD
> > and is then backported to the release branches."
> >
> > If you don't do this, you risk having people do (more, all)
> > development and testing on a release branch and never test -HEAD (or
> > "upstream linux" here). Once you open that particular flood gate, it's
> > hard to close.
>
> But this is exactly the opposite; the patch that broke things is in
> the 'release branch' (3.3.1); it's not in upstream (3.3). Sure, it's
> also on a later upstream, which is also broken.
What is the git commit id of the patch in 3.3.1 that caused this to
break? This is the first time I have heard that 3.3 worked and 3.3.1
did not work. Someone needs to tell me these things...
greg k-h
On 04/16/2012 02:50 PM, Felipe Contreras wrote:
> On Tue, Apr 17, 2012 at 12:27 AM, Greg KH<[email protected]> wrote:
>> On Tue, Apr 17, 2012 at 12:18:13AM +0300, Felipe Contreras wrote:
>>> On Mon, Apr 16, 2012 at 11:58 PM, Greg KH<[email protected]> wrote:
>>>> On Mon, Apr 16, 2012 at 11:11:05PM +0300, Felipe Contreras wrote:
>>>>> On Mon, Apr 16, 2012 at 7:27 PM, Greg KH<[email protected]> wrote:
>>>>>> Just one minor correction in this looney email thread:
>>>>>>
>>>>>> On Sat, Apr 14, 2012 at 01:53:22AM +0300, Felipe Contreras wrote:
>>>>>>> v3.3.x on the other hand are *not* stable. They contain patches
>>>>>>> backported from v3.4, but nobody guarantees they will work. There was
>>>>>>> no v3.3.1-rc1, so the first time the patches compromising v3.3.1 were
>>>>>>> generally tested together is in v3.3.1, at which point if somebody
>>>>>>> finds issues, it's too late; bad patches are *not* going to be removed
>>>>>>> in v3.3.2.
>>>>>>
>>>>>> Of course there was a 3.3.1-rc1, see the linux-kernel archives for the
>>>>>> announcemen and the individual patches. kernel.org has the large patch
>>>>>> itself if you like that format instead.
>>>>>
>>>>> I don't see it here:
>>>>> http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=tags
>>>>>
>>>>> If you really want people to try it, why not tag it?
>>>>
>>>> That would be because I don't keep it in that tree. It is in a quilt
>>>> tree you can find in the stable-queue.git repo, and I have never tagged
>>>> -rc1 releases there. No one has ever asked for it before, so in the
>>>> past 6 years of stable releases, I guess no one ever needed it.
>>>>
>>>> ketchup and tarballs seem to work well for others, perhaps you can use
>>>> that as well (hint, ketchup on top of the linux-stable tree works just
>>>> fine for testing this.)
>>>
>>> Perhaps the current process will be continue to be OK, but I do
>>> believe a tagged v3.3.1-rc1 would have catched the ath9k issue.
>>
>> How exactly would that have helped here?
>
> More people would have given it a try. Not that many people read the
> mailing list, and the ones that do certainly might want to avoid
> applying a big series of patches; even if their mail client makes it
> easy (mine (Gmail) doesn't). A tag, and an announcement to give a try
> would make it *much* easier.
>
>> You point out:
>>
>>> I used to compile my own kernels and use your stable tree, but this a
>>> new laptop and I was using Arch Linux which automatically updated to
>>> v3.3.1, and with no network I had no way to revert to v3.2.x.
>>> Fortunately I had the kernel sources available, but I wonder how many
>>> people were completely stuck.
>>
>> Arch wouldn't have included a -rc in their kernel (unless they are
>> crazy), so this would not have helped your situation at all.
>
> There's *a lot* of people that got affected by the 3.3.1 release; we
> don't need to break a lot of boxes to figure out there's an issue,
> only a few would suffice, even one.
>
>>> If some other 3.x.1 release get broken this way, I would seriously
>>> consider tagging v3.x.1-rc1 as well. It works for Linus' tree.
>>
>> "this way" was for a very tiny subset of hardware, so odds are, if this
>> happens again, it wouldn't be caught this way either. That subset just
>> happened to show up in your machine, but, for example, not in the wide
>> range of hardware I test with here, nor the machines that others test
>> with.
>
> It was certainly not just me. I have seen a lot of people mentioning
> "their wifi is broken", a lot of them using Arch Linux,
> coincidentally.
>
> These issues would most likely not be caught before v3.x.1, and which
> point it's too late, they cannot be reverted to v3.x.2 just like that;
> they have to wait for upstream. Hopefully and probably everything
> would go smooth like this time, but maybe not, we'll have to wait and
> see. With more people using Arch Linux and thus the latest "stable"
> release, I'd say we might see an increase in these kinds of issues.
>
> Cheers.
>
on an arch install only a couple months old with one cache cleaning in
the beginning. the results of packages I could go back to on a bad update.
/var/cache/pacman/pkg $ ls -1 /var/cache/pacman/pkg/linux-*
/var/cache/pacman/pkg/linux-3.2.11-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-3.2.1-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-3.2.12-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-3.2.1-2-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-3.2.13-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-3.2.14-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-3.2.2-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-3.2.4-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-3.2.5-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-3.2.6-2-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-3.2.7-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-3.2.8-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-3.2.9-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-3.3.1-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-3.3.2-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-api-headers-3.1.6-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-api-headers-3.3-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-docs-3.2.11-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-docs-3.2.1-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-docs-3.2.12-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-docs-3.2.1-2-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-docs-3.2.13-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-docs-3.2.14-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-docs-3.2.2-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-docs-3.2.4-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-docs-3.2.5-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-docs-3.2.6-2-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-docs-3.2.7-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-docs-3.2.8-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-docs-3.2.9-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-docs-3.3.1-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-docs-3.3.2-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-firmware-20111101-1-any.pkg.tar.xz
/var/cache/pacman/pkg/linux-firmware-20120205-1-any.pkg.tar.xz
/var/cache/pacman/pkg/linux-firmware-20120227-1-any.pkg.tar.xz
/var/cache/pacman/pkg/linux-headers-3.2.11-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-headers-3.2.1-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-headers-3.2.12-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-headers-3.2.1-2-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-headers-3.2.13-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-headers-3.2.14-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-headers-3.2.2-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-headers-3.2.4-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-headers-3.2.5-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-headers-3.2.6-2-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-headers-3.2.7-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-headers-3.2.8-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-headers-3.2.9-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-headers-3.3.1-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-headers-3.3.2-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-lts-3.0.17-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-lts-3.0.17-2-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-lts-3.0.19-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-lts-3.0.20-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-lts-3.0.27-1-x86_64.pkg.tar.xz
/var/cache/pacman/pkg/linux-lts-headers-3.0.27-1-x86_64.pkg.tar.xz
yes this lists non relavent packages but shows that it is YOU who does
not know your distro well enough.
Give up. I am done.. and Felipe NO MORE PRIVATE EMAILS! keep your
conversations public unless someone asks you to contact them off list
Felipe Contreras wrote:
> But then are you saying that if upstream is broken (3.4-rc2), then
> stable should be broken as well (3.3.1), and remain broken until
> upstream is fixed? I fail to see what would be the point of that.
No, he's saying that when upstream is broken for the same reason as
stable is, it seems wise to:
- report upstream
- fix your local system
- fix any systems you are responsible for
- fix upstream
- only then fix stable.
That way, the user doesn't get the disconcerting experience of getting
and then losing the fix when upgrading first to the next stable
version, then to the next upstream version.
Makes sense?
Hope that helps,
Jonathan
On Wed, Apr 11, 2012 at 8:29 PM, Greg KH <[email protected]> wrote:
> On Wed, Apr 11, 2012 at 07:59:14PM -0400, Sergio Correia wrote:
>> Hello Greg,
>>
>>
>> On Wed, Apr 11, 2012 at 7:11 PM, Greg KH <[email protected]> wrote:
>> > This is the start of the stable review cycle for the 3.3.2 release.
>> > There are 78 patches in this series, all will be posted as a response
>> > to this one. ?If anyone has any issues with these being applied, please
>> > let me know.
>> >
>> > Responses should be made by Fri Apr 13 23:10:16 UTC 2012.
>> > Anything received after that time might be too late.
>> >
>>
>> is there any chance for this one to be included in this review cycle?
>>
>> http://www.spinics.net/lists/linux-wireless/msg87999.html
>
> Have you read Documentation/stable_kernel_rules.txt? ?Based on that, I
> don't think it can, yet, right?
>
I hadn't, but I have now, thanks for the pointer. And yes, you are right.
thanks,
sergio
> thanks,
>
> greg k-h
On Apr 14 Felipe Contreras wrote:
> On Sat, Apr 14, 2012 at 2:05 AM, Jonathan Nieder <[email protected]> wrote:
> > I don't think Stefan meant the above as tongue-in-cheek, for what it's
> > worth. Another stable kernel with different rules really would be an
> > interesting exercise, and would probably fulfill a need for a certain
> > audience.
> >
> > It's not like nobody does that already, anyway. For example, I hear
> > Fedora has a kernel that they maintain well for a different audience,
> > using different rules.
>
> Of course, although the difference with the stable kernel would be
> very small if the only thing added is an extra rule for acceptance:
> "It reverts an earlier
> patch to 'stable'."
It looks like a small difference on the surface, but it isn't. It would
mean "yes, we /do/ forward ports in -stable too in some cases".
In face of the frequent and predictable rate of mainline releases and the
very frequent stable releases, the problems associated with forward ports
very much appear to outweigh the benefits.
So the folks who do the actual work appear to be content with stable =
backports only, and it seems you can't convince them to do forward ports.
Which means that if you really want forward ports, you need to find
somebody else who does forward ports (e.g. a suitable distributor) or do
them yourself.
> But "we do this, and if you don't like it you can do whatever you
> want" is not a valid argument in favor of the status quo,
That would not be a valid argument, no. But it has been mentioned /why/
only backports are done in stable, i.e. what the problems with forward
ports are:
- First of all more work. (Could be compensated by longer periods
between stable releases, which would be an obvious downside too,
and actually defeat much of the purpose of the stable series of
getting out fixes to people.)
- Backports are already risky, which is why only /small/ backports are
allowed into stable. Forward ports bring their own set of risks.
Inevitably, users of the kernel called "stable" will get even
more reason to wonder why this series is not "stable" in a sense of
"absolutely free of regressions".
For example, merging subsystem development trees into the mainline is
actually a forward port. The mainline is at risk not only because
that subsystem tree hasn't had as much exposure to hardware and
workloads as the mainline will have, but additionally because the
merge result is different from the subsystem tree head. (The result
is a forward port.) Mainline development uses for example the
linux-next tree in order to reduce a subset of these forward porting
risks, those associated with conflicting developments in different
subsystems and cross-tree development. Which kinds of tools should
stable use if it were to accept forward ports?
So, rather than managing the risk of forward ports in stable, those
risks are systematically avoided by not ever doing forward ports, not
even small ones.
- The mainline would not immediately benefit from the work put into
forward ports in stable. Worse, fixes in those forward ports might
not ever make it into the mainline. That would be detrimental to all
mainline users, and thus to all stable users. (Stable users are
mainline users because stable is branched off of mainline every 2.5
months.)
These downsides have been mentioned in the thread; please consider them.
It is not about "we¹ do this because we are set in our ways". The stable
rules have pragmatic reasons which are for example founded on lessons
learned from the 2.4/2.5/2.6 history of development, maintenance, and
distribution.
So there are arguments for the status quo, the arguments have been
mentioned several times, and I can't see what would invalidate any of
these arguments.
--------
¹) Greg, if you are still reading this thread: Excuse the 'we', I do not
mean to speak for you.
Besides, Greg, thank you for the work. As a driver maintainer and when
supporting end users, I benefit from it since distributor kernels do not
deviate much from mainline these days, among else thanks to your stable
and longterm series. And it is an important way for me to get certain
driver fixes delivered to users conveniently and timely.
--
Stefan Richter
-=====-===-- -=-- -===-
http://arcgraph.de/sr/
On Apr 13 Stefan Richter wrote:
> there can only be two kinds of regressions in stable:
>
> A) First the regression was introduced into mainline, and accidentally it
> was carried over from there into stable.
>
> B) The regression only happened in stable because a backport from
> mainline to stable went wrong, e.g. a prerequisite to the backport
> was forgotten to be backported beforehand.
>
> AFAIU we are talking about a regression of type A.
>
> It seems you are arguing that stable candidate patches which fix
> regressions of type A should be treated differently from other stable
> candidate patches.
Correction to the last sentence:
Type A has two subtypes: A.1) The regression occurred in mainline between
the branch points v3.M and v3.N and resulted in a regression from v3.M.y
to v3.N.y. A.2) The regression occurred in mainline after branch point
v3.N and resulted in a regression from v3.N to v3.N.y.
A.1) Mainline and stable may be affected.
A.2) Mainline and stable may be affected.
B) Stable is affected, mainline is not.
You seem to be arguing that best practices to fix A.1 issues somehow do
not apply to how to fix A.2 issues. (They do apply though.)
--
Stefan Richter
-=====-===-- -=-- -==-=
http://arcgraph.de/sr/
On Thu, Apr 12, 2012 at 5:46 PM, Greg KH <[email protected]> wrote:
> On Thu, Apr 12, 2012 at 04:32:40PM +0300, Felipe Contreras wrote:
>> On Thu, Apr 12, 2012 at 4:13 AM, Greg KH <[email protected]> wrote:
>> > On Thu, Apr 12, 2012 at 04:03:59AM +0300, Felipe Contreras wrote:
>> >> On Thu, Apr 12, 2012 at 3:29 AM, Greg KH <[email protected]> wrote:
>> >> > On Wed, Apr 11, 2012 at 07:59:14PM -0400, Sergio Correia wrote:
>> >> >> Hello Greg,
>> >> >>
>> >> >>
>> >> >> On Wed, Apr 11, 2012 at 7:11 PM, Greg KH <[email protected]> wrote:
>> >> >> > This is the start of the stable review cycle for the 3.3.2 release.
>> >> >> > There are 78 patches in this series, all will be posted as a response
>> >> >> > to this one. If anyone has any issues with these being applied, please
>> >> >> > let me know.
>> >> >> >
>> >> >> > Responses should be made by Fri Apr 13 23:10:16 UTC 2012.
>> >> >> > Anything received after that time might be too late.
>> >> >> >
>> >> >>
>> >> >> is there any chance for this one to be included in this review cycle?
>> >> >>
>> >> >> http://www.spinics.net/lists/linux-wireless/msg87999.html
>> >>
>> >> I was going to ask for exactly the same thing. My system is completely
>> >> unusable without this patch; not only does the network doesn't work,
>> >> but quite often the kernel is stuck consuming 100% of the CPU.
>> >>
>> >> > Have you read Documentation/stable_kernel_rules.txt? Based on that, I
>> >> > don't think it can, yet, right?
>> >>
>> >> Why not? This patch makes the code go back to a previous state, it is
>> >> obviously more stable than the current state, and the code already
>> >> exists in Linus's tree (in previous releases).
>> >
>> > It does? What is the git commit id of the patch? Based in the email
>> > above, I assumed it had not made it to Linus's tree already.
>>
>> It's a revert of c1afdaff90538ef085b756454f12b29575411214, so so just
>> take a look at the code in c1afdaff90538ef085b756454f12b29575411214^.
>>
>> >> But hey, I guess it's OK that 3.3.x is stuck in and endless loop right
>> >> after booting, because rules are more important than fixing obvious
>> >> breakage.
>> >
>> > What rule did you think I was saying this was not acceptable for?
>>
>> The fact that the patch as not been applied/reviewed/accepted upstream.
>>
>> Personally I don't see what is the problem with reverts; we already
>> know the previous code was working. Sure, in theory it might behave
>> different due to other changes, but that doesn't seem to be the case
>> here, plus, it can't be worst than the current situation of staying in
>> an endless loop.
>
> A revert is the same as a patch. It needs to be in Linus's tree before
> I can add it to the stable releases.
Right, because otherwise people's systems would actually work.
But hey, as I said, following rules is more important, regardless of
what the rules are, and why they are there. The rules that actually
triggered this issue in v3.3.1, as this is not in v3.3.
You could just accept that the patch should have never landed in
v3.3.1 in the first place, but it's much easier to arbitrarily keep
stacking patches without thinking too much about them.
I guess I'll better use Linus's releases for now and go to v3.3.
--
Felipe Contreras
On Mon, Apr 16, 2012 at 11:58 PM, Greg KH <[email protected]> wrote:
> On Mon, Apr 16, 2012 at 11:11:05PM +0300, Felipe Contreras wrote:
>> On Mon, Apr 16, 2012 at 7:27 PM, Greg KH <[email protected]> wrote:
>> > Just one minor correction in this looney email thread:
>> >
>> > On Sat, Apr 14, 2012 at 01:53:22AM +0300, Felipe Contreras wrote:
>> >> v3.3.x on the other hand are *not* stable. They contain patches
>> >> backported from v3.4, but nobody guarantees they will work. There was
>> >> no v3.3.1-rc1, so the first time the patches compromising v3.3.1 were
>> >> generally tested together is in v3.3.1, at which point if somebody
>> >> finds issues, it's too late; bad patches are *not* going to be removed
>> >> in v3.3.2.
>> >
>> > Of course there was a 3.3.1-rc1, see the linux-kernel archives for the
>> > announcemen and the individual patches. kernel.org has the large patch
>> > itself if you like that format instead.
>>
>> I don't see it here:
>> http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=tags
>>
>> If you really want people to try it, why not tag it?
>
> That would be because I don't keep it in that tree. It is in a quilt
> tree you can find in the stable-queue.git repo, and I have never tagged
> -rc1 releases there. No one has ever asked for it before, so in the
> past 6 years of stable releases, I guess no one ever needed it.
>
> ketchup and tarballs seem to work well for others, perhaps you can use
> that as well (hint, ketchup on top of the linux-stable tree works just
> fine for testing this.)
Perhaps the current process will be continue to be OK, but I do
believe a tagged v3.3.1-rc1 would have catched the ath9k issue.
Hopefully that would mean it could have been dropped/reverted for
v3.3.1.
I used to compile my own kernels and use your stable tree, but this a
new laptop and I was using Arch Linux which automatically updated to
v3.3.1, and with no network I had no way to revert to v3.2.x.
Fortunately I had the kernel sources available, but I wonder how many
people were completely stuck.
If some other 3.x.1 release get broken this way, I would seriously
consider tagging v3.x.1-rc1 as well. It works for Linus' tree.
Cheers.
--
Felipe Contreras
On Thu, Apr 12, 2012 at 10:30:21PM +0200, Alexander Holler wrote:
> Am 12.04.2012 22:06, schrieb Greg KH:
> >On Thu, Apr 12, 2012 at 09:57:53PM +0200, Alexander Holler wrote:
> >>Hello,
> >>
> >>Am 12.04.2012 02:29, schrieb Greg KH:
> >>
> >>>>is there any chance for this one to be included in this review cycle?
> >>>>
> >>>>http://www.spinics.net/lists/linux-wireless/msg87999.html
> >>>
> >>>Have you read Documentation/stable_kernel_rules.txt? Based on that, I
> >>>don't think it can, yet, right?
> >>
> >>Hmm, after reading that, I think the text could need an update about
> >>how to submit patches written by others (which can already be found
> >>in the upstream tree).
> >>
> >>At least for me the text reads like only the authors can request
> >>inclusion of a patch into the stable tree.
> >
> >Patches are always welcome, but I think you will find the file already
> >describes this (hint, the first '-' line for the Procedure section
> >handles it).
>
> That reads:
>
> "- Send the patch, after verifying that it follows the above rules, to
> [email protected]. You must note the upstream commit ID in the
> changelog of your submission, as well as the kernel version you wish
> it to be applied to."
>
> Maybe I'm a bit dumb, but what would be my changelog if the patch
> was written by someone else (and e.g. fixes something the author
> wasn't aware of or hasn't described)?
It would be identical to what is in Linus's tree today.
Export the patch from git, add the needed information to the comment
area (i.e. the git commit id and what kernel to apply it to) and send it
to that email address.
Or, even easier, just email the stable@ address the git commit id, and
the kernel version you want it applied to, and cc: the authors of the
patch and maintainers (on the signed-off-by: path).
Hope this helps,
greg k-h
On Tue, Apr 17, 2012 at 12:44:25AM +0300, Felipe Contreras wrote:
> >> Perhaps the current process will be continue to be OK, but I do
> >> believe a tagged v3.3.1-rc1 would have catched the ath9k issue.
> >
> > How exactly would that have helped here?
>
> More people would have given it a try. Not that many people read the
> mailing list, and the ones that do certainly might want to avoid
> applying a big series of patches; even if their mail client makes it
> easy (mine (Gmail) doesn't). A tag, and an announcement to give a try
> would make it *much* easier.
That's totally unrelated. A tag means that these changes would have been
committed (with history etc...). This would not have changed anything of
the running/broken code. It would have failed similarly without any way
for you to revert. The rc1 was announced and was available on kernel.org.
A tag would not have changed anything here, except by making the release
process much more complex by basically having to revert every unsuitable
patch instead of dropping it. It would mean that rc1 would in fact have
been 3.3.1 and that 3.3.1 would have been 3.3.2. You just shift the release
by one number and don't bring any value here.
> >> I used to compile my own kernels and use your stable tree, but this a
> >> new laptop and I was using Arch Linux which automatically updated to
> >> v3.3.1, and with no network I had no way to revert to v3.2.x.
> >> Fortunately I had the kernel sources available, but I wonder how many
> >> people were completely stuck.
> >
> > Arch wouldn't have included a -rc in their kernel (unless they are
> > crazy), so this would not have helped your situation at all.
>
> There's *a lot* of people that got affected by the 3.3.1 release; we
> don't need to break a lot of boxes to figure out there's an issue,
> only a few would suffice, even one.
I'm sorry, but I don't buy this. There are *always* regressions caused by
kernel updates, and even the beginners I know are used to keep a working
kernel on their systems precisely because they know they can be left with
something which does not boot anymore. So if you upgrade your *only* kernel
without keeping a safe one, it's not a mistake, it's a fault, your fault. A
beginner wouldn't have done this. You wouldn't have recovered from a hanging
boot, a panic or a SATA timeout either. Drivers issues are the worst ones
because nobody can exhaustively test them all, not even your distro vendor.
The solution is always to keep a working kernel as an alternate boot.
> >> If some other 3.x.1 release get broken this way, I would seriously
> >> consider tagging v3.x.1-rc1 as well. It works for Linus' tree.
> >
> > "this way" was for a very tiny subset of hardware, so odds are, if this
> > happens again, it wouldn't be caught this way either. ?That subset just
> > happened to show up in your machine, but, for example, not in the wide
> > range of hardware I test with here, nor the machines that others test
> > with.
>
> It was certainly not just me. I have seen a lot of people mentioning
> "their wifi is broken", a lot of them using Arch Linux,
> coincidentally.
Maybe because these are people you know and you taught to blindly update
without keep a safe boot option ?
Now it becomes a lot clearer that you want a user fault to be covered by
a development process. That will never ever work because the only way it
will solve your behavioral issue is to release 100% safe and 100% compatible
kernels each time, which by definition is not possible if anything changes.
Willy
On Thu, Apr 12, 2012 at 3:29 AM, Greg KH <[email protected]> wrote:
> On Wed, Apr 11, 2012 at 07:59:14PM -0400, Sergio Correia wrote:
>> Hello Greg,
>>
>>
>> On Wed, Apr 11, 2012 at 7:11 PM, Greg KH <[email protected]> wrote:
>> > This is the start of the stable review cycle for the 3.3.2 release.
>> > There are 78 patches in this series, all will be posted as a response
>> > to this one. If anyone has any issues with these being applied, please
>> > let me know.
>> >
>> > Responses should be made by Fri Apr 13 23:10:16 UTC 2012.
>> > Anything received after that time might be too late.
>> >
>>
>> is there any chance for this one to be included in this review cycle?
>>
>> http://www.spinics.net/lists/linux-wireless/msg87999.html
I was going to ask for exactly the same thing. My system is completely
unusable without this patch; not only does the network doesn't work,
but quite often the kernel is stuck consuming 100% of the CPU.
> Have you read Documentation/stable_kernel_rules.txt? Based on that, I
> don't think it can, yet, right?
Why not? This patch makes the code go back to a previous state, it is
obviously more stable than the current state, and the code already
exists in Linus's tree (in previous releases).
But hey, I guess it's OK that 3.3.x is stuck in and endless loop right
after booting, because rules are more important than fixing obvious
breakage.
--
Felipe Contreras
On Thu, Apr 12, 2012 at 04:32:40PM +0300, Felipe Contreras wrote:
> On Thu, Apr 12, 2012 at 4:13 AM, Greg KH <[email protected]> wrote:
> > On Thu, Apr 12, 2012 at 04:03:59AM +0300, Felipe Contreras wrote:
> >> On Thu, Apr 12, 2012 at 3:29 AM, Greg KH <[email protected]> wrote:
> >> > On Wed, Apr 11, 2012 at 07:59:14PM -0400, Sergio Correia wrote:
> >> >> Hello Greg,
> >> >>
> >> >>
> >> >> On Wed, Apr 11, 2012 at 7:11 PM, Greg KH <[email protected]> wrote:
> >> >> > This is the start of the stable review cycle for the 3.3.2 release.
> >> >> > There are 78 patches in this series, all will be posted as a response
> >> >> > to this one. ?If anyone has any issues with these being applied, please
> >> >> > let me know.
> >> >> >
> >> >> > Responses should be made by Fri Apr 13 23:10:16 UTC 2012.
> >> >> > Anything received after that time might be too late.
> >> >> >
> >> >>
> >> >> is there any chance for this one to be included in this review cycle?
> >> >>
> >> >> http://www.spinics.net/lists/linux-wireless/msg87999.html
> >>
> >> I was going to ask for exactly the same thing. My system is completely
> >> unusable without this patch; not only does the network doesn't work,
> >> but quite often the kernel is stuck consuming 100% of the CPU.
> >>
> >> > Have you read Documentation/stable_kernel_rules.txt? ?Based on that, I
> >> > don't think it can, yet, right?
> >>
> >> Why not? This patch makes the code go back to a previous state, it is
> >> obviously more stable than the current state, and the code already
> >> exists in Linus's tree (in previous releases).
> >
> > It does? ?What is the git commit id of the patch? ?Based in the email
> > above, I assumed it had not made it to Linus's tree already.
>
> It's a revert of c1afdaff90538ef085b756454f12b29575411214, so so just
> take a look at the code in c1afdaff90538ef085b756454f12b29575411214^.
>
> >> But hey, I guess it's OK that 3.3.x is stuck in and endless loop right
> >> after booting, because rules are more important than fixing obvious
> >> breakage.
> >
> > What rule did you think I was saying this was not acceptable for?
>
> The fact that the patch as not been applied/reviewed/accepted upstream.
>
> Personally I don't see what is the problem with reverts; we already
> know the previous code was working. Sure, in theory it might behave
> different due to other changes, but that doesn't seem to be the case
> here, plus, it can't be worst than the current situation of staying in
> an endless loop.
A revert is the same as a patch. It needs to be in Linus's tree before
I can add it to the stable releases.
greg k-h
Am 12.04.2012 22:06, schrieb Greg KH:
> On Thu, Apr 12, 2012 at 09:57:53PM +0200, Alexander Holler wrote:
>> Hello,
>>
>> Am 12.04.2012 02:29, schrieb Greg KH:
>>
>>>> is there any chance for this one to be included in this review cycle?
>>>>
>>>> http://www.spinics.net/lists/linux-wireless/msg87999.html
>>>
>>> Have you read Documentation/stable_kernel_rules.txt? Based on that, I
>>> don't think it can, yet, right?
>>
>> Hmm, after reading that, I think the text could need an update about
>> how to submit patches written by others (which can already be found
>> in the upstream tree).
>>
>> At least for me the text reads like only the authors can request
>> inclusion of a patch into the stable tree.
>
> Patches are always welcome, but I think you will find the file already
> describes this (hint, the first '-' line for the Procedure section
> handles it).
That reads:
"- Send the patch, after verifying that it follows the above rules, to
[email protected]. You must note the upstream commit ID in the
changelog of your submission, as well as the kernel version you wish
it to be applied to."
Maybe I'm a bit dumb, but what would be my changelog if the patch was
written by someone else (and e.g. fixes something the author wasn't
aware of or hasn't described)?
Sorry, but I don't understand "changelog" in that context. Should I
modify the (description of the) patch?
Regards,
Alexander
On Thu, 12 Apr 2012, Greg KH wrote:
> On Thu, Apr 12, 2012 at 09:43:33PM +0300, Felipe Contreras wrote:
> > On Thu, Apr 12, 2012 at 8:24 PM, Adrian Chadd <[email protected]> wrote:
> > > On 12 April 2012 09:49, Felipe Contreras <[email protected]> wrote:
> > >
> > >>>
> > >>> A revert is the same as a patch. ?It needs to be in Linus's tree before
> > >>> I can add it to the stable releases.
> > >>
> > >> Right, because otherwise people's systems would actually work.
> > >>
> > >> But hey, as I said, following rules is more important, regardless of
> > >> what the rules are, and why they are there. The rules that actually
> > >> triggered this issue in v3.3.1, as this is not in v3.3.
> > >>
> > >> You could just accept that the patch should have never landed in
> > >> v3.3.1 in the first place, but it's much easier to arbitrarily keep
> > >> stacking patches without thinking too much about them.
> > >
> > > Greg is doing the right thing here. We face the same deal in FreeBSD -
> > > people want fixes to go into a release branch first, but if you do
> > > that you break the development flow - which is "stuff goes into -HEAD
> > > and is then backported to the release branches."
> > >
> > > If you don't do this, you risk having people do (more, all)
> > > development and testing on a release branch and never test -HEAD (or
> > > "upstream linux" here). Once you open that particular flood gate, it's
> > > hard to close.
> >
> > But this is exactly the opposite; the patch that broke things is in
> > the 'release branch' (3.3.1); it's not in upstream (3.3). Sure, it's
> > also on a later upstream, which is also broken.
>
> What is the git commit id of the patch in 3.3.1 that caused this to
> break? This is the first time I have heard that 3.3 worked and 3.3.1
> did not work. Someone needs to tell me these things...
Should be
db6a6a78d8602964c9dfb1d8ce18daefd92da0a7 in stable/linux-3.3.y
c1afdaff90538ef085b756454f12b29575411214 in linux/master
c'ya
sven-haegar
--
Three may keep a secret, if two of them are dead.
- Ben F.
On 04/16/2012 02:50 PM, Felipe Contreras wrote:
> On Tue, Apr 17, 2012 at 12:27 AM, Greg KH<[email protected]> wrote:
>> On Tue, Apr 17, 2012 at 12:18:13AM +0300, Felipe Contreras wrote:
>>> On Mon, Apr 16, 2012 at 11:58 PM, Greg KH<[email protected]> wrote:
>>>> On Mon, Apr 16, 2012 at 11:11:05PM +0300, Felipe Contreras wrote:
>>>>> On Mon, Apr 16, 2012 at 7:27 PM, Greg KH<[email protected]> wrote:
>>>>>> Just one minor correction in this looney email thread:
>>>>>>
>>>>>> On Sat, Apr 14, 2012 at 01:53:22AM +0300, Felipe Contreras wrote:
>>>>>>> v3.3.x on the other hand are *not* stable. They contain patches
>>>>>>> backported from v3.4, but nobody guarantees they will work. There was
>>>>>>> no v3.3.1-rc1, so the first time the patches compromising v3.3.1 were
>>>>>>> generally tested together is in v3.3.1, at which point if somebody
>>>>>>> finds issues, it's too late; bad patches are *not* going to be removed
>>>>>>> in v3.3.2.
>>>>>>
>>>>>> Of course there was a 3.3.1-rc1, see the linux-kernel archives for the
>>>>>> announcemen and the individual patches. kernel.org has the large patch
>>>>>> itself if you like that format instead.
>>>>>
>>>>> I don't see it here:
>>>>> http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=tags
>>>>>
>>>>> If you really want people to try it, why not tag it?
>>>>
>>>> That would be because I don't keep it in that tree. It is in a quilt
>>>> tree you can find in the stable-queue.git repo, and I have never tagged
>>>> -rc1 releases there. No one has ever asked for it before, so in the
>>>> past 6 years of stable releases, I guess no one ever needed it.
>>>>
>>>> ketchup and tarballs seem to work well for others, perhaps you can use
>>>> that as well (hint, ketchup on top of the linux-stable tree works just
>>>> fine for testing this.)
>>>
>>> Perhaps the current process will be continue to be OK, but I do
>>> believe a tagged v3.3.1-rc1 would have catched the ath9k issue.
>>
>> How exactly would that have helped here?
>
> More people would have given it a try. Not that many people read the
> mailing list, and the ones that do certainly might want to avoid
> applying a big series of patches; even if their mail client makes it
> easy (mine (Gmail) doesn't). A tag, and an announcement to give a try
> would make it *much* easier.
>
>> You point out:
>>
>>> I used to compile my own kernels and use your stable tree, but this a
>>> new laptop and I was using Arch Linux which automatically updated to
>>> v3.3.1, and with no network I had no way to revert to v3.2.x.
>>> Fortunately I had the kernel sources available, but I wonder how many
>>> people were completely stuck.
>>
>> Arch wouldn't have included a -rc in their kernel (unless they are
>> crazy), so this would not have helped your situation at all.
>
> There's *a lot* of people that got affected by the 3.3.1 release; we
> don't need to break a lot of boxes to figure out there's an issue,
> only a few would suffice, even one.
>
>>> If some other 3.x.1 release get broken this way, I would seriously
>>> consider tagging v3.x.1-rc1 as well. It works for Linus' tree.
>>
>> "this way" was for a very tiny subset of hardware, so odds are, if this
>> happens again, it wouldn't be caught this way either. That subset just
>> happened to show up in your machine, but, for example, not in the wide
>> range of hardware I test with here, nor the machines that others test
>> with.
>
> It was certainly not just me. I have seen a lot of people mentioning
> "their wifi is broken", a lot of them using Arch Linux,
> coincidentally.
Because it was only "tested" through the mailing list on Arch-general.
Like I stated your issue was related to you and not understanding Arch
AND how that kernel was tested before being pushed to the repo's
>
> These issues would most likely not be caught before v3.x.1, and which
> point it's too late, they cannot be reverted to v3.x.2 just like that;
> they have to wait for upstream. Hopefully and probably everything
> would go smooth like this time, but maybe not, we'll have to wait and
> see. With more people using Arch Linux and thus the latest "stable"
> release, I'd say we might see an increase in these kinds of issues.
>
> Cheers.
>
Learn the distro you are using better.
On Mon, Apr 16, 2012 at 7:27 PM, Greg KH <[email protected]> wrote:
> Just one minor correction in this looney email thread:
>
> On Sat, Apr 14, 2012 at 01:53:22AM +0300, Felipe Contreras wrote:
>> v3.3.x on the other hand are *not* stable. They contain patches
>> backported from v3.4, but nobody guarantees they will work. There was
>> no v3.3.1-rc1, so the first time the patches compromising v3.3.1 were
>> generally tested together is in v3.3.1, at which point if somebody
>> finds issues, it's too late; bad patches are *not* going to be removed
>> in v3.3.2.
>
> Of course there was a 3.3.1-rc1, see the linux-kernel archives for the
> announcemen and the individual patches. kernel.org has the large patch
> itself if you like that format instead.
I don't see it here:
http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=tags
If you really want people to try it, why not tag it?
--
Felipe Contreras
On Apr 14 Felipe Contreras wrote:
> On Sat, Apr 14, 2012 at 12:10 PM, Stefan Richter
> <[email protected]> wrote:
> > On Apr 14 Felipe Contreras wrote:
>
> >> Of course, although the difference with the stable kernel would be
> >> very small if the only thing added is an extra rule for acceptance:
> >> "It reverts an earlier patch to 'stable'."
> >
> > It looks like a small difference on the surface, but it isn't. It would
> > mean "yes, we /do/ forward ports in -stable too in some cases".
>
> How? There's a lot reverts in mainline, where do they come from? Are
> they forward ports from some ghost trees?
Indeed, reverts that go into mainline can often be called forward-ports:
A subsystem developer applied the revert to his subsystem tree, Linus
merges that tree, and the merge result is technically a forward-port
(except if the pull resulted in a fast-forward instead of a merge).
However, "fix it in stable before mainline" requires to allow forward-ports
in stable for a different reason: If the fix in mainline gets delayed
until after stable's next branch point, the stable fix needs to be
forward-ported from 3.M.y to 3.N.y.
> If you drop a patch from the stable review queue before it gets into a
> stable release, and then that patch is reverted from mainline, is that
> also a "forward port"?
There is just one fix of one bug, not a fix plus a port of the fix to a
similarly buggy tree.
--
Stefan Richter
-=====-===-- -=-- -===-
http://arcgraph.de/sr/
Just one minor correction in this looney email thread:
On Sat, Apr 14, 2012 at 01:53:22AM +0300, Felipe Contreras wrote:
> v3.3.x on the other hand are *not* stable. They contain patches
> backported from v3.4, but nobody guarantees they will work. There was
> no v3.3.1-rc1, so the first time the patches compromising v3.3.1 were
> generally tested together is in v3.3.1, at which point if somebody
> finds issues, it's too late; bad patches are *not* going to be removed
> in v3.3.2.
Of course there was a 3.3.1-rc1, see the linux-kernel archives for the
announcemen and the individual patches. kernel.org has the large patch
itself if you like that format instead.
greg k-h
On Tue, Apr 17, 2012 at 12:18:13AM +0300, Felipe Contreras wrote:
> On Mon, Apr 16, 2012 at 11:58 PM, Greg KH <[email protected]> wrote:
> > On Mon, Apr 16, 2012 at 11:11:05PM +0300, Felipe Contreras wrote:
> >> On Mon, Apr 16, 2012 at 7:27 PM, Greg KH <[email protected]> wrote:
> >> > Just one minor correction in this looney email thread:
> >> >
> >> > On Sat, Apr 14, 2012 at 01:53:22AM +0300, Felipe Contreras wrote:
> >> >> v3.3.x on the other hand are *not* stable. They contain patches
> >> >> backported from v3.4, but nobody guarantees they will work. There was
> >> >> no v3.3.1-rc1, so the first time the patches compromising v3.3.1 were
> >> >> generally tested together is in v3.3.1, at which point if somebody
> >> >> finds issues, it's too late; bad patches are *not* going to be removed
> >> >> in v3.3.2.
> >> >
> >> > Of course there was a 3.3.1-rc1, see the linux-kernel archives for the
> >> > announcemen and the individual patches. ?kernel.org has the large patch
> >> > itself if you like that format instead.
> >>
> >> I don't see it here:
> >> http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=tags
> >>
> >> If you really want people to try it, why not tag it?
> >
> > That would be because I don't keep it in that tree. ?It is in a quilt
> > tree you can find in the stable-queue.git repo, and I have never tagged
> > -rc1 releases there. ?No one has ever asked for it before, so in the
> > past 6 years of stable releases, I guess no one ever needed it.
> >
> > ketchup and tarballs seem to work well for others, perhaps you can use
> > that as well (hint, ketchup on top of the linux-stable tree works just
> > fine for testing this.)
>
> Perhaps the current process will be continue to be OK, but I do
> believe a tagged v3.3.1-rc1 would have catched the ath9k issue.
How exactly would that have helped here?
You point out:
> I used to compile my own kernels and use your stable tree, but this a
> new laptop and I was using Arch Linux which automatically updated to
> v3.3.1, and with no network I had no way to revert to v3.2.x.
> Fortunately I had the kernel sources available, but I wonder how many
> people were completely stuck.
Arch wouldn't have included a -rc in their kernel (unless they are
crazy), so this would not have helped your situation at all.
So, no, sorry, I'm not going to put -rc kernels in the linux-stable git
tree, they don't fit with our current development flow at this point in
time.
Again, if you want to, you can use ketchup and linux-stable together
quite easily, if you wish to help us with testing.
> If some other 3.x.1 release get broken this way, I would seriously
> consider tagging v3.x.1-rc1 as well. It works for Linus' tree.
"this way" was for a very tiny subset of hardware, so odds are, if this
happens again, it wouldn't be caught this way either. That subset just
happened to show up in your machine, but, for example, not in the wide
range of hardware I test with here, nor the machines that others test
with.
This thread has gone on long enough, this is it from me, sorry.
greg k-h
On Thu, Apr 12, 2012 at 10:05 PM, Linus Torvalds
<[email protected]> wrote:
> On Thu, Apr 12, 2012 at 9:49 AM, Felipe Contreras
> <[email protected]> wrote:
>>>
>>> A revert is the same as a patch. It needs to be in Linus's tree before
>>> I can add it to the stable releases.
>>
>> Right, because otherwise people's systems would actually work.
>
> There are rules for a damn good reason.
>
> The rule for -stable is that the changes *have* to be in upstream, for
> a very simple reason: otherwise bugs get re-introduced.
>
> If -stable starts revertign things that aren't reverted up-stream,
> what do you think happens to the *next* kernel version?
Which is why nobody is trying to change the rules regarding that.
And your same argument applies to the all the thousands of commits on
the next kernel version; what would happen on the next kernel version?
The relevant patch, like the thousands of patches in v3.4-rc*, did not
exist in v3.3, so if you add one on v3.3.1, and remove it on v3.3.2
would be *exactly* the same as if you had not added it at all to the
stable series.
> I don't think you realize how well kernel development has worked over
> the last few years. And the stable rules are part of it.
I do understand how well the development works, which is why I often
use it as an example and guideline, but that doesn't mean it's
perfect.
> So stop complaining. Reverts really *are* just patches, Greg is 100%
> right, and you are simply wrong.
I could argue in favor of exceptions, but I don't think you realize
the fact that this change does not affect your tree *at all*. Adding
and removing a patch in the stable tree is a no-op.
> And the revert is now apparently in John's tree, and will make it to
> David and then me shortly. It will get reverted in stable too once
> that happens. In the meantime, your complaints are to Greg only shows
> that you don't understand why the rules exist, and the fact that you
> *continue* to complain just makes you look stupid.
Is that going to happen before Friday? If not, then v3.3.2 will still
be broken *for no reason*.
--
Felipe Contreras
On Fri, Apr 13, 2012 at 01:58:10AM +0300, Felipe Contreras wrote:
> On Fri, Apr 13, 2012 at 1:12 AM, David Miller <[email protected]> wrote:
> > From: Felipe Contreras <[email protected]>
> > Date: Fri, 13 Apr 2012 01:04:42 +0300
> >
> >> Wrong is wrong, before or after the 3.3.1 tag, this patch is not
> >> 'stable' material, and removing it does not affect upstream at all.
> >
> > What you don't understand is that bug fixes will get lost if you only
> > fix them in -stable, it doesn't matter HOW THEY GOT into -stable.
>
> Let's suppose that c1afdaf was never back-ported from v3.4-rc1, how
> would you have fond out there was an issue with it? There's 10000
> patches in v3.4-rc2, how do you expect to find issues in them?
>
> People found out this issue on v3.4-rc1, so the fix would not have
> been lost, but lets assume it would, v3.3.1 had the issue, the patch
> as reverted in v3.3.2, and v3.4 still had the issue. So what? There's
> already 10000 patches that would never make it to 3.3.x, and many will
> have issues, which is why there would be v3.4.x.
>
> > In fact IT HAS FUCKING HAPPENED that we didn't fix something upstream
> > that got fixed in -stable a time long ago when we didn't have the
> > policy we're using now which you're going so unreasonably ape-shit
> > about.
>
> I see how a *fix* on stable could get lost, but this is not a fix.
Felipe, you don't seem to get it : there are many bugs in each new release.
Given the number of fixes Greg merges into a longterm branch, I'd say that
there are around 1500 bugs waiting to be discovered and fixed in a new
release. Does this mean we need to fix them all at once ? No, because we
don't know about them yet.
The process you're criticizing consists in ensuring that once a bug is known,
it gets fixed in mainline so that it never appears there again. The way the
bug is discovered doesn't matter, even if it's discovered that a fix caused
the bug and that it must be reverted. The fact is mainline is buggy and we
know this because stable is too. So mainline must be fixed first. This
process works because stable users are pressuring developers to push their
fixes to Linus in order to get them. What happened with this bug prooved
the process is working fine.
Another point is that you don't want stable to merge, revert, merge again,
revert again etc... This happened a little bit during 2.6.32 because some
fixes were not really obvious. It's common for some fixes to have to be
adapted for stable branches, and to have side effects, hence the review
cycle. We need to limit these random issues as much as possible if we
don't want users to lose trust in the stable branches. This is extremely
important. So picking random fixes that have not been qualified by all
interested parties in stable is inappropriate. Reverting without evaluating
impacts is one form of picking a random fix.
What you should have done would have been to reply to Greg saying "wait a
minute, we still have an issue with patch XX, I'm trying to get it reverted
in upstream and will send you the commit ID, it would be nice to have it in
3.3.2". It wastes less time for everyone and achieves the same result.
Once again, if you think that the stable branch you're using is not stable
enough for you, pick another one. Greg maintains multiple branches so that
everyone is satisfied. The risk of bugs over time probably looks like
(cos(t)+1)/t. Find an older branch with a much smaller risk of regressions
and be done with it.
Last point, you should note that you're the only one here who doesn't
understand the process. That doesn't make you a fool, but it should tell you
that you probably need to think a bit further before telling people how they
should work, especially when all other ones agree on the benefits of the
process, including Arnd explaining that FreeBSD had been facing the exact
same trouble and now applies the same process. It is not just a small band
of nerds doing this for fun right here, but seems to be more generalized.
Regards
Willy
On Apr 15 Felipe Contreras wrote:
> The question that has not been answered is what makes them different,
It was answered in the grandparent post and in posts before it.
> I'm glad you see it's silly to put bad patches in the stable release
> just so they get properly tracked for mainline, but that's exactly
> what you arguing for.
I am not arguing for anything remotely like that.
--
Stefan Richter
-=====-===-- -=-- -====
http://arcgraph.de/sr/
On Fri, Apr 13, 2012 at 12:20:20AM +0300, Felipe Contreras wrote:
> The relevant patch, like the thousands of patches in v3.4-rc*, did not
> exist in v3.3, so if you add one on v3.3.1, and remove it on v3.3.2
> would be *exactly* the same as if you had not added it at all to the
> stable series.
Except that thanks to its addition in 3.3.1 we know it has broken some
setups. Now we know they're broken, there is a good reason to *ensure*
it is removed in 3.4-rc too. If you revert it from 3.3.2 and forget about
3.4-rc, you'll end up with the faulty patch present again in 3.4. It is
*very* important that -stable reflects what next should look like.
The only frustration from you that I can understand comes from the
multi-hop that the patch has to pass through before hitting Linus,
but for the same reason you want every intermediary maintainer to
ensure that his own tree is fixed so that the patch is not reintroduced
in a future batch.
Again, why don't you apply it into your tree ? Better, fork 3.3-stable
and announce 3.3.2-fc which has this patch. It will help other users too.
Let's just see how long you keep up with out-of-sync fixes.
Regards,
Willy
On 04/16/2012 02:18 PM, Felipe Contreras wrote:
> On Mon, Apr 16, 2012 at 11:58 PM, Greg KH<[email protected]> wrote:
>> On Mon, Apr 16, 2012 at 11:11:05PM +0300, Felipe Contreras wrote:
>>> On Mon, Apr 16, 2012 at 7:27 PM, Greg KH<[email protected]> wrote:
>>>> Just one minor correction in this looney email thread:
>>>>
>>>> On Sat, Apr 14, 2012 at 01:53:22AM +0300, Felipe Contreras wrote:
>>>>> v3.3.x on the other hand are *not* stable. They contain patches
>>>>> backported from v3.4, but nobody guarantees they will work. There was
>>>>> no v3.3.1-rc1, so the first time the patches compromising v3.3.1 were
>>>>> generally tested together is in v3.3.1, at which point if somebody
>>>>> finds issues, it's too late; bad patches are *not* going to be removed
>>>>> in v3.3.2.
>>>>
>>>> Of course there was a 3.3.1-rc1, see the linux-kernel archives for the
>>>> announcemen and the individual patches. kernel.org has the large patch
>>>> itself if you like that format instead.
>>>
>>> I don't see it here:
>>> http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=tags
>>>
>>> If you really want people to try it, why not tag it?
>>
>> That would be because I don't keep it in that tree. It is in a quilt
>> tree you can find in the stable-queue.git repo, and I have never tagged
>> -rc1 releases there. No one has ever asked for it before, so in the
>> past 6 years of stable releases, I guess no one ever needed it.
>>
>> ketchup and tarballs seem to work well for others, perhaps you can use
>> that as well (hint, ketchup on top of the linux-stable tree works just
>> fine for testing this.)
>
> Perhaps the current process will be continue to be OK, but I do
> believe a tagged v3.3.1-rc1 would have catched the ath9k issue.
> Hopefully that would mean it could have been dropped/reverted for
> v3.3.1.
>
> I used to compile my own kernels and use your stable tree, but this a
> new laptop and I was using Arch Linux which automatically updated to
> v3.3.1, and with no network I had no way to revert to v3.2.x.
For an Arch user I am shocked you are able to keep things going on your
setup. Why are you not spending this much effort complaining to Arch
Dev's for not properly testing 3.3.1 before pushing it. I know they did
not put it into testing and was just put through the mailing list for
people to test, with only a few ack's showing up on the list, before
pushing about a day later.
Also how did you not have something to roll back to? Why was there not a
package left in your /var/cache/pacman/pkg cache? Unless you clean out
your cache way to often you should have more than one kernel you could
have "rolled back" with a simple pacman -U linux-version-that-last-worked.
This much effort and bitching on your part to people who had nothing to
do with you getting a bad push from Arch, Arch should have done further
testing of it OR you should have waited a day before updating to see if
anything on your shiny new laptop would have been effected.
> Fortunately I had the kernel sources available, but I wonder how many
> people were completely stuck.
On arch no one should have been stuck if they would have read the wiki's
and been more aware of their systems. Stable or not you are living on
the bleeding edge or as close to it as possible when on Arch, read their
philosophy and methods and their wiki. Then if you have issues with
something Arch pushes to you, would it not be better to spend your
efforts on the distro you are running?
>
> If some other 3.x.1 release get broken this way, I would seriously
> consider tagging v3.x.1-rc1 as well. It works for Linus' tree.
>
> Cheers.
>
Just my .02 from someone not involved in any of this who is just fed up
with you twisting crap around to fit your gripes with something that is
not related to the trees or methods at hand. YOU got a package from YOUR
distro of choice and because YOU were not aware of things when updating
your distro YOU broke it. Not Linus or Greg, but Arch and you not
understanding the kernel did NOT go through the normal testing repo. I
have a system that would have broke in the same manner as yours, running
Arch, but because I am aware I have a few things that are getting lots
of work in the kernel, I need to be aware of the updates, especially
kernel's just being tested through the mailing list. Also why did you
not have at least the LTS as a backup on your Arch install, if you are
clearing out your cache so frequently to not have at least one version
to "roll back"?
Learn your hardware and how it interacts with the distro you choose and
you can better help yourself and ALL these efforts would not just end up
making you seem like a ranting loon!
On Thu, Apr 12, 2012 at 2:34 PM, Linus Torvalds
<[email protected]> wrote:
>
> So just reverting it from stable, *WITHOUT LEARNING THE LESSON*, is
> not a no-op at all, it's a sign of being a f*cking moron.
Btw, the revert is now in my tree (commit 011afa1ed8c4), and marked
for stable. So *now* Greg can revert it from stable too.
But the important lesson to everybody should be that "we don't lose
fixes from -stable". If a problem was found in stable, it needs to be
fixed upstream. In fact, quite often people *do* find problems in
stable, because it tends to have more users more quickly than
mainline. That makes it really really important to make sure that
those problems get fixed upstream, and not hidden in stable due to
some kind of dieseased "it's a no-op to revert it" thinking.
End of story.
Linus
On Thu, Apr 12, 2012 at 3:04 PM, Felipe Contreras
<[email protected]> wrote:
>
> Sure, but removing that patch from the stable tree is not going the
> change that information; we already know the patch is wrong.
.. and we wait until it has been fixed in mainline so that we *know*
that information doesn't get lost.
Exactly like any other patch. Exactly like the rules for -stable says we should.
Stop the idiotic arguing already.
Linus
On Sat, Apr 14, 2012 at 06:43:06PM +0300, Felipe Contreras wrote:
> I understand you use 'stable' as guarantee, and I know it works, but
> do you *need* this guarantee?
>
> And before you go on why you need this guarantee to avoid fixes to be
> lost, this is an *entirely different thing*; we are not talking about
> fixes in 'stable' that don't exist in mainline--for which there is
> evidence that those caused problems in the past, we are talking about
> reverting patches from 'stable' that are not part of the upstream
> release from where the 'stable' branch was forked--*nobody* has showed
> any evidence that this has happened before and caused issues.
Why make a special case for the version from which stable was derived ?
That doesn't make sense at all to me since by definition, *all* patches
that are in stable were not in this version !
Take it simpler if you want : *all* patches in stable need an upstream
commit ID, whether they're backports or reverts. You don't revert a
patch from stable, you backport a revert from upstream.
Willy
On Apr 12 Felipe Contreras wrote:
> But this is exactly the opposite; the patch that broke things is in
> the 'release branch' (3.3.1); it's not in upstream (3.3). Sure, it's
> also on a later upstream, which is also broken.
^^^^^
No, upstream /earlier/ than 3.3.1 contains the defect.
v3.4-rc1 is older than v3.3.1.
3.3 is not 3.3.y's upstream. 3.3 is the branch point from where 3.3.y
began.
torvalds/linux.git #HEAD is 3.3.y's upstream.
A git snapshot shortly before v3.4-rc1 was v3.3.1's upstream.
Furthermore, consider this: You as user of the 3.3.y series are using a
temporary, dead-end side branch. Its maintenance will stop at some point,
and you will be left looking for a different, maintained series to migrate
to. You will be most interested in that series /not/ containing any
regressions that you suffered already through the 3.3.y lifetime.
That other 3.3+n.y kernel to which you will be wanting to migrate to
should obviously be based on a branch point which does not lack any of the
fixes which your last 3.3.y already had.
I.e. you require that stable kernels are branched off of good branch
points. Mainline releases, i.e. releases of branch points, happen on a
time-based scheme (as opposed to a feature-based scheme). So a rule whose
purpose is to ensure that future stable branch points contain all fixes
that earlier stable branches had already must necessarily be a time-based
rule. This time-based rule is "fix mainline before stable".
The rule is there to protect you, as a user of the stable series, from
repeated regressions.
--
Stefan Richter
-=====-===-- -=-- -==-=
http://arcgraph.de/sr/
* Felipe Contreras <[email protected]> wrote:
> On Sat, Apr 14, 2012 at 1:47 PM, Ingo Molnar <[email protected]> wrote:
> >
> > * Felipe Contreras <[email protected]> wrote:
> >
> >> On Fri, Apr 13, 2012 at 1:07 AM, Linus Torvalds
> >> <[email protected]> wrote:
> >> > On Thu, Apr 12, 2012 at 3:04 PM, Felipe Contreras
> >> > <[email protected]> wrote:
> >> >>
> >> >> Sure, but removing that patch from the stable tree is not
> >> >> going the change that information; we already know the
> >> >> patch is wrong.
> >> >
> >> > .. and we wait until it has been fixed in mainline so that
> >> > we *know* that information doesn't get lost.
> >>
> >> So why don't we pick potentially dangerous patches that might
> >> benefit from some testing, put them in 'stable', and if there
> >> are problems, make sure they get fixed in upstream first?
> >>
> >> Or for that matter totally broken patches we want to make sure
> >> they get fixed in upstream.
> >>
> >> Because the priority of the 'stable' tree is *stability*. Is
> >> it not?
> >>
> >> But what you are saying is: *before* the final review, even a
> >> hint that the patch might cause problems is reason enough to
> >> drop it from stable, but *after* the review, if we know the
> >> patch is totally broken, then it's the opposite; we really
> >> want it in.
> >
> > What you don't seem to understand is that there are good reasons
> > why we first fix bugs upstream, then in -stable. Greg explained
> > it to you, Linus explained it to you and so did many others.
> >
> > Having an order of patches *necessarily* means that the
> > development tree will get fixes sooner than the stable tree. In
> > other words, this *necessarily* means that the stable tree - and
> > its users - will have to wait a little bit more to have the fix.
> > In the worst-case this 'have to wait a little bit longer' might
> > span the time between two minor stable kernel releases.
> >
> > You seem to equate this 'have to wait a little bit longer to get
> > the fix' property of the maintenance model with 'we don't care
> > about stable tree users' - that claim is obviously idiotic and
> > most of your arguments in this thread are idiotic as well.
>
> This is a straw man again. Again; we are not talking about
> fixes in 'stable' that don't exist in mainline, we are talking
> about reverting patches from 'stable' that are not part of the
> upstream release from where the 'stable' branch was forked.
You are misunderstanding the Linux kernel development process
again:
> You are avoiding the argument you replying to; yesterday a
> patch was droppable from the stable review queue, but today,
> after the release, now we *need* it to stay there until it's
> fixed in the mainline. What changed?
What changed: the stable kernel was released and a new cycle
started. If something is broken in -stable it needs to be
reverted upstream. Full stop.
There is a minor engineering process that if a -stable commit
does not even apply or does not even boot on Greg's box or on
the handful of boxes that test stable release candidates then it
obviously cannot become part of the -stable queue. That kind of
very short term, memory-less integration testing should not be
confused with the much broader, state-ful, Git commit backed
testing that the upstream kernel gets.
There's a new stable cycle every 7 days on average, there's a
new upstream kernel every 7 days on average, and there's very
good reasons for the stable queue to be memory-less and to not
do your 'drop a patch from the previous stable version but don't
bother dropping it from upstream first' kind of messy operation
on it.
> What makes a patch droppable yesterday, but dependent on
> mainline today?
Time and version release engineering: once a stable kernel is
released any temporary integration testing results are flushed -
the upstream kernel is where we maintain the information of
which patch is broken and which not.
This memory-less process, amongst other things, helps the *next*
major stable kernel become more robust, as it removes the
possibility for version skew between stable and upstream.
If you want a revert you either need to report your problem fast
enough to be caught in -stable integration testing, or you need
to work with upstream to push the fix through properly.
People explained this to you, again and again, you refused to
listen, repeatedly, again and again. There does not appear to be
any rational set of arguments that will make you admit that you
were wrong about this.
Anyway, in this discussion you have demonstrated it again why
you are one of the very few commenters I had to permanently
block on G+ ...
Thanks,
Ingo
Felipe Contreras wrote:
> I guess I should avoid the "stable" series then.
I wish you had understood this much much sooner so that this nonsense
thread could have been avoided.
If you want the very latest fixes then *obviously* you need to use
the most bleeding edge repo. (Linus')
//Peter
On Fri, 13 Apr 2012, Felipe Contreras wrote:
> On Thu, Apr 12, 2012 at 10:05 PM, Linus Torvalds
> <[email protected]> wrote:
> > On Thu, Apr 12, 2012 at 9:49 AM, Felipe Contreras
> > <[email protected]> wrote:
> >>>
> >>> A revert is the same as a patch. It needs to be in Linus's tree before
> >>> I can add it to the stable releases.
> >>
> >> Right, because otherwise people's systems would actually work.
> >
> > There are rules for a damn good reason.
> >
> > The rule for -stable is that the changes *have* to be in upstream, for
> > a very simple reason: otherwise bugs get re-introduced.
> >
> > If -stable starts revertign things that aren't reverted up-stream,
> > what do you think happens to the *next* kernel version?
>
> Which is why nobody is trying to change the rules regarding that.
>
> And your same argument applies to the all the thousands of commits on
> the next kernel version; what would happen on the next kernel version?
>
> The relevant patch, like the thousands of patches in v3.4-rc*, did not
> exist in v3.3, so if you add one on v3.3.1, and remove it on v3.3.2
> would be *exactly* the same as if you had not added it at all to the
> stable series.
>
> > I don't think you realize how well kernel development has worked over
> > the last few years. And the stable rules are part of it.
>
> I do understand how well the development works, which is why I often
> use it as an example and guideline, but that doesn't mean it's
> perfect.
>
> > So stop complaining. Reverts really *are* just patches, Greg is 100%
> > right, and you are simply wrong.
>
> I could argue in favor of exceptions, but I don't think you realize
> the fact that this change does not affect your tree *at all*. Adding
> and removing a patch in the stable tree is a no-op.
>
> > And the revert is now apparently in John's tree, and will make it to
> > David and then me shortly. It will get reverted in stable too once
> > that happens. In the meantime, your complaints are to Greg only shows
> > that you don't understand why the rules exist, and the fact that you
> > *continue* to complain just makes you look stupid.
>
> Is that going to happen before Friday? If not, then v3.3.2 will still
> be broken *for no reason*.
>
>
Ok, -stable got broken because a bad patch was backported from mainline to
stable - obviously that's not good.
But, just reverting that broken patch from stable is *not* good enough.
Then we still have a broken mainline.
The rule that only commits which are in mainline get applied to -stable
makes really, Really, REALLY good sense. It helps ensure that we don't
miss fixes going forward.
Imagine that bad patch xyz got added to mainline and backported to stable
and that the only people that get bitten by it are conservative types that
only run -stable kernels. Then we revert the patch from stable since it
broke stuff and now everyone are happy - but *mainline is still broken*
and there's noone pushing for the problem to get fixed in mainline since
noone affected by the problem run that kernel :-(
By insisting that only patches in mainline get backported to -stable we
ensure that both mainline and stable get fixed (and thus also the *next*
stable kernel that'll be cut from mainline in the future).
Yes it's a pain that -stable has to suffer a broken commit for a while -
but sometimes that's just the little pain you have to accept. The greater
good is that with the current rules both mainline, current -stable and
future -stable will get fixed.
--
Jesper Juhl <[email protected]> http://www.chaosbits.net/
Don't top-post http://www.catb.org/jargon/html/T/top-post.html
Plain text mails only, please.
On Thu, Apr 12, 2012 at 9:49 AM, Felipe Contreras
<[email protected]> wrote:
>>
>> A revert is the same as a patch. ?It needs to be in Linus's tree before
>> I can add it to the stable releases.
>
> Right, because otherwise people's systems would actually work.
There are rules for a damn good reason.
The rule for -stable is that the changes *have* to be in upstream, for
a very simple reason: otherwise bugs get re-introduced.
If -stable starts revertign things that aren't reverted up-stream,
what do you think happens to the *next* kernel version?
We have those -stable rules for a very good reason - we used to not
have them, and the above "oops, we fixed it in stable, but the fix
never made it upstream" happened *all*the*time*.
I don't think you realize how well kernel development has worked over
the last few years. And the stable rules are part of it.
So stop complaining. Reverts really *are* just patches, Greg is 100%
right, and you are simply wrong.
And the revert is now apparently in John's tree, and will make it to
David and then me shortly. It will get reverted in stable too once
that happens. In the meantime, your complaints are to Greg only shows
that you don't understand why the rules exist, and the fact that you
*continue* to complain just makes you look stupid.
Linus
On Sat, Apr 14, 2012 at 12:10 PM, Stefan Richter
<[email protected]> wrote:
> On Apr 14 Felipe Contreras wrote:
>> Of course, although the difference with the stable kernel would be
>> very small if the only thing added is an extra rule for acceptance:
>> "It reverts an earlier patch to 'stable'."
>
> It looks like a small difference on the surface, but it isn't. It would
> mean "yes, we /do/ forward ports in -stable too in some cases".
How? There's a lot reverts in mainline, where do they come from? Are
they forward ports from some ghost trees?
If you drop a patch from the stable review queue before it gets into a
stable release, and then that patch is reverted from mainline, is that
also a "forward port"?
--
Felipe Contreras
On Apr 14 Felipe Contreras wrote:
> I already exemplified how they are very different, but here it goes
> again. The patch "drm/i915: Add lock on drm_helper_resume_force_mode"
> was just tagged in 3.3.2, if I had said yesterday "this patch breaks
> things on my machine", then that patch would have been dropped for
> 3.3.2 even though it's still on mainline--why? Because we know it's
> broken, and broken patches are not supposed to land to stable. But
> today, one day later, we have to wait until it's fixed in master
> first. Why?
>
> What makes a patch droppable yesterday, but dependent on mainline today?
Huh?
Because "yesterday" was a review before stable release:
- A buggy mainline release exists.
- No buggy stable release exists.
whereas "today" is after stable release:
- A buggy mainline release exists.
- A buggy stable release exists.
(The WLAN breakage wich is being talked about was reported after
release, not during review, right?)
[quote re-ordered]
> Again, you can repeat the same thing as much as you want, it still
> doesn't answer what I have asked.
Yeah, sorry for that. All the time I thought you asked why a *revert*
which is applicable to mainline and stable first happens in stable.
But your question was actually what the difference between
- dropping a patch from a queue of candidate patches versus
- adding a reverting patch to repair a defect from a previous release
is.
The former is still part of the decision whether a changeset which
exists in mainline should be backported into stable. Somebody initially
thought it should be, but others should have a look too and amend that
decision if need be. Somebody reports that the change would introduce a
regression. Usually, this disqualifies a patch from being applied to
stable right away, without further work having to be done in stable.
"Drop a stable candidate before release" is a form of "decline a
submission to stable", happening late in the preparations of a stable
release.
The latter is when damage was done; it is now about bug fixing. This
involves debugging of the regression, finding a right approach to
fix it (e.g. revert), write + review + test + commit the fix, port the fix
to all relevant other kernel branches, review + test + commit those ports
too.
"Add a reverting fix" is a form of "add a fix".
You are indeed pointing to a procedural flaw here. In "add a fix" cases,
stable release procedures ensure that if 3.3.3 included the revert, 3.4
will include it to, by the Linus->Greg order of commiting patches. But in
the "drop a candidate before release" case, Greg and the stable reviewers
do not have a similarly fool-proof procedure to ensure that the next branch
point will be free of the regression. Now they need to watch closely that
a fix gets into mainline ideally before the next branch point is going to
be released.
So there is indeed a difficulty involved with dropping patches from the
candidate queue. Fortunately, it is not necessary to subject post-release
reverts to the same difficulty.
> This of course, has *not* been explained.
Others had discussed "not adding a changeset" versus "amending an already
released changeset by another changeset on top of it" already. Now I did
too and apologize to everybody else for redundancy.
--
Stefan Richter
-=====-===-- -=-- -===-
http://arcgraph.de/sr/
On Sun, Apr 15, 2012 at 1:47 AM, Stefan Richter
<[email protected]> wrote:
> On Apr 15 Felipe Contreras wrote:
>> The question that has not been answered is what makes them different,
>
> It was answered in the grandparent post and in posts before it.
Nope.
>> I'm glad you see it's silly to put bad patches in the stable release
>> just so they get properly tracked for mainline, but that's exactly
>> what you arguing for.
>
> I am not arguing for anything remotely like that.
Please, do explain why that is silly then.
--
Felipe Contreras
On Fri, Apr 13, 2012 at 12:34:59AM +0300, Felipe Contreras wrote:
> Now what happens when:
>
> - you realize the fix made matters worst, in fact, so worst that the
> whole thing is unusable in some systems
>
> Presumably we are now in the next round of:
>
> - fix upstream
>
> But v.3.3.2 is due Friday, which makes it very likely that the fix
> won't get in. And what did we gain? If you simplify the situation to
> what you explained above, it seems very reasonable, but that's not the
> whole picture.
BTW, if you're so in need for critical fixes, why do you jump to the latest
version ? It's known and accepted that stable versions stabilize over time
with a few random hiccups during their early stage. You could have chosen
3.0 instead which generally holds some fixes that cooked longer before being
merged.
There's a cost at living on the bleeding edge. It's fun and sometimes it
hurts but that should not be that dramatic that you spend so many emails
telling how stable releases should be maintained.
Willy
On Sun, Apr 15, 2012 at 10:15 AM, Felipe Contreras
<[email protected]> wrote:
>
> This is not a reason, this is just stating what happens without
> explaining *why*.
>
> Q: What changes when a tag is made?
> A: A tag is made
I'll make one more try at explaining to you, but then I'll just set my
mail reader to ignore you, because judging by past performance (not
just in this thread) you will just continue to argue.
I'll explain two things. None of those two things are actually about
"tags", although I will start with the issue that is at least
"coincidental" with tags, namely the fundamental nature of reality.
At no point is the "tag" at all important. You are bringing up all
these red herrings, and trying to actively confuse the issue.
The only thing that matters is "it's been made available to others".
That's fundamentally what a release is. If you know physics, think of
a release (and the tag is associated with it) as "collapsing the wave
function". Before that, the outside world doesn't know the exact state
of your tree (they may have good guesses based on the patches they've
seen) - after that, it's "done". You cannot change it. And more
importantly, other people can see the state, and measure it, and
depend on it.
This true regardless of git, btw, but git actually makes that "it's
been available to others" be a real technical distinction, since the
identity of something cannot be changed (so "availability" ends up
meaning "identity" - something is absolutely fixed in stone - you
can't change it, because what you made available is fixed and outside
of your control).
So once some release has been made, you can't change it. If Greg has
made a 3.3.2 release (the tag is not the important part, although the
tag is *coincidental* with the release, of course), then that is what
it is. You cannot change history. You cannot say "oh, we can go back
in time and undo things". There is no change that is a no-op.
THAT is important. And it seems to be something that you don't
understand. The past is immutable. A "tag" has no real other meaning
than as a particular marker of a particular past state. But the tag
itself is not what makes the past immutable. REALITY is what makes the
past immutable.
And reverting a patch DOES NOT CHANGE HISTORY. It does not magically
go back in time and say "nothing happened". Every time you say that
"reverting a pacth is a no-op", you look more and more stupid, because
you don't seem to understand this fundamental fact.
A revert is nothing but another change. A revert is *exactly* the same
thing as a patch that doesn't revert, but instead fixes. If you cannot
understand that, you're not worth talking to.
And the reason we don't do reverts in stable releases without having
the revert upstream is *EXACTLY* the same reason we don't make other
changes in stable releases without making that change upstream first.
And another really fundamental thing that you don't seem to understand
is that the most important part of "stability" is not actually "bug
free" (which is something we can never attain in any project that is
non-trivial and still evolving), but "reliability". Not even of a
single release, but in *time*. We want to make fundamental and
*reliable* forward progress. And that is why we have the whole "we
don't make changes to the stable tree without having those changes in
upstream first".
And the thing is, it's not just 3.3.3 that follows 3.3.2. It's 3.4.1
too. The reason we have the "no stable fixes before it's been
upstreamed" (and again: a "revert" is *nothing* but another name for a
fix, that gives some additional information about *how* we fixed
something too) is because we want to make nice plodding reliable
forward progress. Mistakes (== bugs) will happen, but the stable rules
are there to make sure that those mistakes get fixed, and don't have
long-term impact. THAT is the "stable" part of the stable series.
Things are monotonic in the big picture, even if we may have some
non-monotonic noise in the details.
If you think that "stable" means "bug free", you are fundamentally
confused about software engineering.
If you think you can go back in time and "undo" things, you are even
more fundamentally confused about reality.
And if you cannot understand what tens of people have tried to explain
to you, you are just f*cking stupid.
Linus
On Sat, Apr 14, 2012 at 10:33:59PM +0300, Felipe Contreras wrote:
> >> Again, what makes a released patch undroppable?
> >
> > Being applied, in other words, having a commit ID in the branch.
>
> Seriously? That's your reason?
>
> Hey, thousands of users out there; the reason why we pushed a patch
> that is known to be broken in v3.3.x is because it already has a
> commit ID.
I think you have a real problem with logics in general as it's not the
first time you're reverting cause and consequence in people's arguments.
I'm basically saying that we don't revert patches any other way than
by backporting the revert and you're saying that every patch has to
be backported. Come on, this discussion makes no sense at all. You're
wasting everyone's time for nothing, just because you want to be right
even after everyone explained you the same thing. You got me bored.
Willy
On Thu, Apr 12, 2012 at 9:56 PM, Jonathan Nieder <[email protected]> wrote:
> Felipe Contreras wrote:
>
>> But then are you saying that if upstream is broken (3.4-rc2), then
>> stable should be broken as well (3.3.1), and remain broken until
>> upstream is fixed? I fail to see what would be the point of that.
>
> No, he's saying that when upstream is broken for the same reason as
> stable is, it seems wise to:
>
> - report upstream
> - fix your local system
> - fix any systems you are responsible for
> - fix upstream
> - only then fix stable.
I'm not sure those steps were followed for this particular patch on
v3.3.1, but lets assume they where. Now what happens when:
- you realize the fix made matters worst, in fact, so worst that the
whole thing is unusable in some systems
Presumably we are now in the next round of:
- fix upstream
But v.3.3.2 is due Friday, which makes it very likely that the fix
won't get in. And what did we gain? If you simplify the situation to
what you explained above, it seems very reasonable, but that's not the
whole picture.
--
Felipe Contreras
On 04/16/2012 10:25 PM, Felipe Contreras wrote:
> Releasing implies many things. There is this notion called theory of
> mind, which children learn at a very young age, that helps us see the
> world through the eyes of other people. If you think people *must* see
> the word "release" and think *exactly* the same thing as you do, and
> apologize when they don't, well, I'd say you are missing a very
> valuable skill.
Maybe we are all a bit autistic then.
> But fine, lets suppose I should have thought exactly the same thing as
> you did, and fine, lets say I was wrong in not doing so. That still
> doesn't help one iota in this discussion. Linus Torvalds pointed the
> exact reason, which allowed us to move forward, because of my
> incompetence, or whatever.
I think this thread has been going on long enough and a fair number of
people tried to explain the process and why it is set up as such. How
long do you want to rephrase and stick to your "theory of mind". Email
filters will probably be created rather sooner than later. Does
Contreras have the same linguistic origin as "contrary"? Sorry, could
not think of anything constructive here.
> The next logical question has still not been answered; does the
> undroppability come from the fact that v3.4 is more important that
> v3.3.1? This time I promise to think at least one day for all the
> possible meanings of each key word you say.
3.4 is not most important, but upstream is and currently those are the
3.4-rcN releases. Every patch has to go upstream first as it was, is,
and will be our past, present, and future.
The only thing I can think of in favor of your arguments is that the
name 'stable' imply that they are without issues, ie. stable. However,
with a bit of ToM you may see that they are maintenance releases.
Gr. AvS
On Thu, Apr 12, 2012 at 2:44 PM, Linus Torvalds
<[email protected]> wrote:
> On Thu, Apr 12, 2012 at 2:34 PM, Linus Torvalds
> <[email protected]> wrote:
>>
>> So just reverting it from stable, *WITHOUT LEARNING THE LESSON*, is
>> not a no-op at all, it's a sign of being a f*cking moron.
>
> Btw, the revert is now in my tree (commit 011afa1ed8c4), and marked
> for stable. So *now* Greg can revert it from stable too.
>
> But the important lesson to everybody should be that "we don't lose
> fixes from -stable". If a problem was found in stable, it needs to be
> fixed upstream. In fact, quite often people *do* find problems in
> stable, because it tends to have more users more quickly than
> mainline. That makes it really really important to make sure that
> those problems get fixed upstream, and not hidden in stable due to
> some kind of dieseased "it's a no-op to revert it" thinking.
>
> End of story.
FWIW if people want stable fixes propagated faster (before Linus sucks
it in or Greg sucks it in after Linus) things like compat-wireless (to
be renamed to compat-drivers) allows us to get these out, but we label
them properly [0]. We in fact have a place holder for even other type
of non-upstream patches that any PHB may come up with as required but
-- the key is to
1) help categorize these properly
2) keep metrics of them
3) prioritize upstream first
The pending-stable/ patches get patches out to the community faster
than Linus / Greg can apply them or before we even get the community
to test them. We get these sucked out by looking for the Cc:
[email protected] The linux-next-cherry-pick/ allows you suck out
non-stable patches and I gladly accept such patches so long as they
are in linux-next and I can suck them out automatically if you Cc:
[email protected]. There are similar tricks for the other types of
patches.
[0] http://wireless.kernel.org/en/users/Download/stable/#Additional_patches_to_stable_releases
Luis
On Apr 13 Felipe Contreras wrote:
> On Fri, Apr 13, 2012 at 11:57 AM, Stefan Richter
> <[email protected]> wrote:
> > On Apr 12 Felipe Contreras wrote:
> >> But this is exactly the opposite; the patch that broke things is in
> >> the 'release branch' (3.3.1); it's not in upstream (3.3). Sure, it's
> >> also on a later upstream, which is also broken.
> > ^^^^^
> > No, upstream /earlier/ than 3.3.1 contains the defect.
>
> Time is not relevant for the point being made, but fine:
Time of occurrence of a regression in mainline and stable is indeed
irrelevant, as there can only be two kinds of regressions in stable:
A) First the regression was introduced into mainline, and accidentally it
was carried over from there into stable.
B) The regression only happened in stable because a backport from
mainline to stable went wrong, e.g. a prerequisite to the backport
was forgotten to be backported beforehand.
AFAIU we are talking about a regression of type A.
It seems you are arguing that stable candidate patches which fix
regressions of type A should be treated differently from other stable
candidate patches.
> But this is exactly the opposite; the patch that broke things is in
> the 'release branch' (3.3.1); it's not in the upstream release from
> where stable began (3.3). Sure, it's also on upstream, which is also
> broken.
(To what is this the opposite?)
So the defect is present in two kernel branches: Linus'es and Greg's.
The fix needs eventually go into both branches. For reasons which have
been enumerated many times in this thread already, Greg takes the fix from
Linus, not the other way around.
If you do not like to wait for Linus and Greg, you simply have to derive
an own kernel which additionally contains your preferred fixes.
The reasons for the Linus->Greg order of maintaining the stable series 100%
apply to fixes for type A regressions as well.
> > Furthermore, consider this: You as user of the 3.3.y series are using a
> > temporary, dead-end side branch. Its maintenance will stop at some point,
> > and you will be left looking for a different, maintained series to migrate
> > to. You will be most interested in that series /not/ containing any
> > regressions that you suffered already through the 3.3.y lifetime.
>
> Of course, I will be interested in that, although most likely I would
> be switching to another stable release (v3.4.1), not the upstream
> release (v3.4), and most distros would do the same.
Indeed.
> Even in the unlikely event that v3.4 is broken, most likely v3.4.1
> would contain the fix.
That would only happen if the upstream fix was published after v3.4 but
before Greg finished cherry-picking from Linus' post-3.4 git head.
> But I'm also interested in v3.3.2 working.
It's obviously a bit late for that, but v3.3.3 seems likely to bring the
fix.
> So you are saying that:
>
> a) v3.3.1 (bad), v3.3.2 (bad), v3.4 (good)
> b) v3.3.1 (bad), v3.3.2 (good), v3.4 (bad)
> c) v3.3.1 (bad), v3.3.2 (good), v3.4 (good)
[...]
Not exactly.
I and others are saying that procedures must ensure that if e.g. v3.3.3
was "good", then v3.4 and hence v3.4.y must be "good" too. ("Good" here
meaning "contains fix xyzabc".)
Furthermore I was saying that due to the time-based instead of
feature-based release schedule, the procedure which gives above guarantee
is a time-based procedure: Greg takes fixes *if* they were published by
Linus == *after* they were published by Linus.
I add: The second reason for this procedure is that v3.x.y is a successor
of v3.x but not of v3.x-1.y. Forward-porting from v3.x-1.y to v3.x.y is
not in scope of the stable series. The reasons why there is no
forward-porting done in stable series, again, have been mentioned several
times in this thread.
--
Stefan Richter
-=====-===-- -=-- -==-=
http://arcgraph.de/sr/
On Mon, Apr 16, 2012 at 01:12:48AM +0300, Felipe Contreras wrote:
> I'm not going to argue the semantics of what is a revert, but I am
> going to show the difference between the two situations:
>
> a) v3.0* (good), v3.1* (good), v3.2* (good), v3.3 (good), v3.3.1
> (bad), v3.3.2 (good), v3.4 (bad)
This situation is not possible thanks to the process : if v3.3.2 has
a fix that was not in 3.3.1, then this fix is also in 3.4.
> b) v3.0* (bad), v3.1* (bad), v3.2* (bad), v3.3 (bad), v3.3.1 (good), v3.4 (bad)
Same here.
> Maybe they are the same from certain point of view, but you just
> argued that what *others* see is what makes the patch unrevertable,
> well, it's obvious that from the point of view of the users the two
> situations are clearly different. I believe it was you who said that
> breaking user experience is the absolute no-no any project could
> make[1]--so from that point of view a) is *much* worst.
>
> But what is done is done, and as you said, you can't change the past,
> now the important thing is what to do next. And here are the two
> options' worst-case scenarios:
>
> a.1) v3.0* (good), ... v3.3 (good), v3.3.1 (bad), v3.3.2 (bad), v3.3.3
> (bad), v3.3.4 (bad), v3.3.5 (bad), v3.4 (good)
This situation is stupid as it means we'd refuse to fix the issue.
> a.2) v3.0* (good), ... v3.3 (good), v3.3.1 (bad), v3.3.2 (good),
> v3.3.3 (good), v3.3.4 (good), v3.3.5 (good), v3.4 (bad)
Not possible.
> These two scenarios are unlikely, but either way, in order to
> guarantee that you don't end up with a.2) You are willing to risk
> going into a.1)
You don't seem to understand that 3.4 is not *after* 3.3.x, but in parallel.
This is more like this :
3.2 ----- 3.3 --------- 3.4-rc* ------------- 3.4 ----- 3.5-rc* --------
\ \
\ `--- 3.4.1 ---- 3.4.2 --
\
`--- 3.3.1 ----- 3.3.2 ---- 3.3.3 ---- 3.3.4 --- 3.3.5 ---
Here you clearly see why everything in 3.3.x must come from upstream and
why it's important that 3.4 has every fix that 3.3 has.
> So, *obviously* v3.4 is more important than v3.3.x. I could argue that
> the users out there would prefer a.1) any day, because it's unlikely
> anyway (v3.4 would be good), but I won't.
No because for them it would mean end of support at one point when 3.3 dies,
with no ability to upgrade.
> All I want now is to agree on a reason, you have finally pointed out
> that the reason for this different treatment is the user's visibility,
It's not the user's visibility, it's published code. Once code is published,
you cannot magically fix it without emitting a new patch for this code and
announcing so that users apply it. These patches are called stable releases.
Users want a good reason to apply these patches, rebuild and reboot, and
that's one reason we absolutely want to have the commit descriptions from
upstream which detail the exact reason for the patch (even if it's a revert).
> but that still doesn't explain why the rules for b) automatically
> apply to a), since it's clearly different from the users's point of
> view; if you agree that v3.4 is more important than v3.3.x, I believe
> we have the reason right there.
It's not "more important", it's important for long-term stability. For
3.3 users, 3.3.x is more important that 3.4. However one thing is certain:
3.4 users are not going to push fixes into 3.3.x, but thanks to the process
we have, 3.3.x users are going to ask for a fix to be pushed upstream. So
users pressure ensure long-term stability.
Willy
On Tue, Apr 17, 2012 at 12:27 AM, Greg KH <[email protected]> wrote:
> On Tue, Apr 17, 2012 at 12:18:13AM +0300, Felipe Contreras wrote:
>> On Mon, Apr 16, 2012 at 11:58 PM, Greg KH <[email protected]> wrote:
>> > On Mon, Apr 16, 2012 at 11:11:05PM +0300, Felipe Contreras wrote:
>> >> On Mon, Apr 16, 2012 at 7:27 PM, Greg KH <[email protected]> wrote:
>> >> > Just one minor correction in this looney email thread:
>> >> >
>> >> > On Sat, Apr 14, 2012 at 01:53:22AM +0300, Felipe Contreras wrote:
>> >> >> v3.3.x on the other hand are *not* stable. They contain patches
>> >> >> backported from v3.4, but nobody guarantees they will work. There was
>> >> >> no v3.3.1-rc1, so the first time the patches compromising v3.3.1 were
>> >> >> generally tested together is in v3.3.1, at which point if somebody
>> >> >> finds issues, it's too late; bad patches are *not* going to be removed
>> >> >> in v3.3.2.
>> >> >
>> >> > Of course there was a 3.3.1-rc1, see the linux-kernel archives for the
>> >> > announcemen and the individual patches. kernel.org has the large patch
>> >> > itself if you like that format instead.
>> >>
>> >> I don't see it here:
>> >> http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=tags
>> >>
>> >> If you really want people to try it, why not tag it?
>> >
>> > That would be because I don't keep it in that tree. It is in a quilt
>> > tree you can find in the stable-queue.git repo, and I have never tagged
>> > -rc1 releases there. No one has ever asked for it before, so in the
>> > past 6 years of stable releases, I guess no one ever needed it.
>> >
>> > ketchup and tarballs seem to work well for others, perhaps you can use
>> > that as well (hint, ketchup on top of the linux-stable tree works just
>> > fine for testing this.)
>>
>> Perhaps the current process will be continue to be OK, but I do
>> believe a tagged v3.3.1-rc1 would have catched the ath9k issue.
>
> How exactly would that have helped here?
More people would have given it a try. Not that many people read the
mailing list, and the ones that do certainly might want to avoid
applying a big series of patches; even if their mail client makes it
easy (mine (Gmail) doesn't). A tag, and an announcement to give a try
would make it *much* easier.
> You point out:
>
>> I used to compile my own kernels and use your stable tree, but this a
>> new laptop and I was using Arch Linux which automatically updated to
>> v3.3.1, and with no network I had no way to revert to v3.2.x.
>> Fortunately I had the kernel sources available, but I wonder how many
>> people were completely stuck.
>
> Arch wouldn't have included a -rc in their kernel (unless they are
> crazy), so this would not have helped your situation at all.
There's *a lot* of people that got affected by the 3.3.1 release; we
don't need to break a lot of boxes to figure out there's an issue,
only a few would suffice, even one.
>> If some other 3.x.1 release get broken this way, I would seriously
>> consider tagging v3.x.1-rc1 as well. It works for Linus' tree.
>
> "this way" was for a very tiny subset of hardware, so odds are, if this
> happens again, it wouldn't be caught this way either. That subset just
> happened to show up in your machine, but, for example, not in the wide
> range of hardware I test with here, nor the machines that others test
> with.
It was certainly not just me. I have seen a lot of people mentioning
"their wifi is broken", a lot of them using Arch Linux,
coincidentally.
These issues would most likely not be caught before v3.x.1, and which
point it's too late, they cannot be reverted to v3.x.2 just like that;
they have to wait for upstream. Hopefully and probably everything
would go smooth like this time, but maybe not, we'll have to wait and
see. With more people using Arch Linux and thus the latest "stable"
release, I'd say we might see an increase in these kinds of issues.
Cheers.
--
Felipe Contreras
Felipe Contreras wrote:
> On Fri, Apr 13, 2012 at 4:42 PM, Stefan Richter wrote:
>> If you do not like to wait for Linus and Greg, you simply have to derive
>> an own kernel which additionally contains your preferred fixes.
>
> Yes, because clearly everybody thinks the process is perfect, and
> criticizing it is heresy.
Close. Not everyone. For example, you do not think the process is
perfect.
I don't think Stefan meant the above as tongue-in-cheek, for what it's
worth. Another stable kernel with different rules really would be an
interesting exercise, and would probably fulfill a need for a certain
audience.
It's not like nobody does that already, anyway. For example, I hear
Fedora has a kernel that they maintain well for a different audience,
using different rules.
Ciao,
Jonathan
On Fri, Apr 13, 2012 at 1:07 AM, Linus Torvalds
<[email protected]> wrote:
> On Thu, Apr 12, 2012 at 3:04 PM, Felipe Contreras
> <[email protected]> wrote:
>>
>> Sure, but removing that patch from the stable tree is not going the
>> change that information; we already know the patch is wrong.
>
> .. and we wait until it has been fixed in mainline so that we *know*
> that information doesn't get lost.
So why don't we pick potentially dangerous patches that might benefit
from some testing, put them in 'stable', and if there are problems,
make sure they get fixed in upstream first? Or for that matter totally
broken patches we want to make sure they get fixed in upstream.
Because the priority of the 'stable' tree is *stability*. Is it not?
But what you are saying is: *before* the final review, even a hint
that the patch might cause problems is reason enough to drop it from
stable, but *after* the review, if we know the patch is totally
broken, then it's the opposite; we really want it in.
--
Felipe Contreras
Hello,
Am 12.04.2012 02:29, schrieb Greg KH:
>> is there any chance for this one to be included in this review cycle?
>>
>> http://www.spinics.net/lists/linux-wireless/msg87999.html
>
> Have you read Documentation/stable_kernel_rules.txt? Based on that, I
> don't think it can, yet, right?
Hmm, after reading that, I think the text could need an update about how
to submit patches written by others (which can already be found in the
upstream tree).
At least for me the text reads like only the authors can request
inclusion of a patch into the stable tree.
Regards,
Alexander
On Thu, Apr 12, 2012 at 2:20 PM, Felipe Contreras
<[email protected]> wrote:
>
> I could argue in favor of exceptions, but I don't think you realize
> the fact that this change does not affect your tree *at all*. Adding
> and removing a patch in the stable tree is a no-op.
You're a fucking moron.
It's not a no-op at all, and you don't seem to understand it.
It's *information*.
It's "that patch didn't work". That's not a no-op. That's actual
useful and worthwhile knowledge.
To quote Thomas Edison: "I have not failed. I've just found 10,000
ways that won't work."
So just reverting it from stable, *WITHOUT LEARNING THE LESSON*, is
not a no-op at all, it's a sign of being a f*cking moron.
I'm stupider for just reading your email. Go away.
Linus
From: Felipe Contreras <[email protected]>
Date: Fri, 13 Apr 2012 01:04:42 +0300
> Wrong is wrong, before or after the 3.3.1 tag, this patch is not
> 'stable' material, and removing it does not affect upstream at all.
What you don't understand is that bug fixes will get lost if you only
fix them in -stable, it doesn't matter HOW THEY GOT into -stable.
In fact IT HAS FUCKING HAPPENED that we didn't fix something upstream
that got fixed in -stable a time long ago when we didn't have the
policy we're using now which you're going so unreasonably ape-shit
about.
And we didn't notice the discrepency until much later.
So the issues are real, there's proven precendence, and you're just
so wrong it's not even funny.
On Thu, Apr 12, 2012 at 04:03:59AM +0300, Felipe Contreras wrote:
> On Thu, Apr 12, 2012 at 3:29 AM, Greg KH <[email protected]> wrote:
> > On Wed, Apr 11, 2012 at 07:59:14PM -0400, Sergio Correia wrote:
> >> Hello Greg,
> >>
> >>
> >> On Wed, Apr 11, 2012 at 7:11 PM, Greg KH <[email protected]> wrote:
> >> > This is the start of the stable review cycle for the 3.3.2 release.
> >> > There are 78 patches in this series, all will be posted as a response
> >> > to this one. ?If anyone has any issues with these being applied, please
> >> > let me know.
> >> >
> >> > Responses should be made by Fri Apr 13 23:10:16 UTC 2012.
> >> > Anything received after that time might be too late.
> >> >
> >>
> >> is there any chance for this one to be included in this review cycle?
> >>
> >> http://www.spinics.net/lists/linux-wireless/msg87999.html
>
> I was going to ask for exactly the same thing. My system is completely
> unusable without this patch; not only does the network doesn't work,
> but quite often the kernel is stuck consuming 100% of the CPU.
>
> > Have you read Documentation/stable_kernel_rules.txt? ?Based on that, I
> > don't think it can, yet, right?
>
> Why not? This patch makes the code go back to a previous state, it is
> obviously more stable than the current state, and the code already
> exists in Linus's tree (in previous releases).
It does? What is the git commit id of the patch? Based in the email
above, I assumed it had not made it to Linus's tree already.
> But hey, I guess it's OK that 3.3.x is stuck in and endless loop right
> after booting, because rules are more important than fixing obvious
> breakage.
What rule did you think I was saying this was not acceptable for?
greg k-h
Look, this is getting ridiculous.
I think you're completely wrapped up in the "git" way of thinking about things.
The patch can't be "dropped". It can't be "reverted". It can't be
"removed". All that Linus can do is apply a reversed patch. Then Greg
can sync up however he does it.
You seem completely wrapped up in some misguided notion that the patch
can be dropped from some series of patches. This isn't OpenWRT - you
don't have a directory of a hundred-odd patches. It's a source
revision system, complete with history. You don't remove a patch from
an existing repository - you commit a fix. That may be a reversed
version of the actual commit - but the effect is the same, the change
is "reverted". It just isn't removed from the history.
I think this is done and dusted. :-)
Adrian
On Sun, Apr 15, 2012 at 12:21 AM, Stefan Richter
<[email protected]> wrote:
> On Apr 14 Felipe Contreras wrote:
>> On Sat, Apr 14, 2012 at 8:55 PM, Stefan Richter
>> <[email protected]> wrote:
>> > Generally, "commit + push out" makes it undroppable. In case of -stable,
>> > commit/ push out/ tag are close and virtually identical.
>> >
>> > After a change was pushed out, the choice narrows down to add a reverting
>> > change for a subsequent release or to rebase to a point before the
>> > defective commit. The latter is not done to -stable, obviously. (3.3.2
>> > is not being restarted from 3.3 if a regression from 3.3 to 3.3.1 was
>> > discovered.)
>>
>> That's irrelevant; whether you rebase and drop the patch, or you
>> revert it, the resulting code is *exactly* the same. It's also the
>> same if Greg drops it from his quilt queue before pushing (assuming
>> that's what he uses).
>
> The result may be the same (sometimes it actually isn't),
If the result is not the same, then that's a different situation; I'm
talking about true reverts/drops in which the result is *exactly* the
same. OK?
>> Let's avoid this semantic red herring, by undroppable I mean
>> unrevertable, or undiscardable, or anything that effectively makes the
>> patch not be there.
>
> How do you "make it not be there"? By adding a reverting patch.
>
> The reverting patch needs to come from somewhere, and the only
> disagreement is basically through which channels the patch should be
> allowed to come, or which qualifications this reverting patch should
> fulfill.
If the patch was tagged in a release, yes, but if the patch was in the
queue, but never tagged, it can be dropped.
The question that has not been answered is what makes them different, and why.
>> >> But *why*? You say you *really* need to problem to fixed in mainline,
>> >> that's why it cannot be dropped, but that applies also to patches in
>> >> the queue *before* the tag is made, doesn't it? If you find a patch in
>> >> the stable review queue causes problems, why don't you leave it there?
>> >
>> > As Willy wrote, that's actually what is done sometimes. I didn't know
>> > that.
>>
>> Don't avoid the question.
>
> Willy answered that, hence I didn't think I have to. The patch can in
> fact be kept in the stable queue as a reminder, it just should not be
> applied to stable as long as there is no resolution for mainline.
>> I don't mean just leave it in the queue, I mean leave it so it gets
>> tagged in the release.
>
> That would be even sillier than the discussion which we are having.
Exactly!
I'm glad you see it's silly to put bad patches in the stable release
just so they get properly tracked for mainline, but that's exactly
what you arguing for.
Now all you have to do is explain why it's silly before the tag, but not after.
--
Felipe Contreras
On Sat, Apr 14, 2012 at 10:41 AM, Stefan Richter
<[email protected]> wrote:
> On Apr 14 Felipe Contreras wrote:
>> I already exemplified how they are very different, but here it goes
>> again. The patch "drm/i915: Add lock on drm_helper_resume_force_mode"
>> was just tagged in 3.3.2, if I had said yesterday "this patch breaks
>> things on my machine", then that patch would have been dropped for
>> 3.3.2 even though it's still on mainline--why? Because we know it's
>> broken, and broken patches are not supposed to land to stable. But
>> today, one day later, we have to wait until it's fixed in master
>> first. Why?
>>
>> What makes a patch droppable yesterday, but dependent on mainline today?
>
> Huh?
>
> Because "yesterday" was a review before stable release:
> - A buggy mainline release exists.
> - No buggy stable release exists.
> whereas "today" is after stable release:
> - A buggy mainline release exists.
> - A buggy stable release exists.
IOW; a tag makes undroppable.
But *why*? You say you *really* need to problem to fixed in mainline,
that's why it cannot be dropped, but that applies also to patches in
the queue *before* the tag is made, doesn't it? If you find a patch in
the stable review queue causes problems, why don't you leave it there?
You *really* need to problem fixed in mainline, don't you?
So yesterday the priority is stability > 'upstream first', but today
it's 'upstream first' > stability. Nobody has put forward an argument
for this shift in priorities--"a tag was made" is not argument.
> (The WLAN breakage wich is being talked about was reported after
> release, not during review, right?)
Yeah, this is a hypothetical thought experiment to demonstrate the
shift in priorities.
> [quote re-ordered]
>> Again, you can repeat the same thing as much as you want, it still
>> doesn't answer what I have asked.
>
> Yeah, sorry for that. All the time I thought you asked why a *revert*
> which is applicable to mainline and stable first happens in stable.
>
> But your question was actually what the difference between
> - dropping a patch from a queue of candidate patches versus
> - adding a reverting patch to repair a defect from a previous release
> is.
Yes, that is one of my arguments.
> The former is still part of the decision whether a changeset which
> exists in mainline should be backported into stable. Somebody initially
> thought it should be, but others should have a look too and amend that
> decision if need be. Somebody reports that the change would introduce a
> regression. Usually, this disqualifies a patch from being applied to
> stable right away, without further work having to be done in stable.
>
> "Drop a stable candidate before release" is a form of "decline a
> submission to stable", happening late in the preparations of a stable
> release.
>
> The latter is when damage was done; it is now about bug fixing. This
> involves debugging of the regression, finding a right approach to
> fix it (e.g. revert), write + review + test + commit the fix, port the fix
> to all relevant other kernel branches, review + test + commit those ports
> too.
Really? So if the patch doesn't make it to stable you don't need to do
any of that? IOW; if the patch doesn't make it to stable, it's OK to
leave it broken for v3.4? There's 10000 patches in v3.4-rc* that are
all about bug fixing, they don't need to go into stable to be fixed,
do they? If a non-stable patch needs to be reverted in mainline, it
gets reverted in mainline, regardless of weather or not it made it to
stable or not.
So, the hypothetical patch that was dropped in the stable review queue
yesterday has to be fixed in mainline too, just like the hypothetical
patch that made it to stable and was found problematic today has to be
fixed in mainline.
Again, what makes a released patch undroppable? It's not the bug
fixing; weather or not it gets into stable, it still has to be fixed
in mainline.
> "Add a reverting fix" is a form of "add a fix".
>
> You are indeed pointing to a procedural flaw here. In "add a fix" cases,
> stable release procedures ensure that if 3.3.3 included the revert, 3.4
> will include it to, by the Linus->Greg order of commiting patches. But in
> the "drop a candidate before release" case, Greg and the stable reviewers
> do not have a similarly fool-proof procedure to ensure that the next branch
> point will be free of the regression. Now they need to watch closely that
> a fix gets into mainline ideally before the next branch point is going to
> be released.
I don't see the procedural flaw here. There's 10000 patches that never
go through the stable review process, and they don't need to. Somehow
v3.4 will be relatively OK. So the dropped patches from the stable
review queue will just receive the same treatment as any other patch.
Just by going through the stable review process the patch already
received more eyes to make the bugs more shallow. There might be a lot
of trees out there where people pick patches from mainline, and they
don't *need* to follow the "upstream first" rule, by reviewing the
patch, and maybe even testing it, and then reporting the problem, they
are helping getting it fixed for v3.4.
You don't *need* to keep the patch in the stable review queue, you
don't *need* to make a stable release with it in order to ensure that
it will fixed in mainline, the information about the patch problems is
not lost.
Seems to me you are abusing the 'stable' branch as a bug tracking
system for certain patches for the next release.
> So there is indeed a difficulty involved with dropping patches from the
> candidate queue. Fortunately, it is not necessary to subject post-release
> reverts to the same difficulty.
It's the other way around.
--
Felipe Contreras
On Thu, Apr 12, 2012 at 07:49:34PM +0300, Felipe Contreras wrote:
> On Thu, Apr 12, 2012 at 5:46 PM, Greg KH <[email protected]> wrote:
> > A revert is the same as a patch. ?It needs to be in Linus's tree before
> > I can add it to the stable releases.
>
> Right, because otherwise people's systems would actually work.
>
> But hey, as I said, following rules is more important, regardless of
> what the rules are, and why they are there. The rules that actually
> triggered this issue in v3.3.1, as this is not in v3.3.
>
> You could just accept that the patch should have never landed in
> v3.3.1 in the first place, but it's much easier to arbitrarily keep
> stacking patches without thinking too much about them.
>
> I guess I'll better use Linus's releases for now and go to v3.3.
Felipe, you're unfair to Greg. You're saying this because you don't know
what it's like to be on the side of the guy who has to decide whether
to apply a patch or not, which basically means whether to risk breaking
systems or to leave broken systems as they are.
Most stable users will switch from a stable version to another one
in a next release, and these users do not want any regression. This
means that we absolutely don't want to risk that a stable version
has a fix that is missing from a newer version. Yes this is a crappy
and annoying process but it's the only way to ensure that fixes don't
get lost during an upgrade.
BTW, it works because if we're discussing this here, it's because
this final fix or revert appears to be missing from mainline. After
the discussion I'm sure it will be there.
Last point is that stable maintainers are not your slaves. If you know
how to patch your mainline kernel to apply a stable patch, you also
know how to apply the patch you're pointing to. It's quite easy and
many of us do that. *All* the kernels I'm using are stable + a few
local patches and I don't see where the problem is. So please be a bit
more constructive. Make your job by ensuring that the fix you want is
in mainline then pass the commit ID to Greg so that he can merge it in
next release. You'll feel relieved and everyone will be glad to benefit
from this fix.
Willy
On Wed, Apr 11, 2012 at 07:59:14PM -0400, Sergio Correia wrote:
> Hello Greg,
>
>
> On Wed, Apr 11, 2012 at 7:11 PM, Greg KH <[email protected]> wrote:
> > This is the start of the stable review cycle for the 3.3.2 release.
> > There are 78 patches in this series, all will be posted as a response
> > to this one. ?If anyone has any issues with these being applied, please
> > let me know.
> >
> > Responses should be made by Fri Apr 13 23:10:16 UTC 2012.
> > Anything received after that time might be too late.
> >
>
> is there any chance for this one to be included in this review cycle?
>
> http://www.spinics.net/lists/linux-wireless/msg87999.html
Have you read Documentation/stable_kernel_rules.txt? Based on that, I
don't think it can, yet, right?
thanks,
greg k-h
Felipe,
On Sun, Apr 15, 2012 at 08:15:23PM +0300, Felipe Contreras wrote:
(...)
> Why is it that the patch from yesterday doesn't reduce the version
> skew, but the patch from today does? You are not explaining *why*.
Severl of us have explained to you multiple times *why*. Either you're
having a parsing problem, or you don't understand anything to versionning
but in all cases you should try to educate yourself a minimum before
repeating the same question in loops.
> It's like I'm asking why the freezer area is different from the normal
> refrigerator area, and you are saying; well it's colder, it's usually
> smaller, and it makes all food taste better. So you are either just
> describing the differences without explaining the reason (being colder
> helps for some foods), or you are providing an unsubstantiated claim;
> you go from a) it's colder, to b) it makes all food taste better. And
> when asked *why* again, you repeat, well, because it's colder. And
> then you immediately go to the conclusion; therefore you should put as
> much food in the freezer as possible.
No, it's quite the opposite, almost everyone in this thread explained to
you that bacteria don't develop in cold and that's the reason it's better
to keep food. But you deliberately (or maybe inconsciously because you
don't know what bacteria are) wipe out these explanations and pretend
nobody explained the reason to you.
Please re-read all responses, everything is there. I thought my explanation
was detailed enough to understand why it works that way, and Ingo explained
it in great details again. So please now make an effort to try to read what
people explain and stop pretending they refuse to respond, that's getting
really irritating.
The fact that you're the only one to ask this in loops should ring a bell,
it seems like many people managed to understand, either before this thread
or during this thread. It's up to you, I don't think that insisting on
people to explain the same things to you in many different ways will solve
your problems, really.
Regards,
Willy
On 12.04.2012, Sergio Correia wrote:
> is there any chance for this one to be included in this review cycle?
>
> http://www.spinics.net/lists/linux-wireless/msg87999.html
Thanks for pointing this out! This patch fixes my network problems
which forced me to go back to a previous kernel.
On Mon, Apr 16, 2012 at 8:32 AM, Ingo Molnar <[email protected]> wrote:
>
> * Felipe Contreras <[email protected]> wrote:
>
>> On Sun, Apr 15, 2012 at 8:49 PM, Linus Torvalds
>> <[email protected]> wrote:
>> > The only thing that matters is "it's been made available to others".
>>
>> Exactly! Now *this* is a reason. [...]
>
> Erm, many people referred to that in this discussion explicitly
> and implicitly - beyond its well-known technical meaning it's
> also the English meaning of the word 'release':
>
> http://www.merriam-webster.com/dictionary/releasing
Releasing implies many things. There is this notion called theory of
mind, which children learn at a very young age, that helps us see the
world through the eyes of other people. If you think people *must* see
the word "release" and think *exactly* the same thing as you do, and
apologize when they don't, well, I'd say you are missing a very
valuable skill.
But fine, lets suppose I should have thought exactly the same thing as
you did, and fine, lets say I was wrong in not doing so. That still
doesn't help one iota in this discussion. Linus Torvalds pointed the
exact reason, which allowed us to move forward, because of my
incompetence, or whatever.
The next logical question has still not been answered; does the
undroppability come from the fact that v3.4 is more important that
v3.3.1? This time I promise to think at least one day for all the
possible meanings of each key word you say.
--
Felipe Contreras
On Thu, Apr 12, 2012 at 8:24 PM, Adrian Chadd <[email protected]> wrote:
> On 12 April 2012 09:49, Felipe Contreras <[email protected]> wrote:
>
>>>
>>> A revert is the same as a patch. It needs to be in Linus's tree before
>>> I can add it to the stable releases.
>>
>> Right, because otherwise people's systems would actually work.
>>
>> But hey, as I said, following rules is more important, regardless of
>> what the rules are, and why they are there. The rules that actually
>> triggered this issue in v3.3.1, as this is not in v3.3.
>>
>> You could just accept that the patch should have never landed in
>> v3.3.1 in the first place, but it's much easier to arbitrarily keep
>> stacking patches without thinking too much about them.
>
> Greg is doing the right thing here. We face the same deal in FreeBSD -
> people want fixes to go into a release branch first, but if you do
> that you break the development flow - which is "stuff goes into -HEAD
> and is then backported to the release branches."
>
> If you don't do this, you risk having people do (more, all)
> development and testing on a release branch and never test -HEAD (or
> "upstream linux" here). Once you open that particular flood gate, it's
> hard to close.
But this is exactly the opposite; the patch that broke things is in
the 'release branch' (3.3.1); it's not in upstream (3.3). Sure, it's
also on a later upstream, which is also broken.
But then are you saying that if upstream is broken (3.4-rc2), then
stable should be broken as well (3.3.1), and remain broken until
upstream is fixed? I fail to see what would be the point of that.
> We had this problem with Squid. People ran and developed on Squid-2.4.
> The head version of Squid-2 was stable, but that isn't what people ran
> in production. They wanted features and bugfixes against Squid-2.2,
> squid-2.4, and not Squid-2.STABLE (which at the time was
> Squid-2.6/Sqiud-2.7.) That .. didn't work. Things diverged quite
> quickly and it got very ugly.
And why do you think the same would happen here if *one patch* is applied?
Plus, git is developed this way; and yes, you might say the gates are
opened when there's a new non-maintenance release, but the same
happens in Linux. It's not the rule of 'first on X' branch that keeps
the gates safe; it's the amount of patches.
> So I applaud Greg for sticking to correct stable release engineering
> here. We over in the BSD world know just how painful that is. :)
So, in your mind "correct" is "never ever do an exception", even if
this strictness leads to less stability. If the objective is not
stability, I would call this the 'backports' tree then, which might or
might not lead to stability.
Rules are not perfect, why not add a new rule "It reverts an earlier
patch to 'stable'.", then you would be both following the rules, and
ensuring more stability :)
Cheers.
--
Felipe Contreras
On Sat, Apr 14, 2012 at 2:05 AM, Jonathan Nieder <[email protected]> wrote:
> Felipe Contreras wrote:
>> On Fri, Apr 13, 2012 at 4:42 PM, Stefan Richter wrote:
>
>>> If you do not like to wait for Linus and Greg, you simply have to derive
>>> an own kernel which additionally contains your preferred fixes.
>>
>> Yes, because clearly everybody thinks the process is perfect, and
>> criticizing it is heresy.
>
> Close. Not everyone. For example, you do not think the process is
> perfect.
So you think the process is *perfect*? I would have expected
reasonable people to know that nothing is perfect, realize the
sarcasm, and meditate for a second. But it seems expecting somebody to
agree the process is not perfect is too much to ask.
> I don't think Stefan meant the above as tongue-in-cheek, for what it's
> worth. Another stable kernel with different rules really would be an
> interesting exercise, and would probably fulfill a need for a certain
> audience.
>
> It's not like nobody does that already, anyway. For example, I hear
> Fedora has a kernel that they maintain well for a different audience,
> using different rules.
Of course, although the difference with the stable kernel would be
very small if the only thing added is an extra rule for acceptance:
"It reverts an earlier
patch to 'stable'."
But "we do this, and if you don't like it you can do whatever you
want" is not a valid argument in favor of the status quo, even though
it's used a lot in open source, and it's true, and there's nothing
wrong with that... I yet have to see an answer to my arguments, and
not a regurgitated answer for something nobody is proposing.
Cheers.
--
Felipe Contreras
On Sat, Apr 14, 2012 at 02:18:33AM +0300, Felipe Contreras wrote:
> On Sat, Apr 14, 2012 at 2:05 AM, Jonathan Nieder <[email protected]> wrote:
> > Felipe Contreras wrote:
> >> On Fri, Apr 13, 2012 at 4:42 PM, Stefan Richter wrote:
> >
> >>> If you do not like to wait for Linus and Greg, you simply have to derive
> >>> an own kernel which additionally contains your preferred fixes.
> >>
> >> Yes, because clearly everybody thinks the process is perfect, and
> >> criticizing it is heresy.
> >
> > Close. ?Not everyone. ?For example, you do not think the process is
> > perfect.
>
> So you think the process is *perfect*? I would have expected
> reasonable people to know that nothing is perfect, realize the
> sarcasm, and meditate for a second. But it seems expecting somebody to
> agree the process is not perfect is too much to ask.
Nobody has said it's perfect. Jonathan said it's "close" to perfect. I
personally think it's the best tradeoff we could find between a perfectly
stable branch and a perfect mainline. We manage to converge towards the
best quality in both branches by accepting a small delay in -stable.
The problem is that *you* don't accept to wait as soon as you know there's
a bug, and *you* don't want to make the necessary efforts to make things
move on. The process is clear and easy and has been explained to you several
times. If *you* want a fix, *you* just have to ask the fix's author to push
it to mainline and ask Greg to include it. There will always be a delay
between the moment a fix is written and the moment it boots on your host
anyway.
How would you have acted with 2.6.32 when there was this bug preventing
machines from living more than 208 days ? You think that just complaining
loudly that it's unacceptable to have this bug would have made things go
better ? Very few people were experiencing it and even fewer people were
able to understand it. Fortunately some users deployed a lot of efforts in
providing traces, configs, etc to narrow the bug down and it eventually got
fixed something like one year after the first report, thanks to everyone's
involvment. Linux is not only Linus or Greg's job, it's everyone's. You
have to take your part when you feel concerned about something. If you
don't want to participate, that's fine. Use a vendor's distro and file
a bug report, they'll do this job themselves and in the mean time they'll
probably provide you with the fix.
> > I don't think Stefan meant the above as tongue-in-cheek, for what it's
> > worth. ?Another stable kernel with different rules really would be an
> > interesting exercise, and would probably fulfill a need for a certain
> > audience.
BTW this has been done a lot in the 2.4 era, by Alan, Andrew, Andrea, MCP,
myself. Yes it's interesting and useful. Experience shows that this ends
at one point because it requires a lot of work. And at this time there were
no such rules and it was very common to see that -ac or -aa kernels were
more stable than mainline for some workloads, which was a bit problematic,
especially when these kernels had to be rebased onto newer versions and
some patches had to be dropped at the risk of losing some stability fixes.
> > It's not like nobody does that already, anyway. ?For example, I hear
> > Fedora has a kernel that they maintain well for a different audience,
> > using different rules.
>
> Of course, although the difference with the stable kernel would be
> very small if the only thing added is an extra rule for acceptance:
> "It reverts an earlier patch to 'stable'."
Reverting patches that were not appropriate for -stable happens from
time to time, but only when the issue is specific to -stable (eg: build
issues). Here what you don't seem to understand is that the bug was
not specific to -stable but was present everywhere. So we had a bug
in mainline that we put in -stable, and we want mainline to be fixed
and we use -stable as a guarantee that mainline will be fixed. And it
works and has never failed yet. That's not hard to understand I think.
> But "we do this, and if you don't like it you can do whatever you
> want" is not a valid argument in favor of the status quo, even though
> it's used a lot in open source, and it's true, and there's nothing
> wrong with that... I yet have to see an answer to my arguments, and
> not a regurgitated answer for something nobody is proposing.
You've had many answers but you seem to selectively filter them. What you're
doing looks to me like "grep agree-with-me /dev/urandom && echo I was right".
Can take some time but it will eventually succeed. But please if you don't
want to listen to what people explain to you, at least stop polluting their
mailboxes with your looping arguments.
Willy
On Fri, Apr 13, 2012 at 10:08 PM, Peter Stuge <[email protected]> wrote:
> Felipe Contreras wrote:
>> I guess I should avoid the "stable" series then.
>
> I wish you had understood this much much sooner so that this nonsense
> thread could have been avoided.
>
> If you want the very latest fixes then *obviously* you need to use
> the most bleeding edge repo. (Linus')
No, I don't want the latest fixes, I want the latest *stable* kernel.
v3.3 is stable, v3.4-rcx are not. v3.4 would take months to cook,
there will be several release candidates, and it won't be released
until the known issues decrease to a reasonable level.
v3.3.x on the other hand are *not* stable. They contain patches
backported from v3.4, but nobody guarantees they will work. There was
no v3.3.1-rc1, so the first time the patches compromising v3.3.1 were
generally tested together is in v3.3.1, at which point if somebody
finds issues, it's too late; bad patches are *not* going to be removed
in v3.3.2. Once a tag is made, all the patches in it are dependent on
the pace of the development of mainline (v3.4-rcx), which is
definitely not stable, specially in the first release candidates.
IOW, the "stable" branch tries to be stable up to a point, then, it
becomes a testing ground for mainline, and a tracking device for
certain mainline issues.
--
Felipe Contreras
On Mon, Apr 16, 2012 at 11:11:05PM +0300, Felipe Contreras wrote:
> On Mon, Apr 16, 2012 at 7:27 PM, Greg KH <[email protected]> wrote:
> > Just one minor correction in this looney email thread:
> >
> > On Sat, Apr 14, 2012 at 01:53:22AM +0300, Felipe Contreras wrote:
> >> v3.3.x on the other hand are *not* stable. They contain patches
> >> backported from v3.4, but nobody guarantees they will work. There was
> >> no v3.3.1-rc1, so the first time the patches compromising v3.3.1 were
> >> generally tested together is in v3.3.1, at which point if somebody
> >> finds issues, it's too late; bad patches are *not* going to be removed
> >> in v3.3.2.
> >
> > Of course there was a 3.3.1-rc1, see the linux-kernel archives for the
> > announcemen and the individual patches. ?kernel.org has the large patch
> > itself if you like that format instead.
>
> I don't see it here:
> http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=tags
>
> If you really want people to try it, why not tag it?
That would be because I don't keep it in that tree. It is in a quilt
tree you can find in the stable-queue.git repo, and I have never tagged
-rc1 releases there. No one has ever asked for it before, so in the
past 6 years of stable releases, I guess no one ever needed it.
ketchup and tarballs seem to work well for others, perhaps you can use
that as well (hint, ketchup on top of the linux-stable tree works just
fine for testing this.)
greg k-h
* Felipe Contreras <[email protected]> wrote:
> On Sun, Apr 15, 2012 at 8:49 PM, Linus Torvalds
> <[email protected]> wrote:
> > On Sun, Apr 15, 2012 at 10:15 AM, Felipe Contreras
> > <[email protected]> wrote:
> >>
> >> This is not a reason, this is just stating what happens without
> >> explaining *why*.
> >>
> >> Q: What changes when a tag is made?
> >> A: A tag is made
>
> [...]
>
> > The only thing that matters is "it's been made available to others".
>
> Exactly! Now *this* is a reason. [...]
Erm, many people referred to that in this discussion explicitly
and implicitly - beyond its well-known technical meaning it's
also the English meaning of the word 'release':
http://www.merriam-webster.com/dictionary/releasing
: [...] to make available to the public [...]
So stop pretending that this is somehow the mistake of *others*
explaining it to you wrong ...
Just admit it already that you were trivially and dead wrong all
along, and that you were pretty difficult and obnoxious about
handling a discussion that went against you.
Being wrong is normal and it happens all the time - and a big
part of the picture is how you handle such situations, to not
try to wriggle out of the responsibility of being wrong and to
not waste other people's time with such self-gratifying mental
obfuscation.
Such as:
> I'm not going to argue the semantics of what is a revert,
> [...]
You should not argue about the semantics about a revert not
because somehow it's not important (it is), but because you've
been wrong about this issue too. You should learn and improve.
Thanks,
Ingo
On Fri, Apr 13, 2012 at 8:34 AM, Willy Tarreau <[email protected]> wrote:
> On Fri, Apr 13, 2012 at 01:58:10AM +0300, Felipe Contreras wrote:
>> On Fri, Apr 13, 2012 at 1:12 AM, David Miller <[email protected]> wrote:
>> > From: Felipe Contreras <[email protected]>
>> > Date: Fri, 13 Apr 2012 01:04:42 +0300
>> >
>> >> Wrong is wrong, before or after the 3.3.1 tag, this patch is not
>> >> 'stable' material, and removing it does not affect upstream at all.
>> >
>> > What you don't understand is that bug fixes will get lost if you only
>> > fix them in -stable, it doesn't matter HOW THEY GOT into -stable.
>>
>> Let's suppose that c1afdaf was never back-ported from v3.4-rc1, how
>> would you have fond out there was an issue with it? There's 10000
>> patches in v3.4-rc2, how do you expect to find issues in them?
>>
>> People found out this issue on v3.4-rc1, so the fix would not have
>> been lost, but lets assume it would, v3.3.1 had the issue, the patch
>> as reverted in v3.3.2, and v3.4 still had the issue. So what? There's
>> already 10000 patches that would never make it to 3.3.x, and many will
>> have issues, which is why there would be v3.4.x.
>>
>> > In fact IT HAS FUCKING HAPPENED that we didn't fix something upstream
>> > that got fixed in -stable a time long ago when we didn't have the
>> > policy we're using now which you're going so unreasonably ape-shit
>> > about.
>>
>> I see how a *fix* on stable could get lost, but this is not a fix.
>
> Felipe, you don't seem to get it : there are many bugs in each new release.
> Given the number of fixes Greg merges into a longterm branch, I'd say that
> there are around 1500 bugs waiting to be discovered and fixed in a new
> release. Does this mean we need to fix them all at once ? No, because we
> don't know about them yet.
>
> The process you're criticizing consists in ensuring that once a bug is known,
> it gets fixed in mainline so that it never appears there again. The way the
> bug is discovered doesn't matter, even if it's discovered that a fix caused
> the bug and that it must be reverted. The fact is mainline is buggy and we
> know this because stable is too. So mainline must be fixed first. This
> process works because stable users are pressuring developers to push their
> fixes to Linus in order to get them. What happened with this bug prooved
> the process is working fine.
Let's list the scenarios:
a) normal patch
v3.3 (good), v3.4 (+) (good)
b) normal stable patch
v3.3 (good), v3.3.1 (+) (good), v3.4 (+) (good)
c) regression patch
v3.3 (good), v3.4 (+) (bad)
d) regression patch, fixed
v3.3 (good), v3.4 (good)
e) stable regression patch
v3.3 (good), v3.3.1 (+) (bad), v3.4 (+) (bad)
e.1) stable regression patch, normal fix
v3.3 (good), v3.3.1 (+) (bad), v3.3.2 (good), v3.4 (good)
e.2) stable regression patch, lost fix
v3.3 (good), v3.3.1 (+) (bad), v3.3.2 (good), v3.4 (+) (bad)
As you can see, even in the worst-case scenarios, there's no
difference between (c) and (e.2). But what you are saying is that it
doesn't matter at which point the issue with the patch is found, (e.2)
has to be avoided *at all costs*, but you don't explain _why_. What is
so different between (c) and (e.2)?
And this is the worst-case scenario, I keep hearing people that this
has happened in the past, but I don' think so, I think what has
happened is:
f) stable patch fix, lost
v3.3 (bad), v3.3.1 (+) (good), v3.4 (bad)
That I can see happening, and the current rules ensure that would not
happen, but (e.2)? I yet have to see any evidence of this happening in
the past.
But lets be realistic; most likely the issue would be and fixed in
upstream (d), so it doesn't matter what happens in stable, the end
result would be the same (e.1). In fact in this particular patch
people found problems in v3.4-rc1, so all evidence points out that we
would have ended up in (e.1), not (e.2).
So, if we expand the possibilities in the current situation, we have:
0) v3.3 (good), v3.3.1 (good), v3.3.2 (good), v3.3.3 (good), v3.4 (+)
(bad), v3.4.1 (good)
1) v3.3 (good), v3.3.1 (+) (bad), v3.3.2 (good), v3.3.3 (good), v3.4
(good), v3.4.1 (good)
2) v3.3 (good), v3.3.1 (+) (bad), v3.3.2 (+) (bad), v3.3.3 (good),
v3.4 (good), v3.4.1 (good)
3) v3.3 (good), v3.3.1 (+) (bad), v3.3.2 (good), v3.3.3 (good), v3.4
(+) (bad), v3.4.1 (good) #unlikely
4) v3.3 (good), v3.3.1 (+) (bad), v3.3.2 (+) (bad), v3.3.3 (+) (bad),
v3.4 (+) (bad), v3.4.1 (good) #unlikely
It looks like the patch is going both to upstream and stable (1),
which is ideal, but when faced with the option between (2) and (3),
you say (3) must absolutely be avoided even though it's basically the
same as (0), which is the norm for thousands of patches that don't get
back-ported to stable (and it's also unlikely to happen).
Why?
Plus, (1) (2) (3) (4) are already bad situations, and should be
avoided at all costs; patches to stable are not supposed to be
potentially dangerous, they are not meant be breaking things.
> Another point is that you don't want stable to merge, revert, merge again,
> revert again etc... This happened a little bit during 2.6.32 because some
> fixes were not really obvious. It's common for some fixes to have to be
> adapted for stable branches, and to have side effects, hence the review
> cycle. We need to limit these random issues as much as possible if we
> don't want users to lose trust in the stable branches. This is extremely
> important. So picking random fixes that have not been qualified by all
> interested parties in stable is inappropriate. Reverting without evaluating
> impacts is one form of picking a random fix.
Yeah, but that is not the case here, the options are clear; (a) go
back to a previous state where power management doesn't work
correctly, (b) stay in the current state where the system goes to a
completely unusable state.
> What you should have done would have been to reply to Greg saying "wait a
> minute, we still have an issue with patch XX, I'm trying to get it reverted
> in upstream and will send you the commit ID, it would be nice to have it in
> 3.3.2". It wastes less time for everyone and achieves the same result.
There's a lot of people affected by this issue, and a lot of noise.
Personally I didn't receive the revert patch, so I could not comment
on it. I think this patch should have been sent to LKML, but one
cannot expect everyone to do the perfect thing all the time.
> Once again, if you think that the stable branch you're using is not stable
> enough for you, pick another one. Greg maintains multiple branches so that
> everyone is satisfied. The risk of bugs over time probably looks like
> (cos(t)+1)/t. Find an older branch with a much smaller risk of regressions
> and be done with it.
I'm not sure I would want to use 'stable' anymore, because clearly,
the main goal doesn't seem to be *stability* as I thought. Apparently
it's supposed to be a testing ground for patches queued for the next
release.
> Last point, you should note that you're the only one here who doesn't
> understand the process. That doesn't make you a fool, but it should tell you
> that you probably need to think a bit further before telling people how they
> should work, especially when all other ones agree on the benefits of the
> process, including Arnd explaining that FreeBSD had been facing the exact
> same trouble and now applies the same process. It is not just a small band
> of nerds doing this for fun right here, but seems to be more generalized.
Ad populum.
The fact that I'm questioning the process doesn't mean I don't
understand it. But if you are not open to criticism, fine.
Cheers.
--
Felipe Contreras
On Sat, Apr 14, 2012 at 8:44 AM, Willy Tarreau <[email protected]> wrote:
> Nobody has said it's perfect. Jonathan said it's "close" to perfect. I
> personally think it's the best tradeoff we could find between a perfectly
> stable branch and a perfect mainline. We manage to converge towards the
> best quality in both branches by accepting a small delay in -stable.
You are saying the same thing; it cannot be improved.
> The problem is that *you* don't accept to wait as soon as you know there's
> a bug,
I'm going to stop right here. You are making an eloquent argument for
something nobody is saying.
This is pure straw man argumentation, see the reply from Stefan
Richter, who seems to be the only person who is actually following the
argument I'm making.
> Reverting patches that were not appropriate for -stable happens from
> time to time, but only when the issue is specific to -stable (eg: build
> issues). Here what you don't seem to understand is that the bug was
> not specific to -stable but was present everywhere. So we had a bug
> in mainline that we put in -stable, and we want mainline to be fixed
> and we use -stable as a guarantee that mainline will be fixed. And it
> works and has never failed yet. That's not hard to understand I think.
I understand you use 'stable' as guarantee, and I know it works, but
do you *need* this guarantee?
And before you go on why you need this guarantee to avoid fixes to be
lost, this is an *entirely different thing*; we are not talking about
fixes in 'stable' that don't exist in mainline--for which there is
evidence that those caused problems in the past, we are talking about
reverting patches from 'stable' that are not part of the upstream
release from where the 'stable' branch was forked--*nobody* has showed
any evidence that this has happened before and caused issues.
Only Stefan Richter is trying to tackle this argument.
Cheers.
--
Felipe Contreras
On 12 April 2012 09:49, Felipe Contreras <[email protected]> wrote:
>>
>> A revert is the same as a patch. ?It needs to be in Linus's tree before
>> I can add it to the stable releases.
>
> Right, because otherwise people's systems would actually work.
>
> But hey, as I said, following rules is more important, regardless of
> what the rules are, and why they are there. The rules that actually
> triggered this issue in v3.3.1, as this is not in v3.3.
>
> You could just accept that the patch should have never landed in
> v3.3.1 in the first place, but it's much easier to arbitrarily keep
> stacking patches without thinking too much about them.
Greg is doing the right thing here. We face the same deal in FreeBSD -
people want fixes to go into a release branch first, but if you do
that you break the development flow - which is "stuff goes into -HEAD
and is then backported to the release branches."
If you don't do this, you risk having people do (more, all)
development and testing on a release branch and never test -HEAD (or
"upstream linux" here). Once you open that particular flood gate, it's
hard to close.
We had this problem with Squid. People ran and developed on Squid-2.4.
The head version of Squid-2 was stable, but that isn't what people ran
in production. They wanted features and bugfixes against Squid-2.2,
squid-2.4, and not Squid-2.STABLE (which at the time was
Squid-2.6/Sqiud-2.7.) That .. didn't work. Things diverged quite
quickly and it got very ugly.
So I applaud Greg for sticking to correct stable release engineering
here. We over in the BSD world know just how painful that is. :)
Adrian
On Sat, Apr 14, 2012 at 8:55 PM, Stefan Richter
<[email protected]> wrote:
> On Apr 14 Felipe Contreras wrote:
>> On Sat, Apr 14, 2012 at 10:41 AM, Stefan Richter
>> <[email protected]> wrote:
>> > On Apr 14 Felipe Contreras wrote:
>> >> I already exemplified how they are very different, but here it goes
>> >> again. The patch "drm/i915: Add lock on drm_helper_resume_force_mode"
>> >> was just tagged in 3.3.2, if I had said yesterday "this patch breaks
>> >> things on my machine", then that patch would have been dropped for
>> >> 3.3.2 even though it's still on mainline--why? Because we know it's
>> >> broken, and broken patches are not supposed to land to stable. But
>> >> today, one day later, we have to wait until it's fixed in master
>> >> first. Why?
>> >>
>> >> What makes a patch droppable yesterday, but dependent on mainline today?
>> >
>> > Huh?
>> >
>> > Because "yesterday" was a review before stable release:
>> > - A buggy mainline release exists.
>> > - No buggy stable release exists.
>> > whereas "today" is after stable release:
>> > - A buggy mainline release exists.
>> > - A buggy stable release exists.
>>
>> IOW; a tag makes undroppable.
>
> Generally, "commit + push out" makes it undroppable. In case of -stable,
> commit/ push out/ tag are close and virtually identical.
>
> After a change was pushed out, the choice narrows down to add a reverting
> change for a subsequent release or to rebase to a point before the
> defective commit. The latter is not done to -stable, obviously. (3.3.2
> is not being restarted from 3.3 if a regression from 3.3 to 3.3.1 was
> discovered.)
That's irrelevant; whether you rebase and drop the patch, or you
revert it, the resulting code is *exactly* the same. It's also the
same if Greg drops it from his quilt queue before pushing (assuming
that's what he uses).
Let's avoid this semantic red herring, by undroppable I mean
unrevertable, or undiscardable, or anything that effectively makes the
patch not be there.
>> But *why*? You say you *really* need to problem to fixed in mainline,
>> that's why it cannot be dropped, but that applies also to patches in
>> the queue *before* the tag is made, doesn't it? If you find a patch in
>> the stable review queue causes problems, why don't you leave it there?
>
> As Willy wrote, that's actually what is done sometimes. I didn't know
> that.
Don't avoid the question.
I don't mean just leave it in the queue, I mean leave it so it gets
tagged in the release.
>> You *really* need to problem fixed in mainline, don't you?
>>
>> So yesterday the priority is stability > 'upstream first', but today
>> it's 'upstream first' > stability. Nobody has put forward an argument
>> for this shift in priorities--
>
> Yesterday, folks cared about mainline too.
Who suggested otherwise? Being priority #2 doesn't mean you don't
care. We always care about mainline, even for patches that are not
proposed for 'stable'.
But if we found yesterday that the patch would break things, it would
not make it to the stable release.
You are again avoiding the argument.
>> "a tag was made" is not argument.
>
> "A faulty commit was pushed out" means that a defective release was made
> and a fix needs to be created and released. This is obviously something
> different from rejecting to commit a submitted change after negative
> review.
>
> Sure, negative review of a stable submission consequently means that
> mainline needs to be checked whether it requires a fix, and if yes, stable
> users will be interested in getting that fix really into the mainline
> rather sooner than later. So suddenly they have to track a mainline bug.
> Still /one/ bug though.
Yes.
> So that is when a stable submission was dropped "yesterday". Now what
> about if a defective change was not dropped but released, and now requires
> a fix (e.g. revert) "today: There are now /two/ bugs: In the mainline
> and in a stable kernel.
What?! So, if an issue has been in the kernel for the last 10 years,
and it's just fixed, that means we suddenly have hundreds of bugs?
You are playing with words; it's *one* bug that is present in multiple releases.
a) if there's a bug in an internal tree, say linux-nokia; it's sill *one* bug
b) if the bug gets propagated to linux-omap; it's still *one* bug
c) if the bug gets into Linus's 'master' branch (before a tag); it's
still *one* bug
d) if it gets tagged into v3.5-rc1; it's still *one* bug
e) if it gets into Greg's stable queue; it's still *one* bug
f) and if it gets into v3.4.1; it's still *one* bug.
This is an unseasoned assertion, and the rest of your comments assume
it is true, so I'm not going to reply to them.
By saying it's "two bugs", you are still avoiding the question of why
they are different. *Why* are they two bugs? What is so different
between e) and f) that makes bad patches undroppable?
I appreciate you are exploring this discussion, but I wonder if you
accept the possibility that there's actually no good reason for this
change in priorities, or if you accept that even a change in
priorities is taking place.
Cheers.
--
Felipe Contreras
On Thu, Apr 12, 2012 at 09:57:53PM +0200, Alexander Holler wrote:
> Hello,
>
> Am 12.04.2012 02:29, schrieb Greg KH:
>
> >>is there any chance for this one to be included in this review cycle?
> >>
> >>http://www.spinics.net/lists/linux-wireless/msg87999.html
> >
> >Have you read Documentation/stable_kernel_rules.txt? Based on that, I
> >don't think it can, yet, right?
>
> Hmm, after reading that, I think the text could need an update about
> how to submit patches written by others (which can already be found
> in the upstream tree).
>
> At least for me the text reads like only the authors can request
> inclusion of a patch into the stable tree.
Patches are always welcome, but I think you will find the file already
describes this (hint, the first '-' line for the Procedure section
handles it).
greg k-h