From justin at specialbusservice.com Wed Feb 6 10:54:19 2019 From: justin at specialbusservice.com (Justin Cormack) Date: Wed, 6 Feb 2019 10:54:19 +0000 Subject: [rustyk8s] Implementing Kubernetes in Rust: mailing list Message-ID: Hi all I have created a mailing list for discussion of re-implementing Kubernetes in Rust. See my blog post for background https://www.cloudatomiclab.com/rustyk8s/ I am mailing as you expressed interest, you can subscribe at http://lists.opendev.org/cgi-bin/mailman/listinfo/rustyk8s I have a bunch more people to invite and please invite anyone you know who is interested, and lets get hacking! Justin From darren at rancher.com Thu Feb 7 17:59:23 2019 From: darren at rancher.com (Darren Shepherd) Date: Thu, 7 Feb 2019 17:59:23 +0000 Subject: [rustyk8s] Hi and whats the plan? Message-ID: I've missed any discussion on this topic might have happened so all I know is based on the https://www.cloudatomiclab.com/rustyk8s/ blog. Is the purpose of rustyk8s to hack on a shared implementation or a place where people can share thoughts on their own k8s implementation in rust they are hacking on? Asking because of this tweet https://twitter.com/justincormack/status/1067185866001575936 makes it seem like a competition (which I'm all for). Darren -------------- next part -------------- An HTML attachment was scrubbed... URL: From justin at specialbusservice.com Thu Feb 7 23:20:47 2019 From: justin at specialbusservice.com (Justin Cormack) Date: Thu, 7 Feb 2019 23:20:47 +0000 Subject: [rustyk8s] Hi and whats the plan? In-Reply-To: References: Message-ID: Hi, Its all pretty free form! We have a few people who should chime in shortly and talk about their experiences with writing Kubernetes client APIs in Rust, and working with the types and so on, which I think is generally helpful, and another who hacked together a bit of a rough prototype as an experiment, which I think will generally be helpful. Overall we have a mix of Rust beginners and experts and also people with varying depth on the k8s APIs, so people may want to collaborate, but I agree we definitely don't all want to sit down on day one and try and work on one thing, just see what you can get to work and ask for help and advice, but ideally in the open so others can look and contribute. Justin On Thu, 7 Feb 2019 at 17:59, Darren Shepherd wrote: > > I've missed any discussion on this topic might have happened so all I know is based on the https://www.cloudatomiclab.com/rustyk8s/ blog. Is the purpose of rustyk8s to hack on a shared implementation or a place where people can share thoughts on their own k8s implementation in rust they are hacking on? Asking because of this tweet https://twitter.com/justincormack/status/1067185866001575936 makes it seem like a competition (which I'm all for). > > > Darren > > _______________________________________________ > rustyk8s mailing list > rustyk8s at lists.opendev.org > http://lists.opendev.org/cgi-bin/mailman/listinfo/rustyk8s From robertc at robertcollins.net Sat Feb 9 01:58:29 2019 From: robertc at robertcollins.net (Robert Collins) Date: Sat, 9 Feb 2019 14:58:29 +1300 Subject: [rustyk8s] Hi and whats the plan? In-Reply-To: References: Message-ID: Hi, I'm one of those people; I've been collaborating with Angus Lees (cc'd) on kubernetes-rs for a while now, though as both of us had very rare time slices it hadn't moved very far in that time (and most of my contributions had been design discussions for the same time reason).... I'm planning on putting a good solid spike in starting around a week from now (got a 2000km road trip to do beforehand). https://github.com/anguslees/kubernetes-rs The broad goals of that repo are to create a crate that permits other crates that: - implement controllers in as little as a few lambdas - implement ops-scripting problems pithily - export 3rd party CRD datatypes for use by others (e.g. kubernetes-rs + some-crd-crate + some other crate that uses the crd). - implement API servers (for k8s api extension by proxied requests) With this crate itself having the following attributes: -type safe modelling of the API (e.g. Enums rather than strings that may-or-may-not-be-correct; and the ability to write get compiler errors on match branches missing arms). -native async interface -unstructured handling as well (safety valve for unknown types) -avoid large redundant code generation (e.g. provide Derive implementations rather than running a codegen which outputs parallel modules) There's a good representative core there, but we aren't quite happy with the ergonomics of the layers - once we are expanding out into full API coverage, and implementing an actual API server itself shouldn't have any major speed bumps. Personally I speculate that the go constraints have deeply distorted the API server - CSP is great for creating correct code, but not great for being able to control that code in a very precise way across the entire system - https://github.com/kubernetes/kubernetes/issues/12208 for instance is *still* open, and its lack drives many other bugs such as the ones around kubelet qps <-> apiserver (which shows up in e.g. https://github.com/kubernetes/kubernetes/issues/23349) So I'd love to have an apiserver that is compact and lean and see what we can do with it. -Rob On Fri, 8 Feb 2019 at 12:21, Justin Cormack wrote: > > Hi, > > Its all pretty free form! We have a few people who should chime in > shortly and talk about their experiences with > writing Kubernetes client APIs in Rust, and working with the types and > so on, which I think is generally helpful, > and another who hacked together a bit of a rough prototype as an > experiment, which I think will generally be > helpful. > > Overall we have a mix of Rust beginners and experts and also people > with varying depth on the k8s APIs, so people > may want to collaborate, but I agree we definitely don't all want to > sit down on day one and try and work on one > thing, just see what you can get to work and ask for help and advice, > but ideally in the open so others can look and > contribute. > > Justin > > On Thu, 7 Feb 2019 at 17:59, Darren Shepherd wrote: > > > > I've missed any discussion on this topic might have happened so all I know is based on the https://www.cloudatomiclab.com/rustyk8s/ blog. Is the purpose of rustyk8s to hack on a shared implementation or a place where people can share thoughts on their own k8s implementation in rust they are hacking on? Asking because of this tweet https://twitter.com/justincormack/status/1067185866001575936 makes it seem like a competition (which I'm all for). > > > > > > Darren > > > > _______________________________________________ > > rustyk8s mailing list > > rustyk8s at lists.opendev.org > > http://lists.opendev.org/cgi-bin/mailman/listinfo/rustyk8s > > _______________________________________________ > rustyk8s mailing list > rustyk8s at lists.opendev.org > http://lists.opendev.org/cgi-bin/mailman/listinfo/rustyk8s From peter at psftw.com Sun Feb 10 01:00:50 2019 From: peter at psftw.com (Peter Salvatore) Date: Sun, 10 Feb 2019 01:00:50 +0000 Subject: [rustyk8s] Elephant in the room? Message-ID: Recently I spent an evening looking around at the existing rust crates related to k8s and one of them really stood out to me: https://github.com/Arnavion/k8s-openapi-codegen At first glance this looks like an incredible library to depend on for building K8S-related things that need to function against multiple versions of the API. The README describing some of the design decisions is also entertaining: https://github.com/Arnavion/k8s-openapi-codegen/tree/master/k8s-openapi Call me impressed! If I wind up building anything, it will likely be based on the above project. :-) Thank you Arnav! -Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: From sylvain at bluxte.net Sun Feb 10 08:29:25 2019 From: sylvain at bluxte.net (Sylvain Wallez) Date: Sun, 10 Feb 2019 09:29:25 +0100 Subject: [rustyk8s] Hi and whats the plan? In-Reply-To: References: Message-ID: <2b739449-68a0-9290-66ac-f3d9612d4b96@bluxte.net> Hi, A comment from the peanut gallery: reimplementing k8s in Rust is an interesting goal (and having a fast and robust api-server that is not based on etcd would be awesome). We have to consider however what would help increase usage of Rust in the k8s ecosystem and make it a strong alternative to Go to develop k8s components. Good client libararies is a must, but helpers/sdk on top of it would make the choice even easier. We started a project at $work a few months ago to build a custom operator and Go was chosen just because of there are great supporting libraries, like https://github.com/kubernetes-sigs/kubebuilder So first thing to do would be to agree on a common k8s api crate, pretty much like tokio is the de-facto standard crate for async networking, to avoid having a fragmented landscape and allow building higher-level functionality on top of it. My 2 cents, Sylvain Le 08/02/2019 à 00:20, Justin Cormack a écrit : > Hi, > > Its all pretty free form! We have a few people who should chime in > shortly and talk about their experiences with > writing Kubernetes client APIs in Rust, and working with the types and > so on, which I think is generally helpful, > and another who hacked together a bit of a rough prototype as an > experiment, which I think will generally be > helpful. > > Overall we have a mix of Rust beginners and experts and also people > with varying depth on the k8s APIs, so people > may want to collaborate, but I agree we definitely don't all want to > sit down on day one and try and work on one > thing, just see what you can get to work and ask for help and advice, > but ideally in the open so others can look and > contribute. > > Justin > > On Thu, 7 Feb 2019 at 17:59, Darren Shepherd wrote: >> I've missed any discussion on this topic might have happened so all I know is based on the https://www.cloudatomiclab.com/rustyk8s/ blog. Is the purpose of rustyk8s to hack on a shared implementation or a place where people can share thoughts on their own k8s implementation in rust they are hacking on? Asking because of this tweet https://twitter.com/justincormack/status/1067185866001575936 makes it seem like a competition (which I'm all for). >> From justin at specialbusservice.com Sun Feb 10 12:42:35 2019 From: justin at specialbusservice.com (Justin Cormack) Date: Sun, 10 Feb 2019 12:42:35 +0000 Subject: [rustyk8s] Hi and whats the plan? In-Reply-To: References: Message-ID: On Sat, 9 Feb 2019 at 01:58, Robert Collins wrote: > > Hi, I'm one of those people; I've been collaborating with Angus Lees > (cc'd) on kubernetes-rs for a while now, though as both of us had very > rare time slices it hadn't moved very far in that time (and most of my > contributions had been design discussions for the same time > reason).... I'm planning on putting a good solid spike in starting > around a week from now (got a 2000km road trip to do beforehand). > > https://github.com/anguslees/kubernetes-rs > > The broad goals of that repo are to create a crate that permits other > crates that: > - implement controllers in as little as a few lambdas > - implement ops-scripting problems pithily > - export 3rd party CRD datatypes for use by others (e.g. > kubernetes-rs + some-crd-crate + some other crate that uses the crd). > - implement API servers (for k8s api extension by proxied requests) > > With this crate itself having the following attributes: > -type safe modelling of the API (e.g. Enums rather than strings that > may-or-may-not-be-correct; and the ability to write get compiler > errors on match branches missing arms). > -native async interface > -unstructured handling as well (safety valve for unknown types) > -avoid large redundant code generation > (e.g. provide Derive implementations rather than running a codegen > which outputs parallel modules) > > There's a good representative core there, but we aren't quite happy > with the ergonomics of the layers - once we are expanding out into > full API coverage, and implementing an actual API server itself > shouldn't have any major speed bumps. This definitely looks like a good set of requirements and starting point. Look forward to seeing the next steps. > Personally I speculate that the go constraints have deeply distorted > the API server - CSP is great for creating correct code, but not great > for being able to control that code in a very precise way across the > entire system - https://github.com/kubernetes/kubernetes/issues/12208 > for instance is *still* open, and its lack drives many other bugs such > as the ones around kubelet qps <-> apiserver (which shows up in e.g. > https://github.com/kubernetes/kubernetes/issues/23349) It doesn't surprise me that no one wants to tackle these issues with the existing code base and they just remain open. And agree, resource control in Go is not how the system and its libraries were designed. From justin at specialbusservice.com Sun Feb 10 12:52:21 2019 From: justin at specialbusservice.com (Justin Cormack) Date: Sun, 10 Feb 2019 12:52:21 +0000 Subject: [rustyk8s] Elephant in the room? In-Reply-To: References: Message-ID: On Sun, 10 Feb 2019 at 01:01, Peter Salvatore wrote: > > Recently I spent an evening looking around at the existing rust crates related to k8s and one of them really stood out to me: https://github.com/Arnavion/k8s-openapi-codegen At first glance this looks like an incredible library to depend on for building K8S-related things that need to function against multiple versions of the API. The README describing some of the design decisions is also entertaining: https://github.com/Arnavion/k8s-openapi-codegen/tree/master/k8s-openapi The mess of pulling out the API definitions from the Go code has been discussed quite a bit, and some improvements in the API spec have been made. Quite a bit of this has been people trying to get usable schemas (eg JSON schema) out for tooling purposes, as well as API bindings. Making sure there are open issues for all these is important too, as removing special cases and hacks will make this all easier. Justin From robertc at robertcollins.net Tue Feb 12 22:17:26 2019 From: robertc at robertcollins.net (Robert Collins) Date: Wed, 13 Feb 2019 11:17:26 +1300 Subject: [rustyk8s] Elephant in the room? In-Reply-To: References: Message-ID: Thats a pretty nice crate and I applaud its design decisions vis-a-vis network / IO abstractions, and the nice Options structs. However its a huge amount of code duplication - the k8s-openapi crate is 3.5M! - consider https://github.com/Arnavion/k8s-openapi-codegen/blob/master/k8s-openapi/src/v1_13/mod.rs - every response function is basically the same - ``` fn try_from_parts(status_code: http::StatusCode, buf: &[u8]) -> Result<(Self, usize), crate::ResponseError> { match status_code { http::StatusCode::OK => { let result = match serde_json::from_slice(buf) { Ok(value) => value, Err(ref err) if err.is_eof() => return Err(crate::ResponseError::NeedMoreData), Err(err) => return Err(crate::ResponseError::Json(err)), }; Ok((GetAdmissionregistrationAPIGroupResponse::Ok(result), buf.len())) }, http::StatusCode::UNAUTHORIZED => Ok((GetAdmissionregistrationAPIGroupResponse::Unauthorized, 0)), _ => Ok((GetAdmissionregistrationAPIGroupResponse::Other, 0)), } } ``` What gus and I are aiming at is having this itself parameterised by an API type, permitting a single implementation to serve for both core types and CRDs. -Rob On Sun, 10 Feb 2019 at 14:01, Peter Salvatore wrote: > > Recently I spent an evening looking around at the existing rust crates related to k8s and one of them really stood out to me: https://github.com/Arnavion/k8s-openapi-codegen At first glance this looks like an incredible library to depend on for building K8S-related things that need to function against multiple versions of the API. The README describing some of the design decisions is also entertaining: https://github.com/Arnavion/k8s-openapi-codegen/tree/master/k8s-openapi > > Call me impressed! If I wind up building anything, it will likely be based on the above project. :-) Thank you Arnav! > > -Peter > _______________________________________________ > rustyk8s mailing list > rustyk8s at lists.opendev.org > http://lists.opendev.org/cgi-bin/mailman/listinfo/rustyk8s From robertc at robertcollins.net Fri Feb 15 01:19:43 2019 From: robertc at robertcollins.net (Robert Collins) Date: Fri, 15 Feb 2019 14:19:43 +1300 Subject: [rustyk8s] slack channel? Message-ID: I just noticed that there isn't a #kubernetes-rs / #kubernetes-rust slack channel in the k8s slack; should I request one? -Rob From justin at specialbusservice.com Sat Feb 16 08:29:04 2019 From: justin at specialbusservice.com (Justin Cormack) Date: Sat, 16 Feb 2019 08:29:04 +0000 Subject: [rustyk8s] slack channel? In-Reply-To: References: Message-ID: Yes it might be a good idea though I might just call it #rust for discoverability... justin On Fri, 15 Feb 2019 at 01:20, Robert Collins wrote: > > I just noticed that there isn't a #kubernetes-rs / #kubernetes-rust > slack channel in the k8s slack; should I request one? > > -Rob > > _______________________________________________ > rustyk8s mailing list > rustyk8s at lists.opendev.org > http://lists.opendev.org/cgi-bin/mailman/listinfo/rustyk8s From robertc at robertcollins.net Sat Feb 16 09:13:28 2019 From: robertc at robertcollins.net (Robert Collins) Date: Sat, 16 Feb 2019 22:13:28 +1300 Subject: [rustyk8s] slack channel? In-Reply-To: References: Message-ID: Ruby is at kubernetes-ruby, as a data point On Sat, 16 Feb 2019, 21:29 Justin Cormack, wrote: > Yes it might be a good idea though I might just call it #rust for > discoverability... > > justin > > On Fri, 15 Feb 2019 at 01:20, Robert Collins > wrote: > > > > I just noticed that there isn't a #kubernetes-rs / #kubernetes-rust > > slack channel in the k8s slack; should I request one? > > > > -Rob > > > > _______________________________________________ > > rustyk8s mailing list > > rustyk8s at lists.opendev.org > > http://lists.opendev.org/cgi-bin/mailman/listinfo/rustyk8s > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robertc at robertcollins.net Mon Feb 18 20:12:55 2019 From: robertc at robertcollins.net (Robert Collins) Date: Tue, 19 Feb 2019 09:12:55 +1300 Subject: [rustyk8s] slack channel? In-Reply-To: References: Message-ID: https://kubernetes.slack.com/archives/C4M06S5HS/p1550339714051700 On Sat, 16 Feb 2019 at 22:13, Robert Collins wrote: > > Ruby is at kubernetes-ruby, as a data point > > On Sat, 16 Feb 2019, 21:29 Justin Cormack, wrote: >> >> Yes it might be a good idea though I might just call it #rust for >> discoverability... >> >> justin >> >> On Fri, 15 Feb 2019 at 01:20, Robert Collins wrote: >> > >> > I just noticed that there isn't a #kubernetes-rs / #kubernetes-rust >> > slack channel in the k8s slack; should I request one? >> > >> > -Rob >> > >> > _______________________________________________ >> > rustyk8s mailing list >> > rustyk8s at lists.opendev.org >> > http://lists.opendev.org/cgi-bin/mailman/listinfo/rustyk8s