October 2019 - Edge-computing - lists.opendev.org

PTG cross-project session scheduling and plans
by Ildiko Vancsa 30 Oct '19

30 Oct '19

Hi, As a few people from the group are already on the road and had to skip the meeting today I’m dropping this mail as a reminder to the PTG planning. We have a few cross-project slots that are currently scheduled for Thursday, namely: * Neutron - Thursday 12:00pm - 12:30pm * Cinder - 1:30pm - A few more cross-project sessions scheduled for Friday * Ironic - Friday 1:30pm - * Airship - some time in the afternoon, check right after lunch Beyond the above you can find the currently planned topics here: https://etherpad.openstack.org/p/PVG-ECG-PTG Please reply to this thread asap if any of the cross-project sessions are colliding with other activities. Please add comments to the etherpad or reply to this thread with further questions or comments. Thanks, Ildikó

2 4

Edge call is on now
by Ildiko Vancsa 15 Oct '19

15 Oct '19

Zoom is fixed. :) https://zoom.us/j/879678938 Thanks, Ildikó

2 1

StarlingX's Use of Keystone for Kubernetes Authentication
by Waines, Greg 11 Oct '19

11 Oct '19

Hey Colleen, Ildiko asked me to summarize StarlingX’s experiences with using Keystone in STX R2.0 . Specifically summarize StarlingX’s experiences with using Keystone for Kubernetes API Authentication and Docker Registry Authentication. Greg. Background: In STX 1.0 * StarlingX managed a Baremetal OpenStack Cloud, * All the OpenStack Services, e.g. neutron, nova, cinder, etc. ... including StarlingX itself, used Keystone to authenticate, In STX 2.0 * StarlingX changed to manage a Baremetal Kubernetes Cluster, with a Containerized OpenStack Cloud running on top of Kubernetes. * The solution is now layered * StarlingX Kubernetes * Keystone * StarlingX ... authenticating with the Platform Keystone * Kubernetes ... authenticating with the Platform Keystone (intent ... but abandoned) * Docker Registry ... authenticating with the Platform Keystone * OPTIONAL Containerized OpenStack * Keystone, * Nova, Neutron, Glance, Cinder, Horizon, Heat * ... all authenticating with the OpenStack Keystone * Barbican, Telemetry (Ceilometer, Panko, Gnocchi, Aodh), Ironic * ... all authenticating with the OpenStack Keystone Use of Keystone in “StarlingX Kubernetes”: * Ironically we were using Keystone for authentication of a Baremetal Kubernetes Cluster * Normally this would be done for a Kubernetes Cluster running on top of OpenStack * i.e. the Kubernetes Cluster using services of the underlying cloud provider * e.g. using the cloud provider’s authentication services, load balancing services, storage services, etc. * ( This doesn’t have any relevance to this discussion ... just thought I’d point it out. ) * StarlingX Services authenticating with Keystone * No issues ... did it in STX 1.0 ... so no changes required. * Docker Registry authenticating with Keystone * This link: https://docs.docker.com/registry/spec/auth/token/ describes how Docker Registry token authentication works * The Authentication/Authorization Service in the above link’s diagram, MUST be a JSON Web Token Server supporting OAUTH2 interface to docker, * Docker provides an EXAMPLE of such a Token Server in its github: https://github.com/docker/distribution/tree/master/contrib/token-server , which can be copied and updated to handle your preferred authn/z backend. * Docker guy we spoke to was kind of surprised someone hadn’t done this already for keystone. * So StarlingX started with this example and built a JSON Web Token Server supporting OAUTH2 interface to docker and with a backend talking to Keystone * Located here: https://opendev.org/starlingx/containers/src/branch/master/registry-token-s… * ... we are still doing this currently. * Kubernetes API authenticating with Keystone * So we were using the Keystone Authentication and Authorization Webhooks and the Client-Keystone-Auth defined here: https://github.com/kubernetes/cloud-provider-openstack * I presented this solution in this presentation at the DENVER OPENSTACK SUMMIT https://www.dropbox.com/s/gnyf3mtyf5janq1/KubernetesMultiTenancyWithKeyston… * See slides 6,7,8 * This did not work out-of-the box circa early-2019, we had to fix the following issues: * Had to add Authentication to Armada (upstreamed) * StarlingX used Armada to manage set of helm charts for openstack * Armada did NOT support kubernetes authentication * StarlingX team added it and upstreamed it * Had to fix lock feature for Authentication in Armada (upstreamed) * After previous upstream commit to support authentication in armada, * A commit went in to Armada to support locking (for atomic transactions I think), and the commit forgot to take into account authentication, * StarlingX team fixed this commit upstream. * Had issues with expiring tokens with the client-keystone-auth * I believe that the client-auth-plugin was using a cached keystone client that was holding on to tokens that eventually expired * We put in some workaround for this ... can’t remember details. * Eventually we “generally” had this working ... except for a specific but-easily-repeatable use case, where the system would hang or fail in client-keystone-auth (being called from the StarlingX systemInventory process) * It was a path in the client-keystone-auth, where the code was trying to refresh the token in the kubernetes client, * Investigation suggested that the client-keystone-auth was not compatible with our version of eventlet(0.18.4) ... which is a few years old, * There was no way to update the token through “proper” methods. Attempting to delete the client caused the code to mysteriously freeze. If you tried to set the client to none on 1 line then set it to a new client in another line, the code would freeze as you try to set the client to none. * we managed to get it to work by digging through the internals of the client and setting a new token that way, but this was somewhat of a hack. * Since eventlet was used everywhere and we were getting close to STX 2.0 release date, we felt like it was too much risk to try and upversion it. * So, we abandoned the “Kubernetes API authentication with Keystone” work. * StarlingX Roadmap wrt Kubernetes API Authentication * Given our struggles in getting Keystone working ... a little uneasy in picking it up again, * Also Authentication requirements from StarlingX 2.0 Users of the Kubernetes Layer only, are asking for things like: * OpenID Connect (OIDC, OAuth2) type authentication, * LDAP * SAML * Integration with specific authentication platforms like: * GitHub, * LinkedIn, * Azure, * Windows Active Directory * I suspect many of these would require development in Keystone to support. * Whereas there are other existing opensource solutions that can provide that now. Greg.

2 1

PTG planning - ACTION NEEDED
by Ildiko Vancsa 10 Oct '19

10 Oct '19

Hi, I’m reaching out with a friendly reminder that it is time now to prepare for the upcoming PTG (https://www.openstack.org/ptg/) which will be held in Shanghai, November 6-8. The Edge Computing Group session is scheduled for all day Friday (November 8). We have time to discuss items that are more relevant for the working group’s activities as well as organizing cross-project sessions. I created an etherpad which has a few items there already: https://etherpad.openstack.org/p/PVG-ECG-PTG Please add ‘+1’s next to the items you are interested in discussing as well as comments or new items to talk about. We will continue the planning on the next weekly call next Tuesday (October 15). It is crucial to identify projects to have sessions with to make sure we get a chance to schedule the collaborative sessions. Please let me know if you have any questions. Thanks and Best Regards, Ildikó

1 0