Cloud Foundry Networking Update: Service Mesh Solutions for Kubernetes

By: | September 4, 2019
Share

This post provides an update to the Cloud Foundry community on the CF Networking team’s investments in service mesh solutions since Cloud Foundry Summit North America 2019.

Last October we shared that with our integration between and Cloud Foundry and Istio Pilot, we were able to offer weighted routing via a new Envoy-based ingress gateway and Cloud Controller APIs or a CLI plugin, enabling developers to have more control in shifting traffic from one version of their application to another. That thread is here.

At Cloud Foundry NA Summit in Philadelphia earlier this year, I gave a presentation at User Day sharing our journey from routing to service mesh in Cloud Foundry, with the ever-present goal of delivering business outcomes for platform operators and the development teams they serve. 

Since Cloud Foundry NA Summit, we’ve extended our integration with Istio and Envoy by routing app-to-app traffic through the Envoy sidecar proxies, and configuring them with default policies for client-side load balancing, timeouts, and retries. These features could remove the need for developers to implement these behaviors in application code, further enabling them to focus on business value. Platform operators can enable these features using istio-release, a BOSH release that can be deployed alongside cf-deployment using an ops file. 

Warning: our integration with Istio is not yet ready for production use cases. The control plane is not yet HA nor is it sufficiently instrumented for production monitoring. In addition, app-to-app features are only supported with fewer that 300 internal routes. We have identified how we can scale our use of sidecars far beyond this, by uniquely configuring the sidecars for each application based on c2c security policies, but have not yet implemented this change.

While we have always intended to bring these features to a level of maturity that can be leveraged in production environments, we have recently decided to cease investments in service mesh solutions for apps pushed with Cloud Foundry to the Diego container scheduler, and have begun working on solutions for applications pushed with Cloud Foundry to Kubernetes. We continue to leverage the Istio and Envoy open source projects for our intended outcomes.

We have decided to cease investments in service mesh solutions for apps pushed with Cloud Foundry to the Diego container scheduler, and have begun working on solutions for applications pushed with Cloud Foundry to Kubernetes.

The first milestone we are targeting is automated ingress routing to an app pushed with Cloud Foundry to a K8s cluster. Our roadmap includes advanced traffic management, security policy controls, and observability for all data paths (ingress, app-to-app, and egress), all using familiar Cloud Foundry workflows.

Related investments by other Cloud Foundry teams

In parallel with the Networking team’s efforts, other Cloud Foundry teams are doing great work toward the same vision:

  • A collaboration between CAPI and CLI teams, responsible for completing the v3 CC API and delivering the v7 CLI, working on the APIs and CLI commands to support declarative configuration of routing rules, starting with percentage-based traffic splitting for external routes. 
  • Engineers from the Windows team have been contributing a Windows port for the Envoy Proxy project, which will enable developers of .NET apps to benefit in the same ways as we have planned for Linux apps. Once the sidecars are in place, the operating system is abstracted. 

We’d love help with all of this. If you’d like to contribute please reach out to us in the #networking channel on Cloud Foundry Slack. 

Let’s meet up

Members of the CF Networking team are attending Cloud Foundry EU Summit in The Hague. Register for the event and let us know if you’d like to meet up, or drop by office hours for the Networking project on Thursday 9/12 at 2:30pm. 

Shannon Coen is the Product Lead, Pivotal Cloud Foundry Networking, Pivotal, Inc.

Shannon Coen Profile Image

Shannon Coen, AUTHOR

SEE ALL ARTICLES