Inter-container communication or container networking on Cloud Foundry has been a big challenge for the Cloud Foundry community. So far, the only mechanism for enforcing policy for inter-container communication is to use GoRouter as a broker for all requests. There are many problems with this approach, and one of the biggest drawbacks has been security risks in addition to significance performance and availability overhead.
There are many use-cases where developers need networking capabilities between applications. The good news is that Cloud Foundry teams are working on the problem. A few days ago Jason Sherron, Director of Product Management at VMware Software, Inc. blogged about a vision for the future of container networking on Cloud Foundry.
Evan Farrar, a software engineer with VMware Software, is working on possible solutions to the problem. I interviewed him to learn more about the challenges regarding container networking on Cloud Foundry. Here is an edited version of that interview:
Can you tell us a bit about yourself?
I’m a Product Manager and Software Engineer living in sunny Los Angeles working for VMware, Inc.
How did you become involved with cloud technologies?
I’ve primarily worked at startups as an engineer and I became an early engineer at Groupon. Every day at Groupon there was another crisis related to slightly misconfigured servers or new record traffic which exceeded what we had provisioned just the day before.
At times we had to work long hours just to keep the site running. The need for an abstraction from physical hardware that I could rely on was quite literally keeping me up at night so I’ve been following this trend toward Platform-as-a-Service at every company I’ve worked at since then.
What are the challenges regarding container networking for apps in CF that you are trying to solve?
There are two very important problems we’re trying to solve. The first is how to allow applications to make requests to another application without incurring the penalty of hitting the gorouter and load balancer. This leads to the second problem, how to isolate those applications that shouldn’t talk to one another.
Can you give us some use-cases where devs do need networking capabilities between apps?
I’ve worked mostly in consumer internet applications and from there I can think of a real example I’ve lived before: One team might have a mobile API handler which needs to handle a user tapping the checkout button on their shopping cart. This would need to reach out to an application that stores the shopping cart contents, one that estimates tax rates based on the contents of the cart and a third inventory application that begins a hold on a piece of inventory until checkout is complete. If each of these requests add even minuscule amounts of latency then it can add up to a poor experience for the customer.
What are the possible solutions?
Fundamentally the solution will be to allow level 3 connectivity directly to the container, giving it its own IP address. There are a lot of container networking solutions right now, but our plan is to create a VXLAN based overlay network to provide containers with IP addresses while still allowing some decoupling from the underlay network that components of Cloud Foundry rely on.
What’s the status of the proposal?
The proposal has been accepted into the Cloud Foundry Incubator program and engineers from VMware and IBM are working on it with me.
What is the mechanism for the proposal to be accepted in CF?
Right now our team is focused on demonstrating connectivity between applications so that we can gather more feedback from the community to harden our solution and ensure we’re building the right thing. After we see how that goes, we will propose to the foundation and the CF-Dev mailing list that the project become an official part of the runtime.
Image Credit: Lars. P https://flic.kr/p/8aDafR
