Engineering

Meet the New Container Networking Stack for Cloud Foundry

by November 18, 2016

Introduction
A few months ago, we shared a vision for container networking for Cloud Foundry. Today, we are excited to introduce you to netman-release – the new, pluggable container networking stack for Cloud Foundry.
As stated in the vision, the main problems that the container networking effort aims to solve are:

Security policies within Cloud Foundry are provided through Application Security Groups (ASGs) which require an application restart to apply policy. Simple, CIDR-based rules are too broad to indicate application intent.
All communication between containers must go through the Gorouter. This exposes internal applications by requiring them to have a public route or configuring ASGs to allow all internal communication.

All About the Platform: Cloud Foundry Unconference

by October 7, 2016

With over 700 people gathering in Frankfurt at Cloud Foundry Summit Europe, I thought this would be a perfect opportunity for platform developers to meet face-to-face – those who work collaboratively across company lines to build the open source Cloud Foundry platform. I teamed up with Paula Kennedy of Pivotal to organize the official one-day Unconference event.
We were able to secure a great lineup of speakers and sponsors including anynines, Armakuni, Google Cloud Platform, Swisscom and Pivotal – this event would not have been possible without their support. The Unconference took place on the Monday before Summit and was fully booked with over 100 participants:

Cloud Foundry Summit Europe unconference is packed full! pic.twitter.

Cloud Foundry Diego Approaching 250,000 Containers in a Single Cluster, And What That Means For the Enterprise

by September 19, 2016

Interest in containers reached a broad audience nearly two years ago. My, how time flies! Since then, adoption has been, well, up and to the left.
Today, Cloud Foundry is announcing it’s retiring its legacy DEA backend in favor of Diego, which is rapidly approaching the scale of 250,000 containers in a single cluster.
Why should you care?
Consider the iPhone. When you first purchased one, it was likely an 8gb phone. Chances are, you thought you’d never use all the space on that phone. Chances are you were dead wrong. Fast forward to last week’s iPhone 7 announcement. Max size for those devices was 256gb. If you’re anything like me, you probably plan to order the largest capacity device available, just to be safe.

EOL Timeline for Legacy Cloud Foundry DEA Backend

by September 19, 2016

Goal
Cloud Foundry’s next generation runtime, Diego, is production ready and has been running production workloads since October 2015.  Work to increase its scalability substantially past that of the DEAs is nearing completion.  After this work completes we will be phasing out support for the DEA, Warden, HM9k, DEA Logging Agent, and Collector components.  In this document, “DEAs” will be used to stand for all these components.
Rollout Plan
Though the specific date is subject to change, we expect Diego to meet its scaling targets of 250,000 containers spanning 1000 Cells during Q3 of 2016.  This is being accomplished primarily by transitioning Diego from etcd to a relational database (Cloud Foundry’s HA MySQL database by default, though postgres is also supported).

Meet the Photon Platform BOSH CPI

by March 31, 2016

Today, we’re excited to continue addressing the needs of developers by announcing the release of the Photon Platform BOSH CPI. We’re building upon the recent release of v0.8 of Photon Controller: a distributed, multi-tenant and elastic platform purposefully-built for modern applications.
BOSH was designed to address the challenges in building both small and large distributed system. A BOSH CPI, or Cloud Provider Interface, is an API that is used to interact with an underlying IaaS to create and manage objects on an infrastructure, including images, VMs and disks.

Introducing CF-Abacus – Part 3 of 3

by February 19, 2016

picture credit: http://www.tecmaths.com
CF-Abacus team: Jean-Sebastien Delfino (IBM), Saravanakumar Srinivasan “Assk” (Independent), Benjamin Cheng (IBM), Hristo Lliev (SAP), Georgi Sabev (SAP), Kevin Yudhiswara (IBM), Piotr Przybylski (IBM), and Rajkiran Balasubramanian (IBM)
Introduction
In part 1 and part 2 of this series we introduced the metering problem that CF-Abacus solves as well as giving a complete overview of its architecture and design. In the last part we discuss the decisions the team took to realize the architecture and give some hints about future directions and conclude with the next steps. We also provide various references that all parties interested with CF-Abacus can use to become more active in the project.

Introducing CF-Abacus – Part 2 of 3

by February 12, 2016

picture credit: http://www.tecmaths.com
CF-Abacus team: Jean-Sebastien Delfino (IBM), Saravanakumar Srinivasan “Assk” (Independent), Benjamin Cheng (IBM), Hristo Lliev (SAP), Georgi Sabev (SAP), Kevin Yudhiswara (IBM), Piotr Przybylski (IBM), and Rajkiran Balasubramanian (IBM)
Introduction
In part 1 of this 3-part series, we motivated the metering problem for a Platform-as-a-Service (PaaS) like CloudFoundry (CF) and introduced and scoped CF-Abacus, which we claim provides a turnkey metering engine that can be used in a scalable manner by any CF installation. In this part, we dive into the architecture of CF-Abacus and also some of the key design points chosen by the team.
Architecture Overview
CF-Abacus uses a pipeline architecture.

Introducing CF-Abacus – Part 1 of 3

by February 4, 2016

picture credit: http://www.tecmaths.com
CF-Abacus team: Jean-Sebastien Delfino (IBM), Saravanakumar Srinivasan “Assk” (Independent), Benjamin Cheng (IBM), Hristo Lliev (SAP), Georgi Sabev (SAP), Kevin Yudhiswara (IBM), Piotr Przybylski (IBM), and Rajkiran Balasubramanian (IBM)
Abstract
Project CF-Abacus is a scalable usage aggregation and metering engine for any CloudFoundry (CF) installations. The paramount goal of the incubation project is to provide a complete turnkey solution that any CF vendor can use to enable their CF environments with usage metering and aggregation in a manner that will scale with number of users, organizations, spaces, and services.

How To Bill on Cloud Foundry

by September 11, 2015

This is a guest blog post by Greg Cobb, Ben Watts, Michael Smykowski and Dieu Cao from Pivotal.
As a Cloud Foundry operator, you may wish to monetize your application runtime and services usage. While Cloud Foundry does not include a built-in billing system, it does provide features to allow operators to construct their own billing tools that meet their individual needs and requirements.
Cloud Foundry exposes usage events for both applications and services. These usage events detail when Cloud Foundry users start, stop, and upgrade apps and services. These events contain additional information about the resource usage of the apps and services. Usage events are guaranteed to only appear for operations that have successfully completed and to be in chronological order.

Garden and runC

by July 31, 2015

Today, Julz Friedman (an engineer from IBM and the Anchor for Cloud Foundry’s Garden project) posted an important update from the Garden project team to the cf-dev mailing list about the future of the Garden project’s approach to the Linux container runtime. TL;DR here is that the Garden project will be transitioning from it’s own backend for Linux containers to using runC from the Open Container Initiative.
As he says in his email, the team did an initial prototype as part of this decision (fair warning, that repo is a spike).