Cloud Foundry explained

So I've been reading up on Cloud Foundry and yet I'm still confused as to what it is. Here is my take anyway on PaaS on CF, and hopefully you guys could tell me if I'm wrong and explain it a bit better.

A traditional PaaS offering like Microsoft Azure or Google AppEngine provides a full platform to develop, test, host and manage your web app. You must however use their API and are restricted to the services that they offer and languages/frameworks that they support.

Cloud Foundry seems to be some kind of "middle-man", whereby it allows your app to use services from many public clouds. How does it accomplish this? Is there a single API that you use, something like LibCloud or JCloud? Can you use one service from one provider, and another service from another provider, for example? And does Cloud Foundry itself offer any services, or is it merely a middle-man allowing you to easily migrate from one platform to another, and use different service combinations from different providers in a single app?

I'm a developer on Cloud Foundry -- and yes, Cloud Foundry is indeed a bit nebulous (no pun intended). Hopefully I can help clarify things a bit.

Cloud Foundry is a platform as a service, but it needs an infrastructure as a service underneath it. Cloud Foundry supports vSphere, vCloud, OpenStack, and Amazon AWS as infrastructures through the BOSH tool. Most web application developers don't care about any of that, but this is really cool for people who have to worry about large IT infrastructure.

Say you're in charge of IT for AcmeCorp. You've got 50,000 employees who all use your internal web service, Fizzbuzz, to help them do their jobs. To support all the employees, you need dozens of instances of the Fizzbuzz application running on several machines with powerful processors and lots of memory, and you need massive amounts of disk space to store information generated by the Foo, Bar, and Baz applications you use internally, too. You've moved well beyond what you would care to manage on your own blade servers, so you decide to lease a datacenter.

Unfortunately, AcmeCorp is horribly dysfunctional. The finance department has a huge say in what datacenter you use, and every couple years they make you switch from one datacenter to another. Every couple years, you have several weeks of downtime while your engineers try to fix bugs in Fizzbuzz exposed by switching between vSphere, vCloud, OpenStack, or whatever.

If your engineers had written Fizzbuzz, Foo, Bar, and Baz against Cloud Foundry rather than directly against the underlying infrastructure, your downtime would have been minimized. You wouldn't have to worry so much about being locked in to a particular datacenter, because that layer of hosting has been abstracted away by Cloud Foundry. Cloud Foundry does support a certain set of services too, including PostgreSQL, MySQL, Mongo, Redis, and RabbitMQ, to name some. If Foo, Bar, and Baz use those services provided by Cloud Foundry, that's one less thing to worry about when you migrate between infrastructures.

Later on down the road, you realize that you can make a fortune by selling Fizzbuzz as a service to other large businesses. You're in really good shape for this: because your engineers rearchitected Fizzbuzz to run on Cloud Foundry, you can simply deploy Cloud Foundry to AWS for as long as it's needed. Customer tried it for six months and decided not to renew the service? No problem, you don't have any datacenter leases to worry about -- just terminate all those EC2 instances and move on. You can easily have one deployment of Cloud Foundry for each instance of Fizzbuzz as a service so that your customers' data is totally isolated from each other.

The icing on the cake is that Cloud Foundry is open source. If you find that it isn't quite suited to your needs, you don't have to just email support and wait around for the Cloud Foundry engineers to implement your dream feature -- you've got the source too, so you can make any changes you need. And it's available under the Apache 2.0 license, so pull requests are gladly accepted, though not required.

I hope that paints a picture of the kinds of problems that Cloud Foundry solves. Feel free to ask for more details in a comment, or you can check out the Cloud Foundry mailing list if that makes more sense for future questions.

I'm a Developer Advocate for Cloud Foundry and want to add a little to Mark's answer to focus on some of the other details you mentioned in your original question.

Firstly, you mention GAE and Azure. Both of these have certain limitations - GAE limits you to specific languages and APIs, for example. Neither are Open Source. CF is extensible (the new version has buildpack support for example, enabling you to choose "any" language runtime), and you can choose to run it where you want.

Mark mentions 4 IaaS providers we can run CF on today, but assuming that the IaaS in question (let's say we include Azure, CloudStack, Google Compute Engine etc as future targets) can support a small number of what we call Cloud Provider Interfaces (CPIs) then you can deploy Cloud Foundry onto those infrastructures too.

You ask how it is possible to use services from different providers. Like Heroku, the forthcoming version of Cloud Foundry (.com) will support a "marketplace" where you can plugin functions from additional vendors, and if you are running your own Cloud Foundry instance you can choose which services to deploy and connect to your apps.

It's pretty cool :-) come talk to us on the mailing list if you want to know more!

would like to add this as a comment regarding API to Andy's answer but unfortunately do not have enough reputation to do it. As far as I understand Cloud Foundry really has no specific API, but it provides a lot of useful information via environment variables (e.g. VCAP_SERVICES, VCAP_APPLICATION, VCAP_CONSOLE_IP, VCAP_APP_PORT), that can be accessed from any language or framework. While a lot of info from such variables are internal to Cloud Foundry, some of them might be quite useful. The main one is VCAP_SERVICES that provides information about services, that are bound to your app.

For example, if I would like to gather information about Azure Cloud Service instance (say, its ID), on which my app is currently running, I would use this class from Azure Management Library.

In its turn Cloud Foundry provides VCAP_APPLICATION env. variable, which will contain following fields:

{"application_users": [],
"instance_id":"97467a9cf508cb75273284b948b6319b",
"instance_index":1,
"application_version":"330b7caf-50e5-48f4-8792-1c80a90b06f1",
"application_name":"helloworld",
"application_uris":["helloworld.vcap.me"],
"started_at":"2013-07-22 10:58:16 +0300",
"started_at_timestamp":1374479896,
"host":"0.0.0.0",
"port":61014,
"limits":{"mem":256,"disk":1024,"fds":16384},
"version":"330b7caf-50e5-48f4-8792-1c80a90b06f1",
"name":"helloworld",
"uris":["helloworld.vcap.me"],
"users":[],
"start":"2013-07-22 10:58:16 +0300",
"state_timestamp":1374479896}

And, finally, a few words about logs, monitoring and diagnostic. This is currently not implemented on CF PaaS level, however I hope this will be implemented (as it is a really useful feature) and maybe some new env. variables (say VCAP_LOGS, VCAP_PERFORMANCE_COUNTERS) will be exposed to our apps.

Certainly CF is an abstraction layer between your IaaS (servers, storage and networking) and your application, giving you portability to move your app among public and private clouds, but it is also much more:

1. A highly horizontally-scalable container-based platform

Apps run in containers allowing for better resource management than assigning apps to hosts (VMs). Warden/Garden is the CF-native container technology although Docker is also supported in recent versions.

2. A self-healing platform providing multiple layers of HA to your application

Health management system resurrects failed app instances, container hosts, platform processes, and VMs, with no outage. Availability zone support provides HA at the infrastructure layer. Rolling updates and canary deploys allow for zero down-time even during deployments or platform upgrades.

3. An opinionated, polyglot application run-time

Using the heroku "buildpack" construct, app language is auto-detected and the appropriate runtime stack is built on top of a vanilla OS image, allowing developers to focus on writing code.

4. Developer on-demand provisioning of stateful data services

Developers can self-provision a slice of a MySQL, RabbitMQ, Redis, etc. cluster with uri/credentials automatically injected into their app's environment.