Pivotal/Cloud Foundry

Cloud Foundry | Cloud Foundry CLI | Apps | Tasks | Logs | OpsManager

Overview

Software Platform that lets you build and run software in a consistent way, in any place.
Runs across any platform such as:

• Amazon Web Services
• Microsoft Azure
• Google Cloud Platform
• Open Stack
• VMWare

Cloud Foundry builds out the infrastructure including the virtual machine and a hardened version of the operating system, Windows or Linux.

The Operations Manager Login URL will be like: https://{opsman_domain}/uaa/login

ie. https://opsmgr-10.haas-59.pez.pivotal.io/uaa/login

The Pivotal Apps Manager Login URL will be like: https://login.{system_domain}/login. You can use the following credentials from the PAS tile to log in: UAA > Admin

ie. https://login.run-10.haas-59.pez.pivotal.io/login

Hierarchy

Fundamental Concepts

Process Changes

Distributed System

Distributed systems are hard to build, test, manage, and scale.

Ephemeral Infrastructure

Virtual machines and containers are temporary

Immutable Infrastructure

Updates to systems and applications are not done in-place but rather new, updated instances are created instead.

Elastic Runtime (CF)

Diego

Schedules tasks and long-running processes (LRPs)

Task - guaranteed to run at most once eg. stage and application

LRP - a long running process, typically represented as a web app. lrps can have multiple instances.

Container - An application instance is run within an immutable container

Cell - Containers are run within a cell.

Garden - Containers are managed by Garden.

Rep - represents the cell in the BBS/auctions

Auction - auctions are held to bid on executing a task or an LRP.

Executor - Manages container allocations on the cell. Also streams stdout and stderr to Metron

Metron - Forwards logs to the loggregator subsystem

BBS - Bulletin Board System is the API to access the Diego database for tasks and LRPs.

Brain - The brain is composed of two components, the auctioneer which holds auctions for tasks and LRPs, and the Converger which reconciles desired LRPs vs Actual LRPs through auctions.

Loggregator

Metron - Forwards logs to the loggregator subsystem

Doppler - Gathers logs from Metron. also has app syslog drains to services like splunk or papertrail.

Traffic Controller - Handles client requests for logs. Also exposes a web socket endpoint called the firehose.

Firehose - A websocket endpoint that exposes app logs, container metrics, and ER component metrics.

Nozzles - Consume the firehose output

Cloud Controller API

The Cloud Controller exposes an API for using and managing the Elastic Runtime (CF)

Cloud Controller Database - The cloud controller persists org/space/app data in the cloud controller database

Blob Store - The cloud controller persists app packages and droplets to the blob store.

CC-Bridge - The CC-Bridge translates app specific messages into the generic language of tasks and LRPs.

Routing

The router routes traffic to appropriate components and all app instances.
The router round robins between application instances

Buildpacks

Buildpacks provide framework and runtime support for your applications. They build immutable droplets (stage your application).

How Diego stages buildpack applications

How Diego stages docker images

High Availability

Availability Zones

Application instances are evenly distributed across availability zones. An Application will stay up despite losing an AZ.

Bosh Managed Processes

Elastic runtime processes are monitored and automatically restarted. This is monitored by Monit. The event will be reported to the health monitor for further investigation.

Resurrector

The Bosh agent continuously reports health of the VM/job. if the VM fails or is destroyed/deleted, it will be automatically recreated when the health monitor resurrector reports it to the BOSH director.

Self Healing Application Instances

Once running, failed application instances will be recreated. The brain monitors application instances to verify that the desired state = actual state. When they do not match, the brain will begin the process of recreating the application instance.

Startup Order

As of PAS 2.2, this is the startup order i've found for CF applications using bosh manifest
consul_server > nats > nfs_server > mysql_proxy > mysql > diego_database > uaa > cloud_controller > ha_proxy > router > mysql_monitor > clock_global > cloud_controller_worker > diego_brain > diego_cell > loggregator_trafficcontroller > syslog_adapter > syslog_scheduler > doppler > credhub

• name: database

instances: 0

serial: true

• name: blobstore

instances: 0

serial: true

• name: control

instances: 0

serial: true

• name: compute

instances: 0

• name: consul_server

instances: 1

serial: true

• name: nats

instances: 1

• name: nfs_server

instances: 1

serial: true

• name: mysql_proxy

instances: 1

• name: mysql

instances: 1

• name: backup-prepare

instances: 0

• name: diego_database

instances: 1

serial: true

• name: uaa

instances: 1

serial: true

• name: cloud_controller

instances: 1

serial: true

• name: ha_proxy

instances: 1

• name: router

instances: 1

serial: true

• name: mysql_monitor

instances: 1

• name: clock_global

instances: 1

• name: cloud_controller_worker

instances: 1

• name: diego_brain

instances: 1

• name: diego_cell

instances: 1

• name: loggregator_trafficcontroller

instances: 1

• name: syslog_adapter

instances: 1

• name: syslog_scheduler

instances: 1

• name: doppler

instances: 1

• name: tcp_router

instances: 0

• name: credhub

instances: 1

• name: push-apps-manager

instances: 0

Quick Reference

Spring-App log level

Doesn't really belong here, but i didn't want to make a new page for one small thing.
To increase the level of logging for a spring app in a CF env, you can use the set-env like so:

cf set-env APP_NAME ENV_VAR_NAME ENV_VAR_VALUE
ie. ~$ cf set-env spring-music logging.level.root TRACE

Restart the app, then run logging

~$ cf logs spring-music