Install Chef Automate with Chef Habitat Builder On-prem
This guide details how to install Chef Automate and deploy Chef Habitat Builder on-prem together. Enterprise customers may wish to set up an on-premises Chef Habitat Builder depot to store Chef Habitat packages for use by their own Chef Habitat Studios and Supervisors.
This guide covers setting up Chef Automate and Chef Habitat Builder on-prem, and bootstrapping Chef Habitat Builder on-prem with curated core seed lists from the Chef Habitat public Builder.
The Chef installer includes everything necessary to get started with Chef Automate and Chef Habitat Builder on-prem. Bootstrapping Chef Habitat Builder requires:
- An outward bound HTTPS connection
- An existing Chef Habitat public Builder account.
System Requirements
This guide demonstrates setup of Chef Habitat Builder on-prem by installing the components with the Chef Automate deployment utility. Contact your Chef representative before using this implementation in production for more specific scale recommendations.
Hardware Requirements
The minimum with which Chef Automate and Chef Habitat Builder can be deployed on a single host is:
- 16 GB of RAM.
- 130 GB of disk space, available to /hab.
- 4 vCPUs.
- Inbound network connectivity from LAN (HTTP/HTTPS) is required for internal clients to access the on-premises Chef Habitat Builder.
For deployments that are expected to see production-scale workload, we recommend:
- 64 GB of RAM.
- 200 GB of diskspace, available to /hab.
- 16 vCPUs.
Roughly 80 GB of the disk space is designated for Chef Automate; the rest is used for Chef Habitat Builder and the artifacts it stores. The current implementation uses MinIO for Chef Habitat artifact storage; we do not support using Artifactory for artifact storage.
Operating System
Chef Automate and Chef Habitat Builder require:
- a Linux kernel of version 3.2 or greater
systemd
as the init systemuseradd
curl
orwget
- The shell that starts Chef Automate should have a max open files setting of at least 65535
- Run the installation and bootstrapping procedures as the superuser or use
sudo
at the start of each command.
Supported Topologies
- Chef Automate and Chef Habitat Builder on the same host
- Chef Automate and Chef Habitat Builder on different hosts
Unsupported Topologies
- high-availability/DR/multinode Builder
Get Started with Chef Automate and Chef Habitat On-prem
Download the Chef Automate Installer
Download and unzip the installer:
curl https://packages.chef.io/files/current/latest/chef-automate-cli/chef-automate_linux_amd64.zip | gunzip - > chef-automate && chmod +x chef-automate
Deploy Chef Automate and Chef Habitat Builder On-prem
Deploying Chef Habitat Builder with Chef Automate requires a Chef Automate license. If you already have a Chef Automate license, you may use it for the deployment. Otherwise, you can accept the 30-day trial license when you first sign in to Chef Automate.
If you are deploying Chef Habitat Builder with Chef Automate in an airgapped environment, follow the documentation on building an airgap bundle.
You can deploy Chef Habitat Builder either with a full Chef Automate installation or with the Chef Automate auth stack only.
When deploying Chef Habitat Builder on the same host as Chef Automate, authentication to Chef Automate will be configured as part of the deployment.
Deploy Chef Automate and Chef Habitat Builder
To deploy Chef Automate and Chef Habitat Builder on the same host, specify both the builder
and automate
products on the command line.
For example:
./chef-automate deploy --product builder --product automate
Accept the license with y
.
Deploy Chef Habitat Builder with Chef Automate Auth
To deploy Chef Habitat Builder on a different host than Chef Automate, specify only the builder
product on the command line.
Chef Automate UI and Auth services only will also be deployed to support authentication.
When deploying Chef Habitat Builder on a different host than Chef Automate the supported authentication options are LDAP or SAML. Follow the standard instructions for Chef Automate LDAP or Chef Automate SAML configuration.
For example:
./chef-automate deploy --product builder
Accept the license with y
.
Please note that the Chef Automate UI will only support managing Users, Groups, and Authorization policies when deployed without the full Chef Automate stack. If you wish to enable all of Chef Automate at a later time, you can update the product configuration to include the complete Chef Automate stack. For example:
Create a new patch configuration toml as
config.toml
and update the products to include bothbuilder
andautomate
:[deployment.v1.svc] products = ["builder", "automate"]
Patch the configuration to deploy the rest of the Chef Automate services:
./chef-automate config patch config.toml
You should see output similar to:
Updating deployment configuration Applying deployment configuration Installed automate-opensearch Installed automate-es-gateway Installed event-service Installed es-sidecar-service Installed event-feed-service Installed secrets-service Installed applications-service Installed notifications-service Installed nodemanager-service Installed compliance-service Installed ingest-service Installed config-mgmt-service Installed data-feed-service Installed event-gateway Started automate-opensearch Started automate-es-gateway Started event-service Started es-sidecar-service Started event-feed-service Started secrets-service Started applications-service Started notifications-service Started nodemanager-service Started compliance-service Started ingest-service Started config-mgmt-service Started data-feed-service Started event-gateway Started automate-load-balancer Success: Configuration patched
Add Chef Habitat Builder to a Chef Automate Installation
Patch an existing Chef Automate installation to add Chef Habitat Builder:
Create a
patch.toml
file to addbuilder
to the list of products to deploy:[deployment.v1.svc] products=["automate", "builder"]
Apply the patch to the Chef Automate installation:
sudo chef-automate config ./patch.toml
The command output shows the Chef Habitat Builder services being added:
Updating deployment configuration Applying deployment configuration Installed automate-minio Installed automate-builder-memcached Installed automate-builder-api Installed automate-builder-api-proxy Started automate-minio Started automate-builder-memcached Started automate-builder-api Started automate-builder-api-proxy Started automate-load-balancer Success: Configuration patched
Sign in to Chef Automate and the Chef Habitat Builder
View your login credentials in the terminal with:
cat automate-credentials.toml
You should see output similar to:
url = "https://automate.example.com" username = "admin" password = "abcdefgh1234567890PASSWORDSTRING"
Navigate to
https://automate.example.com
in your browser. If you cannot access the site in Google Chrome, try Mozilla Firefox or Microsoft Edge.Sign in to Chef Automate using the credentials provided during deployment.
Select Applications in the top navigation bar
Select Chef Habitat Builder in the left sidebar, which redirects the browser to the Chef Habitat Builder login site
Select Sign in with Chef Automate
Sign into Chef Habitat Builder using the same credentials used with Chef Automate
Generate a Chef Habitat on-prem Builder Personal Access Token
You need a Personal Access Token for Chef Habitat on-prem in order to bootstrap packages and to perform authenticated operations with the hab
client.
Select your Gravatar icon on the top right corner of the Chef Habitat Builder web page, and then select Profile. This will take you to a page where you can generate your access token. Make sure to save it securely.
Create the Core Origin
Once you are signed in to the Chef Habitat Builder UI, select the New Origin button and enter core
as the name for the origin.
Access Chef Habitat Builder On-prem With Chef Habitat Command-Line Tools
Use the https://automate.example.com/bldr/v1
URL when accessing your Chef Habitat Builder installation with the Chef Habitat command-line tools.
The Chef Habitat command-line tools recognize the HAB_BLDR_URL
environment variable, which you can set on the command line with:
export HAB_BLDR_URL=https://automate.example.com/bldr/v1/
Access the Chef Habitat Builder On-prem REST API
To access the REST API for your on-prem installation of Chef Habitat Builder, you must specify your Builder authentication token as a bearer token in your request’s
Authorization
header.
For example:
curl -H "Authorization: Bearer <your-habitat-builder-auth-token>" https://automate.example.com/bldr/v1/...
Bootstrap Chef Habitat Builder with Core Packages (Optional)
Prerequisites:
- HTTPS connection
- GitHub account
- Public Chef Habitat Builder account
- Public Chef Habitat Builder personal access token
Use seed lists to populate your on-premises Chef Habitat Builder installation with the packages required by your builds. Sample seed lists exist for the following scenarios:
- Full
core
: the full contents of the upstreamcore
origin. The x86_64 Linux set expands to 12GB, the Linux kernel2 set to 1GB, and the Windows set to 3.5GB. - Core dependencies: a subset of
core
consisting of commonly-used buildtime dependencies. - Effortless: packages used to start with the Effortless pattern. A complete Effortless implementation requires the contents of both the
stable
and theunstable
channel.
Clone the Chef Habitat Builder On-prem Repository
To access the curated seed lists for bootstrapping Chef Habitat Builder on-prem, you will need to clone the Chef Habitat Builder on-prem repository using https.
git clone https://github.com/habitat-sh/on-prem-builder.git
Once the repository is successfully cloned, move into the on-prem-builder
repository:
cd on-prem-builder
The Chef Automate installer uses a self-signed certificate. Copy the SSL public key certificate chain from Chef Automate into /hab/cache/ssl
with this command:
cp /hab/svc/automate-load-balancer/data/automate.example.com.cert /hab/cache/ssl/automate.example.com.cert
Download Seed List Packages from the Public Chef Habitat Builder
Your host must have access to the internet to download the curated seed list packages from the public Chef Habitat Builder. If you have not already done so, create a user account and personal access token on the public Chef Habitat Builder.
Use the hab pkg download
command with a seed list </path/to/seed_list>
to download packages for
your desired architecture <arch>
from a channel <channel>
to a directory <artifact-dir>
:
HAB_AUTH_TOKEN=<your_public_builder_personal_access_token> hab pkg download --target <arch> --channel <channel> --file </path/to/seed_list> --download-directory <artifact-dir>
For example, to use the Effortless seed list to download x86_64-linux
packages from the
stable
channel to the builder_bootstrap
directory:
HAB_AUTH_TOKEN=<your_public_builder_personal_access_token> hab pkg download --target x86_64-linux --channel stable --file package_seed_lists/effortless_x86_64-linux_stable --download-directory builder_bootstrap
Bulk-Upload Seed List Packages to Chef Habitat Builder on-prem
Run the bulkupload
command to upload artifacts from <artifact-dir>
to the <channel>
channel in the on-premises Chef Habitat Builder using the Builder API endpoint:
HAB_AUTH_TOKEN=<your_on-prem_Builder_personal_access_token> hab pkg bulkupload --url https://automate.example.com/bldr/v1 --channel <channel> <artifact-dir> --auto-create-origins
For example,
HAB_AUTH_TOKEN=<your_on-prem_Builder_personal_access_token> hab pkg bulkupload --url https://automate.example.com/bldr/v1 --channel stable builder_bootstrap/ --auto-create-origins
Should produce the output:
Preparing to upload artifacts to the 'stable' channel on https://automate.example.com/bldr/v1
Using builder_bootstrap/artifacts for artifacts and builder_bootstrap/keys signing keys
Found 46 artifact(s) for upload.
Discovering origin names from local artifact cache
Missing origin 'chef'
Origin 'core' already exists
Missing origin 'effortless'
Creating origin chef.
Created origin chef.
Creating origin effortless.
Created origin effortless.
75 B / 75 B | [===========================================] 100.00 % 1.31 MB/s d
Uploading public origin key chef-20160614114050.pub
...
The --auto-create-origins
flag creates each origin listed in the
<artifact-dir>/artifacts
directory. If you omit the --auto-create-origins
flag,
use the Chef Habitat Builder UI to create the necessary origins before running the
bulkupload
command.
To finish up, return to your Chef Habitat Builder on-prem installation and view the packages that you’ve added to your core
origin at https://automate.example.com/bldr/#/origins/core/packages
.
Using Chef Habitat Builder
Because you are using an on-prem installation of Chef Habitat Builder, you must specify the Builder API endpoint of your installation when following the Habitat Builder documentation. This documentation covers using origin keys, using origin secrets, and uploading and promoting packages.
Operating Chef Habitat Builder
Chef Habitat Builder uses the same mechanisms that Chef Automate does for backups, log management, and uninstalling.
Logging errors
To change the log level for Chef Habitat Builder only, create a TOML file that contains the partial configuration below. Uncomment and change settings as needed, and then run chef-automate config patch </path/to/your-file.toml>
to deploy your change.
[builder_api.v1.sys.log]
level = "debug"
scoped_levels = ["tokio_core=error", "tokio_reactor=error", "zmq=error", "hyper=error" ]
Setting up Chef Automate as an OAuth Provider for Chef Habitat Builder (Deprecated)
Warning
To configure Chef Automate as an OAuth Provider for Chef Habitat Builder, create a TOML file with the partial configuration below.
Run chef-automate config patch </path/to/your-file.toml>
to deploy your change.
bldr_client_id
and bldr_client_secret
simply need to match what you configured for the corresponding
values in Chef Habitat Builder (see below). However, we strongly recommend those values follow
best practices
for client_id
and client_secret
in the Oauth2 standard.
[session.v1.sys.service]
bldr_signin_url = "<your Builder fqdn>" # for example, "http://builder.test/"
# This needs to match what you configured OAUTH_CLIENT_ID as when you configured Habitat Builder.
bldr_client_id = "<your Habitat Builder Oauth2 Client ID>"
# This needs to match what you configured OAUTH_CLIENT_SECRET as when you configured Habitat Builder.
bldr_client_secret = "<your Habitat Builder Oauth2 Client Secret>"
In addition, add Chef Automate’s TLS certificate to Builder’s list of accepted certificates.
Locate Chef Automate’s default self-signed certificate by running cat /hab/svc/automate-load-balancer/data/automate.example.com.cert
, copy this default certificate, and then add it to your Builder instance’s list of accepted certificates.
-----BEGIN CERTIFICATE-----
MIIDfDCCAmSgAcaSldKaf...
-----END CERTIFICATE-----
If you are using a certificate signed by a trusted certificate authority instead of the default certificate, you can provide Builder with the root certificate authority for the signed certificate.
For more information, check out this further explanation on how to configure Builder to authenticate via Chef Automate.