Chef Infra Server Overview
The Chef Infra Server API is written in Erlang, which is an excellent language for critical enterprise concerns like concurrency, fault-tolerance, and distributed environments. The Chef Infra Server can scale to the size of any enterprise.
Note
/etc/opscode/chef-server.rb
file. Whenever this file is modified, the
chef-server-ctl reconfigure
command must be run to apply the changes.
See the Chef Infra Server settings guide for
additional information.Server Components
The following diagram shows the various components that are part of a Chef Infra Server deployment and how they relate to one another.
Component | Description |
---|---|
Clients | The Chef Infra Server is accessed primarily by nodes that are under management by Chef, as Chef Infra Client runs occur. It is also accessed by individuals who maintain cookbooks and policy that is stored on the Chef Infra Server, typically from a workstation. And also by individual users with credentials to Chef Infra Server components, such as the Chef management console. |
Load Balancer | Nginx is an open-source HTTP and reverse proxy server that is used as the front-end load balancer for the Chef Infra Server. All requests to the Chef Infra Server API are routed through Nginx. |
Chef Manage | Chef Manage is the web interface for the Chef Infra Server, which uses the Chef Infra Server API for all communication to the Chef Infra Server. |
Chef Infra Server | Erchef is a complete rewrite of the core API for the Chef Infra Server, which allows it to be faster and more scalable than previous versions. The API itself is still compatible with the original Ruby-based Chef Infra Server, which means that cookbooks and recipes that were authored for the Ruby-based Chef Infra Server will continue to work on the Erlang-based Chef Infra Server. Chef Infra Client is still written in Ruby. Note Even though the Chef Infra Server is authored in Erlang, writing code in
Erlang is NOT a requirement for using Chef Infra. |
Bookshelf | Bookshelf is used to store cookbook content—files, templates, and so
on—that have been uploaded to the Chef Infra Server as part of a
cookbook version. Cookbook content is stored by content checksum. If two
different cookbooks or different versions of the same cookbook include
the same file or template, Bookshelf will store that file only once. The
cookbook content managed by Bookshelf is stored in flat files and is
separated from the Chef Infra Server and search index repositories. All cookbooks are stored in a dedicated repository. |
Messages | chef-elasticsearch wraps Elastisearch and exposes its REST API for indexing and search. All messages are added to a dedicated search index repository. |
PostgreSQL | PostgreSQL is the data storage repository for the Chef Infra Server. |
External Cookbooks
The following diagram highlights the specific changes that occur when cookbooks are stored at an external location, such as Amazon Simple Storage Service (S3).
The following table describes the components that are different from the default configuration of the Chef Infra Server when cookbooks are stored at an external location:
Component | Description |
---|---|
Clients | The Chef Infra Server will provide signed URLs for cookbook requests made by the various clients (individual users, knife requests, and from the Chef Infra Client that is installed on nodes under management by Chef Infra). |
Load Balancer | The signed URLs for cookbooks are pointed here, and then routed to cookbook storage, as required. |
Chef Infra Server | Erchef is a complete rewrite of the core API for the Chef Infra Server, which allows it to be faster and more scalable than previous versions. The API itself is still compatible with the original Ruby-based Chef Infra Server, which means that cookbooks and recipes that were authored for the Ruby-based Chef Infra Server will continue to work on the Erlang-based Chef Infra Server. Chef Infra Client is still written in Ruby. Note Even though the Chef Infra Server is authored in Erlang, writing code in
Erlang is NOT a requirement for using Chef Infra. |
Amazon Simple Storage Service (S3) | Bookshelf is used to store cookbook content—files, templates, and so
on—that have been uploaded to the Chef Infra Server as part of a
cookbook version. Cookbook content is stored by content checksum. If two
different cookbooks or different versions of the same cookbook include
the same file or template, Bookshelf will store that file only once. The
cookbook content managed by Bookshelf is stored in flat files and is
separated from the Chef Infra Server and search index repositories. This represents external cookbooks storage at Amazon Simple Storage Service (S3). |
AWS Settings
Required Settings
To configure external cookbook storage using Amazon Simple Storage
Service (S3) set the following configuration settings in the
chef-server.rb
file and run chef-server-ctl reconfigure
:
bookshelf['access_key_id']
The access key identifier. We do not recommend specifying this in the configuration file. Please use
chef-server-ctl set-secret bookshelf access_key_id
from the Secrets Management commands.Default value: generated by default.
bookshelf['external_url']
The S3 URL. Do not include the bucket name in the URL.
bookshelf['secret_access_key']
The secret key. We do not recommend specifying this in the configuration file. Please use
chef-server-ctl set-secret bookshelf secret_access_key
from the Secrets Management commands.Default value: generated by default.
opscode_erchef['s3_bucket']
The name of the Amazon Simple Storage Service (S3) bucket.
Default value:
bookshelf
.bookshelf['vip']
The virtual IP address or host name of the Amazon Simple Service (S3) API. Do not include the bucket name in the URL.
Default value:
127.0.0.1
.bookshelf['enable']
Enable bookshelf. When connecting to S3, this should normally be set to
false
.Default value:
true
.
An example chef-server.rb
configuration:
bookshelf['enable'] = false
bookshelf['vip'] = 's3.us-west-2.amazonaws.com'
bookshelf['external_url'] = 'https://s3.us-west-2.amazonaws.com'
bookshelf['access_key_id'] = '<ACCESS_ID>'
bookshelf['secret_access_key'] = '<ACCESS_KEY>'
opscode_erchef['s3_bucket'] = '<BUCKET_NAME>'
Optional Settings
The following optional settings are also available and may require modification when using an external S3 provider:
opscode_erchef['nginx_bookshelf_caching']
Whether Nginx is used to cache cookbooks. When
:on
, Nginx serves up the cached content instead of forwarding the request.Default value:
:off
.opscode_erchef['s3_parallel_ops_fanout']
Default value:
20
.opscode_erchef['s3_parallel_ops_timeout']
Default value:
5000
.opscode_erchef['s3_url_expiry_window_size']
The frequency at which unique URLs are generated. This value may be a specific amount of time, i.e.
15m
(fifteen minutes) or a percentage of the value ofs3_url_ttl
, i.e.10%
.Default value:
:off
.opscode_erchef['s3_url_ttl']
The amount of time (in seconds) before connections to the server expire. If node bootstraps are timing out, increase this setting.
Default value:
28800
.opscode_erchef['s3_url_type']
The URL style to use (
path
orvhost
) when connecting to S3. Mainly used to manually override the default setting. Note that Amazon may eliminate path-style URLs on some or all S3 buckets in the future.Default value:
vhost
.
External PostgreSQL
The following diagram highlights the specific changes that occur when PostgreSQL is configured and managed independently of the Chef Infra Server configuration.
The following table describes the components in an external PostgreSQL configuration that are different from the default configuration of the Chef Infra Server:
Component | Description |
---|---|
Chef Infra Server | The Chef Infra Server configuration file is updated to point to an independently configured set of servers for PostgreSQL. |
PostgreSQL | PostgreSQL is the data storage repository for the Chef Infra Server. This represents the independently configured set of servers that are running PostgreSQL and are configured to act as the data store for the Chef Infra Server. |
PostgreSQL Settings
Use the following configuration settings in the chef-server.rb file to configure external PostgreSQL for use with the Chef Infra Server:
postgresql['db_superuser']
The PostgreSQL user name. This user must be granted either the
CREATE ROLE
andCREATE DATABASE
permissions in PostgreSQL or be grantedSUPERUSER
permission. This user must also have an entry in the host-based authentication configuration file used by PostgreSQL (traditionally namedpg_hba.conf
).Required when
postgresql['external']
is set totrue
.Default value:
'superuser_userid'
.postgresql['db_superuser_password']
The password for the user specified by
postgresql['db_superuser']
.Required when
postgresql['external']
is set totrue
.The
db_superuser_password
can also be set usingchef-server-ctl set-db-superuser-password
from the Secrets Management commands.postgresql['external']
Set to
true
to run PostgreSQL external to the Chef Infra Server. Must be set once only on a new installation of the Chef Infra Server before the firstchef-server-ctl reconfigure
command is run. If this is set after a reconfigure or set tofalse
, any reconfigure of the Chef Infra Server will return an error.Required.
Default value:
false
.postgresql['port']
The port on which the service is to listen. The port used by PostgreSQL if that port is not 5432.
Optional when
postgresql['external']
is set totrue
.Default value:
5432
.postgresql['vip']
The virtual IP address. The host for this IP address must be online and reachable from the Chef Infra Server via the port specified by
postgresql['port']
. Set this value to the IP address or hostname for the machine on which external PostgreSQL is located whenpostgresql['external']
is set totrue
.Required when
postgresql['external']
is set totrue
.
Optional Settings
The following optional settings are required when configuring external PostgreSQL on Microsoft Azure:
bookshelf['sql_connection_user']
The PostgreSQL user name in
'username@hostname'
format (e.g.'bookshelf@my_postgresql.postgres.database.azure.com'
), whereusername
would normally equal the value ofbookshelf['sql_user']
(default:'bookshelf'
).This setting is required in an external Azure PostgreSQL database-as-a-service configuration. If set to
nil
, Chef Infra Server assumes that the database is not on Azure and the PostgreSQL connection will be made using the value specified inbookshelf['sql_user']
.Default value:
nil
.oc_bifrost['sql_connection_user']
The PostgreSQL user name in
'username@hostname'
format (e.g.'bifrost@my_postgresql.postgres.database.azure.com'
), whereusername
would normally equal the value ofoc_bifrost['sql_user']
(default:'bifrost'
).This setting is required in an external Azure PostgreSQL database-as-a-service configuration. If set to
nil
, Chef Infra Server assumes that the database is not on Azure and the PostgreSQL connection will be made using the value specified inoc_bifrost['sql_user']
.Default value:
nil
.oc_id['sql_connection_user']
The PostgreSQL user name in
'username@hostname'
format (e.g.'oc_id@my_postgresql.postgres.database.azure.com'
), whereusername
would normally equal the value ofoc_id['sql_user']
(default:'od_id'
).This setting is required in an external Azure PostgreSQL database-as-a-service configuration. If set to
nil
, Chef Infra Server assumes that the database is not on Azure and the PostgreSQL connection will be made using the value specified inoc_id['sql_user']
.Default value:
nil
.opscode_erchef['sql_connection_user']
The PostgreSQL user name in
'username@hostname'
format (e.g.'opscode_chef@my_postgresql.postgres.database.azure.com'
), whereusername
would normally equal the value ofopscode-erchef['sql_user']
(default:'opscode_chef'
).This setting is required in an external Azure PostgreSQL database-as-a-service configuration. If set to
nil
, Chef Infra Server assumes that the database is not on Azure and the PostgreSQL connection will be made using the value specified inopscode_erchef['sql_user']
.Default value:
nil
.postgresql['db_connection_superuser']
The PostgreSQL superuser name in
'username@hostname'
format (e.g.'opscode_pgsql@my_postgresql.postgres.database.azure.com'
), whereusername
would normally equal the value ofpostgresql['db_superuser']
with any dashes replaced by underscores.This setting is required in an external Azure PostgreSQL database-as-a-service configuration. If set to
nil
, Chef Infra Server assumes that the database is not on Azure and the PostgreSQL connection will be made using the value specified inpostgresql['db_superuser']
.Default value:
nil
.
An example chef-server.rb
configuration for External PostgreSQL on
Microsoft Azure:
topology 'standalone'
postgresql['external'] = true
postgresql['vip'] = 'my_postgresql.postgres.database.azure.com'
postgresql['db_superuser'] = 'opscode_pgsql'
postgresql['db_superuser_password'] = 'My_postgres_password1!'
postgresql['db_connection_superuser'] = 'opscode_pgsql@my_postgresql.postgres.database.azure.com'
# postgresql['sslmode']='require' # required if 'Enforce SSL connection' is enabled on Azure PostgreSQL
bookshelf['sql_connection_user'] = 'bookshelf@my_postgresql.postgres.database.azure.com'
oc_bifrost['sql_connection_user'] = 'bifrost@my_postgresql.postgres.database.azure.com'
oc_id['sql_connection_user'] = 'oc_id@my_postgresql.postgres.database.azure.com'
opscode_erchef['sql_connection_user'] = 'opscode_chef@my_postgresql.postgres.database.azure.com'
Note
Bookshelf Settings
In instances that require cookbooks to be stored within a SQL backend,
such as in a high availability setup, you must set the storage_type
to
:sql
:
bookshelf['storage_type'] = :sql