Custom Configuration

While Halyard handles the majority of Spinnaker configuration, there will always be feature flags and properties either too new or niche to be supported by Halyard. Furthermore, some users will be more comfortable forgoing Halyard’s configuration generation, or will want to port their old Spinnaker configuration into Halyard. For these users & use-cases, Halyard supports custom Profiles, and custom Service Settings.

Custom Profiles

If you want to supply custom Profiles, for a particular Deployment ($DEPLOYMENT, typically default) place them in ~/.hal/$DEPLOYMENT/profiles/. The will override any profiles generated by Halyard, and will be copied over by Profile name.

For example, if you want to supply a different clouddriver.yml, you can do so by placing it in ~/.hal/$DEPLOYMENT/profiles/clouddriver.yml, and running hal deploy apply.

For any of the Spring based services, any profile matching ${SERVICENAME}-.*\.yml will be copied and applied to $SERVICENAME. Since this approach will completely overwrite the generated service, and you may want to only supply a few flags, you can instead provide them in ${SERVICENAME}-local.yml, which will always be loaded by $SERVICENAME with no further configuration needed. Further Spring Profiles can be activated by configuring their env to include SPRING_PROFILES_ACTIVE entries for your desired service. See the below Custom Service Settings for more information.

You can validate that your Custom Profiles have been picked up and applied to their services by checking ~/.hal/$DEPLOYMENT/history/service-profiles.yml.

Custom Profile for Deck

To supply custom settings or new values that override generated settings for the Deck UI, you can place them in a file called settings-local.js. For example, to enable the artifactsRewrite feature flag, you can create the following file:

~/.hal/default/profiles/settings-local.js:

window.spinnakerSettings.feature.artifactsRewrite = true;

Custom Service Settings

For each service, $SERVICENAME, you can supply a file ~/.hal/$DEPLOYMENT/service-settings/$SERVICENAME.yml to override any settings as found in ~/.hal/$DEPLOYMENT/history/service-settings.yml. For example, if you wanted Echo to listen on port 443 with the JAVA_OPTS="-Xms256m -Xmx512m" environment variables set, you create the following file:

~/.hal/default/service-settings/echo.yml:

port: 443
env:
  JAVA_OPTS: "-Xms256m -Xmx512m"

If you wanted to specify a secure connection to redis, you would create the following file:

~/.hal/default/service-settings/redis.yml:

overrideBaseUrl: redis://someuser:[email protected]:6379

Tweakable Service Settings

Below are all the Service Settings you can configure. Keep in mind the ones generated by Halyard are generally best and are all that we test, so there is no guarantee making changes to these will result in a working Spinnaker cluster.

Value Description
address The address other services will use to lookup this service; e.g. localhost or redis.service.spinnaker.consul.
artifactId A fully-qualified identifier for the Artifact this service is to run - see your Service Settings output path for examples, since this varies by platform/deployment environment.
basicAuthEnabled Whether or not to enable HTTP basic auth for this service.
enabled Whether or not this service is enabled. Be careful, since disabling critical services like consul-client can make your entire installation unusable and un-upgradable.
env Environment variables to make available to this Service. Supplied as YAML key-value pairs.
healthEndpoint The HTTP endpoint that can be queried for health status. Leave empty if you want Halyard to simply check that a TCP connection is openable. This information is typically transmitted to the platform’s discovery system.
host The host that this service will bind to. 0.0.0.0 will always work, but more restrictive bindings may be more secure.
kubernetes You can add Kubernetes specific service settings, see details below.
location The Spinnaker “location” this will be installed in. This is a namespace in Kubernetes, and a zone in GCE.
monitored Whether or not this has monitoring endpoints exposed and understood by spinnaker-monitoring.
overrideBaseUrl The baseURL this service is reachable on. This is already made configurable for Deck & Gate via hal config security, since these are both public-facing and may service from different hosts than they are discoverable on internal to Spinnaker.
port The port number this service is bound to, and will accept requests on.
safeToUpdate Whether or not this service can be shutdown, and spun on a new VM/container. This protects datastores like Vault & Redis from being taken down from Halyard.
scheme The URI scheme for addressing this service, e.g. http vs. https.
skipLifeCycleManagement Whether or not Halyard should skip managing a service’s life cycle.
targetSize The initial number of nodes this service will be created with. This is only respected on the initial deployment, and further edits will be rejected in favor of the prior service version’s size.

Kubernetes

Inside the kubernetes item at the top level you can add a number of custom service settings that impact how the Spinnaker services are deployed onto a Kubernetes cluster. This lets you partially configure how the Kubernetes Deployment, Service, Pods and other Kubernetes resources are deployed by Halyard. This allows you to do things like add a Kubernetes secret or emptydir to be mounted as a Volume and VolumeMount, change the default Kubernetes deployment strategy and more.

At a high level, the configurable items inside the Kubernetes service settings are:

Item Details
deploymentStrategy Defines either RollingUpdate or Recreate deployment strategy, including setting MaxSurge and MaxUnavailable See Deployment Strategy below.
imagePullSecrets Define secrets to use to pull a custom artifact when using an artifactId to swap out docker image. More details can be found in the Image Pull Secrets section.
nodePort When using a serviceType of NodePort, this provides the NodePort value.
nodeSelector Provide a list of nodeSelector key-value pairs to add to the pod specification. See Node Selector below.
podAnnotations Provide a list of annotations to put on the deployed pods. See Annotations and Labels below.
podLabels Provide a list of labels to put on the deployed pods. See Annotations and Labels below.
securityContext Set the securityContext that the Spinnaker services should run using in Kubernetes
serviceLabels Provide a list of labels to put on the deployed services. See Annotations and Labels below.
serviceAccountName Provide a default serviceAccount under which to run Spinnaker services.
serviceType Define a specific serviceType for deployed services, ClusterIP (Default) or NodePort
useExecHealthCheck Disable the exec-based healthcheck if necessary. See useExecHealthCheck for details.
volumes Define a set of Volume and VolumeMount items to be attached to the Pods through the Deployment. See Custom Volumes

imagePullSecrets

If making use of custom artifactIds from an authenticated docker registry imagePullSecrets must be made available in the replicaset definition. To specify imagePullSecrets to custom artifactIds they can be specified as follows:

kubernetes:
  imagePullSecrets:
  - desired-image-pull-secret1
  - desired-image-pull-secret2

podAnnotations, podLabels and serviceLabels

Annotations and labels hold metadata that can be used to specify special behavior within Kubernetes. To apply annotations or labels to the Pod and Service resources for a particular Spinnaker service you can use something like:

kubernetes:
  podAnnotations:
    example/annotation: spinnaker.io
    example/annotation-2: halyard
  podLabels:
    examplePodLabel: examplePodLabelValue
  serviceLabels:
    exampleServiceLabel: exampleServiceLabelValue

These are all optional ways to control what metadata elements exist on the resources deployed by Halyard. For additional information about how these can be used, you can look at the Annotations or Labels and Selectors pages from Kubernetes.

nodeSelector

Node selector annotations will put out nodeSelector values in the Pod specification. They generally follow a similar syntax to the annotations and labels. For example:

kubernetes:
  nodeSelector:
     exampleNodeKey: exampleNodeValue

Additional information regarding Node Selectors can be found in the Kubernetes NodeSelector documentation.

useExecHealthCheck

By default halyard deploys services with an exec-based health check in order to improve compatibility with Istio. This however can break functionality for implementations of Load Balancer service types and Ingress Controllers that rely on having an http health check to validate. Setting kubernetes.useExecHealthCheck: false will switch the check method to be http based for such use cases.

Using Custom Volumes

You can define a set of custom volumes for a particular service that will become Volume and VolumeMount items in the generated pod specifications when Halyard deploys Spinnaker on Kubernetes. This can be useful if you need to add custom CA Certificates or other files to Spinnaker service consistently.

The Kubernetes service settings allow you to mount volumes of type configMap, secret and emptyDir with the below syntax.

kubernetes:
  volumes:
  - id: secret-example-vol
    type: secret
    mountPath: /path/to/location/of/secret
  - id: configmap-example-vol
    type: configMap
    mountPath: /path/to/location/of/configMap
  - id: emptydir-example-vol
    type: emptyDir
    mountPath: /path/to/location/of/emptyDir

Make sure that the id field matches the name of the existing secret or configmap resource in Kubernetes that you need mounted.

Deployment Strategy

You can change the deployment strategy that Kubernetes uses when deploying Spinnaker services. For example, if you wanted to adjust the maxSurge and maxUnavailable percentages for a rolling update of Clouddriver, you would define a clouddriver.yml file in your Halyard service-settings like the one below.

kubernetes:
  deploymentStrategy:
    type: rollingUpdate
    strategySettings:
      maxSurge: 100%
      maxUnavailable: 50%

If you wanted to use a Recreate strategy instead, you would use:

kubernetes:
  deploymentStrategy:
    type: recreate

For additional information regarding the implication of each of these deployment strategies, consult the Updating a Deployment documentation.