Install Terraform Integration in Spinnaker (Halyard)

Learn how to install Armory’s Terraform Integration Plugin in a Spinnaker instance managed by Halyard. Terraform Integration enables your app developers to provision infrastructure using Terraform as part of their delivery pipelines.

Overview of installing Terraform Integration

Installing the Terraform Integration plugin consists of these steps:

Configure Spinnaker: Redis, gitrepo artifact, and optional repos
Install the service: Kubernetes manifests
Install the plugin: Local config files

Compatibility

Spinnaker Version	Terraform Integration Service Version	Terraform Integration Plugin Version
1.30.x	2.30	0.0.2
1.29.x	2.28	0.0.1
1.28.x	2.28	0.0.1

Before you begin

You have read the Terraform Integration Overview.

Spinnaker requirements

You are running open source Spinnaker.
You manage your instance using Halyard. If you are using the Spinnaker Operator, see Install Terraform Integration in Spinnaker (Spinnaker Operator)

Warning

The examples in this guide are for a vanilla Spinnaker installation. You may need to adjust them for your environment.

Terraform Integration requirements

Basic auth credentials for the Git repository where your store your Terraform scripts. The Terraform Integration plugin needs access to credentials to download directories that house your Terraform templates.
- You can configure your Git repo with any of the following:
  - A Personal Access Token (potentially associated with a service account).
  - SSH protocol in the form of an SSH key or an SSH key file
  - Basic auth in the form of a user and password, or a user-password file
A source for Terraform Input Variable files (tfvar) or a backend config. You must have a separate artifact provider that can pull your tfvar file(s). The Terraform Integration plugin supports the following artifact providers for tfvar files and backend configs:
- GitHub
- BitBucket
- HTTP artifact
A dedicated external Redis instance
- Armory requires configuring a dedicated external Redis instance for production usage of the Terraform Integration plugin. This is to ensure that you do not encounter scaling or stability issues in production.

Configure Spinnaker

Configure Redis

Terraform Integration uses Redis to store Terraform logs and plans.

You can only configure the Terraform Integration feature to use a password with the default Redis user.

Configure Redis settings in your configuration and then apply.

spec:
  spinnakerConfig:
    profiles:
      terraformer:
        redis:
          baseUrl: "redis://spin-redis:6379"
          password: "password"

Configure your artifact account

The Terraform Integration uses the following artifact accounts:

Git Repo - To fetch the repo housing your main Terraform files.
GitHub, BitBucket or HTTP - Optional. To fetch single files such as var-files or backend config files.

Configure the Git Repo artifact

Spinnaker uses the Git Repo Artifact Provider to download the repo containing your main Terraform templates.

Edit your configuration to add the following:

spec:
  spinnakerConfig:
    profiles:
      clouddriver:
        artifacts:
          gitRepo:
            enabled: true
            accounts:
            - name: gitrepo
              token: <your-personal-access-token> #  personal access token

For more configuration options, see Configure a Git Repo Artifact Account.

Configure additional repos

This step is optional.

These optional steps describe how to configure GitHub as an artifact provider for the Terraform Integration.

Spinnaker uses the Github Artifact Provider to download any referenced tfvar files.

Configure your GitHub artifact:

spec:
  spinnakerConfig:
    config:
      artifacts:
        github:
          accounts:
          - name: <github-for-terraform> 
            token: <your-github-personal-access-token>
          enabled: true

name: the name for this account; replace github-for-terraform with a unique identifier for the artifact account.
token: GitHub personal access token; this field supports “encrypted” field references.

Spinnaker uses the BitBucket Artifact Provider to download any referenced tfvar files, so it must be configured with the BitBucket token to pull these files.

spec:
  spinnakerConfig:
    config:
      artifacts:
        bitbucket:
          enabled: true
          accounts:
          - name: <bitbucket-for-terraform>
            username: <your-bitbucket-username>
            password: <your-bitbucket-password>

name: the name for this account; replace <bitbucket-for-terraform> with a unique identifier for the artifact account.
username: Your Bitbucket username.
password: Your Bitbucket password; this field supports “encrypted” field references.

Install the service

Configure Kubernetes permissions

The following manifest creates a ServiceAccount, ClusterRole, and ClusterRoleBinding. Apply the manifest in your spinnaker namespace.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: terraformer-sa
  namespace: spinnaker
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: terraformer-cluster-role
rules:
- apiGroups:
  - extensions
  resources:
  - ingresses
  - ingresses/status
  verbs:
  - get
  - list
  - watch
  - create
  - update
  - patch
  - delete
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  - ingresses/status
  verbs:
  - get
  - list
  - watch
  - create
  - update
  - patch
  - delete
- apiGroups:
  - ""
  resources:
  - pods
  - endpoints
  verbs:
  - get
  - list
  - watch
  - create
  - update
  - patch
  - delete
- apiGroups:
  - ""
  resources:
  - services
  - services/finalizers
  - events
  - configmaps
  - secrets
  - namespaces
  - jobs
  verbs:
  - create
  - get
  - list
  - update
  - watch
  - patch
  - delete
- apiGroups:
  - batch
  resources:
  - jobs
  verbs:
  - create
  - get
  - list
  - update
  - watch
  - patch
- apiGroups:
  - apps
  - extensions
  resources:
  - deployments
  - deployments/finalizers
  - deployments/scale
  - daemonsets
  - replicasets
  - statefulsets
  verbs:
  - create
  - get
  - list
  - update
  - watch
  - patch
  - delete
- apiGroups:
  - monitoring.coreos.com
  resources:
  - servicemonitors
  verbs:
  - get
  - create
- apiGroups:
  - spinnaker.armory.io
  resources:
  - '*'
  - spinnakerservices
  verbs:
  - create
  - get
  - list
  - update
  - watch
  - patch
- apiGroups:
  - admissionregistration.k8s.io
  resources:
  - validatingwebhookconfigurations
  verbs:
  - '*'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: terraformer-cluster-role-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: terraformer-cluster-role
subjects:
- kind: ServiceAccount
  name: terraformer-sa
  namespace: spinnaker

Configure the service

Create a ConfigMap to contain your Terraformer Integration service configuration. Be sure to check the spinnaker.yml entry in the data section to ensure the values match your Spinnaker installation.

apiVersion: v1
kind: ConfigMap
metadata:
  name: spin-terraformer-config
  namespace: spinnaker
data:
  terraformer.yml: |
    clouddriver:
      baseUrl: ${services.clouddriver.baseUrl}
    enabled: true
    executor:
      scrapeLogsIntervalSecs: 5
      workers: 3
    git:
      enabled: false
    redis:
      baseUrl: ${services.redis.baseUrl}
     enabled: ${services.redis.enabled}
    server:
      host: ${services.terraformer.host}
      port: ${services.terraformer.port}
    spectator:
      applicationName: ${spring.application.name}
      webEndpoint:
        enabled: false
    spinnaker:
      extensibility:
        plugins: {}
        plugins-root-path: /opt/terraformer/plugins
        repositories: {}
        strict-plugin-loading: false    
  spinnaker.yml: |
    global.spinnaker.timezone: America/Los_Angeles
    services:
      clouddriver:
        baseUrl: http://spin-clouddriver:7002
        enabled: true
        host: 0.0.0.0
        port: 7002
      clouddriverCaching:
        baseUrl: http://spin-clouddriver-caching:7002
        enabled: false
        host: 0.0.0.0
        port: 7002
      clouddriverRo:
        baseUrl: http://spin-clouddriver-ro:7002
        enabled: false
        host: 0.0.0.0
        port: 7002
      clouddriverRoDeck:
        baseUrl: http://spin-clouddriver-ro-deck:7002
        enabled: false
        host: 0.0.0.0
        port: 7002
      clouddriverRw:
        baseUrl: http://spin-clouddriver-rw:7002
        enabled: false
        host: 0.0.0.0
        port: 7002
      deck:
        baseUrl: http://localhost:9000
        enabled: true
        host: 0.0.0.0
        port: 9000
      dinghy:
        baseUrl: http://spin-dinghy:8081
        enabled: true
        host: 0.0.0.0
        port: 8081
      echo:
        baseUrl: http://spin-echo:8089
        enabled: true
        host: 0.0.0.0
        port: 8089
      echoScheduler:
        baseUrl: http://spin-echo-scheduler:8089
        enabled: false
        host: 0.0.0.0
        port: 8089
      echoWorker:
        baseUrl: http://spin-echo-worker:8089
        enabled: false
        host: 0.0.0.0
        port: 8089
      fiat:
        baseUrl: http://spin-fiat:7003
        enabled: false
        host: 0.0.0.0
        port: 7003
      front50:
        baseUrl: http://spin-front50:8080
        enabled: true
        host: 0.0.0.0
        port: 8080
      gate:
        baseUrl: http://localhost:8084
        enabled: true
        host: 0.0.0.0
        port: 8084
      igor:
        baseUrl: http://spin-igor:8088
        enabled: false
        host: 0.0.0.0
        port: 8088
      kayenta:
        baseUrl: http://spin-kayenta:8090
        enabled: false
        host: 0.0.0.0
        port: 8090
      monitoringDaemon:
        baseUrl: http://spin-monitoring-daemon:8008
        enabled: false
        host: 0.0.0.0
        port: 8008
      orca:
        baseUrl: http://spin-orca:8083
        enabled: true
        host: 0.0.0.0
        port: 8083
      redis:
        baseUrl: redis://spin-redis:6379
        enabled: true
        host: 0.0.0.0
        port: 6379
      rosco:
        baseUrl: http://spin-rosco:8087
        enabled: true
        host: 0.0.0.0
        port: 8087
      terraformer:
        baseUrl: http://spin-terraformer:7088
        enabled: false
        host: 0.0.0.0
        port: 7088

Deploy the service

Replace <version> with the Terraform Integration service version compatible with your Spinnaker version.

apiVersion: v1
kind: Service
metadata:
  name: spin-terraformer
  labels:
    app: spin
    cluster: spin-terraformer
spec:
  selector:
    app: spin
    cluster: spin-terraformer
  ports:
    - name: http
      port: 7088
      protocol: TCP
      targetPort: 7088
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: spin-terraformer
  annotations:
    deployment.kubernetes.io/revision: "1"
    moniker.spinnaker.io/application: '"spin"'
    moniker.spinnaker.io/cluster: '"terraformer"'
  labels:
    app: spin
    app.kubernetes.io/managed-by: armory
    app.kubernetes.io/name: terraformer
    app.kubernetes.io/part-of: spinnaker
    app.kubernetes.io/version: <version>  # CHANGE
    cluster: spin-terraformer
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: spin
      cluster: spin-terraformer
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: spin
        app.kubernetes.io/managed-by: armory
        app.kubernetes.io/name: terraformer
        app.kubernetes.io/part-of: spinnaker
        app.kubernetes.io/version: <version> # CHANGE
        cluster: spin-terraformer
    spec:
      affinity: {}
      containers:
        - env:
            - name: SPRING_PROFILES_ACTIVE
              value: local
          image: docker.io/armory/terraformer
          imagePullPolicy: IfNotPresent
          lifecycle: {}
          name: terraformer
          ports:
            - containerPort: 7088
              protocol: TCP
          readinessProbe:
            exec:
              command:
                - wget
                - --no-check-certificate
                - --spider
                - -q
                - http://localhost:7088/health
            failureThreshold: 3
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          resources: {}
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          volumeMounts:
            - mountPath: /opt/spinnaker/config
              name: spin-terraformer-config-file
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext:
        fsGroup: 1000
        runAsUser: 1000
      terminationGracePeriodSeconds: 60
      volumes:
        - name: spin-terraformer-config-file
          secret:
            secretName: spin-terraformer-config-file

Apply the ConfigMap and Deployment manifests in your spinnaker namespace.

Install the plugin

A note about installing plugins in Spinnaker

When Halyard adds a plugin to a Spinnaker installation, it adds the plugin repository information to all services, not just the ones the plugin is for. This means that when you restart Spinnaker, each service restarts, downloads the plugin, and checks if an extension exists for that service. Each service restarting is not ideal for large Spinnaker installations due to service restart times. Clouddriver can take an hour or more to restart if you have many accounts configured.

The Terraform plugin extends Deck, Gate, and Orca. To avoid every Spinnaker service restarting and downloading the plugin, do not add the plugin using Halyard. Instead, follow the local config installation method, in which you configure the plugin in each extended service’s local profile.

Replace <version> with the plugin version [that’s compatible with your Spinnaker instance].

Add the following to gate-local.yml:

proxies:
  - id: terraform
    uri: http://spin-terraformer:7088
    methods:
      - GET
services:
  terraformer:
    enabled: true
    baseUrl: http://spin-terraformer:7088
spinnaker:
  extensibility:
    plugins:
      Armory.Terraformer:
        enabled: true
        version: <version>
    repositories:
      terraformer:
        enabled: true
        url: https://raw.githubusercontent.com/armory-plugins/pluginRepository/master/repositories.json
    deck-proxy:
      enabled: true
      plugins:
        Armory.Terraformer:
          enabled: true
          version: <version>

Add the following to orca-local.yml:

services:
  terraformer:
    enabled: true
    baseUrl: http://spin-terraformer:7088
spinnaker:
  extensibility:
    plugins:
      Armory.Terraformer:
        enabled: true
        version: <version>
    repositories:
      terraformer:
        enabled: true
        url: https://raw.githubusercontent.com/armory-plugins/pluginRepository/master/repositories.json

Save your files and apply your changes by running hal deploy apply.

What’s next

Feedback

Was this page helpful?

Thank you for letting us know!

Sorry to hear that. Please tell us how we can improve.

Last modified August 17, 2023: (525a0c04)