v2.25.1 Armory Continuous Deployment Release (Spinnaker™ v1.25.3)

Release notes for Armory Continuous Deployment 2.25.1

2021/12/15 Release Notes

Note: If you’re experiencing production issues after upgrading Armory Continuous Deployment, rollback to a previous working version and please report issues to http://go.armory.io/support.

For information about what Armory supports for this version, see the [ Armory Continuous Deployment v2.25 compatibility matrix.

Required Halyard or Operator version

​ To install, upgrade, or configure Armory 2.25.1, use one of the following tools:

  • Armory-extended Halyard 1.12 or later
  • Armory Operator 1.2.6 or later ​ For information about upgrading, Operator, see Upgrade the Operator. ​

Security

​ Armory scans the codebase as we develop and release software. For information about CVE scans for this release, see the Support Portal. Note that you must be logged in to the portal to see the information.

This release includes a security fix. For more information, see the Critical Notification that Armory’s Support Team sent out on 14 December 2021, contact your Armory account rep, or see this (login required) Support article: https://support.armory.io/support?id=kb_article_view&sysparm_article=KB0010520. ​

Breaking changes

Breaking changes are kept in this list for 3 minor versions from when the change is introduced. For example, a breaking change introduced in 2.21.0 appears in the list up to and including the 2.24.x releases. It would not appear on 2.25.x release notes.

Suffix no longer added to jobs created by Kubernetes Run Job stage

Spinnaker no longer automatically appends a unique suffix to the name of jobs created by the Kubernetes Run Job stage. Prior to this release, if you specified metadata.name: my-job, Spinnaker updates the name to my-job-[random-string] before deploying the job to Kubernetes. As of this release, the job’s name will be passed through to Kubernetes exactly as supplied.

To continue having a random suffix added to the job name, set the metadata.generateName field instead of metadata.name, which causes the Kubernetes API to append a random suffix to the name.

This change is particularly important for users who are using the preconfigured job stage for Kubernetes or are sharing job stages among different pipelines. In these cases, jobs often running concurrently, and it is important that each job have a unique name. In order to retain the previous behavior, manually update your Kubernetes job manifests to use the generateName field.

Previously, this behavior was opt-in.

Impact

As of Armory 2.22, this behavior is the default. Users can still opt out of the new behavior by setting kubernetes.jobs.append-suffix: true in clouddriver-local.yml. This causes Spinnaker to continue to append a suffix to the name of jobs as in prior releases.

The ability to opt out of the new behavior will be removed in Armory 2.23 (OSS 1.23). The above setting will have no effect, and Spinnaker will no longer append a suffix to job names. We recommended that 2.22 users note which jobs are using the old behavior and prepare to remove the setting before upgrading to Armory 2.23 in the future.

Introduced in: Armory 2.22

​

Zombie Executions

Starting in Spinnaker 2.23.0, ManifestForceCacheRefreshTask was removed, as Kubernetes manifest related stages now do live lookups. While upgrading to Spinnaker 2.23.0 or later, if there is a running pipeline that contains a Kubernetes manifest related stage, it becomes a zombie execution. This causes Orca, Spinnaker’s orchestration service, to fail to complete any Kubernetes manifest related stage in that pipeline.

Workarounds:

To resolve the issue, cancel any zombie executions. For information about how to cancel them, see the Orca Zombie Execution runbook.

Affected versions: 2.23.0 and later

​

ManifestForceCacheRefreshTask removed from Orca

When you upgrade to 2.23.0 or later, you might encounter the following error:

2021-01-29 23:57:19.691 ERROR 1 --- [    scheduler-2] c.netflix.spinnaker.q.redis.RedisQueue   : Failed to read message 8f072714f1df6dbf3af93a4f4fe4cae2, requeuing...
com.fasterxml.jackson.databind.JsonMappingException: No task found for 'com.netflix.spinnaker.orca.clouddriver.tasks.manifest.ManifestForceCacheRefreshTask' (through reference chain: com.netflix.spinnaker.orca.q.RunTask["taskType"])

The ManifestForceCacheRefreshTask task is no longer a required task when deploying a manifest. In earlier releases, forcing the cache to refresh was part of the deployment process for manifests. Because of this change, if a task was running or retried before the upgrade, the error shows up in logs as an exception.

Workaround

Before starting, make sure that you have access to the Redis instance that Orca uses.

To resolve this issue, delete the message from the queue:

  1. Verify that there are pipeline execution failure messages that contain ManifestForceCacheRefreshTask:

    Redis

    hgetall orca.task.queue.messages

    The command returns information similar to the following:

    1) "93ac65e03399a4cfd3678e1355936ab2"
2) "{\"kind\":\"runTask\",\"executionType\":\"PIPELINE\",\"executionId\":\"01EVFCCDG3Q2209E0Z1QTNC0FS\",   \"application\":\"armoryhellodeploy\",\"stageId\":\"01EVFCCDG3TJ7AFPYEJT1N8RDJ\",\"taskId\":\"5\",\"taskType\":\"com.netflix.spinnaker.   orca.clouddriver.tasks.manifest.ManifestForceCacheRefreshTask\",\"attributes\":[{\"kind\":\"attempts\",\"attempts\":1}],   \"ackTimeoutMs\":600000}"

  2. Delete the message(s):

    Redis

    hdel orca.task.queue.messages

    The command returns information similar to the following:

    93ac65e03399a4cfd3678e1355936ab2
(integer) 1
​

Known issues

​

Bake failures

The Packer version included with Rosco disregards package overrides that use the -var-file= option. This may cause bakes to fail.

Affected versions: 2.22.2 and later

Lambda UI issue

There is a UI bug related to the caching agent that prevents Lambda functions from being displayed in the UI when there are no other clusters associated with the Application. In other words, in order for the function to show up in “Functions” tab, there needs to be a cluster (such as an AWS ASG/EC2 instance) deployed for that application.

Affected versions: 2.23.0 (1.23.0) - 2.26.2 Fixed version: 2.26.3

pipelineID for Pipelines-as-Code

There is a known issue where the pipelineID function does not work the first time, which causes pipelines to not be updated. Subsequent changes to the dinghyfile update the pipelines correctly.

Affected versions: 2.21.1 - 2.26.0 Fixed version: 2.26.1

​ ​

Git repo artifact provider cannot checkout SHAs

​ Only branches are currently supported. For more information, see 6363. ​

Server groups

There is a known issue where you cannot edit AWS server groups with the Edit button in the UI. The edit window closes immediately after you open it. ​ Workaround: To make changes to your server groups, edit the stage JSON directly by clicking on the Edit stage as JSON button. ​

Highlighted updates

​ This release includes a security fix. For more information, see the Critical Notification that Armory’s Support Team sent out on 14 December 2021, contact your Armory account rep, or see this (login required) Support article: https://support.armory.io/support?id=kb_article_view&sysparm_article=KB0010520.

Spinnaker Community Contributions

There have also been numerous enhancements, fixes, and features across all of Spinnaker’s other services. See the Spinnaker v1.25.3 changelog for details.

Detailed updates

Bill Of Materials (BOM)

Here’s the BOM for this version.

Expand
version: 2.25.1
timestamp: "2021-12-15 17:23:28"
services:
    clouddriver:
        commit: a764fa3dd360655e8ebd9b81e2217fce554f434c
        version: 2.25.6
    deck:
        commit: 3be7d16e0ba22113b38a7e8a1862f9769a119d10
        version: 2.25.6
    dinghy:
        commit: 7feba3a7b859b9595758c7c9e09782e651d9f0f8
        version: 2.25.5
    echo:
        commit: d4254bb69d38e8bf9216c045c4380933ff4582e1
        version: 2.25.4
    fiat:
        commit: fe60b6210c6ce00167aa42143a4dffebcf03fb9f
        version: 2.25.5
    front50:
        commit: 3d2302240be46ca85600e488c20059a9990f13d4
        version: 2.25.4
    gate:
        commit: eab05d036bef8c391274baa7fb3d294862fad37e
        version: 2.25.7
    igor:
        commit: 670df68838b5183faa5ab42db3559b20bbfb29c9
        version: 2.25.4
    kayenta:
        commit: f859543dc93fe2438cca2b7907fde957dde9f64c
        version: 2.25.4
    monitoring-daemon:
        version: 2.26.0
    monitoring-third-party:
        version: 2.26.0
    orca:
        commit: 8f84752ec39945409533737c25beb2a853fa22d0
        version: 2.25.3
    rosco:
        commit: 8ef53c816490ac4350813003e098d9ccdff33b0b
        version: 2.25.8
    terraformer:
        commit: 4551e4c1976da52d1f96033f2849d97dc4c9131c
        version: 2.25.9
dependencies:
    redis:
        version: 2:2.8.4-2
artifactSources:
    dockerRegistry: docker.io/armory

Armory

Armory Deck - 2.25.3…2.25.6

  • fix(settings): add MPTv2 and Dinghy Events flags back (#776)
  • fix(build): include cd actions & use cd dependencies (#1144)

Armory Kayenta - 2.25.2…2.25.4

  • fix(build): add cd workflow / use io.spinnaker deps (#284)
  • chore(cd): update base service version to kayenta:2021.12.08.00.12.05.release-1.25.x (#291)
  • chore(armory-commons): autobump armory-commons to 3.8.6 (#293)

Dinghy™ - 2.25.1…2.25.5

  • fix(config): bit more err checking on initial config load (#380)
  • fix(crash_on_module_updates): remove call to log.fatal() (#366) (#367) (#428)
  • chore(build): include workflow to onboard dinghy in cd images (#456)

Armory Clouddriver - 2.25.3…2.25.6

  • fix(build): use baseServiceVersion as base for the extension (#482)
  • fix(build): update workflows to produce CD images (#483)
  • fix(build): use the correct dependencies from baseServiceVersion (#485)
  • chore(build): bump armory.settings plugin to 1.7.2 (#495)
  • chore(cd): update base service version to clouddriver:2021.12.07.22.20.09.release-1.25.x (#504)
  • chore(armory-commons): autobump armory-commons to 3.8.6 (#507)

Armory Fiat - 2.25.3…2.25.5

  • fix(build): onboard 2.25.x with astrolabe cd (#265)
  • chore(cd): update base service version to fiat:2021.12.07.22.20.53.release-1.25.x (#277)
  • chore(armory-commons): autobump armory-commons to 3.8.6 (#280)

Terraformer™ - 2.25.0…2.25.9

  • chore(build): add gcloud sdk + anthoscli (#374) (#375)
  • chore(versions): add terraform 13.6 through 14.10 (bp #376) (#377)
  • security(versions): secure terraform binaries (backport #390) (#391)
  • chore(versions): update tf installer to use new hashi keys (#396) (#398)
  • chore(tests): retrieve secrets from s3, not vault (#418) (#422)
  • task(tf_versions): update tf bundled versions script to use new key-server (#417) (#424)
  • task(tf_versions): update tf bundled versions script to use new key (#428) (#430)
  • fix(build): add cd workflow (#446)

Armory Front50 - 2.25.2…2.25.4

  • fix(build): onboard front50 with astrolabe cd build (#326)
  • chore(cd): update base service version to front50:2021.12.07.22.22.25.release-1.25.x (#338)
  • chore(armory-commons): autobump armory-commons to 3.8.6 (#341)

Armory Gate - 2.25.5…2.25.7

  • fix(build): onboard 2.25.x with cd process (#340)
  • feat(gradle): Add stack trace to gradle build (backport #359) (#360)
  • chore(cd): update base service version to gate:2021.12.15.00.04.16.release-1.25.x (#357)

Armory Orca - 2.25.2…2.25.3

  • fix(build): add cd workflow for Astrolabe (#374)
  • chore(cd): update base orca version to 2021.12.07.23.35.56.release-1.25.x (#391)
  • chore(armory-commons): autobump armory-commons to 3.8.6 (#396)

Armory Rosco - 2.25.2…2.25.8

  • Update gradle.yml
  • fix(build): add cd workflows / build using artifact (#320)
  • chore(cd): update base service version to rosco:2021.12.07.23.36.04.release-1.25.x (#330)
  • chore(armory-commons): autobump armory-commons to 3.8.6 (#333)
  • fix(tests): remove tests intended for Spinnaker Cloud (#314) (#322)

Armory Igor - 2.25.2…2.25.4

  • fix(build): include workflows for CD with Astrolabe (#265)
  • chore(cd): update base service version to igor:2021.12.07.23.57.30.release-1.25.x (#273)
  • chore(armory-commons): autobump armory-commons to 3.8.6 (#276)

Armory Echo - 2.25.2…2.25.4

  • fix(build): onboard 2.25.x with astrolabe cd process (#376)
  • chore(build): bump armory gradle 1.7.2 (#378)
  • chore(cd): update base service version to echo:2021.12.07.22.20.45.release-1.25.x (#390)
  • chore(armory-commons): bump armory-commons to 3.8.6 (#394)

Last modified March 3, 2023: (22c29bf4)