Refresh/Update of the Amazon Linux variant

[joe-usa-amzn]

Proposed Commit Message

Adding amazon linux variant to cloud config. 
And Refreshing/Updating module values for Amazon Linux distro only.
Namely cc_ca_cert and cc_yum_add_repo.

Merge type

• Squash merge using "Proposed Commit Message"
• Rebase and merge unique commits. Requires commit messages per-commit each referencing the pull request number (#<PR_NUM>)

[joe-usa-amzn]

Can you take a look @holmanb ?
Thanks.

[blackboxsw]

Thanks for the submission @joe-usa-amzn.

1. Generally we'd like to see unittest coverage for the changes made where possible. So, I'd expect to see some test coverage in tests/unittests/config/tests_cc_ca_certs.py and tests_cc_yum_add_repo.py. In yum_add_repo_case I don't think there is anything signficant or unique to test, but we should at least have something to establish amazon certificate path treatment meets expectations for enable and disable operations.

2. Since we have a lot of duplication of config settings for rhel-based and opensuse-based families of distributions. Let's consolidate both if we can to avoid copy-n-paste boilerplate config for similar settings.
   What I think we can do here is create a DISTRO_FAMILY map to lookup expected config based on whether the distro is in the opensuse or rhel "family".

We could do something like this in cc_ca_certs.py

diff --git a/cloudinit/config/cc_ca_certs.py b/cloudinit/config/cc_ca_certs.py
index 2c02169ac..1122a498e 100644
--- a/cloudinit/config/cc_ca_certs.py
+++ b/cloudinit/config/cc_ca_certs.py
@@ -30,13 +30,6 @@ DISTRO_OVERRIDES = {
         "ca_cert_config": "/etc/ca-certificates/conf.d/cloud-init.conf",
         "ca_cert_update_cmd": ["update-ca-bundle"],
     },
-    "fedora": {
-        "ca_cert_path": "/etc/pki/ca-trust/",
-        "ca_cert_local_path": "/usr/share/pki/ca-trust-source/",
-        "ca_cert_filename": "anchors/cloud-init-ca-cert-{cert_index}.crt",
-        "ca_cert_config": None,
-        "ca_cert_update_cmd": ["update-ca-trust"],
-    },
     "rhel": {
         "ca_cert_path": "/etc/pki/ca-trust/",
         "ca_cert_local_path": "/usr/share/pki/ca-trust-source/",
@@ -58,52 +51,29 @@ DISTRO_OVERRIDES = {
         "ca_cert_config": None,
         "ca_cert_update_cmd": ["rehash_ca_certificates.sh"],
     },
-    "amazon": {
-        "ca_cert_path": "/etc/pki/ca-trust/",
-        "ca_cert_local_path": "/usr/share/pki/ca-trust-source/",
-        "ca_cert_filename": "anchors/cloud-init-ca-cert-{cert_index}.crt",
-        "ca_cert_config": None,
-        "ca_cert_update_cmd": ["update-ca-trust"],
-    },
 }
 
-for distro in (
-    "opensuse-microos",
-    "opensuse-tumbleweed",
-    "opensuse-leap",
-    "sle_hpc",
-    "sle-micro",
-    "sles",
-):
-    DISTRO_OVERRIDES[distro] = DISTRO_OVERRIDES["opensuse"]
-
-for distro in (
-    "almalinux",
-    "centos",
-    "cloudlinux",
-    "rocky",
-):
-    DISTRO_OVERRIDES[distro] = DISTRO_OVERRIDES["rhel"]
-
-distros = [
-    "almalinux",
-    "amazon",
+DISTRO_FAMILY = {
+    "almalinux": "rhel",
+    "amazon": "rhel",
+    "centos": "rhel",
+    "cloudlinux": "rhel",
+    "opensuse-microos": "opensuse",
+    "opensuse-tumbleweed": "opensuse",
+    "opensuse-leap": "opensuse",
+    "rocky": "rhel",
+    "sle_hpc": "opensuse",
+    "sle-micro": "opensuse",
+    "sles": "opensuse",
+}
+
+distros = list(DISTRO_FAMILY.keys()) + [
     "aosc",
-    "centos",
-    "cloudlinux",
     "alpine",
     "debian",
-    "fedora",
     "raspberry-pi-os",
     "rhel",
-    "rocky",
     "opensuse",
-    "opensuse-microos",
-    "opensuse-tumbleweed",
-    "opensuse-leap",
-    "sle_hpc",
-    "sle-micro",
-    "sles",
     "ubuntu",
     "photon",
 ]
@@ -122,6 +92,8 @@ def _distro_ca_certs_configs(distro_name):
     @param distro_name: String providing the distro class name.
     @returns: Dict of distro configurations for ca_cert.
     """
+    if distro_name in DISTRO_FAMILY:
+        distro_name = DISTRO_FAMILY[distro_name]
     cfg = DISTRO_OVERRIDES.get(distro_name, DEFAULT_CONFIG)
     cfg["ca_cert_full_path"] = os.path.join(
         cfg["ca_cert_local_path"], cfg["ca_cert_filename"]

3. rhel and photon distros end up calling cc_ca_certs::remove_default_ca_certs, should "amazon" be listed there too to remove all trusted certificates when #cloud-config user-data contains remove-defaults: true or should amazonlinux just disable the system certs with a leading ! via disable_system_ca_certs function?

4. Please attach /var/log/cloud-init.log for a successful run in AmazonLinux

[blackboxsw]

Can you please attach /var/log/cloud-init.log of a successful run on AmazonLinx 2023 or AmazonLinux2 where some yum repo was configured in user-data with something like:

#cloud-config 
yum_repos:
  my package stream:
    baseurl: http://blah.org/pub/epel/testing/5/$basearch/
    mirrorlist: http://some-url-to-list-of-baseurls
    repo_gpgcheck: 1
    enable_gpgcheck: true
    gpgkey: https://url.to.ascii-armored-gpg-key

I would like to confirm how the Amazonlinux distro is represented in logs and that the resulting files written are relevant.

[joe-usa-amzn]

Fixed cc_certs to have distro families.

Here is the logs from AL2023 with EPEL yum_repo userdata:

2026-02-26 03:00:11,449 - modules.py[DEBUG]: Running module yum-add-repo (<module 'cloudinit.config.cc_yum_add_repo' from '/usr/lib/python3.9/site-packages/cloudinit/config/cc_yum_add_repo.py'>) with frequency once-per-instance
2026-02-26 03:00:11,449 - handlers.py[DEBUG]: start: modules-config/config-yum-add-repo: running config-yum-add-repo with frequency once-per-instance
2026-02-26 03:00:11,449 - util.py[DEBUG]: Writing to /var/lib/cloud/instances/i-01faa8e897d8a7be4/sem/config_yum_add_repo - wb: [644] 25 bytes
2026-02-26 03:00:11,450 - util.py[DEBUG]: Restoring selinux mode for /var/lib/cloud/instances/i-01faa8e897d8a7be4/sem/config_yum_add_repo (recursive=False)
2026-02-26 03:00:11,451 - util.py[DEBUG]: Restoring selinux mode for /var/lib/cloud/instances/i-01faa8e897d8a7be4/sem/config_yum_add_repo (recursive=False)
2026-02-26 03:00:11,452 - helpers.py[DEBUG]: Running config-yum-add-repo using lock (<FileLock using file '/var/lib/cloud/instances/i-01faa8e897d8a7be4/sem/config_yum_add_repo'>)
2026-02-26 03:00:11,453 - util.py[DEBUG]: Writing to /etc/yum.repos.d/epel.repo - wb: [644] 287 bytes
2026-02-26 03:00:11,453 - util.py[DEBUG]: Restoring selinux mode for /etc/yum.repos.d/epel.repo (recursive=False)
2026-02-26 03:00:11,454 - util.py[DEBUG]: Restoring selinux mode for /etc/yum.repos.d/epel.repo (recursive=False)
2026-02-26 03:00:11,454 - handlers.py[DEBUG]: finish: modules-config/config-yum-add-repo: SUCCESS: config-yum-add-repo ran successfully
2026-02-26 03:00:11,454 - modules.py[DEBUG]: Running module ntp (<module 'cloudinit.config.cc_ntp' from '/usr/lib/python3.9/site-packages/cloudinit/config/cc_ntp.py'>) with frequency once-per-instance
2026-02-26 03:00:11,455 - handlers.py[DEBUG]: start: modules-config/config-ntp: running config-ntp with frequency once-per-instance
2026-02-26 03:00:11,455 - util.py[DEBUG]: Writing to /var/lib/cloud/instances/i-01faa8e897d8a7be4/sem/config_ntp - wb: [644] 25 bytes

[joeysk2012]

@blackboxsw Following up, can you take another look?

[blackboxsw]

Thank you @joe-usa-amzn and @joeysk2012.

1. Ideally, I'd like to see the attached full /var/log/cloud-init.log to also see how network configuration is treated (because we touched that in the config/cloud.cfg.tmpl in this PR.

2. can we also please attach /etc/cloud/cloud.cfg from an AL2023 instance where this ran to config that the redaction of network: {config: disabled} aligns with current downstream AL2023 packaging anyway.

Many thanks, when we have the log and the /etc/cloud/cloud.cfg files represented, I think we can merge this.

[joe-usa-amzn]

Hi @blackboxsw

1. I posted the full logs here
2. /etc/cloud/cloud.cfg

[ec2-user@ip-172-31-43-54 ~]$ cat /etc/cloud/cloud.cfg
# The top level settings are used as module
# and base configuration.

# A set of users which may be applied and/or used by various modules
# when a 'default' entry is found it will reference the 'default_user'
# from the distro configuration specified below
users:
  - default

# If this is set, 'root' will not be able to ssh in and they
# will get a message to login instead as the default $user
disable_root: true

mount_default_fields: [~, ~, 'auto', 'defaults,nofail', '0', '2']
resize_rootfs: noblock
ssh_pwauth: false

# This will cause the set+update hostname module to not operate (if true)
preserve_hostname: false

# If you use datasource_list array, keep array items in a single line.
# If you use multi line array, ds-identify script won't read array items.
# Example datasource config
# datasource:
#   Ec2:
#     metadata_urls: [ 'blah.com' ]
#     timeout: 5 # (defaults to 50 seconds)
#     max_wait: 10 # (defaults to 120 seconds)


# The modules that run in the 'init' stage
cloud_init_modules:
  - seed_random
  - bootcmd
  - write_files
  - growpart
  - resizefs
  - disk_setup
  - mounts
  - set_hostname
  - update_hostname
  - update_etc_hosts
  - ca_certs
  - rsyslog
  - selinux
  - users_groups
  - ssh
  - set_passwords

# The modules that run in the 'config' stage
cloud_config_modules:
  - ssh_import_id
  - keyboard
  - locale
  - yum_add_repo
  - ntp
  - timezone
  - disable_ec2_metadata
  - runcmd

# The modules that run in the 'final' stage
cloud_final_modules:
  - package_update_upgrade_install
  - write_files_deferred
  - puppet
  - chef
  - ansible
  - mcollective
  - salt_minion
  - reset_rmc
  - scripts_vendor
  - scripts_per_once
  - scripts_per_boot
  - scripts_per_instance
  - scripts_user
  - ssh_authkey_fingerprints
  - keys_to_console
  - install_hotplug
  - phone_home
  - final_message
  - power_state_change

# System and/or distro specific settings
# (not accessible to handlers/transforms)
system_info:
  # This will affect which distro class gets used
  distro: amazon
  # Default user name + that default users groups (if added/used)
  default_user:
    name: ec2-user
    lock_passwd: True
    gecos: EC2 Default User
    groups: [wheel, adm, systemd-journal]
    sudo: ["ALL=(ALL) NOPASSWD:ALL"]
    shell: /bin/bash
  # Other config here will be given to the distro class and/or path classes
  paths:
    cloud_dir: /var/lib/cloud/
    templates_dir: /etc/cloud/templates/
  ssh_svcname: sshd

[blackboxsw]

Thank you for the attached details.

1. I confirm, the renderd /etc/cloud/cloud.cfg on AmazonLinux aligns with this PR's upstream changes to cloud.cfg.tmpl
2. Unfortunately, the attached logs show that the system configuration on disk in /etc/cloud/cloud.cfg or /etc/cloud/cloud.cfg.d/*.cfg still leave configuration on disk somewhere that prevides network: {config: disabled} in the log:

2026-04-09 20:55:52,146 - stages.py[INFO]: network config is disabled by system_cfg

3. The attached logs show that no yum_add_repo configuration was successfully provided to the instance at launch time as well:

2026-04-09 20:55:54,985 - modules.py[INFO]: Skipping modules 'keyboard,yum_add_repo,ntp,timezone,disable_ec2_metadata,runcmd' because no applicable config is provided.

Can we please get logs which show a successful VM launch proving a succesful network config by cloud-init on a system that doesn't disable network config by default and also contains user-data with valid #cloud-config user-data containing yum_repos configuration?

[blackboxsw]

Thank you @joe-usa-amzn for the updates, it appears though that the attached cloud-init.log shows a system that still has a network: {config: disabled} setting somewhere and the launched VM didn't provide a valid yum_repos configuration or ca_certs in user-data which meant that cc_yum_add_repos module and cc_ca_certs modules were both disabled instead of showing a successful repo setup by cloud-init.

Marking this as request changes as we'll need a success run of the cc_yum_add_repos module and ca_certs with valid user-data as well as showing network setup without network being disabled by system config to confirm both the cloud.cfg.tmpl changes and the cc_yum_add_repo/cc_ca_certs changes.

Examples of ca_certs user-data and yum_repos user-data

Marking request changes, for just the logs showing a full success of network and user-data.

[blackboxsw]

One more blocker too @joe-usa-amzn:
In disable_default_ca_certs(cloud.distro.name) we passed in cloud.distro.name == 'amazon', but end up with a NOOP instead of calling remove_default_ca_certs due to the False check if distro_name in ["rhel", "photon"]:. I think we need to include 'amazon' in that distro_name check to ensure we properly disable CA certs if requested.

[joe-usa-amzn]

One more blocker too @joe-usa-amzn: In disable_default_ca_certs(cloud.distro.name) we passed in cloud.distro.name == 'amazon', but end up with a NOOP instead of calling remove_default_ca_certs due to the False check if distro_name in ["rhel", "photon"]:. I think we need to include 'amazon' in that distro_name check to ensure we properly disable CA certs if requested.

Fixed.

[joe-usa-amzn]

Added update log showing successful run of yum_add_repo and cc_ca_cert:
https://github.com/joe-usa-amzn/cloud-init/blob/amzn-logs/cloud-init.log

The network will always be disabled for ec2 images because we ship the extra config as something baked into the image:

[ec2-user@ip-172-31-36-64 ~]$ cat /etc/cloud/cloud.cfg.d/01_amazon-ec2.cfg
network:
  config: disabled
datasource_list: [ Ec2, None ]
datasource:
  Ec2:
    metadata_urls: ["http://169.254.169.254:80", "http://[fd00:ec2::254]:80"]
    apply_full_imds_network_config: false
ssh_genkeytypes:
  - ed25519
  - ecdsa

Thanks,
@blackboxsw