SUSE Linux
SUSE Public Cloud Infrastructure Update
SUSE has recently made changes to their Public Cloud Update Infrastructure starting June 1, 2020, which is known to cause issues when trying to reach the SUSE repositories as the repositories were no longer present. SUSE has shifted from IP based restrictions to Instance billing code-based restrictions. Per SUSE's official documentation [1], the instances must upgrade the package "cloud-regionsrv-client" to version 9.0.0 or greater. Therefore, it requires users to update the required packages manually. For more information about this change in SUSE Update Infrastructure, please refer to documents [1,2].
[1] Public Cloud Infrastructure Update:
https://www.suse.com/support/kb/doc/?id=000019633
[2] SUSE Update Infrastructure Access Through the Data Center:
https://www.suse.com/c/suse-update-infrastructure-access-through-the-data-center/
Steps: 1) Confirm the instance has a billing product code and is not "null", e.g. "billingProducts”: [“bp-XXXX”]. If an Instance is launched with custom SLES AMI, which is registered before June 2019, such an instance could be more prone to missing “billingProducts” issues:
$ curl -s http://169.254.169.254/latest/dynamic/instance-identity/document | grep -i billing "billingProducts" : [ "bp-6ca54005" ],
2) Since the the above command showed a billing code, we were able to proceed to the next steps to update the "cloud-regionsrv-client" package by following SUSE's documentation [1]
# rpm -qa cloud-regionsrv-client cloud-regionsrv-client-7.0.5-51.1.noarch <------ (old version) # wget --no-check-certificate https://52.15.49.139/late_instance_offline_update_ec2_x86_64_SLE12.tar.gz # tar -xf late_instance_offline_update_ec2_x86_64_SLE12.tar.gz # cd x86_64 # zypper --no-refresh --no-remote --non-interactive in *.rpm # zypper ref # rpm -qa cloud-regionsrv-client cloud-regionsrv-client-9.0.10-52.33.1.noarch <------ (new 9.0 version)
3) Attempt registration:
# registercloudguest --force-new
4) Verify the host is registered:
# SUSEConnect -s
5.A) If the registration fails, share the below output for review:
# rpm -qa | grep -iE "cloud-regionsrv-client|regionServiceClientConfigEC2|regionServiceClientConfigSAPEC2|ec2metadata|SUSEConnect|libzypp|ruby-common" # registercloudguest --force-new # SUSEConnect -s # cat /var/log/cloudregister
5.B) If you are still getting an error such as There are no enabled repositories defined... Use 'zypper addrepo' or 'zypper modifyrepo' commands to add or enable repositories, you can run the following command:
- There are situations when SuSE systems aren't able to install new packages or update them because they lost their AWS repository configuration. This problem can be fixed by issuing the above command as super user (ref: https://www.suse.com/support/kb/doc/?id=000019476 and http://www.scalingbits.com/aws/sap/suse/repos)
$ /usr/sbin/registercloudguest --force-new
Zypper Cheatsheet
https://en.opensuse.org/images/1/17/Zypper-cheat-sheet-1.pdf
https://en.opensuse.org/images/3/30/Zypper-cheat-sheet-2.pdf
Where are repositories located
$ ls -al /etc/zypp/repos.d
Add notes: [+] SUSE - Basic Networking https://documentation.suse.com/sles/15-SP1/html/SLES-all/cha-network.html#sec-network-addresses
[+] openSUSE Documentation - 21.6. Configuring a Network Connection Manually https://www.pks.mpg.de/~mueller/docs/suse10.2/html/opensuse-manual_en/manual/sec.basicnet.manconf.html
[+] make ip route and ip rule permanent on SLES15 SP1 https://forums.suse.com/discussion/comment/58961#Comment_58961
[+] SLE 15 - Configure a static network address while in the Rescue System https://www.suse.com/support/kb/doc/?id=000019454
[+] SUSE LINUX – Administration Guide - Chapter 14. Linux in the Network / 14.3. Manual Network Configuration https://www.novell.com/documentation/suse91/suselinux-adminguide/html/ch14s03.html#sec:netz.conf.dateien
[+] Configure Static IP on SUSE Linux https://sahlitech.com/configure-static-ip-on-suse-linux/
[+] Configure Static Ip using the Terminal https://forums.opensuse.org/showthread.php/431523
[+] SUSE LINUX – Administration Guide https://www.novell.com/documentation/suse91/suselinux-adminguide/html/ch14s05.html
[+] ip rule http://linux-ip.net/html/tools-ip-rule.html
[+] Multi-NIC with cloud-netconfig in EC2 and Azure https://suse.com/c/multi-nic-cloud-netconfig-ec2-azure/
Download image
https://www.suse.com/download/sles/
If you boot up sles machine but can't access public internet or resolve DNS
Check if ipv6 is enabled. Run:
ip addr show
If you see inet6, that means IPv6 is enabled and may be causing routing issues.
Disable IPv6: https://www.suse.com/support/kb/doc/?id=000016980
How to add drivers for Nitro on AWS EC2
For SLES 12 SP4 and earlier based instances:
echo 'add_drivers+=" ena ext4 nvme nvme-core virtio virtio_scsi xen-blkfront xen-netfront "' >> /etc/dracut.conf.d/07-aws-type-switch.conf && mkinitrd
For SLES 12 SP5 and later based instances
echo 'add_drivers+=" ena ext4 nvme nvme-core xen-blkfront xen-netfront "' >> /etc/dracut.conf.d/07-aws-type-switch.conf && mkinitrd
How to add drivers to initrd file
mkinitrd -A -v -m "$xen_drivers" -k "${KERNEL_NAME}" -i "${INITRAMFS_TMP_NAME}"
or
(obsolete -f)
mkinitrd -f -v -m "$xen_drivers" -k "${KERNEL_NAME}" -i "${INITRAMFS_TMP_NAME}"
or
dracut --no-host-only -v --force --force-drivers 'xen-scsifront xen-netfront xen-blkfront nvme-core nvme ena' --omit 'multipath' aws-launch-initrd-4.12.14-94.41-default 4.12.14-94.41-default
Exmaple: (FOR SUSE15 SP2)
$ mkinitrd -f -v -m "xen-scsifront xen-netfront xen-blkfront nvme_core nvme ena" -k vmlinuz-`uname -r` -i /boot/CloudEndure-Migration-initrd-`uname -r`
SLES and Nitro bug "multipath"
Some SLES version have this bug on Nitro regarding multipath.
Multipath is not supported on EC2. Due to this the LVM does not getting detected as /dev/nvme0n1p1 as the device naming under /dev gets completely changed. However, for whatever reason, it is compatible with Xen platform but not with Nitro Platform. Therefore, some SLES instance boots successfully on xen but fails on nitro as the OS is unable to find PV devices of the VG.
The issue can also occur on systems that do not have LVM on the source (i.e. SLES12 SP5). Upon launching as nitro, the UUID's end up pointing to device multipath devices (i.e. dm-0) instead of nvme0n1. This is causing the system to see it as multipath devices and thus will fail to boot.
If you boot up SLES on Nitro and the devices become like this:
# dmsetup info Name: nvme.1d0f-766f6c3039623963363664636632653836633639-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001 State: ACTIVE Read Ahead: 1024 Tables present: LIVE Open count: 2 Event number: 0 Major, minor: 254, 0 Number of targets: 1 UUID: mpath-nvme.1d0f-766f6c3039623963363664636632653836633639-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001
Run multipath command to confirm:
# multipath -ll nvme.1d0f-766f6c3039623963363664636632653836633639-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001 dm-0 NVME,Amazon Elastic Block Store size=16G features='0' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active `- 0:0:1:1 nvme0n1 259:0 active ready running
That means the NVMe local device was identified as multipath device. So you need to add multipath=off in the grub kernel parameter or remove the multipath module from the initrd file.
Permanent solution: edit /etc/default/grub and add the multipath=off parameter in "GRUB_CMDLINE_LINUX_DEFAULT". and then rebuilt initrd.
HOW TO LAUNCH SLES12 SP5 SUCCESSFULLY AS NITRO
[+] Source:
# uname -r 4.12.14-122.173-default # cat /etc/*release NAME="SLES" VERSION="12-SP5" VERSION_ID="12.5" PRETTY_NAME="SUSE Linux Enterprise Server 12 SP5" ID="sles" ANSI_COLOR="0;32" CPE_NAME="cpe:/o:suse:sles:12:sp5" SUSE Linux Enterprise Server 12 (x86_64) VERSION = 12 PATCHLEVEL = 5 # This file is deprecated and will be removed in a future service pack or release. # Please check /etc/os-release for details about this release. # lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 10G 0 disk ├─sda1 8:1 0 1.3G 0 part [SWAP] └─sda2 8:2 0 8.8G 0 part /usr/local sr0 11:0 1 3.8G 0 rom # multipath -ll # lsmod | grep dm_multipath dm_multipath 32768 0 dm_mod 155648 1 dm_multipath scsi_mod 249856 9 scsi_dh_emc,sd_mod,scsi_dh_alua,libata,scsi_dh_rdac,sr_mod,dm_multipath,sg,vmw_pvscsi # ls -l /dev/disk/by-uuid/ total 0 lrwxrwxrwx 1 root root 9 Aug 21 03:42 2019-11-09-00-58-33-00 -> ../../sr0 lrwxrwxrwx 1 root root 10 Aug 21 03:42 428bc961-a28f-430b-8768-1fa14768683d -> ../../sda1 lrwxrwxrwx 1 root root 10 Aug 21 03:42 e633113f-0bf8-47c1-a8b5-55f880d1a608 -> ../../sda2 # lsblk -f NAME FSTYPE LABEL UUID FSAVAIL FSUSE% MOUNTPOINT sda ├─sda1 swap 428bc961-a28f-430b-8768-1fa14768683d [SWAP] └─sda2 btrfs e633113f-0bf8-47c1-a8b5-55f880d1a608 2.7G 63% / sr0 iso9660 SLE-12-SP5-Server-DVD-x86_640372 2019-11-09-00-58-33-00
[+] Steps 1) Run commands to register SLES and install cloud-netconfig-ec2 on the source. This package is helpful in case customer decides to use multiple nics later on (see: https://www.suse.com/c/multi-nic-cloud-netconfig-ec2-azure/ ):
# SUSEConnect -p sle-module-public-cloud/12/x86_64 # zypper install -y cloud-netconfig-ec2 Confirm: # rpm -qa | grep cloud-netconfig-ec2 cloud-netconfig-ec2-1.7-27.1.noarch
2) Install MGN agent.
3) Edit launch template to set instance type to Nitro (t3) and enable public IP.
If you launch the target as Nitro, it will fail 1/2 as it booted to emergency mode due to nvme device name change (see bug explanation above):
[ 11.681603] Adding 1315836k swap on /dev/mapper/nvme.1d0f-766f6c3035636230323137333833373535353965-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001-part1. Priority:-1 extents:1 across:1315836k SSFS [ 11.799105] systemd-journald[722]: Received request to flush runtime journal from PID 1 You are in emergency mode. After logging in, type "journalctl -xb" to view
So now I'm going to try to run the dracut command, to omit the "multipath" module
4) Run:
dracut --no-host-only -v --force --force-drivers 'xen-scsifront xen-netfront xen-blkfront nvme-core nvme ena xen-balloon xen-vbd xen-platform-pci xen-vnif xen-scsi' --omit 'multipath' /var/lib/aws-replication-agent/aws-launch-initrd-`uname -r` uname -r
Wait 10 minutes.
5) Do target launch, it should boot and pass 2/2 successfully:
Welcome to SUSE Linux Enterprise Server 12 SP5 (x86_64) - Kernel 4.12.14-122.173-default (ttyS0).