SSM

From DikapediaV2
Revision as of 14:26, 11 September 2024 by Ardika Sulistija (talk | contribs) (Created page with " ====Run Command Troubleshooting Checklist==== ---- * Verify if the instance is online (SSM agent pre-requisites). Is the Agent up to date? * Verify if SSM document supports the type of instance you want to target (Windows and Linux) * Ask for command ID and verify the status in SSM K2 page from SSM dashboard or from Admiral page. * Check the logs and the stdout location: ** The agent also stores the script (_script.ps1) and outputs (stderr and stdout) in the worker (<...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search


Run Command Troubleshooting Checklist


  • Verify if the instance is online (SSM agent pre-requisites). Is the Agent up to date?
  • Verify if SSM document supports the type of instance you want to target (Windows and Linux)
  • Ask for command ID and verify the status in SSM K2 page from SSM dashboard or from Admiral page.
  • Check the logs and the stdout location:
    • The agent also stores the script (_script.ps1) and outputs (stderr and stdout) in the worker (Orchestration) folders located at:
      • /var/lib/amazon/ssm/<instanceID>/document/orchestration/<RunCommandId>/plug-in/step_number.plugin>/std*
/var/lib/amazon/ssm/< i-xxxxxxxxxxxx>/document/orchestration/<command ID>/awsrunShellScript/[PatchLinux or runInSpecLinux for example]/stdout

/var/lib/amazon/ssm/<i-instanceid>/document/orchestration/<command ID>/awsrunShellScript/[PatchLinux or runInSpecLinux for example]/stderr

# example: "runInSpecLinux"
/var/lib/amazon/ssm/i-0bb46bc5e41a7a36b/document/orchestration/6b141723-0c4b-4726-8770-bc8eed816007/awsrunShellScript/runInSpecLinux/stdout

# If the command action is  aws:runCommand :
/var/lib/amazon/ssm/<i-instanceid>/document/orchestration/<command ID>/awsrunDocument/<stepName>/std*
  • Manual execution from instance?
  • Check the Run command advanced playbooks.
  • Reproduce issue
  • Cut a TT/SIM


Logs


  • /var/log/amazon/ssm

You can view SSM Agent logs on instances in the following locations:

/var/log/amazon/ssm/amazon-ssm-agent.log 

/var/log/amazon/ssm/errors.log

/var/log/amazon/ssm/audits/amazon-ssm-agent-audit-YYYY-MM-DD

Checking command logs:

here is path (replace instance id and command id)

/var/lib/amazon/ssm/<i-instanceid>/document/orchestration/<command ID>/awsrunShellScript/PatchLinux/stdout

/var/lib/amazon/ssm/<i-instanceid>/document/orchestration/<command ID>/awsrunShellScript/PatchLinux/stderr

# If the command action is  aws:runCommand :
/var/lib/amazon/ssm/<i-instanceid>/document/orchestration/<command ID>/awsrunDocument/<stepName>/std*

How to install SSM Agent


Manually install SSM Agent on EC2 instances for Linux https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-manual-agent-install.html


How to reboot managed instance once by using custom Run Command document


Yaml template:

--- 
description: "Command Document Example JSON Template"
mainSteps: 
  - 
    action: "aws:runShellScript"
    inputs: 
      runCommand: 
        - |-
            #!/bin/bash
            REBOOT_INDICATOR=/var/lib/amazon/ssm/ssm-reboot-indicator
            echo 'Rebooting the host via SSM'
            if [ ! -f ${REBOOT_INDICATOR} ]; then
              uptime
              echo 'rebooting... as the reboot file does not exist'
              touch ${REBOOT_INDICATOR}
              exit 194
            else
              echo 'reboot file exist'
              ls -l ${REBOOT_INDICATOR} 
              uptime 
              rm -f ${REBOOT_INDICATOR}
              exit 0
            fi
    name: example
schemaVersion: "2.2"
  • It first checks if the reboot tracker file (/var/lib/amazon/ssm/ssm-reboot-indicator) exists in the 'if' condition. If it does not, then it will create it, run mkfs, and then reboot it by sending out exit 194.
  • Once the system reboots, then it will go through the entire script again. Now since the reboot tracker does exist, it will remove the file and return exit 0.
  • The use of this if-else statement and the reboot tracker file helps prevent the instance from going into a reboot-loop.


--- 
description: "Command Document Example JSON Template"
mainSteps: 
  - 
    action: "aws:runShellScript"
    inputs: 
      runCommand: 
        - |-
            #!/bin/bash
            REBOOT_INDICATOR=/var/lib/amazon/ssm/ssm-reboot-indicator
            echo 'Mounting volume and Rebooting the host via SSM'
            mount -a
            if [ ! -f ${REBOOT_INDICATOR} ]; then
              uptime
              echo 'Creating filesystem on /dev/nvme1n1'
              mkfs -t ext4 /dev/nvme1n1
              echo 'rebooting... as the reboot file does not exist'
              touch ${REBOOT_INDICATOR}
              exit 194
            else 
              echo 'reboot file exist, the server has been rebooted after mounting'
              ls -l ${REBOOT_INDICATOR}
              uptime
              rm -f ${REBOOT_INDICATOR}
              exit 0
            fi
    name: example
schemaVersion: "2.2"