Day 7 of My 100-Day DevOps Challenge: Role of DevOps in Managing Cloud Costs

Introduction

As organizations embrace cloud computing, manageability and cost control become pivotal. While traditional infrastructure demands a dedicated team to handle tasks like database patching, security updates, and server upgrades, cloud platforms operate on a pay-as-you-go model.

For DevOps engineers, this translates to ensuring cost-effectiveness while tracking resource utilization. Today, we’ll build an AWS Resource Tracker using shell scripting. This tool provides insights into AWS resource usage, helping organizations manage their cloud spending more efficiently.

Getting Started with AWS CLI

To automate tasks on AWS, we’ll use the AWS Command Line Interface (CLI).

Step 1: Install AWS CLI

If AWS CLI isn’t installed, follow these steps:

# For Debian-based systems (e.g., Ubuntu)
sudo apt update
sudo apt install awscli -y

# For RHEL-based systems (e.g., CentOS)
sudo yum install aws-cli -y

Confirm the installation:

aws --version

Step 2: Create an AWS Account

Visit AWS Sign-Up to create an account. Once logged in, generate your Access Key ID and Secret Access Key from the Security Credentials section.

Step 3: Configure AWS CLI

Set up your AWS CLI environment by running:

aws configure

You’ll be prompted to input:

• Access Key ID

• Secret Access Key

• Default Region (e.g., us-east-1)

• Default Output Format (json for this exercise)

Building an AWS Resource Tracker Script

Writing the Shell Script

Create a file named aws_resource_tracker.sh using a text editor:

vim aws_resource_tracker.sh

Add the following script:

#!/bin/bash
# AWS Resource Tracker Script

# TRACKING:
# - AWS S3 Buckets
# - AWS EC2 Instances
# - AWS Lambda Functions
# - AWS IAM Users

# Debug mode for troubleshooting
set -x

# S3 Buckets
echo "Listing S3 buckets:"
aws s3 ls > resourceTracker

# EC2 Instances
echo "Listing EC2 instances:"
aws ec2 describe-instances | jq '.Reservations[].Instances[].InstanceId' >> resourceTracker

# Lambda Functions
echo "Listing Lambda functions:"
aws lambda list-functions >> resourceTracker

# IAM Users
echo "Listing IAM users:"
aws iam list-users >> resourceTracker

---

Running the Script

1. Make the Script Executable

chmod 755 aws_resource_tracker.sh

2. Execute the Script

./aws_resource_tracker.sh

Analyzing Script Output

After execution, the resourceTracker file is generated, containing the details of AWS resources:

Listing S3 buckets:
2024-12-12 10:00:00 my-example-bucket

Listing EC2 instances:
"i-1234567890abcdef0"

Listing Lambda functions:
{
    "Functions": [
        {
            "FunctionName": "my-lambda-function",
            "Runtime": "nodejs14.x"
        }
    ]
}

Listing IAM users:
{
    "Users": [
        {
            "UserName": "admin-user",
            "UserId": "A1B2C3D4E5F6"
        }
    ]
}

Why Is This Important?

• Efficiency: Automatically tracks key AWS resources, saving time and effort.

• Cost Control: Identifies unused or overutilized resources, helping optimize costs.

• Troubleshooting: The set -x command highlights execution steps, useful for debugging.

Key Takeaways

1. Automation with AWS CLI: Simplifies resource tracking and reporting.

2. Shell Scripting in DevOps: A fundamental skill for automating repetitive tasks.

3. Debugging Skills: Understanding set -x improves your troubleshooting ability.

4. Cloud Cost Awareness: Tracking resource usage aids in maintaining a cost-effective cloud environment.