Introduction
Did you wake up one morning and decide to dive into a new field? You went on the internet, looking for the highest-paying industries, and discovered that tech is among the top. Intrigued, you started exploring the various roles within the tech industry and stumbled upon the role of DevOps.
You searched for tutorials online and discovered the ability to streamline and automate workflows has captured your interest. If this sounds like you, this article is for you.
Additionally, it would be best if you considered applying for internships, as structured learning and real-world experience can be incredibly advantageous.
Internships not only provide valuable hands-on experience but also allow you to build a portfolio of projects that can showcase your skills to potential employers. One such internship is the HNG Internship.
In this article, I will be explaining how to automate with bash script. Are you ready to journey into the world of efficient and automated workflows? Let's dive in!
Overview
Linux system administration is at the core of many DevOps practices. You’ll often manage Linux-based servers and cloud instances as a DevOps engineer.
Bash scripting complements Linux administration by enabling you to automate routine maintenance tasks, configure servers, etc.
Furthermore, SysOps (system operations) practices are fundamental to this process. SysOps emphasizes the management and operation of systems in a reliable and scalable manner.
By merging Linux system administration with Bash scripting and SysOps practices, DevOps engineers can achieve greater operational efficiency, minimize downtime, and ensure seamless scalability of infrastructure.
What do I need to get ready?
Before diving into automating tasks with Bash scripts, it’s crucial to understand the fundamental operators used in Bash scripting and also basic Linux commands. These operators and commands are the building blocks for writing effective and efficient scripts.
For a detailed explanation of Bash script operators, you can refer to my previous article, Bash Script Operators.
Additionally, having access to a machine with the Linux operating system is essential. Linux provides the robust environment needed to practice and execute Bash scripts effectively.
Whether you’re using a physical machine, a virtual machine, or a cloud instance, having Linux ensures you can follow along with the concepts in this article
The First & Second Rule
At this point, I can feel your excitement to start the automation of your first project but I need you to understand;
The first rule of bash scripting is "Every bash script begins with#!/bin/bash
."
This line, known as a shebang, tells the operating system where to find the Bash interpreter needed to execute the script.
The second rule is "Every bash script must end with a.sh
extension."
It's also important to note that the shell script executes from top to bottom.
Automating tasks with shell scripts
To automate tasks using shell scripts, follow these procedures:
Write Bash Scripts:
- Use a text editor like Vim or Nano to create Bash scripts.
Set Execution Permissions:
- Make the script executable by running the command:
chmod +x <name-of-script>
.
- Make the script executable by running the command:
Use Control Structures:
- Incorporate control structures like loops (
for
,while
), conditionals (if-else
), and functions to manage the script’s flow and handle various scenarios.
- Incorporate control structures like loops (
Schedule Execution:
- Use tools like
cron
to schedule Bash scripts to run at specific times or intervals.
- Use tools like
Test and Debug:
- Thoroughly test your script. Utilize
echo
statements or shell debugging tools to identify and resolve any issues.
- Thoroughly test your script. Utilize
Monitor and Maintain:
Regularly monitor automated tasks for errors.
Update and maintain your scripts as needed to ensure they continue to function correctly.
Sample script
This script is named create_users.sh
#!/bin/bash
list_of_users=$1
log_dir="/var/log"
log_file="/var/log/user_management.log"
password_dir="/var/secure"
password_manager="/var/secure/user_passwords.csv"
# Allow script run with sudo priviledges
if [[ "$(id -u)" -ne 0 ]]; then
echo "The script must be run with root priviledges"
echo "Running as root"
sudo -E "$0" "$@"
exit
fi
# Check if the input file is provided and it it contains any parameters
if [[ -z "$list_of_users" || ! -f "$list_of_users" ]]; then
echo "Usage: $0 <list_of_user>"
exit 1
fi
# Check the log directory exist
if [[ ! -d $log_dir ]]; then
echo "The log directory does not exist. Creating log directory...."
mkdir -p $log_dir
else
echo "The log directory exist"
fi
# Check if the password directory exist
if [[ ! -d $password_dir ]]; then
echo "The password directory does not exist. Creating password directory...."
mkdir -p $password_dir
else
echo "The password directory exist"
fi
# Ensure the log file exists and set the appropriate permissions
touch "$log_file"
chmod -R 755 "$log_file"
# Clear previous logs and initialize the password CSV file with a header
> "$log_file"
echo "username,password" > "$password_manager"
chmod 600 "$password_manager"
# Generate random password function for users
generate_password(){
urandom_data=$(head -c 32 /dev/urandom)
password=$(echo "$urandom_data" | openssl base64 | tr -dc A-Za-z0-9 | head -c 12)
echo "$password"
}
#Create a functio to describe how the logs should appear for verification purposes
logs_info(){
echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$log_file"
}
# Create users and groups function
create_users_groups(){
echo "Create users and assign them to their departments"
# define the parameters
local users=$(echo "$1" | xargs) # Trim leading and trailing spaces
local groups=$(echo "$2" | xargs) # Trim leading and trailing spaces
# Check for empty or invalid usernames or groups
if [[ -z "$users" || -z "$groups" ]]; then
logs_info "Skipping invalid line: $users, $groups"
return 1
fi
# check if user exist
if id -u "$users" &>/dev/null; then
logs_info "User $users already exists. Skipping."
return 1
fi
# Create personal group with the same name as the user
if ! getent group "$users" &>/dev/null; then
groupadd "$users"
if [[ $? -eq 0 ]]; then
logs_info "Created personal group $users for user $users."
else
logs_info "Failed to create personal group $users."
return 1
fi
fi
# Create groups if they don't exist
IFS=',' read -ra group_list <<< "$groups"
for group in "${group_list[@]}"; do
group=$(echo "$group" | xargs) # Trim leading and trailing spaces
if ! getent group "$group" &>/dev/null ; then
groupadd "$group"
if [[ $? -eq 0 ]]; then
logs_info "Created group $group."
else
logs_info "Failed to create group $group."
return 1
fi
fi
done
# Create the user with their name as primary groups
useradd -m "$users" -g "$users"
if [ $? -ne 0 ]; then
echo "Failed to create user $users."
return 1
fi
echo "Created user $users with groups $groups."
# Add the user to additional groups
IFS=',' read -ra group_list <<< "$groups"
for group in "${group_list[@]}"; do
usermod -aG "$group" "$users"
if [[ $? -eq 0 ]]; then
logs_info "Assigned user $users to group $group."
else
logs_info "Failed to assign user $users to group $group."
fi
done
logs_info "Created user $users with groups $group."
# Set permissions and ownership for the home directory
chmod 700 "/home/$users"
chown "$users:$users" "/home/$users"
logs_info "Set permissions for /home/$users."
# Generate a random password for the user
local passwords
passwords=$(generate_password)
echo "$users:$passwords" | chpasswd
if [ $? -ne 0 ]; then
logs_info "Failed to set password for $user."
return 1
fi
logs_info "Generated password for $users."
# Store the password securely
echo "$users,$passwords" >> "$password_manager"
return 0
}
# Main logic
# Check if the file exist
if [[ -e "$list_of_users" ]]; then
echo "The file $list_of_users exists."
else
echo "The file $list_of_users does not exist."
fi
# Check if the file is not empty
if [[ -s "$list_of_users" ]]; then
echo "The file $list_of_users is not empty"
else
echo "The file $list_of_users is empty"
fi
# Calling the function
while IFS=';' read -r users groups; do
echo "Read line: user='$users', groups='$groups'" # Debugging
# Skip empty lines
if [[ -z "$users" && -z "$groups" ]]; then
continue
fi
create_users_groups "$users" "$groups"
done < <(cat "$list_of_users"; echo) # Ensure last line is processed
logs_info "User creation process completed."
exit 0
What was the script meant to solve?
Now that you have seen the script, you might be wondering what this script was meant to solve. Below is a list of tasks the script was created to address:
Create users and groups as specified in a
.txt
file.Set up home directories with appropriate permissions and ownership.
Generate random passwords for the users and log all actions to
/var/log/user_management.log
.Store the generated passwords securely in
/var/secure/user_passwords.csv
.Ensure error handling for scenarios like existing users.
Breakdown of the Script
Computers are not like humans; for them to perform any task, you must be very specific in the instructions you give.
Each line of code in the script is a precise command that tells the computer exactly what to do. This precision ensures that the tasks are executed consistently and accurately every time.
Variables
The first section of the script shows different variables holding various file paths. Variables are containers that hold values. Using variables makes your scripts more flexible and maintainable.
Scripts without variables are considered hardcoded, meaning the values are directly written into the script, which can make them difficult to update and prone to errors.
Also, variables can be global and local. This script made use of both types. The global variable is a variable declared outside a function while a local variable is declared inside the function.
#example of a global variable
password_manager="/var/secure/user_passwords.csv"
# example of a local variable
local users=$(echo "$1" | xargs)
Functions
Functions were used to carry out this task because they allow you to create reusable blocks of code that can perform complex operations.
In plain terms, the logic was to create a system where I could use this script to create users and groups as long as the requirements remained the same.
To achieve this, the following functions were created:
genarate_password()
create_users_group()
Control Structures
The if-else
statements and loops
were used to iterate over the list of users and groups under various conditions.
For example, create a user called Ann if she doesn't exist, else let us know she exist and move to the create the next user
Main Logic
To create the users listed in a file, you need to check if the file exists. You can't assume such a file exists.
If the file exists, you need to confirm if the file holds any content or not.
You might wonder why the main logic is placed at the end. The main logic calls the functions declared earlier in the script.
Functions can only be executed if they are called. In this case, the line
create_user_groups
in the code calls the function defined above.
while IFS=';' read -r user groups; do
create_user_groups "$users" "$groups"
done < "$list_of_users"
Sample file
This file, named users.txt
, contains the information used to create the users and groups.
Ugochi;frontdev,sysops
Temi;QA
Tim;backenddev,QA,devops
Uriel;mobiledev,devops,sysops,cyber
Harry;cyber
Chinwe;marketing,sales
Ann;productmanager
Felix;productdesigner,frontenddev
Kelvin;finance
Stella;finance,projectmanager
Usage
Remember, I mentioned that one of the ways to automate tasks with shell scripts is to make them executable. Once you have finished writing your script, the next step is to change its permissions.
For more information about file permissions, you can refer to my previous article, Linux Permissions: A Beginner's Guide
chmod +x create_users.sh
Run the script and the.txt
file as an argument:
./create_users.sh /home/zenitugo/users.txt
Testing
This is a list of users created, along with the ownership assigned to their home directories.
tail -n 10 /etc/passwd
Ugochi:x:1001:1001::/home/Ugochi:/bin/sh
Temi:x:1002:1004::/home/Temi:/bin/sh
Tim:x:1003:1006::/home/Tim:/bin/sh
Uriel:x:1004:1009::/home/Uriel:/bin/sh
Harry:x:1005:1012::/home/Harry:/bin/sh
Benita:x:1006:1013::/home/Benita:/bin/sh
Ann:x:1007:1016::/home/Ann:/bin/sh
Felix:x:1008:1018::/home/Felix:/bin/sh
Kelvin:x:1009:1020::/home/Kelvin:/bin/sh
Stella:x:1010:1022::/home/Stella:/bin/sh
This is a list of all personal and secondary groups created.
zenitugo@Zenitugo:~$ tail -n 23 /etc/group
Ugochi:x:1001:
frontenddev:x:1002:Ugochi,Felix
sysops:x:1003:Ugochi,Uriel
Temi:x:1004:
QA:x:1005:Temi,Tim
Tim:x:1006:
backenddev:x:1007:Tim
devops:x:1008:Tim,Uriel
Uriel:x:1009:
mobiledev:x:1010:Uriel
cyber:x:1011:Uriel,Harry
Harry:x:1012:
Benita:x:1013:
marketing:x:1014:Benita
sales:x:1015:Benita
Ann:x:1016:
productmanager:x:1017:Ann
Felix:x:1018:
productdesigner:x:1019:Felix
Kelvin:x:1020:
finance:x:1021:Kelvin,Stella
Stella:x:1022:
projectmanager:x:1023:Stella
Here are all the events that occurred, as logged in/var/log/user_management.log
:
2024-07-03 18:14:07 - Created personal group Ugochi for user Ugochi.
2024-07-03 18:14:07 - Created group frontenddev.
2024-07-03 18:14:07 - Created group sysops.
2024-07-03 18:14:07 - Assigned user Ugochi to group frontenddev.
2024-07-03 18:14:07 - Assigned user Ugochi to group sysops.
2024-07-03 18:14:07 - Created user Ugochi with groups sysops.
2024-07-03 18:14:07 - Set permissions for /home/Ugochi.
2024-07-03 18:14:07 - Generated password for Ugochi.
2024-07-03 18:14:07 - Created personal group Temi for user Temi.
2024-07-03 18:14:07 - Created group QA.
2024-07-03 18:14:07 - Assigned user Temi to group QA.
2024-07-03 18:14:07 - Created user Temi with groups QA.
2024-07-03 18:14:07 - Set permissions for /home/Temi.
2024-07-03 18:14:07 - Generated password for Temi.
2024-07-03 18:14:07 - Created personal group Tim for user Tim.
2024-07-03 18:14:07 - Created group backenddev.
2024-07-03 18:14:07 - Created group devops.
2024-07-03 18:14:08 - Assigned user Tim to group backenddev.
2024-07-03 18:14:08 - Assigned user Tim to group QA.
2024-07-03 18:14:08 - Assigned user Tim to group devops.
2024-07-03 18:14:08 - Created user Tim with groups devops.
2024-07-03 18:14:08 - Set permissions for /home/Tim.
2024-07-03 18:14:08 - Generated password for Tim.
2024-07-03 18:14:08 - Created personal group Uriel for user Uriel.
2024-07-03 18:14:08 - Created group mobiledev.
2024-07-03 18:14:08 - Created group cyber.
2024-07-03 18:14:08 - Assigned user Uriel to group mobiledev.
2024-07-03 18:14:08 - Assigned user Uriel to group devops.
2024-07-03 18:14:08 - Assigned user Uriel to group sysops.
2024-07-03 18:14:08 - Assigned user Uriel to group cyber.
2024-07-03 18:14:08 - Created user Uriel with groups cyber.
2024-07-03 18:14:08 - Set permissions for /home/Uriel.
2024-07-03 18:14:08 - Generated password for Uriel.
2024-07-03 18:14:08 - Created personal group Harry for user Harry.
2024-07-03 18:14:08 - Assigned user Harry to group cyber.
2024-07-03 18:14:08 - Created user Harry with groups cyber.
2024-07-03 18:14:08 - Set permissions for /home/Harry.
2024-07-03 18:14:08 - Generated password for Harry.
2024-07-03 18:14:08 - Created personal group Benita for user Benita.
2024-07-03 18:14:08 - Created group marketing.
2024-07-03 18:14:08 - Created group sales.
2024-07-03 18:14:08 - Assigned user Benita to group marketing.
2024-07-03 18:14:08 - Assigned user Benita to group sales.
2024-07-03 18:14:08 - Created user Benita with groups sales.
2024-07-03 18:14:08 - Set permissions for /home/Benita.
2024-07-03 18:14:08 - Generated password for Benita.
2024-07-03 18:14:08 - Created personal group Ann for user Ann.
2024-07-03 18:14:09 - Created group productmanager.
2024-07-03 18:14:09 - Assigned user Ann to group productmanager.
2024-07-03 18:14:09 - Created user Ann with groups productmanager.
2024-07-03 18:14:09 - Set permissions for /home/Ann.
2024-07-03 18:14:09 - Generated password for Ann.
2024-07-03 18:14:09 - Created personal group Felix for user Felix.
2024-07-03 18:14:09 - Created group productdesigner.
2024-07-03 18:14:09 - Assigned user Felix to group productdesigner.
2024-07-03 18:14:09 - Assigned user Felix to group frontenddev.
2024-07-03 18:14:09 - Created user Felix with groups frontenddev.
2024-07-03 18:14:09 - Set permissions for /home/Felix.
2024-07-03 18:14:09 - Generated password for Felix.
2024-07-03 18:14:09 - Created personal group Kelvin for user Kelvin.
2024-07-03 18:14:09 - Created group finance.
2024-07-03 18:14:09 - Assigned user Kelvin to group finance.
2024-07-03 18:14:09 - Created user Kelvin with groups finance.
2024-07-03 18:14:09 - Set permissions for /home/Kelvin.
2024-07-03 18:14:09 - Generated password for Kelvin.
2024-07-03 18:14:09 - Created personal group Stella for user Stella.
2024-07-03 18:14:09 - Created group projectmanager.
2024-07-03 18:14:09 - Assigned user Stella to group finance.
2024-07-03 18:14:09 - Assigned user Stella to group projectmanager.
2024-07-03 18:14:09 - Created user Stella with groups projectmanager.
2024-07-03 18:14:09 - Set permissions for /home/Stella.
2024-07-03 18:14:09 - Generated password for Stella.
2024-07-03 18:14:09 - User creation process completed.
Conclusion
Automation is an essential component of modern DevOps processes. Automating repetitive tasks frees up valuable time to focus on more strategic initiatives, ultimately leading to productivity and innovation.
If you're unsure about your ability to master automation, remember that you don't have to do it alone. Joining a community or group like the HNG Network can provide valuable networking opportunities and support throughout your DevOps journey.