Alex Pandian was the system administrator for a tech company, responsible for managing servers, maintaining network stability, and ensuring that everything ran smoothly.

With many scripts running daily and long-running processes that needed monitoring, Alex was constantly flooded with notifications.

Alex Pandian: “Every day, I have to gothrough dozens of emails and alerts just to find the ones that matter,”

Alex muttered while sipping coffee in the server room.

Alex Pandian: “There must be a better way to streamline all this information.”

Despite using several monitoring tools, the notifications from these systems were scattered and overwhelming. Alex needed a more efficient method to receive alerts only when crucial events occurred, such as script failures or the completion of resource-intensive tasks.

Determined to find a better system, Alex began searching online for a tool that could help consolidate and manage notifications.

After reading through countless forums and reviews, Alex stumbled upon a discussion about ntfy.sh, a service praised for its simplicity and flexibility.

“This looks promising,” Alex thought, excited by the ability to publish and subscribe to notifications using a straightforward, topic-based system. The idea of having notifications sent directly to a phone or desktop without needing complex configurations was exactly what Alex was looking for.

Alex decided to consult with Sam, a fellow system admin known for their expertise in automation and monitoring.

Alex Pandian: “Hey Sam, have you ever used ntfy.sh?”

Sam: “Absolutely, It’s a lifesaver for managing notifications. How do you plan to use it?”

Alex Pandian: “I’m thinking of using it for real-time alerts on script failures and long-running commands, Can you show me how it works?”

Sam: “Of course,”

with a smile, eager to guide Alex through setting up ntfy.sh to improve workflow efficiency.

Together, Sam and Alex began configuring ntfy.sh for Alex’s environment. They focused on setting up topics and integrating them with existing systems to ensure that important notifications were delivered promptly.

Step 1: Identifying Key Topics

Alex identified the main areas where notifications were needed:

• script-failures: To receive alerts whenever a script failed.
• command-completions: To notify when long-running commands finished.
• server-health: For critical server health alerts.

Step 2: Subscribing to Topics

Sam showed Alex how to subscribe to these topics using ntfy.sh on a mobile device and desktop. This ensured that Alex would receive notifications wherever they were, without having to constantly check email or dashboards.

# Subscribe to topics
ntfy subscribe script-failures
ntfy subscribe command-completions
ntfy subscribe server-health

Step 3: Automating Notifications

Sam explained how to use bash scripts and curl to send notifications to ntfy.sh whenever specific events occurred.

“For example, if a script fails, you can automatically send an alert to the ‘script-failures’ topic,” Sam demonstrated.

# Notify on script failure
./backup-script.sh || curl -d "Backup script failed!" ntfy.sh/script-failures

Alex was impressed by the simplicity and efficiency of this approach. “I can automate all of this?” Alex asked.

“Definitely,” Sam replied. “You can integrate it with cron jobs, monitoring tools, and more. It’s a great way to keep track of important events without getting bogged down by noise.”

With the basics in place, Alex began applying ntfy.sh to various real-world scenarios, streamlining the notification process and improving overall efficiency.

Monitoring Script Failures

Alex set up automated alerts for critical scripts that ran daily, ensuring that any failures were immediately reported. This allowed Alex to address issues quickly, minimizing downtime and improving system reliability.

# Notify on critical script failure
./critical-task.sh || curl -d "Critical task script failed!" ntfy.sh/script-failures

Tracking Long-Running Commands

Whenever Alex initiated a long-running command, such as a server backup or data migration, notifications were sent upon completion. This enabled Alex to focus on other tasks without constantly checking on progress.

# Notify on long-running command completion
long-command && curl -d "Long command completed successfully." ntfy.sh/command-completions

Server Health Alerts

To monitor server health, Alex integrated ntfy.sh with existing monitoring tools, ensuring that any critical issues were immediately flagged.

# Send server health alert
curl -d "Server CPU usage is critically high!" ntfy.sh/server-health

As with any new tool, there were challenges to overcome. Alex encountered a few hurdles, but with Sam’s guidance, these were quickly resolved.

Challenge: Managing Multiple Notifications

Initially, Alex found it challenging to manage multiple notifications and ensure that only critical alerts were prioritized. Sam suggested using filters and priorities to focus on the most important messages.

# Subscribe with filters for high-priority alerts
ntfy subscribe script-failures --priority=high

Challenge: Scheduling Notifications

Alex wanted to schedule notifications for regular maintenance tasks and reminders. Sam introduced Alex to using cron for scheduling automated alerts.S

# Schedule notification for regular maintenance
echo "Time for weekly server maintenance." | at 8:00 AM next Saturday ntfy.sh/server-health

Sam gave some more examples to alex,

Monitoring disk space

As a system administrator, you can use ntfy.sh to receive alerts when disk space usage reaches a critical level. This helps prevent issues related to insufficient disk space.

# Check disk space and notify if usage is over 80%
disk_usage=$(df / | grep / | awk '{ print $5 }' | sed 's/%//g')
if [ $disk_usage -gt 80 ]; then
  curl -d "Warning: Disk space usage is at ${disk_usage}%." ntfy.sh/disk-space
fi

Alerting on Website Downtime

You can use ntfy.sh to monitor the status of a website and receive notifications if it goes down.

# Check website status and notify if it's down
website="https://example.com"
status_code=$(curl -o /dev/null -s -w "%{http_code}\n" $website)

if [ $status_code -ne 200 ]; then
  curl -d "Alert: $website is down! Status code: $status_code." ntfy.sh/website-monitor
fi

Reminding for Daily Tasks

You can set up ntfy.sh to send you daily reminders for important tasks, ensuring that you stay on top of your schedule.

# Schedule daily reminders
echo "Time to review your daily tasks!" | at 9:00 AM ntfy.sh/daily-reminders
echo "Stand-up meeting at 10:00 AM." | at 9:50 AM ntfy.sh/daily-reminders

Alerting on High System Load

Monitor system load and receive notifications when it exceeds a certain threshold, allowing you to take action before it impacts performance.

# Check system load and notify if it's high
load=$(uptime | awk '{ print $10 }' | sed 's/,//')
threshold=2.0

if (( $(echo "$load > $threshold" | bc -l) )); then
  curl -d "Warning: System load is high: $load" ntfy.sh/system-load
fi

Notify on Backup Completion

Receive a notification when a backup process completes, allowing you to verify its success.

# Notify on backup completion
backup_command="/path/to/backup_script.sh"
$backup_command && curl -d "Backup completed successfully." ntfy.sh/backup-status || curl -d "Backup failed!" ntfy.sh/backup-status

Notifying on Container Events with Docker

Integrate ntfy.sh with Docker to send alerts for specific container events, such as when a container stops unexpectedly.

# Notify on Docker container stop event
container_name="my_app"
container_status=$(docker inspect -f '{{.State.Status}}' $container_name)

if [ "$container_status" != "running" ]; then
  curl -d "Alert: Docker container $container_name has stopped." ntfy.sh/docker-alerts
fi

Integrating with CI/CD Pipelines

Use ntfy.sh to notify you about the status of CI/CD pipeline stages, ensuring you stay informed about build successes or failures.

# Example GitLab CI/CD YAML snippet
stages:
  - build

build_job:
  stage: build
  script:
    - make build
  after_script:
    - if [ "$CI_JOB_STATUS" == "success" ]; then
        curl -d "Build succeeded for commit $CI_COMMIT_SHORT_SHA." ntfy.sh/ci-cd-status;
      else
        curl -d "Build failed for commit $CI_COMMIT_SHORT_SHA." ntfy.sh/ci-cd-status;
      fi

Notification on ssh login to server

Lets try with docker,

FROM ubuntu:16.04
RUN apt-get update && apt-get install -y openssh-server
RUN mkdir /var/run/sshd
# Set root password for SSH access (change 'your_password' to your desired password)
RUN echo 'root:password' | chpasswd
RUN sed -i 's/PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config
RUN sed 's@session\s*required\s*pam_loginuid.so@session optional pam_loginuid.so@g' -i /etc/pam.d/sshd
COPY ntfy-ssh.sh /usr/bin/ntfy-ssh.sh
RUN chmod +x /usr/bin/ntfy-ssh.sh
RUN echo "session optional pam_exec.so /usr/bin/ntfy-ssh.sh" >> /etc/pam.d/sshd
RUN apt-get -y update; apt-get -y install curl
EXPOSE 22
CMD ["/usr/sbin/sshd", "-D"]

script to send notification,

#!/bin/bash
if [ "${PAM_TYPE}" = "open_session" ]; then
  echo "here"
  curl \
    -H prio:high \
    -H tags:warning \
    -d "SSH login: ${PAM_USER} from ${PAM_RHOST}" \
    ntfy.sh/syed-alerts
fi

With ntfy.sh as an integral part of daily operations, Alex found a renewed sense of balance and control. The once overwhelming chaos of notifications was now a manageable stream of valuable information.

As Alex reflected on the journey, it was clear that ntfy.sh had transformed not just the way notifications were managed, but also the overall approach to system administration.

In a world full of noise, ntfy.sh had provided a clear and effective way to stay informed without distractions. For Alex, it was more than just a tool—it was a new way of managing systems efficiently.