Managing log files created by cron jobs

by

The best way to manage cron logs is to have a wrapper around each job. The wrapper could do these things, at the minimum:

  • initialize environment
  • redirect stdout and stderr to log
  • run the job
  • perform checks to see if job succeeded or not
  • send notifications if necessary
  • clean up logs

Here is a bare bones version of a cron wrapper:

#!/bin/bash

log_dir=/tmp/cron_logs/$(date +'%Y%m%d')
mkdir -p "$log_dir" || { echo "Can't create log directory '$log_dir'"; exit 1; }

#
# we write to the same log each time
# this can be enhanced as per needs: one log per execution, one log per job per execution etc.
#
log_file=$log_dir/cron.log

#
# hitherto, both stdout and stderr end up in the log file
#
exec 2>&1 1>>"$log_file"

#
# Run the environment setup that is shared across all jobs.
# This can set up things like PATH etc. 
#
# Note: it is not a good practice to source in .profile or .bashrc here
#
source /path/to/setup_env.sh

#
# run the job
#
echo "$(date): starting cron, command=[$*]"
"$@"
echo "$(date): cron ended, exit code is $?"

Your cron command line would look like:

/path/to/cron_wrapper command ...

Once this is in place, we can have another job called cron_log_cleaner which can remove older logs. It’s not a bad idea to call the log cleaner from the cron wrapper itself, at the end.

An example:

# run the cron job from command line
cron_wrapper 'echo step 1; sleep 5; echo step 2; sleep 10'

# inspect the log
cat /tmp/cron_logs/20170120/cron.log

The log would contain this after running the wrapped cron job:

Fri Jan 20 04:35:10 UTC 2017: starting cron, command=[echo step 1; sleep 5; echo step 2; sleep 10]
step 1
step 2
Fri Jan 20 04:35:25 UTC 2017: cron ended, exit code is 0

More Related Contents:

  1. grep lines that contain 1 character followed by another character
  2. When should I wrap quotes around a shell variable?
  3. How to convert DOS/Windows newline (CRLF) to Unix newline (LF) in a Bash script
  4. How do I write a bash script to restart a process if it dies?
  5. How do I write standard error to a file while using “tee” with a pipe?
  6. What’s the magic of “-” (a dash) in command-line parameters?
  7. What does $@ mean in a shell script?
  8. Better way to rename files based on multiple patterns
  9. Forcing bash to expand variables in a string loaded from a file
  10. How to instruct cron to execute a job every second week?
  11. What is the difference between “#!/usr/bin/env bash” and “#!/usr/bin/bash”?
  12. How can I programmatically create a new cron job?
  13. Extract file basename without path and extension in bash [duplicate]
  14. How do you find the original user through multiple sudo and su commands?
  15. How to concatenate multiple lines of output to one line?
  16. How can I remove the last character of a file in unix?
  17. How to run a shell script on a Unix console or Mac terminal?
  18. How to check if a file is empty in Bash?
  19. Should PATH contain directories or full paths to binaries?
  20. Execute a command without keeping it in history [closed]
  21. How do I set up cron to run a file just once at a specific time? [closed]
  22. When to wrap quotes around a shell variable?
  23. Restarting cron after changing crontab file? [closed]
  24. Running shell script in parallel
  25. Difference between shell and environment variables
  26. Why crontab uses OR when both day of month and day of week specified?
  27. What’s the point of eval/bash -c as opposed to just evaluating a variable?
  28. What are the rules for valid identifiers (e.g. functions, vars, etc) in Bash?
  29. Need to remove the count from the output when using “uniq -c” command
  30. Creating temporary files in bash

Leave a Comment