How do I determine if a web page exists with shell scripting?

by

Under a *NIX, you can use curl to issue a simple HEAD request (HEAD only asks for the headers, not the page body):

curl --head http://myurl/

Then you can take only the first line, which contains the HTTP status code (200 OK, 404 Not Found, etc.):

curl -s --head http://myurl/ | head -n 1

And then check if you got a decent response (status code is 200 or 3**):

curl -s --head http://myurl/ | head -n 1 | grep "HTTP/1.[01] [23].."

This will output the first line if the status code is okay, or nothing if it isn’t. You can also pipe that to /dev/null to get no output, and use $? to determine if it worked or no:

curl -s --head http://myurl/ | head -n 1 | grep "HTTP/1.[01] [23].." > /dev/null
# on success (page exists), $? will be 0; on failure (page does not exist or
# is unreachable), $? will be 1

EDIT -s simply tells curl to not show a “progress bar”.

More Related Contents:

  1. How to select lines between two marker patterns which may occur multiple times with awk/sed
  2. How to use multiple arguments for awk with a shebang (i.e. #!)?
  3. How to read a file into a variable in shell?
  4. What are the uses of the exec command in shell scripts? [closed]
  5. How can I reverse the order of lines in a file?
  6. How to make grep only match if the entire line matches?
  7. How can I execute a command stored in a variable?
  8. How/When does Execute Shell mark a build as failure in Jenkins?
  9. Compare two files line by line and generate the difference in another file
  10. Alternative to `sed -i` on Solaris
  11. Floating-point arithmetic in UNIX shell script
  12. Run java jar file on a server as background process
  13. How do you tell if a string contains another string in POSIX sh?
  14. How does the #! shebang work?
  15. Concise and portable “join” on the Unix command-line
  16. Unix find command, what are the {} and \; for?
  17. How can I redirect the output of the “time” command?
  18. Compare a string using sh shell
  19. Shell one liner to prepend to a file
  20. How is % (percent sign) special in crontab?
  21. What does the line “#!/bin/sh” mean in a UNIX shell script?
  22. Trim leading and trailing spaces from a string in awk
  23. Can I call a function of a shell script from another shell script?
  24. Remove part of path on Unix
  25. What’s the difference between ${varname} and $varname in a shell scripts
  26. Read file into String and do a loop in Expect Script
  27. How to add text at the end of each line in unix
  28. Call a Unix Script from Excel Vba
  29. How to print 5 consecutive lines after a pattern in file using awk [duplicate]
  30. Unix shell script find out which directory the script file resides?

Leave a Comment