How to test if a remote file exists from a shell script using curl.

curl_file_exists.sh

url='http://example.com/index.html'
# "HEAD" request
#  (most servers support "HEAD", but some don't)
if curl -sfILo/dev/null "$url"; then
  echo "URL exists"
else
  echo "URL does not exist"
fi

curl_file_exists_get.sh

url='http://example.com/index.html'
# "GET" with empty range-request
#  (lightweight if the server supports range-requests, ends
#   up fetching the whole document if the server doesn't...)
if curl -sfLo/dev/null -r0-0 "$url"; then
  echo "URL exists"
else
  echo "URL does not exist"
fi

Curl needs the L option in case the website has redirects, because in the above, a site that returns a 301 will be result in telling you that the URL exists, even if the remote file you're checking for doesn't. Also, the () subshell is unneeded.

url='http://example.com/index.html'
if curl -o/dev/null -sfIL "$url" 2>&1; then
  echo "URL exists"
else
  echo "URL does not exist"
fi

And in case the site doesn't allow checking the header with the I option, you can do:
curl -o/dev/null -sfL -r 0-0 "$url"

True, the -L switch produces what is probably the expected behavior for most people. And the sub-shell was indeed entirely unnecessary. As for the HEAD vs GET difference, in most cases, HEAD will work correctly and is lighter-weight on the network and the server, but it does bear mentioning that it may not work in all cases. The GET-with-range trick is an interesting work-around to try to reduce network load, but it also may not work with all servers.