Use ffprobe

Example command

$ ffprobe -v error -select_streams v:0 -show_entries stream=codec_name -of default=nokey=1:noprint_wrappers=1 input.mp4

Result

h264

Option descriptions

• -v error Omit extra information except for fatal errors.

• -select_streams v:0 Select only the first video stream. Otherwise the codec_name for all other streams in the file, such as audio, will be shown as well.

• -show_entries stream=codec_name Only output the codec_name instead of all stream info.

• -of default=nokey=1:noprint_wrappers=1 Select the default output format style and omit the key and wrapper info. Otherwise, without these options, it will output:

    [STREAM]
    codec_name=h264
    [/STREAM]
  

Also see

• ffprobe documentation
• FFmpeg Wiki: FFprobe Tips

Bash script example

Only re-encode if video is not H.264:

#!/bin/bash

mkdir h264vids

for f in *.mkv
do
  audioformat=$(ffprobe -loglevel error -select_streams v:0 -show_entries stream=codec_name -of default=nw=1:nk=1 "$f")
  if [ "$audioformat" = "h264" ]; then
    ffmpeg -i "$f" -c:v copy -c:a aac -movflags +faststart h264vids/"${f%.*}.mp4"
  else
    ffmpeg -i "$f" -c:v libx264 -c:a aac -movflags +faststart h264vids/"${f%.*}.mp4"
  fi
done

This is a simple script and will ignore additional video streams if you input has more than one.