As already stated in other answers, this behavior is intended. There is some kind of workaround if you can compile (or at least link) the application yourself. Then you can pass -Wl,-rpath <yourDynamicLibraryPath>
to gcc or -rpath <yourDynamicLibraryPath>
to ld and you won’t have to specify LD_LIBRARY_PATH
at all on execution.