From this page (untested by me):
Somewhere in your program’s initialization (before you access any classes from a referenced assembly) do this:
AppDomain.CurrentDomain.AppendPrivatePath(@"bin\DLLs");
Edit: This article says AppendPrivatePath is considered obsolete, but also gives a workaround.
Edit 2: Looks like the easiest and most kosher way to do this is in the app.config file (see here):
<configuration>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<probing privatePath="bin\DLLs" />
</assemblyBinding>
</runtime>
</configuration>