The short answer is that beyond the obvious things, there is not much you can do.
The obvious things that you might want to consider (roughly in order of increasing difficulty and decreasing plausibility) include:
- Static link so there is no DLL to attack.
- Strip all symbols.
- Use a .DEF file and an import library to have only anonymous exports known only by their export ids.
- Keep the DLL in a resource and expose it in the file system (under a suitably obscure name, perhaps even generated at run time) only when running.
- Hide all real functions behind a factory method that exchanges a secret (better, proof of knowledge of a secret) for a table of function pointers to the real methods.
- Use anti-debugging techniques borrowed from the malware world to prevent reverse engineering. (Note that this will likely get you false positives from AV tools.)
Regardless, a sufficiently determined user can still figure out ways to use it. A decent disassembler will quickly provide all the information needed.
Note that if your DLL is really a COM object, or worse yet a CLR Assembly, then there is a huge amount of runtime type information that you can’t strip off without breaking its intended use.
EDIT: Since you’ve retagged to imply that C# and .NET are the environment rather than a pure Win32 DLL written in C, then I really should revise the above to “You Can’t, But…”
There has been a market for obfuscation tools for a long time to deal with environments where delivery of compilable source is mandatory, but you don’t want to deliver useful source. There are C# products that play in that market, and it looks like at least one has chimed in.
Because loading an Assembly requires so much effort from the framework, it is likely that there are permission bits that exert some control for honest providers and consumers of Assemblies. I have not seen any discussion of the real security provided by these methods and simply don’t know how effective they are against a determined attack.
A lot is going to depend on your use case. If you merely want to prevent casual use, you can probably find a solution that works for you. If you want to protect valuable trade secrets from reverse engineering and reuse, you may not be so happy.