It is just syntax.
<% %>
is simply short for <script runat="server"> </script>
aka code render blocks.
<%# %>
are binding expressions (plus the above).
<%= %>
is the above + a Response.Write()
.
<%: %>
is the above + a Response.Write()
wrapped in Html.Encode
(new in .NET 4.0).
<%$ %>
is an ASP.NET expression, used to bind configuration or resource file data during runtime.