The following should solve your problem, using built-in template tags :
<ul>
<li>
{% for obj in objects %}
<a>{{ obj }}</a>
{# if the the forloop counter is divisible by 4, close the <li> tag and open a new one #}
{% if forloop.counter|divisibleby:4 %}
</li>
<li>
{% endif %}
{% endfor %}
</li>
</ul>