I think that this could help you. It is from an answer I found on a Google Group about recursive elements in a tree.
The suggestion is from Brendan Owen: http://jsfiddle.net/brendanowen/uXbn6/8/
<script type="text/ng-template" id="field_renderer.html">
{{data.label}}
<ul>
<li ng-repeat="field in data.fields" ng-include="'field_renderer.html'"></li>
</ul>
</script>
<ul ng-controller="NestedFormCtrl">
<li ng-repeat="field in formData" ng-include="'field_renderer.html'"></li>
</ul>
The proposed solution is about using a template that uses the ng-include directive to call itself if the current element has children.
In your case, I would try to create a template with the ng-switch directive (one case per type of label like you did) and add the ng-include at the end if there are any child labels.