I never use live; I consider the benefits of using delegate to be so substantial as to be overwhelming.
The one benefit of live is that its syntax is very close to that of bind:
$('a.myClass').live('click', function() { ... });
delegate, however, uses a slightly more verbose syntax:
$('#containerElement').delegate('a.myClass', 'click', function() { ... });
This, however, seems to me to be much more explicit about what is actually happening. You don’t realise from the live example that the events are actually being captured on document; with delegate, it is clear that the event capturing happens on #containerElement. You can do the same thing with live, but the syntax becomes increasingly horrid.
Specifying a context for your events to be captured also improves performance. With the live example, every single click on the entire document has to be compared with the selector a.myClass to see if it matches. With delegate, that is only the elements within #containerElement. This will obviously improve performance.
Finally, live requires that your browser looks for a.myClass whether or not it currently exists. delegate only looks for the elements when the events are triggered, giving a further performance advantage.
NB delegate uses live behind the scenes, so you can do anything with live that you can do with delegate. My answer deals with them as they are commonly used.
Note also that neither live nor delegate is the best way to do event delegation in modern jQuery. The new syntax (as of jQuery 1.7) is with the on function. The syntax is as follows:
$('#containerElement').on('click', 'a.myClass', function() { ... });