The <a>nchor element is simply an anchor to or from some content. Originally the HTML specification allowed for named anchors (<a name="foo">) and linked anchors (<a href="#foo">).
The named anchor format is less commonly used, as the fragment identifier is now used to specify an [id] attribute (although for backwards compatibility you can still specify [name] attributes). An <a> element without an [href] attribute is still valid.
As far as semantics and styling is concerned, the <a> element isn’t a link (:link) unless it has an [href] attribute. A side-effect of this is that an <a> element without [href] won’t be in the tabbing order by default.
The real question is whether the <a> element alone is an appropriate representation of a <button>. On a semantic level, there is a distinct difference between a link and a button.
A button is something that when clicked causes an action to occur.
A link is a button that causes a change in navigation in the current document. The navigation that occurs could be moving within the document in the case of fragment identifiers (#foo) or moving to a new document in the case of urls (/bar).
As links are a special type of button, they have often had their actions overridden to perform alternative functions. Continuing to use an anchor as a button is ok from a consistency standpoint, although it’s not quite accurate semantically.
If you’re concerned about the semantics and accessibility of using an <a> element (or <span>, or <div>) as a button, you should add the following attributes:
<a role="button" tabindex="0" ...>...</a>
The button role tells the user that the particular element is being treated as a button as an override for whatever semantics the underlying element may have had.
For <span> and <div> elements, you may want to add JavaScript key listeners for Space or Enter to trigger the click event. <a href> and <button> elements do this by default, but non-button elements do not. Sometimes it makes more sense to bind the click trigger to a different key. For example, a “help” button in a web app might be bound to F1.