Just for the sake of argument.
Both are http request (xhr), and its at least partially untrue it uses more server resources (depends totally on technology, will explain later).
Short polling.
Lot of request that are processed as they come on server. Creates a lot of traffic (uses resources, but frees them as soon as response is send back):
00:00:00 C-> Is the cake ready?
00:00:01 S-> No, wait.
00:00:01 C-> Is the cake ready?
00:00:02 S-> No, wait.
00:00:02 C-> Is the cake ready?
00:00:03 S-> Yes. Have some lad.
00:00:03 C-> Is the other cake ready? ..
Long polling
One request goes to server and client is waiting for the response to come (its unresolved). In case of Server with php/apache would mean a spawned thread to handle, that reserve resources, till its done. So the traffic is smaller, but you eat up your resources fast (or rather you block resources). But if you use for example Node (or any other async approach – c++ qt for example), you can potentially minimize the resource usage a lot (store response object for http request and use it when the work is ready)
12:00 00:00:00 C-> Is the cake ready?
12:00 00:00:03 S-> Yes.Have some lad.
12:00 00:00:03 C-> Is the cake ready?
If you compare that to short polling, you will see that potentially in short poll you used more transfer, but during those 3s you actually take 1,5s of processing time (means something could execute in between your calls). In case for long poll the same resources were used all the time. Now usually php with all libs starts with 4MB memory – then you have a framework 4-20MB. Assume you have 1024MB RAM available (free). Say lets be pessimistic and assume that you will use 25 MB per one php instace. It means you can get only as much as 40 long polled connection scripts.
Its precisely the reason why you could serve potentially a lot more with Node, as node would not spawn its instances (unless you want to use workers etc), so with same memory you could probably get easily to 10k connections hanging. You would get a spike in the CPU as they will come, and when they will potentially be released, but when they are idle its like they are not there (you pay only for the memory structures you would keep in node/c++).
Websocket
Now if you want to send few things, whenever they are in or out of client, go for the websockets (ws protocol). First call is size of http request, but later you send just the messages, from the client to server (new questions) and server to client(answers or pushes – can even do broadcast for all connected clients). There are php websocekts libs but again, use some different technology – node or c++ preferably.
Some libs, like socket.io have a hierarchy of its own, so when websocket fails, it goes back to long or short polling.
When to use.
Short polling – well, never ^^.
Long polling – potentially when you are exchanging single call with server, and server is doing some work in background. Also when you won’t query server on the same page anymore. Also when you are not using php as layer to handle the long polled connection (node/c++ can be a simple middle layer). Note long polling can be really beneficial, but only when you make it so.
Websocket – you potentially will exchange more then one or two calls with server, or something might come from server you did not expected / asked, like notification of email or something. You should plan different “rooms”, depend on functionalities. Embrace the event based nature of javascript ;]