There are three ways to do what you want
Server Script
This includes something like php, asp, jsp…. But you said no to that
Server Side Includes
Your server is serving up the pages so why not take advantage of the built in server side includes? Each server has its own way to do this, take advantage of it.
Client Side Include
This solutions has you calling back to the server after page has already been loaded on the client.