How to display vertical text in table headers with auto height / without text overflow?

If you use a pseudo element and vertical-padding, you may basicly draw a square box or <td> :
http://jsfiddle.net/qjzwG/319/

.verticalTableHeader {
    text-align:center;
    white-space:nowrap;
    transform-origin:50% 50%;
    transform: rotate(90deg);

}
.verticalTableHeader:before {
    content:'';
    padding-top:110%;/* takes width as reference, + 10% for faking some extra padding */
    display:inline-block;
    vertical-align:middle;
}

If you want to keep <td> ith a small width, table-layout:fixed + width might help.
http://jsfiddle.net/qjzwG/320/

.verticalTableHeader {
    text-align:center;
    white-space:nowrap;
    transform: rotate(90deg);

}
.verticalTableHeader p {
    margin:0 -100% ;
    display:inline-block;
}
.verticalTableHeader p:before{
    content:'';
    width:0;
    padding-top:110%;/* takes width as reference, + 10% for faking some extra padding */
    display:inline-block;
    vertical-align:middle;
}
table {
    text-align:center;
    table-layout : fixed;
    width:150px
}

If you want table to still be able to grow from it’s content but not from width of <th> , using a wrapper with a hudge negative margin opposite to dir/direction of document might do : apparently, the closest to your needs, http://jsfiddle.net/qjzwG/320/

<table border="1">
    <tr>
      <th class="verticalTableHeader"><p>First</p></th>
      <th class="verticalTableHeader"><p>Second-long-header</p></th>
      <th class="verticalTableHeader"><p>Third</p></th>
    </tr>

.verticalTableHeader {
    text-align:center;
    white-space:nowrap;
    transform: rotate(90deg);
}
.verticalTableHeader p {
    margin:0 -999px;/* virtually reduce space needed on width to very little */
    display:inline-block;
}
.verticalTableHeader p:before {
    content:'';
    width:0;
    padding-top:110%;
    /* takes width as reference, + 10% for faking some extra padding */
    display:inline-block;
    vertical-align:middle;
}
table {
    text-align:center;
}

HTML from demo and base :

<table border="1">
    <tr>
      <th class="verticalTableHeader">First</th>
      <th class="verticalTableHeader">Second</th>
      <th class="verticalTableHeader">Third</th>
    </tr>
    <tr>
      <td>foo</td>
      <td>foo</td>
      <td>foo</td>
    </tr>
    <tr>
      <td>foo</td>
      <td>foo</td>
      <td>foo</td>
    </tr>
    <tr>
      <td>foo</td>
      <td>foo</td>
      <td>foo</td>
    </tr>
</table>

For older IE , you need to use writing-mode (CSS) :http://msdn.microsoft.com/en-us/library/ie/ms531187%28v=vs.85%29.aspx