font-size vs line-height vs actual height

First, I recommend reading my answer in Inline elements and line-height. To summarize, there are various heights related to inline boxes:

• Height of the inline box, given by line-height
• Height of the line box, which in simple cases is also given by line-height, but not here.
• Height of the content area of the inline box, which is implementation dependent. This is the area painted by the red background.

The other height in your case is the height of the parent div. This is determined by §10.6.3. In this case, since the box establishes an inline formatting context with one line,

The element’s height is the distance from its top content edge to […] the bottom edge of the last line box

So the height of the parent block is given by the height of the line box.

What happens here is that the height of the line box is not the line-height of your inline box. And that’s because the line-height of the parent block is also taken into account:

On a block container element whose content is composed of inline-level elements, ‘line-height’ specifies the minimal height of line boxes within the element.

The minimum height consists of a minimum height
above the baseline and a minimum depth below it, exactly as if each
line box starts with a zero-width inline box with the element’s font
and line height properties.

We call that imaginary box a “strut.”

If you set parent’s line-height to 0, and child’s vertical-align to e.g top, then the height of the parent will exactly be the line-height of the child.

.outer {
  margin-top: 50px;
  background-color: green;
  width: 150px;
  font-family: "Times New Roman";
  line-height: 0;
}
.letter-span-1 {
  background-color: red;
  line-height: 40px;
  font-size: 40px;
  vertical-align: top;
}
.letter-span-2 {
  background-color: red;
  line-height: 15px;
  font-size: 40px;
  vertical-align: top;
}
.letter-span-3 {
  background-color: red;
  line-height: 65px;
  font-size: 40px;
  vertical-align: top;
}

<span class="letter-span-1">XxÀg</span>
<div class="outer">
  <span class="letter-span-1">XxÀg</span>
</div>
The parent block is 40px tall.
<div class="outer">
  <span class="letter-span-2">XxAg</span>
</div>
The parent block is 15px tall.
<div class="outer">
  <span class="letter-span-3">XxÀg</span>
</div>
The parent block is 65px tall.

If you don’t set a line-height to the parent, it will be normal.

Tells user agents to set the used value to a “reasonable” value based
on the font of the element[…]. We recommend a used value for
‘normal’ between 1.0 to 1.2.

That means that there will be a minimum height for the parent which will be its font-size (which you don’t specify, and the default is implementation-dependent) multiplied by that factor (implementation-dependent).

You should also consider the vertical-align of the span. By default it’s baseline, and that may create a gap below. The image in web-tiki’s answer is especially useful:

That’s because vertical-align determines how the span will be aligned with the strut, and with baseline the alignment can depend on font-size and end up increasing the height of the line box. The line box height is the distance between the top of the uppermost and the bottom of the lowermost boxes in the line.

If you don’t want the height of the parent div to be increased by that, you need some other vertical-align, like top, middle, or bottom. Then the font-size of the span shouldn’t affect the height of the div.

To summarize, the height of the div depends on

• Its line-height
  • … which by default depends on div’s font-size
• Span’s line-height
  • … which by default depends on span’s font-size
• Possibly span’s font-size, depending on span’s vertical-align
• And obviously height, min-height, max-height, etc.