Yes – because it does begin with the empty string. Indeed, the empty string logically occurs between every pair of characters.
Put it this way: what definition of “starts with” could you give that would preclude this? Here’s a simple definition of “starts with” that doesn’t:
“x starts with y if the first y.Length
characters of x match those of y.”
An alternative (equivalent) definition:
“x starts with y if x.Substring(0, y.Length).Equals(y)
“