This an effect of the maximal munch principle, which has the lexical analyzer take as many characters as possible to form a valid token. This is covered in draft C++ standard section 2.5
[lex.pptoken] which says:
Otherwise, the next preprocessing token is the longest sequence of
characters that could constitute a preprocessing token, even if that
would cause further lexical analysis to fail.
Any cases such as the one you cite above need a specific exception carved out such as this case for <::
, we can see an example in the following code:
template<typename T> class SomeClass;
class Class;
SomeClass<::Class>* cls;
which is covered in this question, the exception is listed in the bullet just above the maximal munch rule:
Otherwise, if the next three characters are <:: and the subsequent character is neither : nor >, the <
is treated as a preprocessor token by itself and not as the first character of the alternative token <:.
and of course the non-nested >
which you cite in your question.
Note we can see that >=
is a preprocessor token from section 2.13
[lex.operators] which says:
The lexical representation of C++ programs includes a number of preprocessing tokens which are used in
the syntax of the preprocessor or are converted into tokens for operators and punctuators:
and includes >=
in the list.
The >> fix
We can see from the proposal that fixed the >>
case: N1757: Right Angle Brackets which says (emphasis mine):
Ever since the introduction of angle brackets, C++ programmers have
been surprised by the fact that two consecutive right angle brackets
must be separated by whitespace:#include <vector> typedef std::vector<std::vector<int> > Table; // OK typedef std::vector<std::vector<bool>> Flags; // Error
The problem is an immediate consequence of the the “maximum munch”
principle and the fact that >> is a valid token (right shift) in C++.This issue is a minor, but persisting, annoying, and somewhat
embarrassing problem. If the cost is reasonable, it seems therefore
worthwhile to eliminate the surprise.The purpose of this document is to explain ways to allow >> to be
treated as two closing angle brackets, as well as to discuss the
resulting issues. A specific option is proposed along with wording
that would implement the proposal in the current working paper.
Also points out the >=
case:
It is also worth noting that the problem can also occur with the >>=
and >= tokens. For examplevoid func(List<B>= default_val1); void func(List<List<B>>= default_val2);
Both of these forms are currently ill-formed. It may be desirable to
also address this issue, but this paper does not propose to do so.
Note, this change broke backward compatibility with C++03.