Why does this bison code produce unexpected output?

by

Short answer:

yylval.strval = yytext;

You can’t use yytext like that. The string it points to is private to the lexer and will change as soon as the flex action finishes. You need to do something like:

yylval.strval = strdup(yytext);

and then you need to make sure you free the memory afterwards.

Longer answer:

yytext is actually a pointer into the buffer containing the input. In order to make yytext work as though it were a NUL-terminated string, the flex framework overwrites the character following the token with a NUL before it does the action, and then replaces the original character when the action terminates. So strdup will work fine inside the action, but outside the action (in your bison code), you now have a pointer to the part of the buffer starting with the token. And it gets worse later, since flex will read the next part of the source into the same buffer, and now your pointer is to random garbage. There are several possible scenarios, depending on flex options, but none of them are pretty.

So the golden rule: yytext is only valid until the end of the action. If you want to keep it, copy it, and then make sure you free the storage for the copy when you no longer need it.

In almost all the lexers I’ve written, the ID token actually finds the identifier in a symbol table (or puts it there) and returns a pointer into the symbol table, which simplifies memory management. But you still have essentially the same memory management issue with, for example, character string literals.

More Related Contents:

  1. String input to flex lexer
  2. Lex program input doesn't terminate [closed]
  3. Regular expression for a string literal in flex/lex
  4. What memcpy_s return error 404 mean [closed]
  5. Algorithm to find the smallest number in an array in C [closed]
  6. Explanation of output of this C code
  7. Splitting a string into words
  8. Can x86’s MOV really be “free”? Why can’t I reproduce this at all?
  9. Can I share a file descriptor to another process on linux or are they local to the process?
  10. Return a `struct` from a function in C
  11. Array-size macro that rejects pointers
  12. Assign one struct to another in C
  13. Function to dynamically allocate matrix
  14. What happens to a float variable when %d is used in a printf?
  15. What is a “packed” structure in C?
  16. What is the function of “(void) (&_min1 == &_min2)” in the min macro in kernel.h?
  17. How to prevent system hang before watchdog timer task kicks in
  18. Why does integer division by zero result in a floating point exception?
  19. Segmentation fault reversing a string literal [duplicate]
  20. What does -D_XOPEN_SOURCE do/mean?
  21. Passing an ellipsis to another variadic function [duplicate]
  22. Why do you have to link the math library in C?
  23. Does C support raw string literals?
  24. Open directory using C
  25. AES (aes-cbc-128, aes-cbc-192, aes-cbc-256) encryption/decryption with openssl C
  26. Use a dope vector to access arbitrary axial slices of a multidimensional array?
  27. Exceeding array bound in C — Why does this NOT crash?
  28. What’s the difference between “int” and “int_fast16_t”?
  29. How to read terminal’s input buffer immediately after keypress
  30. Is sprintf(buffer, “%s […]”, buffer, […]) safe?

Leave a Comment