How does strtok() split the string into tokens in C?

by

the strtok runtime function works like this

the first time you call strtok you provide a string that you want to tokenize

char s[] = "this is a string";

in the above string space seems to be a good delimiter between words so lets use that:

char* p = strtok(s, " ");

what happens now is that ‘s’ is searched until the space character is found, the first token is returned (‘this’) and p points to that token (string)

in order to get next token and to continue with the same string NULL is passed as first
argument since strtok maintains a static pointer to your previous passed string:

p = strtok(NULL," ");

p now points to ‘is’

and so on until no more spaces can be found, then the last string is returned as the last token ‘string’.

more conveniently you could write it like this instead to print out all tokens:

for (char *p = strtok(s," "); p != NULL; p = strtok(NULL, " "))
{
  puts(p);
}

EDIT:

If you want to store the returned values from strtok you need to copy the token to another buffer e.g. strdup(p); since the original string (pointed to by the static pointer inside strtok) is modified between iterations in order to return the token.

More Related Contents:

  1. Split string with delimiters in C
  2. Using strtok() in nested loops in C?
  3. Nested strtok function problem in C [duplicate]
  4. Split string into tokens and save them in an array
  5. How to split a string to 2 strings in C
  6. C’s strtok() and read only string literals
  7. How to split a string into tokens in C?
  8. String termination C/C++ char = 0
  9. Removing digits and all punctuation except apostrophe from a string in C
  10. ‘ ‘, ‘\n’ , scanf() and output screen
  11. C prog. Pointers and strings [closed]
  12. How do I concatenate const/literal strings in C?
  13. How to convert a string to integer in C?
  14. How do I trim leading/trailing whitespace in a standard way?
  15. Easiest way to split a string on newlines in .NET?
  16. Using the equality operator == to compare two strings for equality in C [duplicate]
  17. Why is “a” != “a” in C?
  18. Reading string from input with space character? [duplicate]
  19. How to convert integer to string in C? [duplicate]
  20. How can I compare strings in C using a `switch` statement?
  21. C – Difference between “char var[]” and “char *var”?
  22. Why does strchr take an int for the char to be found?
  23. Append Char To String in C?
  24. How to extract numbers from string in c?
  25. How to check if a string is a number?
  26. How to format strings using printf() to get equal length in the output
  27. Literal string initializer for a character array
  28. How does strcmp() work?
  29. Why do strings in C need to be null terminated?
  30. split a string on newlines in .NET

Leave a Comment