Fast way to tokenize a string in c

I don’t know about strtok_r, but strtok is probably the fastest way to tokenize a string. Perhaps you were doing it wrong? Maybe that is why it appeared slow for you.

Here is how you tokenize a string in C…

#include <string.h>
#include <stdio.h>

int
main (void)
{
    char  string[] = "root.ahmed.andre";
    char *token = strtok (string, ".");

    while (token) {
        // Do what you want with the token here...
        puts (token);

        // Get the next token
        token = strtok (NULL, ".");
    }
}

And just for the sake of argument, the code below tokenizes your string 1,000,000 times and displays how long it took to do so. For me, it took 90 ms. That’s blazing fast.

#include <string.h>
#include <stdio.h>
#include <sys/time.h>

int
main (void)
{
    struct timeval tv;
    long int       start;
    long int       end;
    int            i;

    // Get start time in milliseconds
    gettimeofday (&tv, NULL);
    start = (tv.tv_sec * 1000) + (tv.tv_usec / 1000);

    for (i = 0; i < 1000000; i++) {
        char  string[] = "root.ahmed.andre";
        char *token = strtok (string, ".");

        while (token) {
            token = strtok (NULL, ".");
        }
    }

    // Get end time in milliseconds
    gettimeofday (&tv, NULL);
    end = (tv.tv_sec * 1000) + (tv.tv_usec / 1000);

    // Print execution time in milliseconds
    printf ("\nDone in %ld ms!\n\n", end - start);

    return 0;
}

Leave a Comment