Array values won't tokenize

Question

char *p = I2CinBuffer; // array contains "string" "," "string"  

but

char *p = I2CinBuffer; // The data to be parsed e.g. 6,100
char *str;       // Temp store for each data chunk
int count = 0;  

while ((str = strtok_r(p, ",", &p)) != NULL)    // seperate at each "," delimiter
{ 
    inParse[count] = str;      // Add chunk to array  
    count++;      
}

does not enter while loop.

Many thanks but the hard coding of the values "6,100" give correct results as it stands. However, I am now aware that the Arduino code is working in C strings and I2CInBuffer is a C string on which the "strtok" or "strok_r" functions will NOT work. My question now is how to parse the C string I2CinBuffer to obtain 2 values separated by a comma i.e. "6, 100".

Answer 1

You're not using strtok_r properly. You have to treat the first call to strtok_r differently to all subsequent calls, and I don't know what you're trying to do with the &p parameter - that could never work since you're breaking the source pointer with that.

According to the manual page:

The strtok_r() function is a reentrant version strtok(). The saveptr argument is a pointer to a char * variable that is used internally by strtok_r() in order to maintain context between successive calls that parse the same string.

On the first call to strtok_r(), str should point to the string to be parsed, and the value of saveptr is ignored. In subsequent calls, str should be NULL, and saveptr should be unchanged since the previous call.

Unless you're doing concurrent tokenising of multiple strings there is no reason to use the reentrant version of strtok. Instead you should just go:

char *p = I2CinBuffer; // The data to be parsed e.g. 6,100
int count = 0;  

char *str = strtok(p, ",");
while (str != NULL)    // seperate at each "," delimiter
{ 
    inParse[count] = str;      // Add chunk to array  
    count++;      
    str = strtok(NULL, ",");
}

Note how the first call, outside the while loop, passes the string to parse. All subsequent calls (inside the while loop) pass NULL as the string to parse so it continues from the last point in the string rather than starting the string afresh every time from the start (with a broken string since strtok() is a destructive function).

A sequence of calls to strtok() that operate on the same string maintains a pointer that determines the point from which to start searching for the next token. The first call to strtok() sets this pointer to point to the first byte of the string. The start of the next token is determined by scanning forward for the next nondelimiter byte in str. If such a byte is found, it is taken as the start of the next token. If no such byte is found, then there are no more tokens, and strtok() returns NULL. (A string that is empty or that contains only delimiters will thus cause strtok() to return NULL on the first call.)

It is considered destructive because:

The end of each token is found by scanning forward until either the next delimiter byte is found or until the terminating null byte ('\0') is encountered. If a delimiter byte is found, it is overwritten with a null byte to terminate the current token, and strtok() saves a pointer to the following byte; that pointer will be used as the starting point when searching for the next token. In this case, strtok() returns a pointer to the start of the found token.

So in other words, your string that started out as:

foo,bar,baz\0

ends up, after tokenizing, as:

foo\0bar\0baz\0

So after the first tokenizing, if you try and pass the string again as the first parameter, instead of "foo,bar,baz", it actually only sees "foo" since the string now ends after foo.

If you wanted to use the reentrant version you'd need to create a separate pointer variable for the save pointer and pass that as the third argument:

char *p = I2CinBuffer; // The data to be parsed e.g. 6,100
int count = 0;  
char *saveptr;

char *str = strtok_r(p, ",", &saveptr);
while (str != NULL)    // seperate at each "," delimiter
{ 
    inParse[count] = str;      // Add chunk to array  
    count++;      
    str = strtok(NULL, ",", &saveptr);
}

Note that you still need to use the same "pass the string on the first call and pass NULL on all other calls" methodology since it is still destructive.

Answer 2

My question now is how to parse the C string I2CinBuffer to obtain 2 values separated by a comma i.e. "6, 100".

Example code:

int parseit (char * p, int * results, const int maxSize)
{
  int count = 0;  
  char *str = strtok (p, ",");   // separate at each "," delimiter
  while (str != NULL && count < maxSize)   
    { 
    results [count++] = atoi (str);      // Add to array  
    str = strtok(NULL, ",");
    }  // end of while
  return count;
}  // end of parseit

void setup ()
{
  Serial.begin (115200);
  Serial.println ();
  const int MAX_INTEGERS = 5;
  int inParse [MAX_INTEGERS];
  char  I2CinBuffer [50];

  strcpy (I2CinBuffer, "6,100"); 
  int found = parseit (I2CinBuffer, inParse, MAX_INTEGERS);
  for (int i = 0; i < found; i++)
    {
    Serial.print (i);
    Serial.print (" = ");
    Serial.println (inParse [i]);
    }  // end of for

}  // end of setup

void loop ()
{
  // rest of code here
}  // end of loop

Results from running it:

0 = 6
1 = 100