1
char *p = I2CinBuffer; // array contains "string" "," "string"  

but

char *p = I2CinBuffer; // The data to be parsed e.g. 6,100
char *str;       // Temp store for each data chunk
int count = 0;  

while ((str = strtok_r(p, ",", &p)) != NULL)    // seperate at each "," delimiter
{ 
    inParse[count] = str;      // Add chunk to array  
    count++;      
}

does not enter while loop.

Many thanks but the hard coding of the values "6,100" give correct results as it stands. However, I am now aware that the Arduino code is working in C strings and I2CInBuffer is a C string on which the "strtok" or "strok_r" functions will NOT work. My question now is how to parse the C string I2CinBuffer to obtain 2 values separated by a comma i.e. "6, 100".

1

You're not using strtok_r properly. You have to treat the first call to strtok_r differently to all subsequent calls, and I don't know what you're trying to do with the &p parameter - that could never work since you're breaking the source pointer with that.

According to the manual page:

The strtok_r() function is a reentrant version strtok(). The saveptr argument is a pointer to a char * variable that is used internally by strtok_r() in order to maintain context between successive calls that parse the same string.

On the first call to strtok_r(), str should point to the string to be parsed, and the value of saveptr is ignored. In subsequent calls, str should be NULL, and saveptr should be unchanged since the previous call.

Unless you're doing concurrent tokenising of multiple strings there is no reason to use the reentrant version of strtok. Instead you should just go:

char *p = I2CinBuffer; // The data to be parsed e.g. 6,100
int count = 0;  

char *str = strtok(p, ",");
while (str != NULL)    // seperate at each "," delimiter
{ 
    inParse[count] = str;      // Add chunk to array  
    count++;      
    str = strtok(NULL, ",");
}

Note how the first call, outside the while loop, passes the string to parse. All subsequent calls (inside the while loop) pass NULL as the string to parse so it continues from the last point in the string rather than starting the string afresh every time from the start (with a broken string since strtok() is a destructive function).

A sequence of calls to strtok() that operate on the same string maintains a pointer that determines the point from which to start searching for the next token. The first call to strtok() sets this pointer to point to the first byte of the string. The start of the next token is determined by scanning forward for the next nondelimiter byte in str. If such a byte is found, it is taken as the start of the next token. If no such byte is found, then there are no more tokens, and strtok() returns NULL. (A string that is empty or that contains only delimiters will thus cause strtok() to return NULL on the first call.)

It is considered destructive because:

The end of each token is found by scanning forward until either the next delimiter byte is found or until the terminating null byte ('\0') is encountered. If a delimiter byte is found, it is overwritten with a null byte to terminate the current token, and strtok() saves a pointer to the following byte; that pointer will be used as the starting point when searching for the next token. In this case, strtok() returns a pointer to the start of the found token.

So in other words, your string that started out as:

foo,bar,baz\0

ends up, after tokenizing, as:

foo\0bar\0baz\0

So after the first tokenizing, if you try and pass the string again as the first parameter, instead of "foo,bar,baz", it actually only sees "foo" since the string now ends after foo.

If you wanted to use the reentrant version you'd need to create a separate pointer variable for the save pointer and pass that as the third argument:

char *p = I2CinBuffer; // The data to be parsed e.g. 6,100
int count = 0;  
char *saveptr;

char *str = strtok_r(p, ",", &saveptr);
while (str != NULL)    // seperate at each "," delimiter
{ 
    inParse[count] = str;      // Add chunk to array  
    count++;      
    str = strtok(NULL, ",", &saveptr);
}

Note that you still need to use the same "pass the string on the first call and pass NULL on all other calls" methodology since it is still destructive.

  • Many thanks but the hard coding of the values "6,100" give correct results as it stands. I am now aware that the Arduino code is working in C strings. I2CInBuffer is a C string. – impartit Jul 5 '15 at 19:55
  • @impartit The big question, then, is what is I2CinBuffer and how is it generated? – Majenko Jul 5 '15 at 20:19
0

My question now is how to parse the C string I2CinBuffer to obtain 2 values separated by a comma i.e. "6, 100".

Example code:

int parseit (char * p, int * results, const int maxSize)
{
  int count = 0;  
  char *str = strtok (p, ",");   // separate at each "," delimiter
  while (str != NULL && count < maxSize)   
    { 
    results [count++] = atoi (str);      // Add to array  
    str = strtok(NULL, ",");
    }  // end of while
  return count;
}  // end of parseit

void setup ()
{
  Serial.begin (115200);
  Serial.println ();
  const int MAX_INTEGERS = 5;
  int inParse [MAX_INTEGERS];
  char  I2CinBuffer [50];

  strcpy (I2CinBuffer, "6,100"); 
  int found = parseit (I2CinBuffer, inParse, MAX_INTEGERS);
  for (int i = 0; i < found; i++)
    {
    Serial.print (i);
    Serial.print (" = ");
    Serial.println (inParse [i]);
    }  // end of for

}  // end of setup

void loop ()
{
  // rest of code here
}  // end of loop

Results from running it:

0 = 6
1 = 100
  • Thanks Nick that's great also but I don't wish to hard code the values "6, 100". The two comma separated values will vary and be received into I2CinBuffer from an external python script by a 'write_List' function using I2C. – impartit Jul 6 '15 at 6:44
  • Doing a strcpy was purely an example for showing the technique. In practice the contents of that would be what you received from wherever. – Nick Gammon Jul 6 '15 at 6:46
  • I2CinBuffer already contains "6, 100" so how will "strcpy (I2CinBuffer, "6,100");" work. – impartit Jul 6 '15 at 16:16
  • If it already contains what you want, just omit that line. Be aware, as other answers have stated, that strtok destructively tokenizes. It replaces each delimiter with 0x00 so that afterwards, the variable has been altered. In other words, you can't tokenize it twice. This may or may not matter to you. As an example, if you wanted to display what you received, for debugging purposes, do that before calling strtok, not afterwards. – Nick Gammon Jul 6 '15 at 21:11
  • I am told by Adafruit support that "strtok won't work on a string object which is what I2CinBuffer is to the Arduino". The data in I2CinBuffer receives the comma separated values from a Python write_list function (Adafruit_I2C.py library file). If I hard card the values as "strcopy(I2CinBuffer, "6,100") the existing strok_r function works just fine. Do you have time to look at the original source code at "letsmakerobots.com/blog/unixguru/….? The author does not respond to my emails. – impartit Jul 8 '15 at 15:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.