Serial printing from flash memory (F() macro, PROGMEM, sprintf_P, SPTR)

Question

I have been testing various techniques on an Arduino Uno for serial printing from flash memory instead of RAM. At the same time, I have been using the freeMemory() function from Adafruit.

I created a program that prints the word "test", with various methods:

Simple print:

Serial.print("test");
Serial.print(freeMemory());

This call reports free RAM = 2267

Using the F() Macro:

Serial.print(F("test"));
Serial.print(freeMemory());

This call reports free RAM = 2299

Using sprintf_P()

char buf[100];
sprintf_P(buf,PSTR("test"));
Serial.print(buf);
Serial.print(freeMemory());

This call reports free RAM = 2267

Using strcpy_P()

 char buf[100];
 strcpy_P(buf, (PGM_P)F("test"));
 Serial.print(buf);
Serial.print(freeMemory());

This call reports free RAM = 2267

Using strcpy_P and const char[] PROGMEM

char buf[100];
const char buff[] PROGMEM = "test";
strcpy_P(buf, buff);
Serial.print(buf);
Serial.print(freeMemory());

This call reports free RAM = 2267

The basic point of this question is to gather all the possible flash memory printing techniques. Also it would be really useful if you could explain in detail what exactly each of the keywords F(), (PGM_P)F, PSTR, const PROGMEM does.

IMPORTANT EDIT, 1:

Depending on where and how I use the call Serial.print(freeMemory()); the results change. For example, at the last example (PROGMEM, strcpy_P) when I included another Serial.print(freeMemory()); before the call of the printing functions, then both freeMemory() calls report 2299!

Another interesting problem is that the freeMemory() reports 2299 as freeRam, while the SRAM of my Arduino Uno is only 2 kbytes.

EDIT 2:

Following the suggestion in comments, I changed "test" to "testtesttesttesttesttesttesttesttesttesttesttesttesttesttesttest". Now all different methods report a free ram of 2299, including the simple Serial.print().

freeMemory():

#ifdef __arm__
// should use uinstd.h to define sbrk but Due causes a conflict
extern "C" char* sbrk(int incr);
#else  // __ARM__
extern char *__brkval;
#endif  // __arm__
int freeMemory() {
  char top;
#ifdef __arm__
  return &top - reinterpret_cast<char*>(sbrk(0));
#elif defined(CORE_TEENSY) || (ARDUINO > 103 && ARDUINO != 151)
  return &top - __brkval;
#else  // __arm__
  return __brkval ? &top - __brkval : &top - __malloc_heap_start;
#endif  // __arm__
}

Answer 1

First, the freeMemory() function that you used from Adafruit was originated from a GitHub repository.

I don't know about the __arm__ implementation, but for the AVR one, it is incomplete for handling the corner case when the program does not use the malloc() function. If your program never uses malloc(), the formula used in freeMemory() will produce the wrong result. A simple corrected version for AVR that I used is here:

extern unsigned int __heap_start;
extern char *__brkval;

int freeMemory() {
  char top_of_stack;

  if (__brkval == 0) {
    Serial.println(((int)&top_of_stack - (int)&__heap_start));
  } else {
    Serial.println((int)&top_of_stack - (int)__brkval);
  }
}

explain in detail what exactly does each of the keywords F(), (PGM_P)F, PSTR, const PROGMEM.

PSTR("string literal")

PSTR came into Arduino from part of avr-libs (a C library) defined in avr/pgmspace.h. It is actually a macro, not a function as many think it was, and is defined as:

#define     PSTR(s)   ((const PROGMEM char *)(s))

It looks a little bit intimidating, but it is actually quite simple. It not only tells the avr-gcc that the string literal s should be kept in program memory (i.e., flash memory in the case of Arduino), but it allows the program to convert it into a const char *s PROGMEM variable, so that it can be pass around into some PROGMEM-aware functions as a parameter.

 // This is a PROGMEM-aware function in C
 printMsg_P(const char* str) {
     char buf[strlen_P(str)+1];
     strcpy_P(buf, str);
     puts(buf);
 }

 // This handles the normal string array function
 printMsg(const char* str) {
     puts(str);
 }

F("string literal")

F() is also a macro, but it is not part of the avr/progmem.h. It is actually part of the String class defined in WString.h together with a definition of a class called __FlashStringHelper:

class __FlashStringHelper;
#define F(string_literal) (reinterpret_cast<const __FlashStringHelper *>(PSTR(string_literal)))

To understand the F() macro and the difference between F() and PSTR() take a little bit explanation.

As you can see from the definition of F(), it casts a string literal into a const char* variable using PSTR(), and then recast it into a class __FlashStrngHelper. If you further look at the __FlashStringHelper, it is actually an empty class without any construct or method. It is literally empty, so why is this useful? And why can't it just pass the string literal around by just using PSTR()?

This has to do with C++'s function overload. Function overload is a feature of C++ where multiple methods could have the same function name, but each with different parameters. Within all the Serial.print() methods, there is a method Serial.print(const char* str) that will take in a string literal and print it to serial monitor, but it is not PROGMEM-aware. If you pass in a PROGMEM string literal Serial.print(PSTR("string literal"));, the same method will accept what you pass in because it meet the type checking (remember PSTR is actually const char*), but it will not print out the correct string for PSTR("string literal"), because it simply expecting a normal string.

In C, as we see before, we solve the problem by using two functions with different names for handling normal string and PROGMEM string, but Serial is a C++ class. It needs another function overloaded method to be PROGMEM-aware, and PSTR() is clearly not the solution as we just mentioned.

WSting.h and Print.cpp create an overload method that is PROGMEM-aware by accepting a class as parameter, so that you can pass a F() wrapped string literal which has a data type of class __FlashStringHelper into the Serial.print() and get the print out correctly from program memory. See the source code yourself.

Serial.print(const __FlashStringHelper *ifsh)

So in summary, the F() and PSTR() are alike. Both are telling the compiler to keep the string literal in the program memory, and allow the string literal to be pass around into some function as parameter. But it was designed for different purpose from different programming paradigm (C versus C++). F() is Serial.print()-friendly, while PSTR() is not. If you are getting PSTR() in a function and yet you want to send it to Serial.print(), you will have to explicitly cast it into __FlashStringHelper like this:

Serial.print((__FlashStringHelper*)PSTR("test"));

PGM_P

PGM_P is a macro defined as:

#define     PGM_P   const char *

Whether you want to use PGM_P or const char * is a personal preference. For example, this is copy from PROGMEM (Arduino reference):

const char *const string_table[] PROGMEM = {string_0, string_1, string_2, string_3, string_4, string_5};

can be written as:

PGM_P const string_table[] PROGMEM = {string_0, string_1, string_2, string_3, string_4, string_5};

This is a lengthy explanation, but I hope this will help with better understanding the subject and therefore be able to use the PROGMEM and all the associated macros effectively.

Answer 2

explain in detail what exactly does each of the keywords F(), (PGM_P)F, PSTR, const PROGMEM.

• F() wraps the string literal in PSTR(...) and then casts it to __FlashStringHelper. This forces it to remain in flash and also gives it a type that the Arduino core can identify as a string in flash for overloading.
• PGM_P is a "Pointer to ProGraM mememory". This is the type that all the _P variants of functions require. It's basically const char * provided by avr/pgmspace.h
• const PROGMEM is actually two things. const means you're not changing the value, and PROGMEM is another name for __ATTR_PROGMEM__ which itself is an alias for __attribute__((__progmem__)) which is a flag for the compiler to tell it to keep the variable in flash and not copy it to RAM.

So F() and PGM_P are used when you are passing a string literal to a function that expects one of those formats. PROGMEM is used when you're setting up a constant "variable" (oxymoron there, I know...) that points to data in flash.