My program is using too much memory

Question

I work in a warehouse and I was tasked to make a device which will show the location of any product when Product ID is given. All of our product has 3 things. Product ID, Rack Number and Gap number.

So the basic program is like that When I enter Product ID: 1640 The program will show in display that it is located in RCK 113 GAP 88

For this I used Arduino Mega, 4x3 Keypad and 16x2 LCD Display with I2C adapter. I found a Arduino Keypad Calculator code on the Internet and modify my code from there. So currently my code is working fine and everything

What I noticed is Arduino mega has only 2% memory left with all that string value (For that I couldn't use Arduino Uno at first). Now due to new products are added I need more space or ways to store data elsewhere.

I can not use internet or WIFI as Another team has already made a device using ESP8266 Nodemcu which shows data from a Google sheet but since my device stores data in itself it is very fast (almost instant) to show any result on the other hand the device with NODEMCU ESP8266 can be easily updated and doesn't bother about size yet it is very slow to show result and needs good wifi connection so it is not efficient in warehouse environment. So every staff use my device on the go and other device as a backup

Currently I have around 1000 product address stored in my device but a new shipment of almost 3000 product is coming and my mega has only 2% memory left.

So any genius here can you help me to compress my program more ? or any fault I made during my coding ? Also this is my first post and sorry for my bad English.

My Code is given below. It is huge please take your time to understand it.

Even I can't copy my code here

Body is limited to 30000 characters; you entered 132142.

This is a sample of my code for you to understand

Here is full code uploaded. Download

What I want to know is, if it is possible to add a Arduino SD card module and keep portion of my string memory in there.

#include <LiquidCrystal_I2C.h>
#include <Keypad.h> 

const byte ROWS = 4; 
const byte COLS = 4;
unsigned int integerValue=0; 
//define the keymap
char keys [ROWS] [COLS] = {
  {'1', '2', '3', 'X'},
  {'4', '5', '6', '*'},
  {'7', '8', '9', '='},
  {'/', '0', '-', '+'}
};
byte rowPins[ROWS] = {9, 8, 7, 6}; 
byte colPins[COLS] = {5, 4, 3, 2};

//create the keypad
Keypad myKeypad = Keypad( makeKeymap(keys), rowPins, colPins, ROWS, COLS );
LiquidCrystal_I2C lcd(0x27, 16, 2);

//variables declaration
boolean valOnePresent = false;
boolean result = false;
String num1;
int ans;
boolean getvalue = false;


void setup(){
  Serial.begin(9600);
  lcd.init(); // initialize the lcd
  lcd.backlight();
  frstscreen();
  //lcd.clear();

}

void loop(){
  char key = myKeypad.getKey();

  if (key != NO_KEY && (key=='1'||key=='2'||key=='3'||key=='4'||key=='5'||key=='6'||key=='7'||key=='8'||key=='9'||key=='0')){
      num1 = num1 + key;
      int numLength = num1.length();
      lcd.setCursor(15 - numLength, 1);
      lcd.print(num1);
      valOnePresent = true;
  }

  else if (valOnePresent == true && key != NO_KEY && key == '='){

      ans = num1.toInt();
      
      lcd.clear();
      lcd.setCursor(15,1);
      lcd.autoscroll();
      lcd.print(ans);
      lcd.noAutoscroll();
      result = true;
      
  }
  else if (key != NO_KEY && key == 'X'){
    lcd.clear();
    valOnePresent = false;
    num1 = "";
    frstscreen();
    getvalue = false;
}

    integerValue = ans;


if (result == true) {

if (integerValue ==  9   ) { IDScrn(); lcd.print("109");lcd.setCursor(12,1);lcd.print("63 "); delayfunc();}
if (integerValue == 15    ) { IDScrn(); lcd.print("107");lcd.setCursor(12,1);lcd.print("60 "); delayfunc();}
if (integerValue == 15    ) { IDScrn(); lcd.print("109");lcd.setCursor(12,1);lcd.print("63 "); delayfunc();}
if (integerValue == 20    ) { IDScrn(); lcd.print("109");lcd.setCursor(12,1);lcd.print("63 "); delayfunc();}
if (integerValue == 22    ) { IDScrn(); lcd.print("110");lcd.setCursor(12,1);lcd.print("68 "); delayfunc();}
if (integerValue == 23    ) { IDScrn(); lcd.print("109");lcd.setCursor(12,1);lcd.print("63 "); delayfunc();}

//TOP and components

if (integerValue ==  1001   ) { IDScrn(); lcd.print("RCK 115 COL A                  "); delayfunc();}
if (integerValue == 1002   ) { IDScrn(); lcd.print("RCK 115 COL A                  "); delayfunc();}
if (integerValue == 1003   ) { IDScrn(); lcd.print("RCK 115 COL A                  "); delayfunc();}


//Stock and CBOX

if (integerValue ==   376    ) { ComScrn(); lcd.print(" Stock Box 1"); delayfunc();}
if (integerValue ==   15     ) { ComScrn(); lcd.print(" Stock Box 1"); delayfunc();}
if (integerValue ==   919    ) { ComScrn(); lcd.print(" Stock Box 1"); delayfunc();}



//CBOX

if (integerValue ==   245    ) { ComScrn(); lcd.print(" 39K COL I "); delayfunc();}
if (integerValue ==   245    ) { ComScrn(); lcd.print(" 10K COL I "); delayfunc();}
if (integerValue ==   245    ) { ComScrn(); lcd.print(" 47k COL I "); delayfunc();}
if (integerValue ==   245    ) { ComScrn(); lcd.print(" 1k  COL I "); delayfunc();}
if (integerValue ==   245    ) { ComScrn(); lcd.print(" 2k  COL I "); delayfunc();}
if (integerValue ==   245    ) { ComScrn(); lcd.print(" 4.7k  COL I "); delayfunc();}
if (integerValue ==   245    ) { ComScrn(); lcd.print(" 27k COL I "); delayfunc();}
if (integerValue ==   766    ) { ComScrn(); lcd.print(" 0 ohm   COL I "); delayfunc();}
if (integerValue ==   766    ) { ComScrn(); lcd.print(" 220 ohm COL I "); delayfunc();}
if (integerValue ==   766    ) { ComScrn(); lcd.print(" 560 ohm COL I "); delayfunc();}
if (integerValue ==   766    ) { ComScrn(); lcd.print(" 470 ohm COL I "); delayfunc();}
if (integerValue ==   766    ) { ComScrn(); lcd.print(" 330 ohm COL I "); delayfunc();}
if (integerValue ==   766    ) { ComScrn(); lcd.print(" 100 ohm COL I "); delayfunc();}
if (integerValue ==   1291     ) { ComScrn(); lcd.print(" Cbox 1  GAP 101 "); delayfunc();}
if (integerValue ==   1314     ) { ComScrn(); lcd.print(" Cbox 1  GAP 101 "); delayfunc();}

if (getvalue == false) { noproduct(); }

} 
 }
void frstscreen()
{
  lcd1stline();
  lcd.print("Warehouse Stock");
  lcd.setCursor(0,1);
  lcd.print("Enter ID: ");
}
void pressA(){

  lcd.clear();
  lcd.setCursor(0,1);
  lcd.print("Press A again");
}
void noproduct(){

  lcd1stline();
  lcd.print("Not Found");
  lcd.setCursor(0,1);
  lcd.print("Press A again");
  result= false;
}
void delayfunc(){
  result = false;
  delay (2000);
  pressA();
  getvalue = true;

  
}
void lcd1stline(){
lcd.clear();
lcd.setCursor(0,0);
}
void IDScrn(){
  lcd.clear();
  lcd.setCursor(0,0);
  lcd.print("ID: ");
  lcd.setCursor(4,0);
  lcd.print(integerValue);
  lcd.setCursor(0,1);
  lcd.print("RCK     GAP");
  lcd.setCursor(4,1);
}

void ComScrn(){
    lcd.clear();
  lcd.setCursor(0,0);
  lcd.print("ID: ");
  lcd.setCursor(4,0);
  lcd.print(integerValue);
  lcd.setCursor(1,1);
}

Answer 1

There are lots of repetitions in your code. This is something you should generally avoid, not only because of memory usage, but also, and most importantly, because of maintainability.

The most common way of getting rid of repetitions is to write a loop that walks through an array. For the first part of your program (the rack/gap scheme), that array could look like this:

// Data structure describing a recorded location.
struct Location {
    uint16_t id;
    uint8_t rack;
    uint8_t gap;
};

// List of recorded product locations.
const Location locations[] = {
    {  9, 109, 63 },
    { 15, 109, 63 },
    { 20, 109, 63 },
    { 22, 110, 68 },
    { 23, 109, 63 },
    // etc...
};

// Number of items in the above list.
const size_t nb_locations = sizeof locations / sizeof locations[0];

Note that the structure uses the smallest possible integers for each field, in order to save memory. The main benefit, however, is that the data is now in a format that is easier to update: a simple list of numbers.

Then, all the code that handles this data becomes a loop, instead of a repetition. For example, the following function can print the location of any item on the LCD. It returns true or false to let you know whether it did find the requested item:

// If the item is found, print its location and return true.
// Otherwise return false.
bool print_location(uint16_t id) {
    for (size_t i = 0; i < nb_locations; i++) {
        if (locations[i].id == id) {
            IDScrn();
            lcd.print(locations[i].rack);
            lcd.setCursor(12,1);
            lcd.print(locations[i].gap);
            delayfunc();
            return true;  // found
        }
    }
    return false;  // not found
}

This scheme still has the problem that all the data is copied to RAM at initialization. If you have 4,000 items, that would require 16,000 bytes of RAM, or about twice the RAM of your Mega. You can save RAM by keeping all constant data in flash, using the PROGMEM qualifier. You have 32 times more flash than RAM. You would then need, however, to bring each item back to RAM in order to use it, which can be done with the memcpy_P() function. Here is a version of the previous code that stores the array in flash:

// List of recorded product locations.
PROGMEM const Location locations[] = {
    {  9, 109, 63 },
    { 15, 109, 63 },
    { 20, 109, 63 },
    { 22, 110, 68 },
    { 23, 109, 63 },
    // etc...
};

// Number of items in the above list.
const size_t nb_locations = sizeof locations / sizeof locations[0];

// If the item is found, print its location and return true.
// Otherwise return false.
bool print_location(uint16_t id) {
    for (size_t i = 0; i < nb_locations; i++) {
        Location location;  // RAM copy of locations[i]
        memcpy_P(&location, &locations[i], sizeof location);
        if (location.id == id) {
            IDScrn();
            lcd.print(location.rack);
            lcd.setCursor(12,1);
            lcd.print(location.gap);
            delayfunc();
            return true;  // found
        }
    }
    return false;  // not found
}

This approach should enable you to store all the database in the flash of your Mega, and maybe even on an Uno. That being said, I second the commenters that say that using an SD card should make upgrading the database a lot more convenient.

Answer 2

Did you try using the (PROGMEM) F macro. It is described in the official Arduino documentation at the end of https://www.arduino.cc/reference/en/language/variables/utilities/progmem/

This forces all strings to be stored in Flash memory instead of the scarce SRAM memory.

You can try with a few strings to see if the memory lowers and calculate if it would be enough (to fit in Flash). If even Flash is not enough, than SD card would be an alternative.

Also some refactorings that might clean up the code (not necessarily reduce the memory used):

• instead of the many if (integerValue == xxx) constructs, use switch/case statements.
• A possible bug are the identical values within (like 245 and 766 used many times, but for different products)
• Move the function lcd.print("...") inside function ComScrn and pass the string (using F(..))
• Similar as above but for IDScrn, using two strings and set the cursor inside IDScrn as well.
• (key != NO_KEY && key == 'X') ... if key is 'X' it can never be NO_KEY so the first check can be removed.
• if (result == true) can be written as if (result).

Answer 3

What I want to know is, if it is possible to add a Arduino SD card module and keep portion of my string memory in there

Well, given that two contributors have already gave a handful of advices to both improving your code and the benefits of using an SD card. I find nothing else to say except answering your very specific question.

The answer is simple, and it is a yes. There is no problem in dividing your data into two parts, one part on your microcontroller/Arduino and the other part on your SD card, you will have to know which part is where so you can access it without any problems and almost instantly as well.

I hope that answers your question fully.

Answer 4

The Arduino Uno uses a rather old, weak processor. It is still a perfectly cromulent microcontroller - it does just fine - but much newer ones do exist.

You mentioned the ESP8266. Did you know it has 40kB of RAM compared to the Uno's 2kB? And the successor, the ESP32, has 320kB of RAM.

Since your application is basically a miniature computer, with a keyboard and screen, have you considered using something that's designed as a miniature computer? A Raspberry Pi (any model) has more gigabytes of RAM than you know what to do with. But it does use a lot more power. But you can also write the software in a normal Linux way (if you happen to like Linux programming). Or, you could consider writing an app for your Android cellphone.

If you insist on using the Uno, another option is to add an SPI RAM expansion chip, for example this 1-euro chip with 32kB (they do go bigger). Now, the Uno has no built-in RAM expansion ability, so you can't just access it with pointers or something, but you can still use the SPI bus to tell the chip to read or write certain bytes, kinda like you can with an SD card.

After writing this, I realized SPI RAM is not so useful for your application, because you don't want to store the data only while the device is turned on. You could use a SPI FRAM or SPI EEPROM chip instead which can store data while it's powered off. The Arduino IDE doesn't have a way to automatically put data into any of these chips... so to input the data you'd probably have to send it over the serial port, into a separate program you wrote, which writes it to the chip. Or you could make a user interface that allows the user to edit the data directly on the device.

Or you can use an SD card, yes. Honestly that's probably the simplest way.