2

Sorry in advance for the length of my question; I want to provide proper context. I wrote an audio visualization program in Java that (in addition to making "pretty pictures" based off of the amplitudes of different frequencies being sent from the computer) sends a color to an Arduino Uno based off the sound it is "hearing."

I know the algorithm for generating this color is functional (in addition to sending that color to the Arduino, it also changes the color of an item on the DrawingCanvas). When the color is sent to the Arduino (as 3 bytes representing red, green, and blue), though, the physical LEDs don't respond correctly (they become a random color for a few seconds then switch to another random color for a bit – the behavior is hard to describe, but certainly not desired!).

I have been able to send colors to the Arduino using the same client program, but when the audio visualizer sends colors by the same method, this odd behavior occurs. My thought is that, because the audio visualizer sends color profiles dozens of times per second, the Arduino might be getting overloaded with data. Is that possible?

Here are the relevant pieces of code from the client, the audio visualizer, and the Arduino:

Client:

public class Client extends WindowController{
  //USB is a class I wrote that extends PApplet. Unless baud rates are the issue, I don't think we need to worry about this.
  static USB port = new USB("COM3", 9600);
  byte[] code = new byte[LENGTH];
  static byte killTerm = -100;
  private AudioVisualizer vis;

public void begin() {
  resize(1500,900);
  for (int k=0;k<LENGTH;k++){
    code[k]=killTerm;
  }
  vis = new AudioVisualizer(10,30,1200,300,canvas);
}

public void onMouseClick(Location point) {
  runVis = !runVis;
  if(runVis) {
    vis.start();
    code[0]=-22;
    sendCodeUSB(code);
  } else
    vis.stop();
}

public static void sendCodeUSB(byte[] code) {
  for (int k=0; k<code.length; k++) {
    port.writeProfile(code[k]);
  }
}

AudioVisualizer

public class AudioVisualizer implements Runnable {
  Thread thread = null;
  DrawingCanvas canvas;
  byte[] code = new byte[3];
  private int xPos;
  private int yPos;
  private int width;
  private int height;
  private int scale;
  private Location topLeft;
  private Location bottomRight;
  private FilledRect background;
  private int brightness=0;
  private boolean needOff = false;

  public AudioVisualizer(int x, int y, int w, int h, DrawingCanvas myCanvas) {
    canvas=myCanvas;
    xPos=x;
    yPos=y;
    topLeft = new Location (xPos,yPos);
    bottomRight = new Location (xPos+width, yPos+height);
    width=w;
    height=h;
    background = new FilledRect(topLeft,width,height,canvas);
    background.setColor(Color.BLACK);
    initiateMonitor();
    scale = height/30;
  }

  public void start() {
    if (thread == null) {
      thread = new Thread(this);
      thread.start();
    }
  }

  public void stop() {
    thread=null;
  }

  public void run() {
    while (thread != null) {
      try {
        Thread.sleep(40);
      } catch (InterruptedException e) {
      }
      graphAmps();
    }
    thread = null;
  }

  public void graphAmps() {
    //there is a whole bunch of stuff here that is unrelated that I removed.
    //basically, just know that graphAmps() calls avgColors()
    avgColors();
  }

  public void avgColors(){
    //there is stuff here that generates avgRed,avgGreen,and avgBlue (known to be working)
    Color c = new Color (avgRed,avgGreen,avgBlue);
    background.setColor(c);
    //so we don't spam the Arduino more than we need to
    if(avgRed>10 || avgGreen>10 || avgBlue>10) {
      sendArduinoColor(c);
      needOff = true;
    } else if (!(avgRed>10 || avgGreen>10 || avgBlue>10) && needOff) {
      sendArduinoColor(c);
    }
  }

  public void sendArduinoColor(Color c) {
    //takes the color and sets code = an array of 3 bytes with values from 0-100 (100 is the highest brightness I ever want the LEDs to be)
    //I am sure this is done correctly so I've left it out.
    constrainArduinoColor(c);
    //calls the static method from the client and passes in the byte array with the rgb values
    Client.sendCodeUSB(code);
  }
}

Arduino

#include <EEPROM.h>

int redPin = 11;
int bluePin = 10;
int greenPin = 9;
//declared here for debugging ease of access
int r;
int g;
int b;
//are we going through the Spectrum Cycle loop for the first time?
boolean sclFirstTime = true;
//temporary initialization
boolean waiting = true;
int index = 0;
const int LENGTH = 20;
byte profile[LENGTH];
byte avProfile[3];
byte killTerm = -100;
byte audioVisualizationCode = -22;

void setup() {
  //start the serial communication
  Serial.begin(9600);
  //initialize the profile to all -1 values
  clearProfile();
}

void loop() {
  //if there is nothing over serial and nothing is in the EPROM, we are waiting
  if (!eepromProfileAvailable() && Serial.available()==0){
    waiting = true;
  } else if (eepromProfileAvailable() && Serial.available()==0) {
    //if there is something in EEPROM and nothing over Serial, we aren't waiting and load the EEPROM profile
    waiting = false;
    loadEEPROMProfile();
    //load the EPROM color profile into the array
  } else if (Serial.available()>0) {
    //if something enters over Serial, we aren't waiting, clear the profile, and load the new profile with the serial info
    waiting = false;
    clearProfile();
    while (Serial.available()<LENGTH) {}
    // Wait until there are LENGTH number of Bytes waiting
    for(int n=0; n<LENGTH; n++) {
      profile[n] = Serial.read(); // place them in the profile.
    }
    Serial.flush();
    //clear out the serial to prep for next time
  }
  //if we have a profile (either from EEPROM or Serial), save it, check for special codes (if found, apply the effect),
  //and if there are no special codes, the profile is a static color profile, and set LEDS to that.
  if(!waiting) {
    saveEEPROMProfile(); //omitted but working
    if (profile[0]==101) {
      while (Serial.available()==0)
        spectrumCycle(); //omitted but working
    } else if (profile[0]==-22) {
      audioVis();
    } else {
      anodeSafteyOut(redPin,profile[0]);
      anodeSafteyOut(greenPin,profile[1]);
      anodeSafteyOut(bluePin,profile[2]);
    }
  }
}

//a bunch of preset color effects omitted
//we are using a common Anode LED, so low value is 255 instead of 0
void anodeSafteyOut(int pin, int c) {
  if(c>=0) {
    c = 255 - c;
    //as a safety measure because there are no resistors (yes, I know im breaking an intergalactic law by not using any), make sure we aren't writing something too bright
    if (c>=155) {
      analogWrite(pin,c);
    }
  }
}

//wait for 3 bytes to come in, assign them to the avProfile, set the colors to that, do it again
//THIS IS PROBABLY WHERE THE PROBLEM IS
void audioVis() {
  while (Serial.available()<3);
  for (int k=0;k<3;k++)
    avProfile[k]=Serial.read();
  anodeSafteyOut(redPin,avProfile[0]);
  anodeSafteyOut(greenPin,avProfile[1]);
  anodeSafteyOut(bluePin,avProfile[2]);
  audioVis();
}

So that's a lot, and I've left a lot out that I didn't think was necessary, though I'm happy to provide the full programs if needed. Thanks for your help!

Here is constrainColorArduino:

public void constrainArduinoColor(Color c) {
  int r = c.getRed();
  int b = c.getBlue();
  int g = c.getGreen();
  int r1 = (int)(r*0.4);
  int b1 = (int)(b*0.4);
  int g1 = (int)(g*0.4);
  byte r2=0;
  byte b2=0;
  byte g2=0;
  for (int k=0; k<r1; k++)
    r2++;
  for (int k=0; k<b1; k++)
    b2++;
  for (int k=0; k<g1; k++)
    g2++;
  System.out.println(r+" "+g+" "+b);
  System.out.println(r1+" "+g1+" "+b1);
  if (r2>100) r2=100;
  if (b2>100) b2=100;
  if (g2>100) g2=100;
  if (r2<0) r2=0;
  if (b2<0) b2=0;
  if (g2<0) g2=0;
  System.out.println(r2+" "+g2+" "+b2);
  code[0]=r2;
  code[1]=g2;
  code[2]=b2;
}
  • constrainArduinoColor(c)? – Mikael Patel Jan 28 '16 at 17:07
  • takes a color, finds the red green and blue ints, creates bytes based off of those numbers, applies them to the array code. – KentAshfield Jan 28 '16 at 17:10
  • but where is the mapping from color c to the code[]? – Mikael Patel Jan 28 '16 at 17:12
  • added the method to the bottom of my post – KentAshfield Jan 28 '16 at 17:24
  • true but im sending a byte. for example, if i code was {r,g,b} and that is sent over where rgb are each bytes and r is initialized to 100 and g and b are 0, the LEDs become all red. – KentAshfield Jan 28 '16 at 17:30
1

My thought is that, because the audio visualizer sends color profiles dozens of times per second, the Arduino might be getting overloaded with data. Is that possible?

Close, but the source of the problem is probably this line

  Serial.flush(); //clear out the serial to prep for next time

Because your audio visualizer keeps sending data, it's quite likely that by the time you get around to claiming a record, at least some part of another one (or more) has also arrived. When you discard that with flush(), you are probably most of the time not going to be discarding exactly a record's worth of bytes, but rather some arbitrary number more or less. This introduces a synchronization problem.

You've already had some advice on solving synchronization problems. There are a variety of workable methods, binary and otherwise. One simple one that works in a lot of situations is to encode bytes textually in hex and terminate the record with a newline. It does have just over a factor of two overhead, but it has the debug/test advantage of being human readable, and you can basically cover the overhead by switching from 9600 to 19200 baud.

0

Your idea that there's a synchronization issue seems plausible. Whenever any byte gets skipped, color assignments shift from RGB to GBR or BRG, etc.

While sending a start code before the color profile is a possible approach to fixing the problem, it may be that occasionally one of the RGB values matches the start code, which could result in temporary desynchronization.

For an alternative method that doesn't require a start code, consider encoding the data into four bytes. In the first byte, send the high two bits for RGB; in the second, the next two bits of each color; etc. Set the low two bits of each byte to 11, 10, 01, 00 for first, second, third, fourth.

That is, each byte you send would be encoded BBGGRRdd where the capital letters are appropriate bits from RGB brightness levels and dd indicates which pairs of bits are being sent.

Here's some (untested) code that implements the decode:

byte code = Serial.read();
byte shif = 2*(code & 3);
byte mask = ~(3<<shif);
code >>= 2;
for (byte i=0; i<3; ++i) {
  profile[i] = (profile[i] & mask) | ((code & 3)<<shif);
  code >>= 2;
}

The following shows a function sendData3() that encodes three bytes from an array of RGB data into four bytes to be sent to the Arduino. It also shows a convenience function sendDataRGB(). A call like sendDataRGB(100, 0, 42); puts three R, G, B values into a three-byte array and calls sendData3() to send it off. [Substitute as appropriate in place of Serial.write]

void sendData3 (byte *dataIn) { // dataIn[] has R,G,B levels
  byte code;
  byte mask=3;          // mask for low 2 bits of a byte
  for (int b=4; b; ) {
    --b;
    byte shif = 2*b;    // For moving bits to align with mask
    code = 0;
    for (int k=3; k; ) { // Loop to handle dIn[2], dIn[1], dIn[0]
      --k;
      code |= (dataIn[k]>>shif) & 3; // Pick off 2 color bits
      code <<= 2;            // Roll up
    }
    Serial.write (code | b);    // Send encoded bits
  }
}

void sendDataRGB (byte R, byte G, byte B) {
  byte dataEg[] = { R, G, B };
  sendData3(dataEg);
}
  • I'll first try sending an extra sentinel byte outside of the range of any of the rgb bytes (like -7, for example), and if that doesn't work out, I'll try this. Hopefully, this will resolve the issue, but I also wonder if this will only fix the color mutation problem and not the latency one. Thanks for the detailed explanation. I look forward to getting back to you. – KentAshfield Jan 28 '16 at 19:01
  • Just finished trying my method of ending each color transmission with an end term, and there was no improvement in behavior so I would like to attempt your method. Could you explain how I would encode the rgb values into 4 byes? I've never done something like that so some additional detail/example would be very much appreciated. If I wanted to send the byte color (100, 0, 42) how would that work, for example? – KentAshfield Jan 29 '16 at 1:33
  • @KentAshfield, I added a C++ example of encoding the data. Call routine sendData3 or sendDataRGB to encode and write an RGB triplet in four bytes. ¶ I haven't looked very closely at most of your host-computer code; there, it might make sense to substitute some simple code that cycles through five or six simple colors at about 2 seconds each, and at the Arduino end, have a simpler program without EEPROM stuff, just for testing communications – James Waldby - jwpat7 Jan 29 '16 at 2:25
  • *jwpat7 thanks for the guidance. Before I jump into the encoding, I might try raising the minimum threshold for sending a color profile from the audio visualizer to something that would correspond to a more prominent call in the music (like a beat, kick, snare, or hat). Doing so would reduce the amount of data being sent to something much more reasonable without sacrificing much of the effect after some tweaks to my equalizer. I'll keep you posted. – KentAshfield Jan 30 '16 at 3:50
  • 1
    Ok great news. When I set up the "simplest scenario" with all excess code removed and a higher minimum threshold, the program worked mostly as desired (equalizers need more fine tuning and my color generator favors blue too much but those are somewhat trivial issues for now). I guess now I just need to go step by step adding in the other code segments again and seeing where the visualizer runs into trouble. – KentAshfield Jan 30 '16 at 5:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.