Arduino FFT results of "beautiful" signal seem inconsistent

Question

I have a BEAUTIFUL doppler radar signal here from a 24.050-24.250 GHZ sensor measuring a very small object at 143 fps speed (speed from another 10 GHZ radar) in a measuring area of about 300mm-500mm in front of the sensor(s). The signal was limited to around 3V with a zener diode.

I took several measurements (at this speed and object size and distance from the sensors) and there were about 8-20 wave peaks on the average above the trigger threshold level (2.58V) until the end of the trigger threshold level box where the peaks start falling off.

Looking at the graph, one can discard the first 3 peaks and the last 2-3 peaks above the trigger threshold which leaves a decent area (orange box) for sampling of the peaks for frequency/period. So in this case we have about 17 "good" peaks inside the orange box to be measured. In this case the orange box (measuring sample area) is 5 x 500 us = 2500 us in length.

I know that an FFT would be the best way to measure, but the one FFT program I used with Arduino gave me inconsistent results. Not sure how to set the sample size (must be in power of 2 increment from 2 to 256). And the maximum sampling frequency is 10kHz (hardware limitations).

I set sample sizes to 4 and 16 and 128 at 10Khz frequency, but get inconsistent results. I suspect logarithmic results, but even those do not add up. At times I get within 2-3% but at times it's off by 30% with no apparent correlation.

What I know:

• with this 24 GHZ radar (no angles) the speed = frequency / 44 to arrive to km/h (km/h x 1.097 = fps)
• in this case the o-scope measured 6613 Hz with a 151.2 us period, but that was for the entire area above the 2.58V trigger threshold. But it also had some much more varying readings as noticeable in the first 2-3 peaks after the trigger and the last 2-3 peaks that are falling off to below the trigger threshold. I excluded these from the orange box.

How do I set up this FFT, or another FFT, or similar method, that would determine the dominant frequency from data inside the orange box? The orange box sample size will always seems to be between the 8-20 number of peaks range. I measured frequency and periods from "positive rising edge to positive rising edge".

I tried with an Adafruit nRF52840 Express (16 Mhz clock) and an Adafruit 32u4 Bluefruit (8 Mhz clock).

I am using this Arduino FFT library: https://github.com/kosme/arduinoFFT

Some broad info about this FFT itself: https://www.norwegiancreations.com/2017/08/what-is-fft-and-how-can-you-implement-it-on-an-arduino/

Thank you.

    #include "arduinoFFT.h"
 
#define SAMPLES 4         //Must be a power of 2     128, 256?
#define SAMPLING_FREQUENCY 10000 //Hz, must be less than 10000 due to ADC
 
arduinoFFT FFT = arduinoFFT();

 
unsigned int sampling_period_us;
unsigned long microseconds;
 
double vReal[SAMPLES];
double vImag[SAMPLES];
 
void setup() {
    Serial.begin(115200);

    sampling_period_us = round(1000000*(1.0/SAMPLING_FREQUENCY));
}
 
void loop() {

 
  if (analogRead(A0) >600) {
   
    /*SAMPLING*/
    for(int i=0; i<SAMPLES; i++)
    {
        microseconds = micros();    //Overflows after around 70 minutes!
     
        vReal[i] = analogRead(A0);
        vImag[i] = 0;
     
        while(micros() < (microseconds + sampling_period_us)){
        }
    }
 
    /*FFT*/
    FFT.Windowing(vReal, SAMPLES, FFT_WIN_TYP_HAMMING, FFT_FORWARD);
    FFT.Compute(vReal, vImag, SAMPLES, FFT_FORWARD);
    FFT.ComplexToMagnitude(vReal, vImag, SAMPLES);
    double peak = FFT.MajorPeak(vReal, SAMPLES, SAMPLING_FREQUENCY);


    /*PRINT RESULTS*/
    Serial.println(peak);     //Print out what frequency is the most dominant.
 
  //  for(int i=0; i<(SAMPLES/2); i++)
   // {
        /*View all these three lines in serial terminal to see which frequencies has which amplitudes*/
         
        //Serial.print((i * 1.0 * SAMPLING_FREQUENCY) / SAMPLES, 1);
        //Serial.print(" ");
      // Serial.println(vReal[i], 1);    //View only this line in serial plotter to visualize the bins
  //  }


 }
 
    //delay(1000);  //Repeat the process every second OR:
   // while(1);       //Run code once
}

EDIT: @Dorian After several days of tribulations and frustrations, I read and re-read your additional info and helped me clarify things better. I think I have some hardware limitations. By the way...yes I do have a 27pf capacitor to the GND with a 3.3K resistor and a 3V zener in parallel at the OP AMP's outlet.

But let me explain first: I decided to add a comparator after the sensor and OP AMP. Now I have a baseline voltage at HIGH (3.1V). Previously, as I posted, the OP AMP had baseline of 1.65V and when motion is detected the voltage bounces between HIGHs (2.5V-3.1V) and LOWs (1V-0V) for about 10-20 times until it settles at the 1.65V baseline again. I suspected that the HIGH and LOW thresholds were not that clear. In order to get a clear HIGH or LOW I added the comparator. These readings go to an MCU's digital Capture pin capable of 62.5ns ticks (16 MHZ). The Arduino program that counts the pulses is verified to work. In the specs the MCU at 3.3V has 1V and below as LOW and 2.5V and above as HIGH.

See the new oscope reading (attached). I get accurate pulse captures (matching the oscope), but I only get about 3 or 4 pulse readings instead of the 10-20 showing on the scope. I used LOW capture (below 1V). But even if I switch to TOGGLE capture (double) I seems to be getting only the last 3-4 complete waves (1/3rd) and seem to be missing the first 2/3rds. Or may getting every 2nd or 3rd?

My OP AMP is fast at 50 MHZ and I used to measure 100us-800us pulses with it accurately. And my comparator (after the OP AMP) is a MAX 9052 with a 400ns propagation delay. 40-80ns rise/fall time. So I should not be missing any pulses. The Arduino program cannot be that slow either.

EDIT #2:

//the best resolution you can achieve is 1/16us.

#include <Arduino.h>

#define FREQ_MEASURE_PIN 8  // actual 6 is PIN D11 and 8 is PIN D12 on Adafruit Feather Express nRF52840 LE, pin 11 is D12 itsybitsy 52840
#define PPI_CHANNEL 1u
#define GPIOTE_CHANNEL 1   //just picked channel #1


volatile int pulseCount = 0;
volatile unsigned long duration; //Pulse2 - Pulse1 = count of time ticks at 62.5 ns each tick (16 Mhz clock scaled at 0)     (expected value example: 792.5620  => 7925620)
volatile unsigned long pulseTime;   // first signal reading
volatile unsigned long lastTime;   //  most current signal reading


void setup() {

    Serial.begin(115200);
   
  initCounter();
}

void loop() 
{  

  
  if  ((pulseCount == 2) && (duration>0))  {

    Serial.println(duration);   //

   }

}



//must use extern "C" because IRQ will not compile since it's "C" coded
extern "C"  
{
void GPIOTE_IRQHandler(void)
{
                                   
  lastTime = pulseTime;
  
  if (NRF_GPIOTE->EVENTS_IN[GPIOTE_CHANNEL] == 1)    //if an input pulse is detected on pin 6 then...
  {
    NRF_GPIOTE->EVENTS_IN[GPIOTE_CHANNEL] = 0;     //  reset event detection to zero to ready for next event detection? 
    pulseTime = NRF_TIMER2->CC[0];                     // save Timer value at same time when signal was read on pin 6

    //pulseCounter
    pulseCount++;
    duration =(((pulseTime*10000) - (lastTime*10000))/16); //multiply by 10000 to avoid float, and divide by 16 for 16 Mhz clock tick = 62.5 ns per tick

   // goes from here to Loop if pulse = 2 (complete PULSE) so a reading can be sent 
    }
  }
}



void initCounter()
{    
  NRF_P0->PIN_CNF[FREQ_MEASURE_PIN] = GPIO_PIN_CNF_DIR_Input << GPIO_PIN_CNF_DIR_Pos |        //sets up PIN 6 for event read
                                      GPIO_PIN_CNF_INPUT_Connect << GPIO_PIN_CNF_INPUT_Pos |  //pos going signal
                                    //  GPIO_PIN_CNF_PULL_Pulldown << GPIO_PIN_CNF_PULL_Pos |   //pulldown LOW so no floating of pin voltage 
                                   //   GPIO_PIN_CNF_SENSE_High << GPIO_PIN_CNF_SENSE_Pos;      //a HIGH POS signal is the trigger
  
                                      GPIO_PIN_CNF_PULL_Pullup << GPIO_PIN_CNF_PULL_Pos |   //Pullup or Pulldown LOW so no floating of pin voltage 
                                      GPIO_PIN_CNF_SENSE_Low << GPIO_PIN_CNF_SENSE_Pos;      //a HIGH POS signal is the trigger .... or a LOW POS
                                      
                                      
  NRF_PPI->CHEN |= 1 << PPI_CHANNEL;
  NRF_PPI->CH[PPI_CHANNEL].EEP = (uint32_t)&NRF_GPIOTE->EVENTS_IN[GPIOTE_CHANNEL];
  NRF_PPI->CH[PPI_CHANNEL].TEP = (uint32_t)&NRF_TIMER2->TASKS_CAPTURE[0];   //sets up Timer channel

  NRF_GPIOTE->CONFIG[GPIOTE_CHANNEL] = GPIOTE_CONFIG_MODE_Event << GPIOTE_CONFIG_MODE_Pos |         //sets up pin for event capture 
                                    FREQ_MEASURE_PIN << GPIOTE_CONFIG_PSEL_Pos |                   //sets up event capture on pin 6
                                    GPIOTE_CONFIG_POLARITY_HiToLo << GPIOTE_CONFIG_POLARITY_Pos;   //sets event capture signal trigger type:  LoToHi   or HiToLo or  Toggle                                    
  NRF_GPIOTE->INTENSET = (1 << GPIOTE_CHANNEL);    //  Interrupt routine setup on a pin
  NVIC_EnableIRQ(GPIOTE_IRQn);                     //  Interrupt routine setup if signal detected on pin 6

  NRF_TIMER2->TASKS_STOP = 1;    //stops timer
  NRF_TIMER2->TASKS_CLEAR = 1;    //clear timer to zero
  NRF_TIMER2->MODE = TIMER_MODE_MODE_Timer << TIMER_MODE_MODE_Pos;   //sets up TIMEr mode as "Timer"
  NRF_TIMER2->BITMODE = TIMER_BITMODE_BITMODE_32Bit << TIMER_BITMODE_BITMODE_Pos;    // sets values for Timer input
  NRF_TIMER2->PRESCALER = 0;       //read at max resolution (at MCU speed)
  NRF_TIMER2->TASKS_START = 1;      // starts the Timer
}

EDIT #3: ================================================

I added the OP AMP circuit below. This circuit worked fine with an inductive sensor using eddy-currents. I measured @ 700-800us single-shot pulses very accurately, but did not have the R1 (22k) resistor. Added the R1 resistor and the GND immediately after it when replaced the inductive sensor with the 24 GHZ radar sensor because I was not getting readings. This circuit provides the first graph I posted, centered at around 1.6V. I took the comparator after the OP AMP off as not to complicate the circuit, and also it had some artifacts.

In summary: the suspicion is that the Arduino program I posted for the nRF52 is not fast enough to capture 100us pulses in fast succession. I tried to use BLE only and uncommented the SerialPrint and SerialBegin, etc....but even with BLE I got the same limited pulse readings. For example: there were 10-20 pulses, I was only able to capture the last 1/3 of the pulses (3-4). I tried HIGH and LOW (HiToLo/LoToHi/Toggle) as well.

The radar sensor is: 3.3V supply. IF out (signal read) is between 60mV-250mV. 60mv being the zero signal state.

Here are some sample period (wave pulse) readings in micro seconds:

97.0000
53.2500
45.2500
42.0625
40.8125
**40.9375**

And the actual period that it should read based on reference measurement:  **38.4435**us

==========================
Another set of period (wave pulse) readings in micro seconds:

95.6875
55.0000
46.7500
41.7500
**40.2500**

And the actual period that it should read based on reference measurement: **39.1703**us

So the last reading is quite close, but not proportionally close where I can "estimate" the actual value reliably. So it's not for example off by 5.13% every time, but it's off by 3-6%.

Graph of example wave pulses (smaller means shorter pulse):

The ZENER at the output is there to clip the waves from going over 3.3V which can damage the MCU's input capture pin.

Thank you for your assistance!

For information only: THESE WERE THE OSCOPE READINGS WITH THE COMPARATOR CIRCUIT ADDED ON TO THE OP AMP'S OUTPUT BUT I SINCE REMOVED THE COMPARATOR CIRCUIT:

EDIT #4: It's been quite a delay as I needed to do something else after not getting anywhere with this project. Then today as I was struggling again I think I arrived to the solution. It requires more work, but at least I have a cobbled together version that seems to work well.

The problem seem to have been with the signal itself being a sine-wave or a quasi-square wave. Using an Arduino AVR sketch for the internal comparator that is being assisted by the comparator's ISR to "square" the incoming signal with timers and interrupts by toggling another pin into the Ref pin of the comparator, I was able to get reliable readings. I actually ended up using an external comparator with the AVR and an nRF52 to count to periods. This is the program and looks another person also went through hell: https://electronics.stackexchange.com/questions/357131/random-and-unpredictable-analog-comparator-behaviour/566974#566974

I prefer to use the nRF52 because of the BLE integration, but I think I can make the AVR work first stand-alone by figuring out how to edit the library to reverse the interrupt from Low to High to High to Low...since the output of the comparator is opposite of my signal input.

Of course I still need to process the readings to sort out the dominant frequency, but manually I was able to calculate 5 out of 5 events within 2-5% of the valid value.

However, it seems that the key in the input signal being read reliably is the additional pin that is used to "convert" the sine wave into a square as part of the comparator's interrupt routine.

I still need to wrap my head around how the AVR program's additional pin toggled LOW/HIGH connected to VREF of the comparator "shapes" the sine wave and acts as a feedback for hysteresis. I believe that which ever way I arrive to a "nice" square wave for the AVR's or nRF52's input counter to read will be a winner. But I will open an new topic if I get stuck. Thank you for your help and any inputs suggestions are welcomed.

EDIT #5: Based on a suggestion in another post I decided to use a 74xx14 Schmitt Trigger at the output of the op amp and it worked really well. The square pulses were well defined (4ns rise time in specs) that I was able to identify every one of the pulses (usually between 5-20) in the software. Manual calculations lead to accurate frequencies too.

Now I need to figure a way to identify the dominant frequency that has to be done on software. I tried several FFT/FHT and ADC averaging programs, but none of them worked right. Will bring this up in a separate question. Thank you all for your help. Seems like the root problem was signal conditioning on my part.

Answer 1

To have a meaningful result from the Fourier transform the input signal must filtered to a bandwidth at most half of the sample rate.

Your 10khz maximum sample rate which I suppose is given by the maximum sample rate of the ADC is less than double of 6.61 kHz , the base frequency of your signal.

Even more, you limit the signal adding high frequency components to the signal that already doesn't fit into the required bandwidth.

You better stick to zero cross detection using both rising and falling edges to have more data to average.

You might also check which part of the signal has the correct speed data against the real speed or to check for a pattern, I see the middle period is lower then at the beginning and the end of the signal.

Later edit. Some things you should check and explanation on why is pointless to use FFT on this.

First, I see you have a clean middle centered signal between -0.7 and 3V. Maybe there is an output capacitor on the sensor that auto limits itself between Zener voltage and the lower clamp diode. The middle is somewhere at 1.5V. But the middle of the logic voltage is somewhere at half VDD, 2.5V.

Maybe that's why you have variations of pulse length from the beginning to the end of the train pulse because the decision line is above the middle line.

Of course setting the decision around the middle line might give a lot of noise but you can either use an envelope detector to see when the pulse begins or a Schmidt trigger so only signal above the hysteresis will pass.

I think that truly detecting the passes around the middle point will give you more reliable and consistent results.

About using discrete Fourier transform (fast or not) for finding the base frequency of the signal, there is no way to obtain a reasonable resolution using such a short signal.

The DFT is discreete not only in time but in frequency to. Anybody who has some basic knowledge about DFT knows that the maximum resolution in frequency that you can obtain in a t timeframe is 1/t, here for 2.5ms the frequency will be in steps of 400Hz, if you have maximum 8kHz signal you will have 20 fixed frequency values from 0 to 8kHz no matter how many samples you take, no matter on what software you make the processing or platform. It's a dead end.

Answer 2

I cannot really understand why you can't catch pulses at 6613 Hz. That is not that fast, even for an Uno.

I did the following experiment with an Uno: I use Timer 2 to generate a short burst of pulses on pin 11 (PB3 = OC2A). This pin is connected to pin 8 (ICP1), where I use Timer 1 to capture the falling edges. In the input capture ISR I count the number of edges received, and I also record the timestamps of the first and the last edge. This way I can compute the received frequency.

The result of the experiment is that this way I can reliably measure frequencies up to 125 kHz. This is about 19 times the frequency of your burst. The measurement becomes unreliable if I keep increasing the frequency.

Here is my test code:

// CPU cycles per half-period of the pulse burst.
const uint16_t half_period = 64;

// Statistics recorded by the input capture ISR.
volatile uint16_t capture_count;
volatile uint16_t timer_start;
volatile uint16_t timer_end;

// Input capture ISR.
ISR(TIMER1_CAPT_vect) {
    uint16_t now = ICR1;
    uint16_t count = capture_count;
    if (count == 0)
        timer_start = now;
    else
        timer_end = now;
    capture_count = count+1;
}

void setup() {
    Serial.begin(9600);
    Serial.print("Expected frequency: ");
    Serial.print((float) F_CPU / (2*half_period));
    Serial.println(" Hz");
    Serial.println("Measured frequency:");
    Serial.flush();

    // Configure Timer 2 for fast PWM.
    DDRB  |= _BV(PB3);    // digital 11 = PB3 = OC2A as output
    TCCR2A = _BV(COM2A0)  // toggle OC2A on compare match
           | _BV(WGM21);  // mode 2 = CTC, TOP = OCR2A
    TCCR2B = 0;           // keep timer stopped
    OCR2A  = half_period - 1;

    // Configure Timer 1 for input capture.
    TCCR1A = 0;           // normal mode
    TCCR1B = _BV(CS10);   // clock @ F_CPU
    TIFR1  = _BV(ICF1);   // clear input capture flag
    TIMSK1 = _BV(ICIE1);  // enable input capture interrupt
}

void loop() {
    // Reset the counter.
    capture_count = 0;

    // Send a burst of pulses.
    TCCR2B = _BV(CS20);   // clock Timer 2 @ F_CPU
    delayMicroseconds(1000);
    TCCR2B = 0;           // stop Timer 2

    // Report results.
    uint16_t timer_cycles = timer_end - timer_start;
    uint16_t pulse_count = capture_count - 1;
    Serial.print(pulse_count);
    Serial.print(" pulses in ");
    Serial.print(timer_cycles);
    Serial.print(" timer cycles -> ");
    Serial.print((float) pulse_count / timer_cycles * F_CPU);
    Serial.println(" Hz");
    delay(500);
}

And here is the output:

Expected frequency: 125000.00 Hz
Measured frequency:
255 pulses in 32640 timer cycles -> 125000.00 Hz
255 pulses in 32640 timer cycles -> 125000.00 Hz
256 pulses in 32768 timer cycles -> 125000.00 Hz
256 pulses in 32768 timer cycles -> 125000.00 Hz
255 pulses in 32640 timer cycles -> 125000.00 Hz
256 pulses in 32768 timer cycles -> 125000.00 Hz
256 pulses in 32768 timer cycles -> 125000.00 Hz
256 pulses in 32768 timer cycles -> 125000.00 Hz
255 pulses in 32640 timer cycles -> 125000.00 Hz
256 pulses in 32768 timer cycles -> 125000.00 Hz
256 pulses in 32768 timer cycles -> 125000.00 Hz