Circular buffer with AudioIn #54

Activity

1. dhowe

   changed the title [-]Circular buffer using audioIn[/-] [+]Circular buffer with audioIn[/+] on May 1, 2020
2. kevinstadler commented on May 2, 2020

   

   kevinstadler

   on May 2, 2020

   Collaborator

   More actions

   An interesting problem! You don't even need to create your own circular buffer, you can just use the Waveform class to create one that's large enough for the number of seconds you want to display, and then selectively read from that large array. Below is some working code for you, note how it might look like there are artefacts (samples jumping up and down as they're scrolling along) but that is simply an effect of being unable to draw every sample all the time -- you would need to draw 44 thousands pixels for every second of audio. If you want to smooth out the graphics you might have to do something like averaging over all the adjacent samples that go into a pixel...

   import processing.sound.*;
   
   
   AudioIn input;
   Waveform waveform;
   
   void setup() {
     size(640, 480);
     noStroke();
     input = new AudioIn(this, 0);
   
     int numberOfSecondsToView = 3;
     int numberOfSamplesToView = numberOfSecondsToView * new Sound(this).sampleRate();
   
     waveform = new Waveform(this, numberOfSamplesToView);
     waveform.input(input);  
   }
   
   void draw() {
     background(125, 255, 125);
     fill(255, 0, 150);
   
     // get the most recent samples
     float[] samples = waveform.analyze();
   
     // only draw a subset of the samples
     for (int i = 0; i < width; i++) {
       // map the pixel position in [0,width - 1] to samples array
       // indices in [0,samples.length - 1]
       int index = (int) map(i, 0, width-1, 0, samples.length - 1);
       // for every pixeldraw a rectangle up or down from the mid-point (the height can be 'negative'!)
       rect(i, height / 2, 1, map(samples[index], -1, 1, -height/2, height/2));
     }
   }
   
3. dhowe commented on May 3, 2020

   

   dhowe

   on May 3, 2020

   Last edited by dhowe

   Author

   More actions

   Nice! I tried your averaging suggestion, dividing the width into buckets and averaging the sample values for each, but found the negative and positive values cancel each other out, so below uses the max absolute value of the samples in a bucket:

     background(245);
   
     float[] samples = waveform.analyze();
     for (int i = 0; i < width; i++) {
       values[i] = 0;
       for (int j = 0; j < samplesPerPixel; j++) {
         float val = abs(samples[i * samplesPerPixel + j]);
         if (val > values[i]) values[i] = val;
       }
     }
     
     for (int i = 0; i < width; i++) {
       float h = values[i] * height;
       rect(i, height / 2 - h / 2, 1, h);
     }
   
4. dhowe

   changed the title [-]Circular buffer with audioIn[/-] [+]Circular buffer with AudioIn[/+] on May 3, 2020
5. dhowe commented on May 4, 2020

   

   dhowe

   on May 4, 2020

   Last edited by dhowe

   Author

   More actions

   This full version uses the same method, but attempts to only update the new pixels. Still doesn't seem quite right to me though. Any thoughts?

   import processing.sound.*;
   
   AudioIn input;
   Waveform waveform;
   float[] values;
   int lastFrameMs;
   int samplesPerPixel;
   
   void setup() {
     size(800, 600);
     input = new AudioIn(this);
     int targetBufferSz = new Sound(this).sampleRate() * 3; // secs
     samplesPerPixel = floor(targetBufferSz / (float) width);
     waveform = new Waveform(this, samplesPerPixel * width);
     values = new float[width];
     waveform.input(input);
     lastFrameMs = millis();
   }
   
   void draw() {
     float[] samples = waveform.analyze();
   
     // calculate the # of new samples
     int elapsed = millis() - lastFrameMs;
     int shift = (int)((elapsed / 1000f) * width);
   
     // shift the array to the left 
     values = shiftLeft(values, shift);
   
     // fill in the new values on the right
     for (int i = width - shift; i < width; i++) {
       values[i] = 0;
       for (int j = 0; j < samplesPerPixel; j++) {
         float val = abs(samples[i * samplesPerPixel + j]);
         if (val > values[i]) values[i] = val;
       }
     }
   
     // draw the values as lines
     background(245);
     for (int i = 0; i < width; i++) {
       float h = values[i] * height;
       rect(i, height / 2 - h / 2, 1, h);
     }
   
     lastFrameMs = millis();
   }
   
   float[] shiftLeft(float[] arr, int shift) {
     float[] tmp = new float[arr.length];
     System.arraycopy(arr, shift, tmp, 0, arr.length - shift);
     return tmp;
   }
   
6. kevinstadler commented on May 5, 2020

   

   kevinstadler

   on May 5, 2020

   Collaborator

   More actions

   The problem is that the Processing millis() might not be perfectly in sync with the synthesis engine, so that's not a reliable way to determine how many of the samples in the waveform buffer are new in comparison to the last call. This information (number of new frames) is of course available in the underlying synthesis engine, there's just no way to access it from the current Sound library release.

   I added some new functions to the library and uploaded a test build here, just extract the zip in your Documents/Arduino/libraries folder and you should be good to go: sound.zip

   The main method of interest for you is:

   • int waveform.getLastAnalysisOffset() -- after calling analyze(), the int returned by this function (in the range [0, nsamples-1]) will tell you the offset into the underlying circular buffer from which the first element of the float[] array returned by analyze() is taken. The absolute value of this is not relevant for you, but you can calculate exactly how many new samples have been added since the last call by taking the difference between the offsets after each call!

   I've also added two more methods that allow you to view the current state of the underlying circular buffer directly, namely:

   • float[] waveform.analyzeCircular()
   • float[] waveform.analyzeCircular(float[] target)

   The Waveform class will continuously overwrite data in the array, so if you want to find out where the newest data has just been written to you can use the new waveform.getLastAnalysisOffset() method from above, and actually use its absolute return value to access the most recent data. (You don't have to use this information, just replace analyze() with analyzeCircular() in the code I posted above and you should already get good results!)

   Hope that helps, let me know if you find anything wrong with the test build!
7. dhowe commented on May 5, 2020

   

   dhowe

   on May 5, 2020

   Last edited by dhowe

   Author

   More actions

   So running the following code in draw (with a 1-sec buffer) , I get the following number of (what I think should be) new samples for each of the first 100 frames (sometimes 0, occasionally negative) -- does this look correct? I guess I would need to handle the wraparounds, but the zeros are still perplexing.

     public void draw()
     {
       float[] samples = waveform.analyze();
       int offset = waveform.getLastAnalysisOffset();
       int newSamples = offset - lastOffset;
       println(frameCount, newSamples);
       ...
       lastOffset = offset;
     }
   

   1 0
   2 0
   3 0
   4 0
   5 0
   6 0
   7 0
   8 0
   9 0
   10 0
   11 0
   12 0
   13 0
   14 0
   15 0
   16 0
   17 0
   18 0
   19 0
   20 0
   21 0
   22 0
   23 580
   24 2148
   25 336
   26 1152
   27 8
   28 1784
   29 1024
   30 512
   31 1024
   32 512
   33 1024
   34 824
   35 712
   36 512
   37 512
   38 1024
   39 512
   40 1024
   41 512
   42 576
   43 960
   44 512
   45 1024
   46 512
   47 1024
   48 512
   49 512
   50 1024
   51 512
   52 1024
   53 512
   54 1024
   55 1024
   56 512
   57 512
   58 1024
   59 512
   60 1024
   61 512
   62 1024
   63 512
   64 1024
   65 512
   66 512
   67 1024
   68 512
   69 512
   70 1024
   71 512
   72 1024
   73 1024
   74 0
   75 1024
   76 1024
   77 512
   78 1024
   79 512
   80 -42976
   81 0
   82 1536
   83 512
   84 752
   85 272
   86 1024
   87 1024
   88 512
   89 1024
   90 0
   91 1536
   92 0
   93 1024
   94 1024
   95 512
   96 512
   97 1024
   98 1024
   99 512
   
8. dhowe commented on May 5, 2020

   

   dhowe

   on May 5, 2020

   Last edited by dhowe

   Author

   More actions

   Getting closer...

   import processing.sound.*;
   
   AudioIn input;
   Waveform waveform;
   float[] values, rms;
   int lastOffset, samplesPerPixel;
   
   void setup()
   {
     size(800, 600);
   
     input = new AudioIn(this);
     int targetBufferSz = new Sound(this).sampleRate() * 3;
     samplesPerPixel = floor(targetBufferSz / (float) width);
     waveform = new Waveform(this, samplesPerPixel * width);
     values = new float[width];
     rms = new float[width];
     waveform.input(input);
     lastOffset = 0;
   }
   
   void draw()
   {
     float[] samples = waveform.analyze();
   
     // calculate the # of new samples
     int offset = waveform.getLastAnalysisOffset();
     int newSamples = (offset - lastOffset);
     if (newSamples < 0) newSamples += samples.length;
     int shift = ceil((newSamples / (float)samples.length) * values.length);
     lastOffset = offset;
   
     // shift the array to the left
     values = shiftLeft(values, shift);
     rms = shiftLeft(rms, shift);
   
     // fill in the new values on the right
     for (int i = width - shift; i < width; i++) {
       values[i] = absAvg(samples, i);
       rms[i] = rmsAvg(samples, i);
     }
   
     // draw the values as rects
     background(245);
     noStroke();
     for (int i = 0; i < width; i++) {
       fill(49, 48, 205);
       float h = values[i] * height;
       rect(i, height / 2 - h / 2, 1, h);
       fill(98, 101, 222);
       h = rms[i] * height;
       rect(i, height / 2 - h / 2, 1, h);
     }
   }
   
   float rmsAvg(float[] samples, int startIdx)
   {
     float sumSq = 0;
     for (int j = 0; j < samplesPerPixel; j++) {
       float val = samples[startIdx * samplesPerPixel + j];
       sumSq += (val * val);
     }
     float mean = sumSq / samplesPerPixel;
     return (float) Math.sqrt(mean);
   }
   
   float absAvg(float[] samples, int startIdx)
   {
     float sum = 0;
     for (int j = 0; j < samplesPerPixel; j++) {
       sum += Math.abs(samples[startIdx * samplesPerPixel + j]);
     }
     return (sum * 2) / (float) samplesPerPixel;
   }
   
   float[] shiftLeft(float[] arr, int shift)
   {
     float[] tmp = new float[arr.length];
     System.arraycopy(arr, shift, tmp, 0, arr.length - shift);
     return tmp;
   }
   
9. kevinstadler

   added

   questionQuestion about setting up or using the libraryQuestion about setting up or using the library

   on Nov 17, 2020
10. kevinstadler commented on Sep 16, 2023

    

    kevinstadler

    on Sep 16, 2023

    Collaborator

    More actions

    This will be addressed as a special case of #55
11. kevinstadler

    closed this as completedon Sep 16, 2023
12. kevinstadler

    added

    enhancementNew feature or requestNew feature or request

    and removed

    questionQuestion about setting up or using the libraryQuestion about setting up or using the library

    on Sep 16, 2023
13. kevinstadler

    closed this as not plannedon Sep 16, 2023
14. dhowe commented on Nov 22, 2025

    

    dhowe

    on Nov 22, 2025

    Author

    More actions

    @kevinstadler trying to get this to work again, but waveform.getLastAnalysisOffset seems to have been removed from the API... any ideas on how to proceed?