Large refactor with demonstration of problem and inefficieny

refactor.pde

/*
  ===========================================================================================
  ArduinoPlotter_processingListener is the source processing script that corresponds to the 
  Plotter library for Arduino. The library supports plots against time as well as 2-variable 
  "X vs Y" graphing. 
  -------------------------------------------------------------------------------------------
  The library transfers information via the serial port to a listener program written with the
  software provided by Processing. No modification is needed to this program; graph placement,
  axis-scaling, etc. are handled automatically. 
  Multiple options for this listener are available including stand-alone applications as well 
  as the source Processing script.

  The library, these listeners, a quick-start guide, and usage examples are available at:
  
  https://github.com/devinconley/ArduinoPlotter

  -------------------------------------------------------------------------------------------
  ArduinoPlotter_processingListener
  v1.0.0
  https://github.com/devinconley/ArduinoPlotter
  by Devin Conley
  ===========================================================================================
 */

import processing.serial.*;
Serial port;

//CONSTANTS
final int[] COLORS = {#00FF00,#FF0000,#0000FF, #FEFF00, #FF9900, #FF00FF}; //int color codes
final char STARTER_KEY = '^';
final char OUTER_KEY = '#';
final String INNER_KEY = "@";
final float AXIS_COVERAGE = 0.75;
final int LABEL_SZ = 14; // text sizes
final int TITLE_SZ = 16;
final int NUM_SZ = 10; 
final int MARGIN_SZ = 20; // between plots
final int BG_COL = 75; // 
final int PLOT_COL = 115;
final int TICK_LEN = 6;
final int NUM_TICKS = 5;
final float PT_SZ = 1.5;

// Setup and config Globals
int h;
int w;
float sub_height;
float sub_width;
int num_graphs;
int total_vars;
int max_points;
String config_code = "This will not be matched!";
boolean configured = false;
int last_config;

// Graph-specific setup globals
float[][] pos_graphs;             // [g] stored as {+x, +y} from origin
String[] titles;                  // [g]
boolean[] xvy;                    // [g]
int[] first_index_graphs;         // [g] -> index?
int[] sz_graphs;                  // [g] num of values tracked in each graph
int[] num_points;                 // [g]
double[][] extreme_graphs___;     // [g] {min_x, max_x, min_y, max_y}
int[][] extreme_graphs_c_;        // [g] {min_x, max_x, min_y, max_y} tracks since last set (how do we know?) (has it ever been set?)
int[] pos_x;                      // [g]
String[][] labels;                // [g]
double[][][][] data;              // [g] [ 0 .. num_points[g] ] [ f_i_g[g] .. (f_i_g[g] + sz_graphs[g]) ] [ 0 = timestamp (for xvt), 1 = data point ]



volatile boolean waiting_for_packet = true;
volatile boolean waiting_for_draw = false;


int number_of_data_points = 0;



void setup() {
  size(800, 800);
  surface.setResizable(true);
  h = height;
  w = width;
  String portID = Serial.list()[0];
  port = new Serial(this, portID, 115200);  
  port.bufferUntil('#');
  frameRate(100);
  textSize(16);
  background(BG_COL);
  strokeWeight(PT_SZ);
}
  
void draw() {
  //PLOT ALL
  try {
    if (configured) {
      background(BG_COL);
      int x = 0;
      while(waiting_for_packet) {
        x++;
      }
      for (int which_graph = 0; which_graph < num_graphs; which_graph++) {
      	if (xvy[which_graph]) {
      	  plot_xy(which_graph);
      	} else {
      	  plot_time(which_graph);
      	}
    waiting_for_packet = true;
    //waiting_for_draw = false;
      if (debug_plot)
        println("---------------------------------------------");
      }
    }
    
    if ( h != height || w != width) {
      h = height;
      w = width;
      setupGraphPosition();
    }
    

    
  } catch (Exception e) {}
}


void plot_background(int graph) {
  // Plot Background
  fill(PLOT_COL);
  stroke(255);
  rect(pos_graphs[graph][0],pos_graphs[graph][1],sub_width,sub_height);
  // Title
  textSize(TITLE_SZ);
  fill(255);
  textAlign(CENTER, TOP);
  text(titles[graph], pos_graphs[graph][0] + sub_width/2, pos_graphs[graph][1] + TITLE_SZ);
}


void plot_xy(int g) {
  int graph = g;
  int k = first_index_graphs[graph];   

  // Calculations for offset and scaling of graph
  double x_scale  = AXIS_COVERAGE * sub_width /  (extreme_graphs___[graph][1] - extreme_graphs___[graph][0]);
  double y_scale  = AXIS_COVERAGE * sub_height / (extreme_graphs___[graph][3] - extreme_graphs___[graph][2]);

  double x_offset = x_scale  *  extreme_graphs___[graph][0] - 0.5  *  (1.0 - AXIS_COVERAGE)  *  sub_width;
  double y_offset = y_scale  *  extreme_graphs___[graph][3] + 0.5  *  (1.0 - AXIS_COVERAGE)  *  sub_height;


  
  if (debug_plot_xy) {
    println("plot_xy:\t\txvy:\tx_scale: " + x_scale + "\ty_scale: " + y_scale + "\tx offset: " + x_offset + "\ty_offset: " + y_offset);
  }
  plot_background(graph);
  
  // X and Y labels
  textSize(LABEL_SZ);
  textAlign(LEFT, TOP);      text(labels[g][k+1], pos_graphs[graph][0]             + 10, pos_graphs[graph][1] + 10);
  textAlign(RIGHT, BOTTOM);  text(labels[g][k]  , pos_graphs[graph][0] + sub_width - 10, pos_graphs[graph][1] + sub_height - 3  *  NUM_SZ);
  

  drawTicks(graph);
  
  // ** add support for multiple paths in x-y here **
  stroke(COLORS[0]);
  
  int target = num_points[graph];
  
  for (int j = 0; j < target; j++) {
    //int color_index = j % 6;
    //stroke(COLORS[color_index++]);
    float x = (float)(pos_graphs[graph][0] +             (data[g][j][k][1]    *  x_scale - x_offset));
    float y = (float)(pos_graphs[graph][1] + y_offset -   data[g][j][k+1][1]  *  y_scale           );
    point(x,y);

    if (debug_plot_xy /*&& j >= (target - 5)*/) {
      println("plot_xy:\t\t" + "xvy:\t" + j + " " + "\t(x,y): " + data[g][j][k][1] + "," + data[g][j][k+1][1]);
    }
  }
  


}

void plot_time(int g) {
  int k = first_index_graphs[g];   

  // Calculations for offset and scaling of graph
  double x_scale =                   sub_width /  (extreme_graphs___[g][1] - extreme_graphs___[g][0]);
  double y_scale = AXIS_COVERAGE  *  sub_height / (extreme_graphs___[g][3] - extreme_graphs___[g][2]);
  
  double x_offset = x_scale  *  extreme_graphs___[g][0];
  //double x_offset = x_scale  *  extreme_graphs___[g][0] - 0.5  *  (1.0 - AXIS_COVERAGE)  *  sub_width;
  double y_offset = y_scale  *  extreme_graphs___[g][3] + 0.5  *  (1.0 - AXIS_COVERAGE)  *  sub_height;
  
  if (debug_plot_time) {
    println("plot_time:\txvt:\tx_scale: " + x_scale + "\ty_scale: " + y_scale + "\tx offset: " + x_offset + "\ty_offset: " + y_offset);
  }
  

  plot_background(g);
  
  // Plot legend
  float textPos = pos_graphs[g][1] + LABEL_SZ;
  textAlign(RIGHT, TOP);
  textSize(LABEL_SZ);
  
  // Plot each line
  for (int i = 0; i < sz_graphs[g]; i++) {
      fill(COLORS[i]);
      text(labels[g][k + i],pos_graphs[g][0] + sub_width - 10,textPos);
      textPos += (LABEL_SZ + 3);
      stroke(COLORS[i]);
      
      // FIXME int target = extreme_graphs_c_[graph][2] - 1;
      int target = num_points[g];

      for (int j = 0; j < target; j++) {
        float x = (float)(pos_graphs[g][0] +           (data[g][j][k+i][0]*x_scale - x_offset ));
        float y = (float)(pos_graphs[g][1] + y_offset - data[g][j][k+i][1]*y_scale            );
      
      point(x,y);
      
      if (debug_plot_time /* && j >= (target - 5) */) {
        println("plot_time:\t" + "xvt:\t" + j + " " + "(x,y): " + data[g][j][k+i][0] + "," + data[g][j][k+i][1]);
        }
      }
      

      
  }
  // Draw axis
  noFill();
  stroke(255);
  rect(pos_graphs[g][0],pos_graphs[g][1],sub_width,sub_height);
  drawTicks(g); // draw ticks over any data (only an issue with time plot)
}

void drawTicks(int graph) {
  // Label graph with numbered tick marks
  stroke(255);
  fill(255);
  textSize(NUM_SZ);
  textAlign(LEFT, CENTER);
  // Draw ticks along y-axis
  float temp_x         = pos_graphs[graph][0] - TICK_LEN/2;
  
  float tick_offset    = 0.5  *  (1.0 -  AXIS_COVERAGE)*  sub_height; 
  float tick_interval  =                 AXIS_COVERAGE *  sub_height / (NUM_TICKS - 1);
  
  float val            = (float) extreme_graphs___[graph][3];
  float val_interval   = (float)(extreme_graphs___[graph][3] - extreme_graphs___[graph][2]) / (NUM_TICKS - 1);
  
  
  for (float temp_y = pos_graphs[graph][1] + tick_offset;  temp_y <= pos_graphs[graph][1] + sub_height - tick_offset; temp_y += tick_interval) {
    line(temp_x, temp_y, temp_x + TICK_LEN, temp_y);
    text(Float.toString(val), temp_x + TICK_LEN + 5, temp_y);
    val -= val_interval;
  }
  

  // Draw along x-axis (will be diff for each type of graph)
  
  float temp_y = pos_graphs[graph][1] + sub_height - TICK_LEN/2;
  
  // if XvY graph, evenly spaced ticks within coverage
  if (xvy[graph]) {
    val_interval   = (float)(extreme_graphs___[graph][1] - extreme_graphs___[graph][0]) / (NUM_TICKS - 1);
    val            = (float) extreme_graphs___[graph][0];
    tick_offset    = 0.5  *  (1.0 -  AXIS_COVERAGE)   *  sub_width;
    tick_interval  =                 AXIS_COVERAGE    *  sub_width / (NUM_TICKS - 1);
  // if a time graph, evenly spaced ticks across all
  } else {
    val_interval   = (float)(extreme_graphs___[graph][1] - extreme_graphs___[graph][0]) / (NUM_TICKS + 1);
    val            = (float) extreme_graphs___[graph][0] + val_interval;
    tick_offset    = sub_width / (NUM_TICKS + 1);
    tick_interval  = tick_offset;
  }
  
  textAlign(CENTER, BOTTOM);
  
  for (temp_x  = pos_graphs[graph][0]              + tick_offset;
       temp_x <= pos_graphs[graph][0] + sub_width  - tick_offset; 
       temp_x += tick_interval) {
    line(temp_x, temp_y, temp_x, temp_y + TICK_LEN);
    text(Float.toString(val), temp_x, temp_y - 5);
    val += val_interval;
  }
  

}

void initialize_data() {
        for (int graph = 0; graph < num_graphs; graph++) {
        
        for (int i = 0; i < max_points; i++) {
          pos_x[i] = 0;
        
          for (int j = 0; j < total_vars; j++) {
            if (i == 0) {
            labels[graph][j] = "";
            }
            for (int k = 0; k < 2; k++) {
              data[graph][i][j][k] = -0.0;
            }
          }
        }
        
        
        titles[graph] = "";
        num_points[graph] = 0;
        xvy[graph] = false;
        sz_graphs[graph] = 0;
        first_index_graphs[graph] = 0;
        
        for (int i = 0; i < 4; i++) {
          extreme_graphs___[graph][i] = -0.0;
          extreme_graphs_c_[graph][i] = 0;
          
        }
      }
}

void setup_serial(String[] array_main) {
  try {
      String[] array_sub = array_main[0].split(INNER_KEY);
      // Check for size of full transmission against expected to flag bad transmission
      num_graphs = Integer.parseInt(array_sub[0]);
      if (array_main.length != num_graphs+1) {
        throw new Exception();
      }
      configured = false;
      if (debug_serial) print("no match");
      

      setupGraphPosition();
      
      // Pull more values and reset datastore arrays as appropriate
      total_vars = Integer.parseInt(array_sub[1]);
      max_points = Integer.parseInt(array_sub[2]);
      
      labels = new String[num_graphs][total_vars];
      data = new double[num_graphs][max_points][total_vars][2];
      pos_x = new int[max_points];
      
      num_points = new int[num_graphs];
      titles = new String[num_graphs];
      xvy = new boolean[num_graphs];
      sz_graphs = new int[num_graphs];
      extreme_graphs___ = new double[num_graphs][4];
      extreme_graphs_c_ = new int[num_graphs][4];
      first_index_graphs = new int[num_graphs];
      

      initialize_data();

      // Iterate through the individual graph data blocks to get graph specific info
      
      int k = 0; // k tracks overall variable index
      
      for (int g = 0; g < num_graphs; g++) {
        
        array_sub = array_main[g+1].split(INNER_KEY);
        
        titles[g]     =                  array_sub[0];
        xvy[g]        = Integer.parseInt(array_sub[1]) == 1;
        num_points[g] = Integer.parseInt(array_sub[2]);
        sz_graphs[g]  = Integer.parseInt(array_sub[3]);
        
        first_index_graphs[g] = k;
        
        if (debug_serial)
          println(xvy[g]);
        
        int p = 4; // first label of this graph falls at index 4
        for (int j = 0; j < sz_graphs[g]; j++) {
          labels[g][k] = array_sub[p];
          p += 2;
          k++;
        }
      }
      
      // Set new config code
      config_code = array_main[0];
      
    
      if (debug_serial)
        println(config_code);
      
      last_config = millis();
      
  }   catch (Exception e) {
    //println("exception....");
  }
  
}






void check_for_new_extremes(String[] array_sub, int g, int temp_time, int p, boolean is_xvy) {

  int start = first_index_graphs[g];
  int sz    = sz_graphs[g];
  
  int top_index;
  int bot_index;
      
  for (int j = start; j < start + sz; j++) {
    //data[g][pos_x[g]][j][0] = temp_time; // if we do this here, we can use this for debugging
    data[g][pos_x[g]][j][1] = Double.parseDouble(array_sub[p]);
  
  if (!is_xvy) {
    data[g][pos_x[g]][j][0] = temp_time; // but it should be done here, because this only applies to xvt graphs
    p += 2;
    top_index = 2;
    bot_index = 3;
  } else {

    top_index = p-5;
    bot_index = p-4;
  }
  // Check for new extremes
       if (data[g][pos_x[g]][j][1] <= extreme_graphs___[g][top_index]) { extreme_graphs___[g][top_index] = data[g][pos_x[g]][j][1]; extreme_graphs_c_[g][top_index] = 0; if (debug_check_for_new_extremes) {println("check_for_new:\t" + (is_xvy ? "xvy:\t" : "xvt:\t") + "e_g_c_[" + g + "][" + top_index + "] to: " + extreme_graphs_c_[g][top_index]);}} 
  else if (data[g][pos_x[g]][j][1] >= extreme_graphs___[g][bot_index]) { extreme_graphs___[g][bot_index] = data[g][pos_x[g]][j][1]; extreme_graphs_c_[g][bot_index] = 0; if (debug_check_for_new_extremes) {println("check_for_new:\t" + (is_xvy ? "xvy:\t" : "xvt:\t") + "e_g_c_[" + g + "][" + bot_index + "] to: " + extreme_graphs_c_[g][bot_index]);}}
  
  if (is_xvy)
    p += 2;
  }
          
  
}


// FIXME: this can overflow e_g_c_
boolean[] check_if_needs_calc(int g, boolean is_xvy) {
  boolean[] needs_calc = {false, false, false, false};
  int j;
  int stop;
  if (is_xvy)
    j = 0;
  else
    j = 2;
    
  if (is_xvy)
    stop = 4;
  else
    stop = 4;


  for (; j < stop; j++) {
    if (extreme_graphs_c_[g][j] > num_points[g]) {
      needs_calc[j] = true;
    } else {
      extreme_graphs_c_[g][j]++;
      if (debug_check_if_needs_calc) {
        println("check_if_needs:\t" + (is_xvy ? "xvy:\t" : "xvt:\t") + "e_g_c_[" + g + "][" + j + "]++ up to " + extreme_graphs_c_[g][j]);
      }
    }
  }

  
  if (debug_check_if_needs_calc) {
    println("check_if_needs:\t" + (is_xvy ? "xvy:\t" : "xvt:\t") + "needs_calc = {" + needs_calc[0] + " " +  needs_calc[1] + " " + needs_calc[2] + " " + needs_calc[3] + "}");
  }
  return needs_calc;
}


void update_extremes(boolean[] needs_calc, int g, boolean is_xvy) {
  if (needs_calc[0] || needs_calc[1] || needs_calc[2] || needs_calc[3]) {
    
    int j = first_index_graphs[g];
    int sz = sz_graphs[g];
  
    if (is_xvy)
      sz = 0;
      

    if (debug_update_extremes) {
        println("update_extremes:" + (is_xvy? "xvy:\t" : "xvt:\t") + "j_0: " + j + "\t\t\tsz: " + sz);
    }
      for (; j < first_index_graphs[g] + sz; j++) {
        for (int k = 0; k < num_points[g]; k++) {
          
          if (is_xvy) {
            j++;  // preadd to handle the x-case, which is not used in yvt graphs
          } 
          
          
          
            if (debug_update_extremes) {
              println("update_extremes:" + (is_xvy? "xvy:" : "xvt:") + "\tj: " + j + "  k: " + k);

              if (needs_calc[0]) println("update_extremes:" + (is_xvy? "xvy:" : "xvt:") + "\tneeds_calc[0]" + " " + data[k][j-1][1] + " < " + extreme_graphs___[g][0] + "?");
              if (needs_calc[1]) println("update_extremes:" + (is_xvy? "xvy:" : "xvt:") + "\tneeds_calc[1]" + " " + data[k][j-1][1] + " < " + extreme_graphs___[g][1]+ "?");
              if (needs_calc[2]) println("update_extremes:" + (is_xvy? "xvy:" : "xvt:") + "\tneeds_calc[2]" + " " + data[k][j][1] + " < " + extreme_graphs___[g][2]+ "?");
              if (needs_calc[3]) println("update_extremes:" + (is_xvy? "xvy:" : "xvt:") + "\tneeds_calc[3]" + " " + data[k][j][1] + " > " + extreme_graphs___[g][3]+ "?");
              
            }
                   if (is_xvy && needs_calc[0] && data[g][k][j-1][1]  < extreme_graphs___[g][0])  {extreme_graphs___[g][0] = data[g][k][j-1][1];    extreme_graphs_c_[g][0] = 0; if (debug_update_extremes) println("set 0 to: " + extreme_graphs___[g][0]);} 
              else if (is_xvy && needs_calc[1] && data[g][k][j-1][1]  > extreme_graphs___[g][1])  {extreme_graphs___[g][1] = data[g][k][j-1][1];    extreme_graphs_c_[g][1] = 0; if (debug_update_extremes) println("set 1 to: " + extreme_graphs___[g][1]);}
                   if (          needs_calc[2] && data[g][k][j][1]    < extreme_graphs___[g][2])  {extreme_graphs___[g][2] = data[g][k][j][1];      extreme_graphs_c_[g][2] = 0; if (debug_update_extremes) println("set 2 to: " + extreme_graphs___[g][2]);}
              else if (          needs_calc[3] && data[g][k][j][1]    > extreme_graphs___[g][3])  {extreme_graphs___[g][3] = data[g][k][j][1];      extreme_graphs_c_[g][3] = 0; if (debug_update_extremes) println("set 3 to: " + extreme_graphs___[g][3]);}       
        }
      }
  }
}


void advance_or_reset(int[] pos_x, int g) {
          // Advance pos_x and rollback pos_x if exceeds max points for specific graph
          
          if (debug_advance_or_reset)
            println("adv_or_reset:" + (xvy[g] ? "\txvy:" : "\txvt:") + "\tpos_x from " + pos_x[g] + " to " +  (pos_x[g] + 1));
          pos_x[g]++;
          if (pos_x[g] >= num_points[g]) {
            pos_x[g] = 0;
          if (debug_advance_or_reset)
            println("adv_or_reset:" + (xvy[g] ? "\txvy:" : "\txvt:") + "\tpos_x to 0");
          }
}

void serialEvent(Serial ser) {
  // Listen for serial data until #, the end of transmission key
    
    String temp = ser.readStringUntil(OUTER_KEY);
    String[] array_main = temp.split("\n");
    String temp2 = temp.replaceAll("\n"," ").replaceAll("\r"," ").replaceAll("\f"," ");
    
    if (array_main[0].length() <= 1)
      return;
    

    if (debug_serial) {
      println("serialEvent:\t\t\tpacket " + number_of_data_points + ":  >>>" + temp2 + "<<<");
    }
    // ********************************************************* //
    // ************* PLOT SETUP FROM CONFIG CODE *************** //
    // ********************************************************* //
    
    // If config code has changed, need to go through setup again
    if (!config_code.equals(array_main[0])) {
      
      if (debug_serial) {
        println("serialEvent:\tconfig_code is: " + config_code + " and array_main[0] is: " + array_main[0]);
        println("serialEvent:\tRUNNING setup_serial(...)");
      }
      setup_serial(array_main);

    } else {
      // Matching a code means we have configured correctly
      configured = true;
    
      // *********************************************************** //
      // ************ NORMAL PLOTTING FUNCTIONALITY **************** //
      // *********************************************************** //
      int temp_time = millis() - last_config;
      
      for (int g = 0; g < num_graphs; g++) {

      	String[] array_sub = array_main[g+1].split(INNER_KEY);
      
        int p = 5;
        

      	if (xvy[g]) {
      	  // Plot x vs y graph
      	  
          check_for_new_extremes(array_sub, g, temp_time, p, xvy[g]);
      	  
      	  // Check for extremes going out of scope, to need a full new max/min calc
          boolean[] needs_calc = check_if_needs_calc(g,xvy[g]);
          
          update_extremes(needs_calc, g, xvy[g]);
         
          advance_or_reset(pos_x, g);

          if (debug_serial) {
            //println("serialEvent:\txvy:\t" + "data[" + pos_x[g] + "][" + first_index_graphs[g] + "][0] is: " + data[pos_x[g]][first_index_graphs[g]][0]);
            println("serialEvent:\txvy:\t" + "g: " + g);
            println("serialEvent:\txvy:\t" + "temp_time: " + temp_time);
            //println("serialEvent:\txvy:\t" + "pos_x[e_g]: " + pos_x[g]);
            println("serialEvent:\txvy:\t" + "num_points[e_g]: " + num_points[g]);
            println("serialEvent:\txvy:\t" + "f_i_g[each]: " + first_index_graphs[g]);
            println("serialEvent:\txvy:\t" + "extreme_graphs___[g][0] (before): " + extreme_graphs___[g][0]);
            
          }
      	} else {
      	  // TIME GRAPH HANDLER

          //FIXME
          check_for_new_extremes(array_sub, g, temp_time, p, xvy[g]);
      
          // Check for extremes going out of scope, to need a full new max/min calc
          boolean[] needs_calc = check_if_needs_calc(g,xvy[g]);

          update_extremes(needs_calc, g, xvy[g]);
          
      	  // Max timestamp of graph will be now
      	  extreme_graphs___[g][1] = temp_time;
      	  
          advance_or_reset(pos_x, g);
      	  
          if (debug_serial) {
            println("serialEvent:\txvt:\t" + "data[" + g + "][" + pos_x[g] + "][" + first_index_graphs[g] + "][0] is: " + data[g][pos_x[g]][first_index_graphs[g]][0]);
            println("serialEvent:\txvt:\t" + "g: " + g);
            println("serialEvent:\txvt:\t" + "temp_time: " + temp_time);
            println("serialEvent:\txvt:\t" + "pos_x[e_g]: " + pos_x[g]);
            println("serialEvent:\txvt:\t" + "num_points[e_g]: " + num_points[g]);
            println("serialEvent:\txvt:\t" + "f_i_g[each]: " + first_index_graphs[g]);
            println("serialEvent:\txvt:\t" + "extreme_graphs___[g][0] (before): " + extreme_graphs___[g][0]);
            
          }
      	  // Min timestamp will be whatever this new pos_x has (about to be replaced)

          if (data[g][pos_x[g]][first_index_graphs[g]][0] != -0.0) { extreme_graphs___[g][0] =                            data[g][pos_x[g]][first_index_graphs[g]][0]; } 
          else {                                                     extreme_graphs___[g][0] = temp_time - ((temp_time -  data[g][pos_x[g]][first_index_graphs[g]][0]) / pos_x[g]) * num_points[g]; }

      	} // end xt graph
        if (debug_serial) {
            println("---------------------------------------------");
        }
      }
          number_of_data_points++;
          waiting_for_packet = false;
          waiting_for_draw = true;
          

    }
}


// Helper method to calculate bounds of graphs
void setupGraphPosition() {
  // Determine orientation of each graph      
  int num_high = 1;
  int num_wide = 1;
  // Increase num subsections in each direction until all graphs can fit
  while (num_high * num_wide < num_graphs) {
    if (num_wide > num_high) {
      num_high++;
    } else if (num_high > num_wide) {
      num_wide++;
    } else if (height >= width ) {
      num_high++;
    } else {
      // Want to increase in width first
      num_wide++;
    }	
  }
 
 // Set bounding box for each subsection
  pos_graphs = new float[num_graphs][2];
  int k = 0; // k tracks overall graph index
  sub_width = round(w/num_wide);
  sub_height = round(h/num_high);
  for (int i = 0; i < num_high; i++) {
    for (int j = 0; j < num_wide; j++) {
      if (k < num_graphs) {
      	pos_graphs[k][0] = j*sub_width + MARGIN_SZ/2;
      	pos_graphs[k][1] = i*sub_height + MARGIN_SZ/2;
      }
      k++;
    }
  }
  sub_width -= MARGIN_SZ;
  sub_height -= MARGIN_SZ;

}









///////////////////////////////////////////  DEBUG AND DUMP
///////////////////////////////////////////


void dump_all() {
  dump_scalars();
  dump_titles();
  dump_xvy();
  dump_f_i_g();
  dump_sz_graphs();
  dump_num_points();
  dump_pos_x();
  dump_pos_graphs();
  dump_data();
  println("----------------------------------------------------------");
  
}

void dump_scalars() {
  println("\th:\t\t\t\t\t" + h + "\n\tsub_h:\t\t\t\t" + sub_height + "\n\tw:\t\t\t\t\t" + w + "\n\tsub_w:\t\t\t\t" + sub_width);
  println("num_graphs:\t\t\t\t" + num_graphs + "\ntotal vars:\t\t\t\t" + total_vars + "\nmax points:\t\t\t\t" + max_points);
  println("number_of_data_points:\t" + number_of_data_points);
  
}

void dump_titles() {
  for(int g = 0; g < num_graphs; g++) println("titles" + "[" + g + "]:\t\ttitle:\t" + titles[g]);
}

void dump_xvy() {
  for(int g = 0; g < num_graphs; g++) println("xvy" + "[" + g + "]:" + "\t\tis xvy?:\t" + xvy[g]);
}

void dump_f_i_g() {
  for(int g = 0; g < num_graphs; g++) println("first_index_graphs" + "[" + g + "]:\t" + first_index_graphs[g]);
}

void dump_sz_graphs() {
  for(int g = 0; g < num_graphs; g++) println("sz_graphs" + "[" + g + "]:\t\t\t" +  sz_graphs[g]);
}

void dump_num_points() {
  for(int g = 0; g < num_graphs; g++) println("num_points" + "[" + g + "]:\t\t\t" +  num_points[g]);
}

void dump_pos_x() {
  for(int g = 0; g < num_graphs; g++) println("pos_x" + "[" + g + "]:\t\t\t\t" + pos_x[g]);
}

void dump_pos_graphs() {
  for(int g = 0; g < num_graphs; g++) println("pos_graphs" + "[" + g + "]" + "\t\t\tx: " + pos_graphs[g][0] + "\ty: " + pos_graphs[g][1]);
}

void dump_extreme_graphs___(int g) {
  println("extreme_graphs___" + "[" + g + "]" + " =" + "\tx_min:\t" + extreme_graphs___[g][0] + "\tx_max:\t" + extreme_graphs___[g][1] + "\ty_min:\t" + extreme_graphs___[g][2]+ "\ty_max:\t" + extreme_graphs___[g][3]);
}

void dump_extreme_graphs_c_(int g) {
  println("extreme_graphs_c_" + "[" + g + "]" + " =" + "\tx_min:\t" + extreme_graphs_c_[g][0] + "\tx_max:\t" + extreme_graphs_c_[g][1] + "\ty_min:\t" + extreme_graphs_c_[g][2] + "\ty_max:\t" + extreme_graphs_c_[g][3]);
}

void dump_labels(int g) {
  for (int a = first_index_graphs[g] ; a < first_index_graphs[g] + sz_graphs[g]; a++) { println("labels" + "[" + g + "]["  + a + "]:" + labels[g][a]); }
}

void dump_data() {
  for (int g = 0; g < num_graphs; g++) {
    println("----------------------------------------------------------");
    dump_extreme_graphs_c_(g);
    dump_extreme_graphs___(g);
    dump_labels(g);
      // this is reversed, because it shows the sequence of sequential points better.
      for (int v = 0; v < total_vars; v++) {
        for (int p = 0; p < num_points[g]; p++) {
        if (data[g][p][v][0] != -0.0)
          println("data[" + g + "][" + p + "][" + v + "][" + 0 + "]:\t\t\t" + data[g][p][v][0]);
        if (data[g][p][v][1] != -0.0)
          println("data[" + g + "][" + p + "][" + v + "][" + 1 + "]:\t\t\t" + data[g][p][v][1]);
      }
    }
  }
}




boolean debug = true;
boolean debug_data = debug && false;
boolean debug_graphing = debug && false;
boolean debug_serial = debug && false;
boolean debug_startup = debug && false;
boolean debug_extremes = debug && false;
boolean debug_update_extremes = debug && false;
boolean debug_check_if_needs_calc = debug && false;
boolean debug_plot = debug && false;
boolean debug_advance_or_reset = debug && false;
boolean debug_check_for_new_extremes = debug && false;
boolean debug_plot_time = debug && false;
boolean debug_plot_xy = debug && false;

I apologize about the bizzare spacing and alignment: I tend to do that, and it's partially how I found some of the bugs and refactoring opportunities.

Regardless of the state of bugs, I have done some pretty big refactoring:

Shared code from plot_xy and plot_time has been re-factored out:

• plot_background is identical for both cases, so it was pulled out

serialEvent has been broken up:

• setup_serial (and the associated try block)
• check_if_needs_calc (this has some xvy dependent code, but otherwise is identical to the original)
• advance_or_reset (this is identical to the original)
• check_for_new_extremes (this is different in a few ways...)
  • first, I set data[][][0] in all cases, which doesn't affect anything.
  • then, I follow the (necessary, it turns out) conditional on is_xvy to either pre- or post- increment p.
  • this allows me to use p-5 and p-4 for both graph cases
• update_extremes (subtle and important changes)
  • I switched the style of one of the two graphs to be index by [j-1] and [j] instead of [j] and [j+1]. I also pre-increment j to handle the xvy case.
  • I brought the for loop from the time graph to both, but skew the xvy graph with sz=0 so that the loop has no effect in xvy case. (this may need to be fixed, actually)

Overall, I have already dramatically increased code re-use. serialEvent is much simpler overall, now.

There is also a ton of unnecessary debugging stuff going on, but I haven't removed it yet because I haven't fixed all of the bugs.

See these lines.

Notice that the original code implies that we should print all of the points, regardless of their state (initialized or not).

In fact, you can see the dramatic effect this has if you simply initialize data[][][] with random data, as I have done here.

If we use target instead, then we would only display values who have been filled in (except there is still a bug which incorrectly fills in data points 0 and 1, which I have yet to quash).

Here is an example of the randomized data being shown. There should only be 5 data points in this graph total.

Note: There is another bug visible here, where the y-axis units are a data value instead of their actual text. I'm not sure why this happens, yet.

This is what that should have looked like (this is leaving the randomization in, but switching the definition of target):

Note: these graphs are microvolts versus time, but I am using AddXYGraph, because I originally wanted to avoid some problems with time scaling. So the time numbers are reported in milliseconds (microseconds scaled up? I forget) from the arduino.

Hi Drew, thanks for reaching out.

As you noticed, I did do a huge overhaul of the library itself recently in favor of simpler usage and templated type support. The listener was definitely due up next for a refactor, so it's awesome to see you are already working on it.

The general code consolidation looks good, and I like the breakdown of the serial event function.

On bugs

• Good catch on the uninitialized values being plotted.
• I'm also looking into why the graph labels sometimes get filled with unexpected characters or values. (I'm looking for a more elegant solution than just brute force parsing the labels on every serial event)
• Can you elaborate a bit on the issue you were having with the time scale?

Moving forward, if you forked the repo and worked in that, it may be easier to keep in-sync on this effort

Thanks for the work on this,
Devin

Devin:

I managed to get some more work done. I have done so many things, it's hard to remember.

I have added:

• a ton of dump functions
• a ton of debug lines
• changed data to be indexed by graph id
• changed labels to be indexed by graph id
• changed initialization routines to set values to -0.0 (float)
• I fixed things that involved cross-contamination (indexing failures that caused one graph to be updated with data from another)
• I fixed issues where time graphs would disappear (background/ticks would, at least) when another graph went over max_points.
• I fixed issues appearing from small number of points (before, 3 and 2 both had problems -- now only a 1 point graph has problems).

I'm sure there are other changes, I've totally lost track of them.

I am not sure if the problem of spurrious labels is solved or not. I did weed out most cases by:

• a check if array_main[0].length <= 1, which abandons certain kinds of serial packet malformations (that occurred often with me)

I have updated the gist. At this point, I would be looking to add features that I need:

• To overcome float representation restrictions, I need to be able to add a 'scale' configuration option to graphs so that I can send values which are pre-scaled (by, say, 10000), so that I can avoid some stupid float restriction. For example, if I just send values like 07.4990 (Volts), I will get 7.49 (or something similar) sent to the client. So instead, I send a value pre-scaled by 1000, 74990.0, and I'm currently just dropping down in units (so, instead of volts, I would send millivolts here). Obviously this is silly, so I would want the client to divide the result by 1000 and display that.
• I would probably want to add the ability to define both dimensions and units, so that I could still display data when my multimeter is autoranging. I could just reset and display a new graph when the range changes (and I will be doing this anyway, at the beginning). So obviously going from the 1 Volts range to any Amps range should reset the data (I would remove and then add a new graph here). But going from milliVolts to Volts might represent the same measurement, and should just change scale on the graph.

However, I think I might hack out most (or all?) of the graphing code and use Grafica. It has some very pretty graphs and includes graphs which can be panned-and-scanned, and probably can be zoomed in our out, which would be awesome features to have.

So let me know what you want me to do. I can give you the code changes I made minus all

• variable renames
• debug statements
• dump code

So basically it will only have refactoring and bug fixes.

Or I can just fork re-commit a history of how I got to where I am now, but with logical commits (variable renames only, add dump code only, add debug statements only, bug fixes one by one, etc).

Let me know what you would prefer.

There are still oustanding issues. For instance, for some reason my time axis starts out negative, so weird things happen there. This is directly related to incorrect handling/usage of extremes_graph[g][0], which goes negative and then whose value gets amplified by some nonsense. Debugging for all of these problems is available (along with many more things).

Here is a screenshot. The first and second graphs are the same data, reported in x_y and then x_t modes.

The third is your sine-wave graph thing.

Note this is after the total number of points had been exceeded (hence why the time axes are all positive numbers). Up until that point, they were negative and then positive.

Drew,
Thanks for the update. Looks like a lot of good fixes and improvements in here and I definitely want to start merging these changes into master.

I am not super concerned about a thorough commit history but it I do want to break this down into multiple pull requests. There are a lot of changes here and I think it would be easier to merge them in as smaller logical chunks (ie. code consolidation, drawing bug fixes, data initialization bug fix, etc. )

A few specific notes:

• On the scaling factor, I'm wondering what limitation you are hitting. Data is sent on the serial port as a double, then stored as a double in the listener application. The number 74990.0 should be handled fine, otherwise that's a type-casting bug somewhere.
• I am actually very interested in the idea of using a scaling system for reducing data transfer costs though... I can't think of a reason we couldn't get away with transferring data via integer with a single scaling factor per packet instead of by double...
• On dimensions and ranging, if your multimeter switches from millivolts to volts, scaling will be fixed as soon as the last millivolts measurement rolls off the screen and graph extremes are recalculated. If the multimeter is regularly alternating between millivolts and volts, I would argue the user should be responsible for handling that
• On negative time values, this is as-designed. When a graph is first initialized, time-zero falls on the right edge of the rolling graph, meaning the rest of the axis is shown as negative. I'm not opposed to displaying any negative value as "0.0" though

Thanks again

Hi Drew - hope all is well.

Just wanted to check in on this quickly. Planning on overhauling the listener over the next few weeks and would like to get your pulls requests merged in before I start making more changes.

Thanks,
Devin