Skip to content

Instantly share code, notes, and snippets.

@phrb

phrb/reconfig_review_workplan.md

Last active October 20, 2017 22:09
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save phrb/3f6af593ef799919a5b19dfdf2e4176c to your computer and use it in GitHub Desktop.
Save phrb/3f6af593ef799919a5b19dfdf2e4176c to your computer and use it in GitHub Desktop.
Download ZIP
Raw
reconfig_review_workplan.md

ReConFig 2017 Paper Revision Work Plan

Reviewer 1

  1. Describe in more detail if you just used OpenTuner or if you extended the tool and did significant programming.

We mentioned we just used OpenTuner. To make it more clear, add a sentence explaining we just used the tool, or that we did not extend it. (autotuner description section) DONE

  1. Go into more details on how you connected OpenTuner to LegUp and Quartus. Did you analyze the report (ASCII) files and search for these metrics?

We can explain that OpenTuner calls LegUp and Quartus and we obtain metrics from output files. (autotuner description section) DONE

  1. Table II does not give the exact weights, just High/Medium/Low. Why?

The exact weights are in the legend. The number will not fit in the cell, should we decrease font? (make the caption bold) DONE

  1. Why is the "performance" scenario only optimizing FMax, not Cycles?

We need to ask Sai, but the scenario is also optimizing Cycles, just with a smaller weight. Add a setence such as: "We wanted to find weights for all metrics (...metrics list...) that made sense in a global optimization scenario, because OpenTuner considers a single value for optimization (fitness)."

Reviewer 2

  1. Besides the experiments and results, the real contributions of this paper are not fully clear.

Make more clear that the methodology and approach are also contributions, and that our code is open and usable. Add a sentence to the introduction: "We showed that autotuning is useful in different scenarios, even when the starting point provided by an expert configuration" DONE

  1. I would expect contributions in terms of the autotuning algorithms, in terms of the framework or approach.

Add a sentence to introduction/abstract: We wrote the problem as an autotuning problem. To do that we had to solve the problem of combining metrics and determining the search space.

  1. The authors don’t reveal the execution time of the autotuning process and the number of configurations evaluated to achieve the best results reported in the paper.

Add clarifications to the introduction: We do reveal the time it took. How can we make that more clear? We have considered that the number of configurations was irrelevant so far, only focusing on the time, but we test around 30 configurations. DONE

  1. How did you verify the functionality of the different designs? By using an HDL simulator?

Add to abstract: Yes, by using Quartus. We mention that in the paper, how can we make it more clear? DONE

  1. What could you say in terms of the efficiency of the autotuning schemes used?

Maybe add a sentence highlighting that the speedups achieved in 1.5h are worth using this approach. We are using very simple applications, so we left aside the discussion about the efficiency of the autotuner. We could mention that analysis as future work.

Reviewer 3

  1. You dont show what exactly your scientific contribution is. Can you clarify it?

Our contribution is the validation of an autotuning methodology for HLS for FPGAs. We could improve the clarity of the sentences of the introduction and conclusion that mention the contributions. DONE

Reviewer 4

  1. Do the authors plan to use their method for real applications?

Add to future work: Yes. We can add that to future work. DONE

  1. The authors mentioned "Values of “–” mean that the Default start could not find a set of HLS parameters that produced a valid output during any of the 1.5h tuning runs." Does it mean that the application is not synthesized ? It needs more explanation and why this happened in that case?

Improve description of "-" cases. No, it means that the application was not "compiled" to verilog. We do not know why the applications failed, it can be related to LegUp's functioning and we initially considered it was out of the scope of our experiments.

  1. In the conclusion, the authors said "The autotuner decreased WNS by 7% on average and up to 16% in the Balanced scenario ..." As far as I understood this sentence is partially correct because this percentage of improvement is due to the characteristics of the application and neither because of the scenario nor the autotuner. So it is better to mention this percentage by linking it to which application.

Add to results: Instead of showing the average, use the range (from x% to y%), because the average of application decreases has no meaning on this case.

The numbers presented are averages for all applications and the maximum decrease in WNS for the balanced scenario. Since we wanted to measure the performance of the autotuner in the entire benchmark, we did not focus on specific applications.

  1. For the figures, it is not clear when one metric is improved then what will happen to other metrics in the same case because they could have values that may cause to refuse that combination of HLS parameters. To be more clear, The authors said " Regarding individual metrics, the values for FMax were worse overall, with 14% smaller FMax in gsm and 62% greater Cycles, ..... with 73% and 72% smaller DSP for dfmul and dfdiv respectively, 33% smaller Blocks and BRAM in dfdiv and smaller values overall for Regs and LUTs. " So this means for dfdiv application, when we succeeded to improve DSP by 72 % then we also got 62% greater for Cycles. Then, is it acceptable for the designer a solution like that?

Add to discussions: "Selecting different weights according to the optimization objective is acceptable to the designer."

All the metrics are optimized at the same time, so the decreases are simultaneous. The designer can choose which metric has the greatest weight in their specific application. DONE

  1. In general, the authors did not discuss the experimental results in the highlight of the application because we are not comparing just scenarios but we are comparing them with respect to the application. We could find one application like adpcm is doing better for area in the balanced scenario than in the area scenario (LUT=0.73 in balanced while 0.78 in area scenario and for Reg =0.91 in balanced while 0.96 in area scenario)

There is no space for this extended discussion. DONE

  1. "achieved smaller absolute values that the Random start." ==> achieved smaller absolute values than the Random start.

Fix that. DONE

  1. Check for the references to be written in upper case for some words like FPGA as in numbers 4, 22, 27, 28

Fix that. DONE

@dmilojicic
Copy link

Excellent job, gentlemen!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment