%Minor Revision - TUFFC-09469-2019
%Transactions on Ultrasonics, Ferroelectrics, and Frequency
%Control (July 23, 2019 9:29 PM)
%To: arthur.hugeat@femto-st.fr, julien.bernard@femto-st.fr,
%gwenhael.goavec@femto-st.fr, pyb2@femto-st.fr, pierre-yves.bourgeois@femto-st.fr,
%jmfriedt@femto-st.fr
%CC: giorgio.santarelli@institutoptique.fr, lewin@ece.drexel.edu
%
%Dear Mr. Arthur HUGEAT
%
%Congratulations! Your manuscript
%
%MANUSCRIPT NO. TUFFC-09469-2019
%MANUSCRIPT TYPE: Papers
%TITLE: Filter optimization for real time digital processing of radiofrequency
%signals: application to oscillator metrology
%AUTHOR(S): HUGEAT, Arthur; BERNARD, Julien; Goavec-Mérou, Gwenhaël; Bourgeois,
%Pierre-Yves; Friedt, Jean-Michel
%
%has been reviewed and it has been suggested that it be accepted for publication
%after minor revisions. In your revision, you must respond to the reviewer’s
%comments at the end of this e-mail or attached.
%
%Your revised manuscript must be submitted within the next THREE WEEKS. If you
%are not able to submit your manuscript in this time frame, you must contact the
%Editor in Chief (Peter Lewin, lewinpa@drexel.edu).
%
%Please resubmit your revised manuscript to the Transactions on Ultrasonics,
%Ferroelectrics, and Frequency Control Manuscript Central website at
%http://mc.manuscriptcentral.com/tuffc-ieee. From the “Author Center” select
%“Manuscripts with Decisions” and under the appropriate manuscript ID select
%“create a revision”.
%
%To expedite the review of your resubmission:
%
%(1) Include or attach a point by point response to reviewer’s comments and
%detail all changes made in your manuscript under “Response to Decision Letter”.
%Failure to address reviewers comments can still lead to a rejection of your
%manuscript.
%(2) Submit a PDF of the revised manuscript using the “Formatted (Double Column)
%Main File - PDF Document Only” file type with all changes highlighted in yellow
%under “File Upload”.
%(3) Original TeX, LaTeX, or Microsoft Word file of the final manuscript as
%Supporting Document.
%(4) High quality source files of your figures in Word, Tiff, Postscript,
%EPS, Excel or Power Point (if figures are not already embedded in your source
%file above) as Supporting Document.
%(5) Author photos and biographies (papers only) as Supporting Document.
%(6) Graphical Abstract to accompany your text abstract on IEEE Xplore (image,
%animation, movie, or audio clip) uploaded as Multimedia.
%
%*Please make sure that all final files have unique file names in order for
%them to be processed correctly by IEEE*
%Please note that a PDF is NOT sufficient for publication, the PDF is used
%for review.
%
%During the resubmission process if you do not see a confirmation screen and
%receive a confirmation e-mail, your revised manuscript was not transmitted
%to us and we will not be able to continue to process your manuscript.
%
%Please refer to the policies regarding the voluntary page charges and
%mandatory page charges in the "Guideline for Authors" at
%http://ieee-uffc.org/publications/transactions-on-uffc/information-for-authors
%Note over-length charge of US$175 per page is applied for published pages in
%excess of 8 pages.
%
%Sincerely,
%
%Giorgio Santarelli
%Associate Editor in Chief
%Transactions on Ultrasonics, Ferroelectrics, and Frequency Control
%
%****************************************************
%REVIEWERS' COMMENTS:

\documentclass[a4paper]{article}
\usepackage{fullpage,graphicx,amsmath, subcaption}
\begin{document}
{\bf Reviewer: 1}

%Comments to the Author
%In general, the language/grammar is adequate.

{\bf
On page 2,  "...allowing to save processing resource..." could be improved.       % r1.1 - fait
}

The sentence was split and now reads ``number of coefficients irrelevant: processing
resources are hence saved by shrinking the filter length.''

{\bf
On page 2, "... or thanks at a radiofrequency-grade..." isn't at all clear what   % r1.2 - fait
the author meant.}

Grammatical error: this sentence now reads ``or by sampling a wideband (125~MS/s)
Analog to Digital Converter (ADC) loaded by a 50~$\Omega$ resistor.''

{\bf
On page 2, the whole paragraph "The first step of our approach is to model..."   % r1.3 - fait
could be improved.
}

Indeed this paragraph has be written again and now reads as\\
``The first step of our approach is to model the DSP chain. Since we aim at only optimizing
the filtering part of the signal processing chain, we have not included the PRN generator or the
ADC in the model: the input data size and rate are considered fixed and defined by the hardware.
The filtering can be done in two ways, either by considering a single monolithic FIR filter
requiring many coefficients to reach the targeted noise rejection ratio, or by
cascading multiple FIR filters, each with fewer coefficients than found in the monolithic filter.
''

{\bf
I appreciate that the authors attempted and document two optimizations: that      % r1.4 - fait
of maximum rejection ratio at fixed silicon area, as well as minimum silicon
area for a fixed minimum rejection ratio. For non-experts, it might be very
useful to compare the results of both optimization paths to the performance and
resource-utilization of generic low-pass filter gateware offered by device
manufacturers. I appreciate also that the authors have presented source code
for examination online.
}

To compare the performance of our FIR filters and the performance of device
manufacturers generic filter, we have added a paragraph and a table at the
end of experiments section. We compare the resources consumption with the same
FIR coefficients set.

{\bf
Reviewer: 2
}

%Comments to the Author
%In the Manuscript, the Authors describe an optimization methodology for filter
%design to be used in phase noise metrology. The methodology is general and can
%be used for many aspects of the processing chain. In the Manuscript, the Authors
%focus on filtering and shifting while the other aspects, in particular decimation,
%will be considered in a future work. The optimization problem is modelled
%theoretically and then solved by means of a commercial software. The solutions
%are tested experimentally on the Redpitaya platform with synthetic and real
%white noises. Two cases are considered as a function of the number of filters:
%maximum rejection given a fixed amount of resources and minimum resource
%utilization given a fixed amount of rejection.
%The Authors find that filtering improves significantly when the number of
%filters increases.
%A lot of work has been done in generalizing and automating the procedure so
%that different approaches can be investigated quickly and efficiently. The
%results presented in the Manuscript seem to be just a case study based on
%the particular criterion chosen by the Authors. Different criteria, in
%general, could lead to different results and it is important to consider
%carefully the criterion adopted by the Authors, in order to check if it
%is adequate to compare the performance of filters and if multi-stage
%filters are really superior than monolithic filters.

{\bf
By observing the results presented in fig. 10-16, it is clear that the            % r2.1
performances of multi-stage filters are obtained at the expense of their
selectivity and, in this sense, the filters presented in these figures
are not equivalent. For example, in Fig. 14, at the limit of the pass band,
the attenuation is almost 15 dB for n = 5, while it is not noticeable for
n = 1.
}

We have added on Figs 10--16 (now Fig 9(a)--(c)) the templates used to defined
the bandpass and the bandstop of the filter.

% We are aware of this non equivalence but we think that difference is not due to
% the cascaded filters but due to the definition of rejection criterion on the passband.
% Indeed, in this article we have choose to take the summation of absolute values divide
% by the bandwidth but this criterion is maybe too permissive and when we cascade
% some filters this impact is more important.
%
% However if we change the passband
% criterion by the summation of absolute value in passband, weighting given to the
% passband ripples are too strong and the solver are too restricted to provide
% any interesting solution but the ripples in passband will be minimal. And if we take the maximum absolute value in
% passband, the rejection evaluation are too close form the original criterion and
% the result will not be improved.
%
% In this article, we will highlight the methodology instead of the filter conception.
% Even if our rejection criterion is not the best, our methodology was not impacted
% by this. So to improve the results, we can choose another criterion to be more
% selective in passband but it is not the main objective of our article.

We are aware of this equivalence but to limit this ripples in passband we need to
enforce the criterion in passband. If we takes a strong constraint like the sum of
absolute values in passband. This criterion si too selective because it considers
all bin on passband while on stopband we consider only the bin with the minimal
rejection. The figure~\ref{fig:letter_sum_criterion} exhibits the results with this
criterion for the case MAX/1000. With this criterion, the solver find an optimal
solution with only two filters in expend of the resource consumption.



If we relax a little the criterion on passband with taking only the maximum absolute
value, we will penalize the ripple peak on passband. The figure~\ref{fig:letter_max_criterion}
shows the results for the case MAX/1000. There as almost no difference with the
article results. Indeed the only little change are on the case $i = 4$ and $i = 5$
which they have some minor differences on coefficients choices.

\begin{figure}[h!tb]
  \centering
  \begin{subfigure}{0.48\linewidth}
    \includegraphics[width=\linewidth]{images/letter_sum_criterion}
    \caption{Results for the case MAX/1000 with as criterion on passband the sum absolute values}
    \label{fig:letter_sum_criterion}
  \end{subfigure}
  \begin{subfigure}{0.48\linewidth}
    \includegraphics[width=\linewidth]{images/letter_max_criterion}
    \caption{Results for the case MAX/1000 with as criterion on passband the maximum absolute value}
    \label{fig:letter_max_criterion}
  \end{subfigure}
\end{figure}

Finally, if we ponder the maximum absolute on passband, we should improve the result.
We have arbitrary pondered by 5 the maximum. Even with this weighting, the solver
choose the same coefficient set.

To conclude, find a better criterion to avoid the ripples on the passband is difficult.
In this article we are focused on the methodology so even if our criterion could
be improved, our methodology still the same and it works independently of rejection criterion.

% %Peut etre refaire une serie de simulation dans lesquelles on impose une coupure
% %non pas entre 40 et 60\% mais entre 50 et 60\% pour demontrer que l'outil s'adapte
% %au critere qu'on lui impose, et que la coupure moins raide n'est pas intrinseque
% %a la cascade de filtres.
% %AH: Je finis les corrections, je poste l'article revu et pendant ce temps j'essaie de
% %relancer des expérimentations. Si j'arrive à les finir à temps, je les intégrerai
%
% densité spectrale de la bande passante
% sum des valeurs absolues / largeur de la bande passante (1/N) vs max dans la bande de coupure
%
% JMF : il n'a pas tord, la coupure est bcp moins franche a 5 filtres qu'a 1. Ca se voyait
% moins avant de moyenner les fonctions de transfert, mais il y a bien une 15aine de dB
% quand on cascade 5 filtres !
%
% Dire que la chute n'est pas du à la casacade mais à notre critère de rejection

{\bf
The reason is in the criterion that considers the average attenuation in          % r2.2
the pass band. This criterion does not take into account the maximum attenuation
in this region, which is a very important parameter for specifying a filter
and for evaluating its performance. For example, with this criterion, a
filter with 0.1 dB of ripple is considered equivalent to a filter with
10 dB of ripple. This point has a strong impact in the optimization process
and in the results that are obtained and has to be reconsidered.
}

See above: Choose a criterion is difficult and depending on the context. The main
contribution on this paper is the methodology not the criterion to quantify the
rejection.

% The manuscript erroneously stated that we considered the mean of the absolute
% value within the bandpass: the manuscript has now been corrected to properly state
% the selected criterion, namely the {\em sum} of the absolute value, so that any
% ripple in the bandpass will reduce the chances of a given filter set from being
% selected. The manuscript now states ``Our criterion to compute the filter rejection considers
% % r2.8 et r2.2 r2.3
% the maximum magnitude within the stopband, to which the {sum of the absolute values
% within the passband is subtracted to avoid filters with excessive ripples}.''

{\bf
I strongly suggest to re-run the analysis with a criterion that takes also        % r2.3 -fait
into account the maximum allowed attenuation in pass band, for example by
fixing its value to a typical one, as it has been done for the transition
bandwidth.
}

See above: the absolute value within the passband will reject filters with
excessive ripples, including excessive attenuation, within the passband.

% TODO: test max(stopband) - max(abs(passband))

{\bf
In addition, I suggest to address the following points:                           % r2.4 - fait
- Page 1, line 50: the Authors state that IIR have shorter impulse response
than FIR. This is not true in general. The sentence should be reconsidered.
}

We have not stated that the IIR has a shorter impulse response but a shorter lag.
Indeed while a typical FIR filter will have 32 to 128~coefficients, few IIR filters
have more than 5~coefficients. Hence, while a FIR requires 128 inputs before providing
the first output, an IIR will start providing outputs only 5 time steps after the initial
input starts feeding the IIR. Hence, the issue we address here is lag and not impulse
response. We aimed at making this sentence clearer by stating that ``Since latency is not an issue
in a openloop phase noise characterization instrument, the large
numbre of taps in the FIR, as opposed to the shorter Infinite Impulse Response (IIR) filter,
is not considered as an issue as would be in a closed loop system in which lag aims at being
minimized to avoid oscillation conditions.''

{\bf
- Fig. 4: the Author should motivate in the text why it has been chosen           % r2.5 - fait
this transition bandwidth and if it is a typical requirement for phase-noise
metrology.
}

The purpose of the paper is to demonstrate how a given filter shape can be achieved by
minimizing varous resource criteria. Indeed the stopband and bandpass boundaries can
be questioned: we have selected this filter shape as a typical anti-aliasing filter considering
the the dataflow is to be halved. Hence, selecting a cutoff frequency of 40\% the initial
Nyquist frequency prevents noise from reaching baseband after decimating the dataflow by a
factor of 2. Such ideas are now stated explicitly in the text as ``Throughout this demonstration,
we arbitrarily set a bandpass of 40\% of the Nyquist frequency and a bandstop from 60\%
of the Nyquist frequency to the end of the band, as would be typically selected to prevent
aliasing before decimating the dataflow by 2. The method is however generalized to any filter
shape as long as it is defined from the initial modelling steps: Fig. \ref{fig:rejection_pyramid}
as described below is indeed unique for each filter shape.''

{\bf
- The impact of the coefficient resolution is discussed. What about the           % r2.6 - fait
resolution of the data stream? Is it fixed? If so, which value has been
used in the analysis? If not, how is it changed with respect to the
coefficient resolution?
}

We have now stated in the beginning of the document that ``we have not included the PRN generator
or the ADC in the model: the input data size and rate are considered fixed and defined by the
hardware.'' so indeed the input datastream resolution is considered as a given.

{\bf
- Page 3, line 47: the initial criterion can be omitted and, consequently,        % r2.7  - fait
Fig. 5 can be removed.
}

Juste mettre une phrase pour dire que la mean ne donnait pas de bons résultats

{\bf
- Page 3, line 55: ``maximum rejection'' is not compatible with fig. 4.             % r2.8  - fait
It should be ``minimum''
- Page e, line 55, second column: ``takin''                                       % r2.9  - fait
- Page 3, line 58: ``pessimistic'' should be replaced with ``conservative''       % r2.10 - fait
- Page 4, line 17: ``meaning'' $\rightarrow$ ``this means''                       % r2.11 - fait
}

All typos and grammatical errors have been corrected.

{\bf
- Page 4, line 10: how $p$ is chosen? Which is the criterion used to choose       % r2.12 - fait
these particular configurations? Are they chosen automatically?
}
% C'est le nombre de coefficients et un taille raisonnable
% Troncature de la pyramide

See below: we have added a better description of $p$ during the transformation explanation.
``This variable $p$ is defined by the user, and represents the number of different
set of coefficients generated (remember, we use \texttt{firls} and \texttt{fir1}
functions from GNU Octave) based on the targeted filter characteristics and implementation
assumptions (estimated number of bits defining the coefficients)''

{\bf
- Page 4, line 31: how does the delta function transform model from non-linear    % r2.13
and non-quadratic to a quadratic?}

The first model is non-quadratic but when we introduce the $p$ configurations,
we can estimate the function $F$ by computing
the rejection for each configuration, so the model become quadratic because we have
some multiplication between variables. With the definition of $\delta_{ij}$ we can
replace the multiplication between variables by multiplication with binary variable and
this one can be linearise as follow:\\
$y$ is a binary variable \\
$x$ is a real variable bounded by $X^{max}$ \\
\begin{equation*}
  m = x \times y \implies
  \left \{
  \begin{split}
    m & \geq 0 \\
    m & \leq y \times X^{max} \\
    m & \leq x \\
    m & \geq x - (1 - y) \times X^{max} \\
  \end{split}
  \right .
\end{equation*}
Gurobi does the linearization so we don't explain this step to keep the model more
simple. However, to improve the transformation explanation we have rewrote the
paragraph ``This model is non-linear and even non-quadratic...''.

% JMF : il faudra mettre une phrase qui explique, ca en lisant cette reponse dans l'article
% je ne comprends pas comment ca repond a la question
%
% AH: Je mets l'idée en français, je vais essayer de traduire ça au mieux.
%
% Le problème n'est pas linéaire car nous multiplions des variables
% entre elles. Pour y remédier, on considère que $\pi_{ij}^C$ et que $C_{ij}$ deviennent
% des constantes. On introduit donc la variable binaire $\delta_{ij}$ qui nous indique
% quel filtre est sélectionné étage par étage. Malgré cela, notre programme est encore
% quadratique car pour la contrainte~\ref{eq:areadef2}, il reste une multiplication entre
% $\delta_{ij}$ et $\pi_i^-$. Mais comme $\delta_{ij}$ est binaire, il est possible
% de linéariser cette multiplication pour peu qu'on puisse borner $\pi_i^-$. Dans notre
% cas définir la borne est facile car $\pi_i^-$ représente une taille de donnée,
% nous définission donc $0 < \pi_i^- \leq 128$ car il s'agit de la plus grande valeur
% qu'on puisse traiter. De plus nous utiliserons Gurobi qui se chargera de faire la
% linéarisation pour nous.


{\bf
- Captions of figure and tables are too minimal.                                  % r2.14
}
We have change the captions of tables and figures.

{\bf
- Figures can be grouped: fig. 10-12 can be grouped as three subplots (a, b, c)   % r2.15 - fait
of a single figure. Same for fig. 13-16.
}
We add two sub figure to group the fig.10-12 and fig. 13-16

{\bf
- Please increase the number of averages for the spectrum. Currently the noise    % r2.16 - fait
of the curves is about 20 dBpk-pk and it doesn’t allow to appreciate the
differences among the curves. I suggest to reduce the noise below 1 dBpk-pk.
}

Indeed averaging had been omitted during post-processing and figure generation: we
are grateful to the reviewer for emphasizing this point which has now been corrected. All spectra
now exhibit sub-dBpk-pl line thickness.

We believe these updates to the manuscript have improved the presentation and made clearer
some of the shortcomings of the initial draft: we are greatful to the reviewers for pointing
out these issues.

Best wishes, A. Hugeat

%In conclusion, my opinion is that the methodology presented in the Manuscript
%deserve to be published, provided that the criterion is changed according
%the indications mentioned above.
\end{document}
%****************************************************
%
%For information about the IEEE Ultrasonics, Ferroelectrics, and Frequency
%Control Society, please visit the website: http://www.ieee-uffc.org. The
%website of the Transactions on Ultrasonics, Ferroelectrics, and Frequency
%Control is at: http://ieee-uffc.org/publications/transactions-on-uffc