QUESTDB/Response_Letter/Response_Letter.tex

\documentclass[10pt]{letter}
\usepackage{UPS_letterhead,xcolor,mhchem,mathpazo,ragged2e,hyperref}
\newcommand{\alert}[1]{\textcolor{red}{#1}}
\definecolor{darkgreen}{HTML}{009900}


\begin{document}

\begin{letter}%
{To the Editors of WIREs Comput. Mol. Sci.}

\opening{Dear Editors,}

\justifying
Please find attached a revised version of the manuscript entitled 
\begin{quote}
	\textit{``QUESTDB: a database of highly-accurate excitation energies for the electronic structure community''}.
\end{quote}
We thank the reviewers for their constructive comments.
Our detailed responses to their comments can be found below.
For convenience, changes are highlighted in red in the revised version of the manuscript. 

We look forward to hearing from you.

\closing{Sincerely, the authors.}

%%% REVIEWER 1 %%%
\noindent \textbf{\large Authors' answer to Reviewer \#1}

\begin{itemize}

	\item 
	{This focus article by V\'eril et al. proposes a detailed analysis of the performances of current electronic structure methods to describe excitation energies. 
	This review is based on the authors' past and current efforts to develop a large dataset of accurate vertical excitation energies for various types of molecules, to which the result of other methods can be compared. 
	This last point is particularly crucial as the use of experimental data to benchmark electronic transitions can be quite challenging (as experimental values do not only reflect the change of electronic character but also encodes contribution from the nuclear degrees of freedom). 
	In this work, the authors summarize their detailed benchmark and introduce a very exciting web platform that allows any user to test and compare the different methods for specific properties, as they want. 
	This web platform allows moving beyond the typical benchmark proposed in articles and doing way more with the data collected. 
	I warmly welcome such a contribution in the field and recommend this work for publication in WIRES Comput. Mol. Sci.
	I have a few comments/suggestions that the authors may want to consider.}
	\\
	\alert{We thank the reviewer for his/her kind comments.}

	\item 
	{For any benchmark presentation, I believe that the limits of validity should be identified and made clear to the reader to prevent any misuse or misunderstanding. 
	In this review, I feel that some more points might need to be stressed to achieve this. 
	The comparison between methods presented here is performed for Franck-Condon (ground-state) geometries only. 
	It is then essential to stress that the ranking of the electronic structure methods (as nicely illustrated in Fig. 5) might change significantly when moving away from the Franck-Condon region. 
	Most electronic structure methods for excited states cannot provide a uniform description of potential energy surfaces, being more accurate in certain regions than in others.}
	\\
	\alert{We have added a comment in the benchmarks section.}
	
	\item 
	{The discussion is based on electronic energies. 
	While electronic energies are important, investigation of excited electronic state often requires additional quantities due to the importance played by nuclear motion and coordinates. 
	Some of this is already discussed in substance in the last paragraph of the conclusion, but offering some information about which methods can provide nuclear gradients or Hessians would be valuable for the reader.}
	\\
	\alert{A sentence has been added to clarity this point in the concluding section.}
 
	\item 
	{While many different molecules are discussed in this work, I think that a caveat about the performance of some of the methods presented for other families of molecules might be welcome. 
	(The authors mention a few different types of molecules in the introduction, but it might be interesting to stress them at the end of this work too.)}
	\\
	\alert{Another comment has been added in the concluding section.}
	
\end{itemize}

%%% REVIEWER 2 %%%
\noindent \textbf{\large Authors' answer to Reviewer \#2}

\begin{itemize}

	\item 
	{It is my pleasure to review "QUESTDB: a database of highly-accurate excitation energies for the electronic structure community" by V\'eril et al for consideration in WIREs as a Focus Article. 
	The manuscript is easy to follow and very comprehensive. 
	I have no hesitation in recommend it for publication.

	Firstly, I have to commend the authors on making their paper easy to follow. 
	I am only passingly familiar with many of the methods they test. 
	Despite this, I gained a good understanding of why they had made the choices they made, and why they should be right. 
	That is not a minor undertaking.

	The manuscript is also structured exceedingly well with a logical path from start to finish. 
	It explains why the QUEST work is important, how it was done, and what remains to be done.

	I do have a number of minor suggestions, identified below. 
	Once these are addressed, I would argue that the paper is more than ready for publication and will make an outstanding contribution to the field.}
	\\
	\alert{We would like to thank the reviewer for his/her support.}

	\item 
	{MGCDB84 by Madirossian and Head-Gordon [10.1080/00268976.2017.1333644] seems to be missing (or at least not highlighted) from the list of ground state benchmark sets despite being one of the largest.}
	\\
	\alert{The reviewer is right. The work of Madirossian and Head-Gordon is now mentioned and cited in the Introduction.}
	
	\item 
	{p6: The third paragraph is a lot to take in and should probably be split into two or three paragraphs to make reading a bit easier.}
	\\
	\alert{This paragraph has been split.}
 
	\item 
	{p7 line 47-48: The wording of the final sentence on this page makes spin-symmetry breaking sound acceptable. 
	While I don't disagree that some (perhaps most) quantum chemists would agree, for those of us on the opposing side I'd like to see a caveat (e.g. which is, unlike common wisdom about).}
	\\
	\alert{This comment has been reworded accordingly.}

	\item 
	{p10-11: Final/start paragraph of p10/p11 should probably be split into two or three.}
	\\
	\alert{These paragraphs have been split.}
	
	\item 
	{Table 1: Are the error bars in eV? This does not seem to be stated.}
	\\
	\alert{Yes, they are and it is now mentioned. Thank you for spotting this.}
	
	\item 
	{Table 2: I would suggest to use italic numbers for "unsafe" TBE so that readers can easily identify them.}
	\\
	\alert{There is already a label Y/N to identify the unsafe TBEs, so we would prefer to leave it as it is.
	Moreover, these data can be also extracted from the website (there is a box to select only the safe TBEs) or the excel spreadsheet provided as supplementary material.}
		
	\item 
	{Table 4: This is purely a matter of taste and 100\% optional given that the authors provide online access to data. 
	But, I think readers might benefit from having the table reported as two horizontal panels, with one reporting up to CC3 and the other the rest. 
	The reason being that many people now read on a screen, where rotation is more annoying than on paper.}
	\\
	\alert{Again, these data are also reported as supplementary material, so we propose to leave Table 4 as it is.}
		
	\item 
	{Figure 5: This is an excellent figure!}
	\\
	\alert{Thank you!}
		
	\item 
	{p27: the second full paragraph should probable by split in two or three (yes, this is a theme).}
	\\
	\alert{Not a problem. This paragraph has been split.}
		
	\item 
	{Finally, on a technical note I suspect that the authors may need to test some more (lower quality) methods to get sufficient statistical accuracy for dieting. 
	The distributions in Figure 5 suggest quite a lot of correlation between the lower accuracy methods tested so far (e.g. CIS(D), CC2 and ADC(2) have very similar error distributions).}
	\\
	\alert{Yes, we are aware of this. We would like to thank the reviewer for pointing this out.}
%	We have added a sentence at the end of the manuscript to mention it.}	
\end{itemize}
 
\end{letter}
\end{document}