Following our previous work focussing on compounds containing up to 3 non-hydrogen atoms [\emph{J. Chem. Theory Comput.}{\bfseries 14} (2018) 4360--4379], we present here highly-accurate vertical transition energies
diacetylene, furan, glyoxal, imidazole, isobutene, methylenecyclopropene, propynal, pyrazine, pyridazine, pyridine, pyrimidine, pyrrole, tetrazine, thioacetone, thiophene, thiopropynal, and triazine. To obtain these energies, we
use equation-of-motion coupled cluster theory up to the highest technically possible excitation order for these systems (CC3, EOM-CCSDT, and EOM-CCSDTQ), selected configuration interaction (SCI) calculations (with tens
of millions of determinants in the reference space), as well as the multiconfigurational $n$-electron valence state perturbation theory (NEVPT2) method. All these approaches are applied in combination with diffuse-containing
atomic basis sets. For all transitions, we report at least CC3/\emph{aug}-cc-pVQZ vertical excitation energies as well as CC3/\emph{aug}-cc-pVTZ oscillator strengths for each dipole-allowed transition. We show that CC3
almost systematically delivers transition energies in agreement with higher-level methods with a typical deviation of $\pm0.04$ eV, except for transitions with a dominant double excitation character where the error is much larger.
The present contribution gathers a large, diverse and accurate set of more than 200 highly-accurate transition energies for states of various natures (valence, Rydberg, singlet, triplet, $n \ra\pis$, $\pi\ra\pis$\ldots).
We use this series of theoretical best estimates to benchmark a series of popular methods for excited state calculations: CIS(D), ADC(2), CC2, STEOM-CCSD, EOM-CCSD, CCSDR(3), CCSDT-3, and CC3. The results
of these benchmarks are compared to the available literature data.
Accurately describing transition energies between the electronic ground state (GS) and excited states (ES) remains an important challenge in quantum chemistry. When dealing with large compounds in complex environments,
one is typically limited to the use of time-dependent density-functional theory (TD-DFT), \cite{Cas95,Ulr12b,Ada13a} a successful yet far from flawless approach. In particular, to perform TD-DFT calculations, one must
choose an ``appropriate'' exchange-correlation functional, which is difficult yet primordial as the impact of the exchange-correlation functional is exacerbated within TD-DFT as compared to DFT. \cite{Lau13} Such selection
can, of course, rely on the intrinsic features of the various exchange-correlation functional families, \eg, it is well-known that range-separated hybrids provide a more physically-sound description of long-range charge-transfer
transitions than semi-local exchange-correlation functionals. \cite{Dre04,Pea08} However, to obtain a quantitative assessment of the accuracy that can be expected from TD-DFT calculations, benchmarks cannot be avoided.
This is why so many assessments of TD-DFT performance for various ES properties are available. \cite{Lau13}
While several of these benchmarks rely on experimental data as reference (typically band shapes \cite{Die04,Die04b,Avi13,Cha13,Lat15b,Mun15,Vaz15,San16b} or 0-0 energies
\cite{Die04b,Goe10a,Jac12d,Chi13b,Win13,Fan14b,Jac14a,Jac15b,Loo19b}), using theoretical best estimates (TBE) based on state-of-the-art computational methods \cite{Sch08,Sau09,Sil10b,Sil10c,Sch17,Loo18a}
are advantageous as they allow comparisons on a perfectly equal footing (same geometry, vertical transitions, no environmental effects, etc). In such a case, the challenge is in fact to obtain accurate TBE, as the
needed top-notch theoretical models generally come with a dreadful scaling with system size and, in addition, typically require large atomic basis sets to deliver transition energies close to the complete basis set (CBS) limit. \cite{Gin19}
More than 20 years ago, Serrano-Andr\`es, Roos, and collaborators compiled an impressive series of reference transition energies for several typical conjugated organic molecules (butadiene, furan, pyrrole, tetrazine, \ldots).
\cite{Ful92,Ser93,Ser93b,Ser93c,Lor95b,Mer96,Mer96b,Roo96,Ser96b} To this end, they relied on experimental GS geometries and the complete-active-space second-order perturbation theory ({\CASPT}) approach with the largest
active spaces and basis sets one could dream of at the time. These {\CASPT} values were later used to assess the performance of TD-DFT combined with various exchange-correlation functionals, \cite{Toz99b,Bur02} and remained for
a long time the best theoretical references available on the market. However, beyond comparisons with experiments, which are always challenging when computing vertical transition energies, \cite{San16b} there was no approach
available at that time to ascertain the accuracy of these transition energies. Nowadays, it is common knowledge that CASPT2 has the tendency of underestimating vertical excitation energies in organic molecules.
A decade ago, Thiel and coworkers defined TBE for 104 singlet and 63 triplet valence ES in 28 small and medium conjugated CNOH organic molecules. \cite{Sch08,Sil10b,Sil10c} These TBE were computed on MP2/6-31G(d) structures
with several levels of theories, notably {\CASPT} and various coupled cluster (CC) variants ({\CCD}, {\CCSD}, and {\CCT}). Interestingly, while the default theoretical protocol used by Thiel and coworkers to define their
first series of TBE was {\CASPT}, \cite{Sch08} the vast majority of their most recent TBE (the so-called ``TBE-2'' in Ref.~\citenum{Sil10c}) were determined at the {\CCT} level of theory with the \emph{aug}-cc-pVTZ (aVTZ) basis set,
often using a basis set extrapolation technique. More specifically, CC3/TZVP values were corrected for basis set incompleteness errors by the difference between {\CCD}/{\AVTZ} and {\CCD}/TZVP results. \cite{Sil10b,Sil10c}
Many works exploited Thiel's TBE for assessing low-order methods,
\cite{Sil08,Goe09,Jac09c,Roh09,Sau09,Jac10c,Jac10g,Sil10,Mar11,Jac11a,Hui11,Del11,Tra11,Pev12,Dom13,Dem13,Sch13b,Voi14,Har14,Yan14b,Sau15,Pie15,Taj16,Mai16,Ris17,Dut18,Hel19,Haa20} highlighting further
their value for the electronic structure community. In contrast, the number of extensions/improvements of this original set remains quite limited. For example, K\'ann\'ar and Szalay computed, in 2014, {\CCSDT}/TZVP reference energies
for 17 singlet states of six molecules. \cite{Kan14} Three years later, the same authors reported 46 {\CCSDT}/{\AVTZ} transition energies in small compounds containing two or three non-hydrogen atoms (ethylene, acetylene, formaldehyde,
formaldimine, and formamide). \cite{Kan17}
Following the same philosophy, two years ago, we reported a set of 106 transition energies for which it was technically possible to reach the full configuration interaction (FCI) limit by performing high-order CC (up to {\CCSDTQP}) and selected
CI (SCI) calculations on {\CCT}/{\AVTZ} GS structures. \cite{Loo18a} We exploited these TBE to benchmark many ES methods. \cite{Loo18a} Among our conclusions, we found that {\CCSDTQ} yields near-{\FCI} quality excitation energies,
whereas we could not detect any significant differences between {\CCT} and {\CCSDT} transition energies, both being very accurate with mean absolute errors (MAE) as small as $0.03$ eV compared to {\FCI}.
Although these conclusions agree well with earlier studies, \cite{Wat13,Kan14,Kan17} they obviously only hold for single excitations, \ie, transitions with $\Td$ in the $80$--$100\%$ range. Therefore, we also recently proposed a set of
20 TBE for transitions exhibiting a significant double-excitation character (\ie, with $\Td$ typically below $80\%$). \cite{Loo19c} Unsurprisingly, our results clearly evidenced that the error in CC methods is intimately related to the $\Td$ value.
For example, for the ES with a significant yet not dominant double excitation character [such as the infamous $A_g$ ES of butadiene ($\Td=75\%$)] CC methods including triples deliver rather accurate estimates (MAE of $0.11$ eV with {\CCT}
and $0.06$ eV with {\CCSDT}), surprisingly outperforming second-order multi-reference schemes such as {\CASPT} or the generally robust $n$-electron valence state perturbation theory ({\NEV}). In contrast, for ES with a dominant double
excitation character, \eg, the low-lying $(n,n)\ra(\pis,\pis)$ excitation in nitrosomethane ($\Td=2\%$), single-reference methods (not including quadruples) have been found to be unsuitable with MAEs of $0.86$ and $0.42$ eV for {\CCT}
and {\CCSDT}, respectively. In this case, multireference methods are in practice required to obtain accurate results. \cite{Loo19c}
A clear limit of our 2018 work \cite{Loo18a} was the size of the compounds put together in our set. These were limited to $1$--$3$ non-hydrogen atoms, hence introducing a potential ``chemical'' bias. Therefore, we have decided, in the
present contribution, to consider larger molecules with organic compounds encompassing 4, 5, and 6 non-hydrogen atoms. For such systems, performing {\CCSDTQ} calculations with large one-electron basis sets is elusive. Moreover,
the convergence of the {\SCI} energy with respect to the number of determinants is obviously slower for these larger compounds, hence extrapolating to the {\FCI} limit with an error of $\sim0.01$ eV is rarely achievable in practice.
Consequently, the ``brute-force'' determination of {\FCI}/CBS estimates, as in our earlier work, \cite{Loo18a} is definitely out of reach here. Anticipating this problem, we have recently investigated bootstrap CBS extrapolation techniques.
\cite{Loo18a,Loo19c} In particular, we have demonstrated that, following an ONIOM-like scheme, \cite{Chu15} one can very accurately estimate such limit by correcting high-level values obtained in a small basis by the difference between
{\CCT} results obtained in a larger basis and in the same small basis.\cite{Loo18a} We globally follow such strategy here. In addition, we also perform {\NEV} calculations in an effort to check the consistency of our estimates. This is particularly
critical for ES with intermediate $\Td$ values. Using this protocol, we define a set of more than 200 \emph{aug}-cc-pVQZ reference transition energies, most being within $\pm0.03$ eV of the {\FCI} limit. These reference energies are obtained
on {\CCT}/{\AVTZ} geometries and additional basis set corrections (up to quadruple-$\zeta$ at least) are also provided for {\CCT}. Together with the results obtained in our two earlier works, \cite{Loo18a,Loo19c} the present TBE will
hopefully contribute to climb a rung higher on the ES accuracy ladder.
Unless otherwise stated, all transition energies are computed in the frozen-core approximation (with a large core for the sulfur atoms). Pople's {\Pop} and Dunning's \emph{aug}-cc-pVXZ (X $=$ D, T, Q, and 5) atomic basis sets are systematically
employed in our excited-state calculations. In the following, we employ the aVXZ shorthand notations for these diffuse-containing basis sets. Various statistical quantities are reported in the remaining of this paper: the mean signed error (MSE),
mean absolute error (MAE), root mean square error (RMSE), standard deviation of the errors (SDE), as well as the positive [\MaxP] and negative [\MaxN] maximum errors. Here, we globally follow the same procedure as in Ref.~\citenum{Loo18a},
so that we only briefly outline the various theoretical methods that we have employed in the subsections below.
Consistently with our previous work, \cite{Loo18a} we systematically use {\CCT}/{\AVTZ} GS geometries obtained without applying the frozen-core approximation. The Cartesian coordinates (in bohr) of each compound can be found in the
Supporting Information (SI). Several structures have been extracted from previous contributions, \cite{Bud17,Jac18a,Bre18a} whereas the missing structures were optimized using DALTON \cite{dalton} and/or CFOUR, \cite{cfour} applying
Because sCI methods are less widespread than the other methods mentioned in the Introduction, we shall detail further their main features
All the SCI calculations have been performed in the frozen-core approximation with the latest version of QUANTUM PACKAGE \cite{Gar19} using the Configuration Interaction using a Perturbative Selection made Iteratively (CIPSI) algorithm to select the
most important determinants in the FCI space. Instead of generating all possible excited determinants like a conventional CI calculation, the iterative CIPSI algorithm performs a sparse exploration of the FCI space via a selection of the most relevant
determinants using a second-order perturbative criterion. At each iteration, the variational (or reference) space is enlarged with new determinants. CIPSI can be seen as a deterministic version of the FCIQMC algorithm developed by Alavi and
coworkers. \cite{Boo09} We refer the interested reader to Ref.~\citenum{Gar19} where our implementation of the CIPSI algorithm is detailed.
Excited-state calculations are performed within a state-averaged formalism which means that the CIPSI algorithm select determinants simultaneously for the GS and ES. Therefore, all electronic states share the same set of determinants with different CI coefficients.
Our implementation of the CIPSI algorithm for ES is detailed in Ref.~\citenum{Sce19}. For each system, a preliminary SCI calculation is performed using Hartree-Fock orbitals in order to generate SCI wavefunctions with at least 5,000,000 determinants.
State-averaged natural orbitals are then computed based on this wavefunction, and a new, larger SCI calculation is performed with this new set of orbitals. This has the advantage to produce a smoother and faster convergence of the SCI energy towards the FCI limit.
For the largest systems, an additional iteration is sometimes required in order to obtain better quality natural orbitals and hence well-converged calculations.
The total SCI energy is defined as the sum of the (zeroth-order) variational energy (computed via diagonalization of the CI matrix in the reference space) and a second-order perturbative correction which takes into account the external determinants, \ie,
the determinants which do not belong to the variational space but are linked to the reference space via a non-zero matrix element. The magnitude of this second-order correction, $E^{(2)}$, provides a qualitative idea of the ``distance" to the FCI limit.
For maximum efficiency, the total SCI energy is linearly extrapolated to $E^{(2)}=0$ (which effectively corresponds to the FCI limit) using the two largest SCI wavefunctions. These extrapolated total energies (simply labeled as FCI in the remaining of the paper)
are then used to computed vertical excitation energies. Although it is not possible to provide a theoretically-sound error bar, we estimate the extrapolation error by the difference in excitation energy between the largest SCI wavefunction and its corresponding
extrapolated value. We believe that it provides a very safe estimate of the extrapolation error. Additional information about the SCI wavefunctions and excitation energies as well as their extrapolated values can be found in the SI.
The {\NEV} calculations have been performed with MOLPRO \cite{molpro} within the partially-contracted scheme ({\PNEV}), which is theoretically superior to its strongly-contracted version due to the larger number of perturbers and greater flexibility. \cite{Ang01,Ang01b,Ang02}
These {\NEV} calculations are performed on top of a state-averaged complete-active-space self-consistent field (SA$n$-CASSCF) calculation (where $n$ is the number of states that has been averaged). Large active spaces carefully chosen and tailored for the desired transitions
have been selected. The definition of the active space considered for each system as well as the number of states in the state-averaged calculation is provided in the SI.
For the other levels of theory, we apply a variety of programs, namely, CFOUR,\cite{cfour} DALTON,\cite{dalton} GAUSSIAN,\cite{Gaussian16} ORCA,\cite{Nee12} MRCC,\cite{Rol13,mrcc} and Q-CHEM. \cite{Sha15} CFOUR is used for
{\CCT}, \cite{Chr95b,Koc97} CCSDT-3, \cite{Wat96,Pro10}{\CCSDT}\cite{Nog87} and {\CCSDTQ}\cite{Kuc91}; Dalton for {\CCD}, \cite{Chr95,Hat00}{\CCSD},\cite{Pur82} CCSDR(3), \cite{Chr96b} and {\CCT}\cite{Chr95b,Koc97}; Gaussian
for CIS(D); \cite{Hea94,Hea95} ORCA for the similarity-transformed EOM-CCSD ({\STEOM})\cite{Noo97,Dut18}; MRCC for {\CCSDT}\cite{Nog87} and {\CCSDTQ}; \cite{Kuc91} and Q-Chem for {\AD}. \cite{Dre15}
Default program settings were applied. We note that for {\STEOM} we only report states that are characterized by an active character percentage of $98\%$ or larger.
In the following, we present results obtained for molecules containing four, five, and six (non-hydrogen) atoms. In all cases, we test several atomic basis sets and push the CC excitation order as high as technically possible.
Given that the {\SCI} energy converges rather slowly for these systems, we provide an estimated error bar for these extrapolated {\FCI} values (\emph{vide supra}). In most cases, these extrapolated FCI reference data
are used as a ``safety net'' to demonstrate the overall consistency of the various approaches rather than as definitive reference values (see next Section). As a further consistency check, we also report {\NEV}/{\AVTZ} excitation
energies for all states. We underline that, except when specifically discussed, all ES present a dominant single-excitation character (see also next Section), so that we do not expect serious CC breakdowns. This is especially
true for triplet ES that are known to be characterized by very large $\Td$ values in the vast majority of the cases. \cite{Sch08} Consequently, we concentrate most of our computational effort on the obtention of accurate transition
energies for singlet states. To assign the different ES, we use literature data, as well as the usual criteria, \ie, relative energies, spatial and spin symmetries, compositions from the underlying molecular orbitals, and oscillator strengths.
This allows clear-cut assignments for the vast majority of the cases. There are however some state/method combinations for which strong mixing between ES of the same symmetry makes unambiguous assignments almost impossible.
$^e${Experimental 0-0 energies from Ref.~\citenum{Hai79} (singlet ES, vacuum UV experiment) and Ref.~\citenum{All84} (triplet ES, EELS). In the latter contribution, the $2.7$ eV value for the $^3\Sigma_u^+$
The ES of these three closely related linear molecules containing two triple bonds have been quite rarely theoretically investigated, \cite{Fis03,Pat06,Luo08,Loo18b,Loo19a} though (rather old) experimental measurements of their
0-0 energies are available for several ES. \cite{Cal63,Job66a,Job66b,Bel69,Fis72,Har77,Hai79,All84} Our main results are collected in Tables \ref{Table-1} and S1. We consider only low-lying valence $\pi\ra\pis$ transitions, which
are all characterized by a strongly dominant single excitation nature ($\Td > 90\%$, \emph{vide infra}). For cyanoacetylene, the {\FCI}/{\Pop} estimates come with small error bars, and one notices an excellent agreement between
these values and their {\CCSDTQ} counterparts, a statement holding for the Dunning's double-$\zeta$ basis set results for which the {\FCI} uncertainties are however larger. Using the {\CCSDTQ} values as references, it appears
that the previously obtained {\CASPT} estimates\cite{Luo08} are, as expected, too low and that the {\CCT} transition energies are slightly more accurate than their CCSDT counterparts, although all CC estimates of Table \ref{Table-1}
come, for a given basis set, in a very tight energetic window. There is also a very neat agreement between the CC/{\AVTZ} and {\NEV}/{\AVTZ}. All these facts provide strong evidences that the CC estimates can be fully trusted for
these three linear systems. The basis set effects are quite significant for the valence ES of cyanoacetylene with successive drops of the transition energies by approximately $0.10$ eV, when going from {\Pop} to {\AVDZ}, and from
{\AVDZ} to {\AVTZ}. The lowest triplet state appears less basis set sensitive, though. As expected, extending further the basis set size (to quadruple- and quintuple-$\zeta$) leaves the results pretty much unchanged. The same
observation holds when adding a second set of diffuse functions, or when correlating the core electrons (see the SI). Obviously, both cyanogen and diacetylene yield very similar trends, with limited methodological effects and quite
large basis set effects, except for the $^1\Sigma_g^+\ra{}^3\Sigma_u^+$ transitions. We note that all CC3 and CCSDT values are, at worst, within $\pm0.02$ eV of the {\FCI} window, \ie, all methods presented in Table \ref{Table-1}
provide very consistent estimates. For all the states reported in this Table, the average absolute deviation between {\NEV}/{\AVTZ} and {\CCT}/{\AVTZ} ({\CCSDT}/{\AVTZ}) is as small as $0.02$ ($0.03$) eV, the lowest absorption and
emission energies of cyanogen being the only two cases showing significant deviations. As a final note, all our vertical absorption (emission) energies are significantly larger (smaller) than the experimentally measured 0-0 energies,
as they should. We refer the interested reader to previous works, \cite{Fis03,Loo19a} for comparisons between theoretical ({\CASPT} and {\CCT}) and experimental 0-0 energies for these three compounds.
These three related compounds present a three-member $sp^2$ carbon cycle conjugated to an external $\pi$ bond. While the ES of methylenecyclopropene have regularly been investigated with theoretical tools in the past,
\cite{Mer96,Roo96,Car10b,Lea12,Gua13,Dad14,Gua14,Sch17,Bud17} the only investigations of vertical transitions we could find for the two other derivatives are a detailed {\CASPT} study of Serrano-Andr\'es and
coworkers in 2002, \cite{Ser02} and a more recent work reporting the three lowest-lying singlet states of cyclopropenone at the {\CASPT}/6-31G level.\cite{Liu14b}
$^b${Electron impact experiment from Ref.~\citenum{Har74}. Note that the $5.5$ eV peak was assigned differently in the original paper, and we follow here the analysis of Serrano-Andr\'es, \cite{Ser02}
whereas the $6.1$ eV assignment was ``supposed'' in the original paper; experimental $\lambda_{\mathrm{max}}$ have been measured at $3.62$ eV and $6.52$ eV for the $^1B_1$ ($n \ra\pis$) and
$^e${$\lambda_{\mathrm{max}}$ in pentane at $-78^o$C from Ref.~\citenum{Sta84};}
$^f${Significant state mixing with the $^1A_1$($\pi\ra3p$) transition, yielding unambiguous attribution difficult;}
$^g${As can be seen in the SI, our {\FCI}/{\AVDZ} estimates are $3.45\pm0.04$ and $4.79\pm0.02$ eV for the two lowest triplet states of methylenecyclopropene hinting that the CC3 and CCSDT
Our results are listed in Tables \ref{Table-2} and S2. As above, considering the {\Pop} basis set, we notice very small differences between {\CCT}, {\CCSDT}, and {\CCSDTQ}, the latter method giving transition energies
systematically falling within the {\FCI} extrapolation incertitude, except in one case (the lowest totally symmetric state of methylenecyclopropene for which the {\CCSDTQ} value is ``off'' by $0.02$ eV only). Depending on the state, it is
either {\CCT} or {\CCSDT} that is closest to {\CCSDTQ}. In fact, considering the {\CCSDTQ}/{\Pop} data listed in Table \ref{Table-2} as reference, the MAE of {\CCT} and {\CCSDT} are $0.019$ and $0.016$ eV, respectively,
hinting that the improvement brought by the latter, more expensive method is limited for these three compounds. For the lowest $B_2$ state of methylenecyclopropene, one of the most challenging cases ($\Td=85\%$),
it is clear from the {\FCI} value that only {\CCSDTQ} is energetically close, the {\CCT} and {\CCSDT} results being slightly too large by $\sim0.05$ eV. It seems reasonable to believe that the same observation can be made for the corresponding state of
cyclopropenethione, although in this case the FCI error bar is too large, which prevents any definitive conclusion. Interestingly, at the {\CCT} level of theory, the rather small {\Pop} basis set provides data within $0.10$ eV of the CBS limit for $80\%$ of
the transitions. There are, of course, exceptions to this rule, \eg, the strongly dipole-allowed $^1A_1(\pi\ra\pis)$ ES of cyclopropenone and the $^1B_1(\pi\ra3s)$ ES of methylenecyclopropene which are significantly
over blueshifted with the Pople basis set (Table S2). For cyclopropenone, our {\CCSDT}/{\AVTZ} estimates do agree reasonably well with the {\CASPT} data of Serrano-Andr\'es, except for the $^1B_2(\pi\ra\pis)$ state
that we locate significantly higher in energy and the three Rydberg states that our CC calculations predict at significantly lower energies. The present {\NEV} results are globally in better agreement with the CC values,
though non-negligible deviations pertain. Even if comparisons with experiment should be made very cautiously, we note that, for the Rydberg states, the present CC data are clearly more consistent with the electron
impact measurements\cite{Har74} than the original {\CASPT} values. For cyclopropenethione, we typically obtain transition energies in agreement or larger than those obtained with {\CASPT}, \cite{Ser02} though there is no obvious relationship between the
valence/Rydberg nature of the ES and the relative {\CASPT} error. The average absolute deviation between our {\NEV} and {\CCT} results is $0.08$ eV only. Finally, in the case of methylenecyclopropene, our values logically agree
very well with the recent estimates of Schwabe and Goerigk, \cite{Sch17} obtained at the {\CCT}/{\AVTZ} level of theory on a different geometry. As anticipated, the available {\CASPT} values \cite{Mer96,Roo96} appear too
low as compared to the present {\NEV} and {\CCSDT} values. For this compound, the available experimental data are based on the wavelength of maximal absorption determined in condensed phase. \cite{Sta84} Hence, only a
qualitative match is reached between theory and experiment.
Let us now turn our attention to the excited states of three pseudo-linear $\pi$-conjugated systems that have been the subject to several investigations in the past, namely, acrolein, \cite{Aqu03,Sah06,Car10b,Lea12,Gua13,Mai14,Aza17b,Sch17,Bat17}
butadiene, \cite{Dal04,Sah06,Sch08,Sil10c,Li11,Wat12,Dad12,Lea12,Ise12,Ise13,Sch17,Shu17,Sok17,Chi18,Cop18,Tra19,Loo19c} and glyoxal. \cite{Sta97b,Koh03,Hat05c,Sah06,Lea12,Poo14,Sch17,Aza17b,Loo18b}
Among these works, it is worth highlighting the detailed theoretical investigation of Saha, Ehara, and Nakatsuji, who reported a huge number of ES for these three systems using a coherent theoretical protocol based on the symmetry-adapted-cluster
configuration interaction (SAC-CI) method. \cite{Sah06} In the following, these three molecules are considered in their most stable \emph{trans} conformation. Our results are listed in Tables \ref{Table-3} and S3.
$^e${Electron impact experiment from Refs.~\citenum{Fli78} and \citenum{Doe81} for the singlet states and from Ref.~\citenum{Mos73} for the two lowest triplet transitions;
$^h${Electron impact experiment from Ref.~\citenum{Ver80} except for the second $^1B_g$ ES for which the value is from another work; \cite{Rob85b} note that
for the lowest $^1B_g$ ($^1B_u$) ES, a range of $4.2$--$4.5$ ($7.4$--$7.9$) eV is given in Ref.~\citenum{Ver80}. }
Acrolein, due to its lower symmetry and high density of ES with mixed characters, is challenging from a theoretical point of view, and {\CCSDTQ} calculations were technically impossible despite all our efforts. For the lowest $n \ra\pis$
transitions of both spin symmetry, the {\FCI} estimates come with a tiny error bar, and it is obvious that the CC excitation energies are slightly too low, especially with {\CCSDT}. Nevertheless, at the exception of the second singlet
and triplet $A''$ ES, the {\CCT} and {\CCSDT} transition energies are within $\pm0.03$ eV of each other. These $A''$ ES are also the only two transitions for which the discrepancies between {\CCT} and {\NEV} exceed $0.20$ eV.
This hints at a good accuracy for all other transitions. This statement is additionally supported by the fact that the present CC values are nearly systematically bracketed by previous {\CASPT} (lower bound)\cite{Aqu03} and SAC-CI (upper bound)\cite{Sah06} results,
consistently with the typical error sign of these two models. For the two lowest triplet states, the present {\CCT}/{\AVTZ} values are also within $\pm0.05$ eV of recent MRCI estimates ($3.50$ and $3.89$ eV). \cite{Mai14} As
can be seen in Table S3, the {\AVTZ} basis set delivers excitation energies very close to the CBS limit: the largest variation when upgrading from {\AVTZ} to {\AVQZ} ($+0.04$ eV) is obtained for the second $^1A'$ Rydberg ES. As experimental data
The nature and relative energies of the lowest bright $B_u$ and dark $A_g$ ES of butadiene have puzzled theoretical chemists for many years. It is beyond the scope of the present study to provide an exhaustive list of previous calculations
and experimental measurements for these two hallmark ES, and we refer the readers to Refs.~\citenum{Wat12} and \citenum{Shu17} for a general and broader overview. For the $B_u$ transition, we believe
that the most solid TBE is the $6.21$ eV value obtained by Watson and Chan using a computational strategy similar to ours. \cite{Wat12} Our {\CCSDT}/{\AVTZ} value of $6.24$ eV is obviously compatible with their reference value, and our TBE/CBS
value is actually $6.21$ eV as well (\emph{vide infra}). For the $A_g$ state, we believe that our previous basis set corrected {\FCI} estimate of $6.50$ eV \cite{Loo19c} remains the most accurate available to date. These two values are slightly lower
than the semi-stochastic heath-bath CI data obtained by Chien \emph{et al.} with a double-$\zeta$ basis and a slightly different geometry: $6.45$ and $6.58$ eV for $B_u$ and $A_g$, respectively. \cite{Chi18} For these two thoroughly studied ES,
one can of course find many other estimates, \eg, at the SAC-CI, \cite{Sah06}{\CCT}, \cite{Sil10c,Sch17}{\CASPT}, \cite{Sil10c} and {\NEV}\cite{Sok17} levels. Globally, for butadiene, we find an excellent coherence between the {\CCT}, {\CCSDT},
and {\CCSDTQ} estimates, that all fall in a $\pm0.02$ eV window. Unsurprisingly, this does not apply to the already mentioned $^1A_g$ ES that is $0.2$ and $0.1$ eV too high with the two former CC methods, a direct consequence of the large
electronic reorganization taking place during this transition. For all the other butadiene ES listed in Table \ref{Table-3}, both {\CCT} and {\CCSDT} can be trusted. We also note that the {\NEV} estimates are within $0.1$--$0.2$ eV of the CC values,
except for the lowest $B_u$ ES for which the associated excitation energy is highly dependent on the selected active space (see the SI). Finally, as can be seen in Table S3, {\AVTZ} produces near-CBS excitation energies for most ES.
However, a significant basis set effect exists for the Rydberg $^1B_u (\pi\ra3p)$ ES with an energy lowering as large as $-0.12$ eV when going from {\AVTZ} to {\AVQZ}. For the record, we note that the available electron impact data
\cite{Mos73,Fli78,Doe81} provide the very same ES ordering as our calculations.
Globally, the conclusions obtained for acrolein and butadiene pertain for glyoxal, \ie, highly consistent CC estimates, reasonable agreement between {\NEV} and {\CCT} values, and limited basis set effects beyond {\AVTZ}, except for the $^1B_u
(n \ra 3p)$ Rydberg state (see Tables \ref{Table-3} and S3). This Rydberg state also exhibits an unexpectedly large deviation of $0.04$ eV between {\CCT} and {\CCSDTQ}. More interestingly, glyoxal presents a genuine low-lying double ES
of $^1A_g$ symmetry. The corresponding $(n,n)\ra(\pis,\pis)$ transition is totally unseen by approaches that cannot model double excitations, \eg, TD-DFT, {\CCSD}, or {\AD}. Compared to the {\FCI} values, the {\CCT} and {\CCSDT}
estimates associated with this transition are too large by $\sim1.0$ and $\sim0.5$ eV, respectively, whereas both the {\CCSDTQ} and {\NEV} approaches are much closer, as already mentioned in our previous work. \cite{Loo19c} For the
other transitions, the present {\CCT} estimates are logically consistent with the values of Ref.~\citenum{Sch17} obtained with the same approach on a different geometry, and remain slightly lower than the SAC-CI estimates of Ref.~\citenum{Sah06}.
Once more, the experimental data \cite{Ver80,Rob85b} are unhelpful in view of the targeted accuracy.
Let us now turn towards six other four-atom compounds. There are several earlier studies reporting estimates of the vertical transition energies for both acetone \cite{Gwa95,Mer96b,Roo96,Wib98,Toz99b,Wib02,Sch08,Sil10c,Car10,Pas12,Ise12,Gua13,Sch17}
and isobutene. \cite{Wib02,Car10,Ise12} To the best of our knowledge, for the four other compounds, the previous computational efforts were mainly focussed on the 0-0 energies of the lowest-lying states. \cite{Koh03,Hat05c,Sen11b,Loo18b,Loo19a}
There are also rather few experimental data available for these six derivatives. \cite{Bir73,Jud83,Bra74,Sta75,Joh79,Jud83,Jud84c,Rob85,Pal87,Kar91b,Xin93}
$^c${Two lowest singlet states: various experiments summarized in Ref.~\citenum{Rob85}; three next singlet states: REMPI experiments from Ref.~\citenum{Xin93}; lowest triplet: trapped electron measurements from Ref.~\citenum{Sta75};}
$^d${0-0 energy reported in Ref.~\citenum{Kar91b};}
For acetone, one should clearly distinguish the valence ES, for which both methodological and basis set effects are small, and the Rydberg transitions that are both very basis set sensitive, and upshifted by ca.~$0.04$ eV with {\CCSDTQ} as
compared to {\CCT} and {\CCSDT}. For this compound, the 1996 {\CASPT} transition energies of Merch\'an and coworkers listed on the right panel of Table \ref{Table-4} are clearly too low, especially for the three valence ES. \cite{Mer96b}
As expected, this error can be partially ascribed to the computational set-up, as the Urban group obtained {\CASPT} excitation energies of $4.40$, $4.09$ and $6.22$ eV for the $^1A_2$, $^3A_2$, and $^3A_1$ ES, \cite{Pas12} in much
better agreement with ours. Their estimates of the three $n \ra3p$ transitions, $7.52$, $7.57$, and $7.53$ eV for the $^1A_2$, $^1A_1$, and $^1B_2$ ES, also systematically fall within $0.10$ eV of our current CC values, whereas for these
three ES, the current {\NEV} values are clearly too large.
In contrast to acetone, both valence and Rydberg ES of thioacetone are rather insensitive to the excitation order of the CC expansion, as illustrated by the maximal discrepancies of $\pm0.02$ eV between the {\CCT}/{\Pop} and {\CCSDTQ}/{\Pop} results.
While the lowest $n \ra\pis$ transition of both spin symmetries are rather basis set insensitive, all the other states need quite large one-electron bases to be correctly described (Table S4). As expected, our theoretical vertical transition energies
show the same ranking but are systematically larger than the available experimental 0-0 energies.
For the isoelectronic isobutene molecule, we have considered two singlet Rydberg and one triplet valence ES. For these three cases, we note, for each basis, a very nice agreement between {\CCT} and {\CCSDT}, the
CC results being also very close to the {\FCI} estimates obtained with the Pople basis set. The similarity with the {\CCSD} results of Caricato and coworkers \cite{Car10} is also very satisfying.
For the three remaining compounds, namely, cyanoformaldehyde, propynal, and thiopropynal, we report low-lying valence transitions with a definite single excitation character. The basis set effects are clearly under control (they are only
significant for the second $^1A''$ ES of cyanoformaldehyde) and we could not detect any variation larger than $0.03$ eV between the {\CCT} and {\CCSDT} values for a given basis, indicating that the CC values are very accurate.
As we have seen for the 15 four-atom molecules considered here, we found extremely consistent transition energies between CC and {\FCI} estimates in the vast majority of the cases. Importantly, we confirm our previous conclusions obtained on
smaller compounds: \cite{Loo18a} i) {\CCSDTQ} values systematically fall within (or are extremely close to) the {\FCI} error bar, ii) both {\CCT} and {\CCSDT} are also highly trustable when the considered ES does not exhibit a strong double
excitation character. Indeed, considering the 54 ``single'' ES cases for which {\CCSDTQ} estimates could be obtained (only excluding the lowest $^1A_g$ ES of butadiene and glyoxal), we determined negligible MSE $< 0.01$ eV, tiny
MAE ($0.01$ and $0.02$ eV), and small maximal deviations ($0.05$ and $0.04$ eV) for {\CCT} and {\CCSDT}, respectively. This clearly indicates that these two approaches provide chemically-accurate estimates (errors below $1$
kcal.mol$^{-1}$ or $0.043$ eV) for most electronic transitions. Interestingly, some of us have shown that {\CCT} also provides chemically-accurate 0-0 energies as compared to experimental values for most valence transitions. \cite{Loo18b,Loo19a,Sue19}
When comparing the {\NEV} and {\CCT} ({\CCSDT}) results obtained with {\AVTZ} for the 91 (65) ES for which comparisons are possible (again excluding only the lowest $^1A_g$ states of butadiene and glyoxal),
one obtains a MSE of $+0.09$ ($+0.09$) eV and a MAE of $0.11$ ($0.12$) eV. This seems to indicate that {\NEV}, as applied here, has a slight tendency to overestimate the transition energies. This contrasts with {\CASPT} that is known to
generally underestimate transition energies, as further illustrated and discussed above and below.
We now consider five-membered rings, and, in particular, five common derivatives that have been considered in several previous theoretical studies (\emph{vide infra}): cyclopentadiene, furan, imidazole, pyrrole, and thiophene.
As the most advanced levels of theory employed in the previous section, namely {\CCSDTQ} and {\FCI}, become beyond reach for these compounds (except in very rare occasions), one has to rely on the nature of the ES and the
For furan, \textit{ab initio} calculations have been performed with almost every available wavefunction method.\cite{Ser93b,Nak96,Tro97b,Chr98b,Chr98c,Wan00,Gro03,Pas06b,Sch08,She09b,Li10c,Sil10b,Sil10c,Sau11,Hol15,Sch17}
However, the present work is, to the best of our knowledge, the first to disclose {\CCSDT} values as well as {\CCT} energies obtained with a quadruple-$\zeta$ basis set. Our results for ten low-lying ES states are listed in Tables \ref{Table-5} and S5.
All singlet (triplet) transitions are characterized by $\Td$ values in the $92$--$94\%$ ($97$--$99\%$) range. Consistently, the maximal discrepancy between {\CCT} and {\CCSDT} is small ($0.04$ eV). In addition, there is a decent consistency
between the present data and the {\NEV} results of both Ref.~\citenum{Pas06b} and of the present work, as well as the MR-CC values of Ref.~\citenum{Li10c}. This holds for almost all transitions, but the $^1B_2$ ($\pi\ra3p$) excitation that we predict
to be significantly higher than in most previous works, even after accounting for the quite large basis set effects ($-0.10$ eV between the {\AVTZ} and {\AVQZ} estimates, see Table S5). We believe that our estimate is the most accurate to date
for this particularly tricky ES. Interestingly, the recent {\AT} values of Ref.~\citenum{Hol15} are consistently smaller by ca.~$-0.2$ eV as compared to {\CCSDT} (see Table \ref{Table-6}), in agreement with the error sign we observed in smaller
compounds for {\AT}. \cite{Loo18a} Again, we note that the experimental data \cite{Vee76b,Fli76,Rob85b} provide the same state ordering as our calculations.
$^g${Electron impact from Ref.~\citenum{Vee76b}: for the $^1A_1$ state two values ($6.44$ and $6.61$ eV) are reported, whereas for the two lowest triplet states, Two values ($3.99$ eV and $5.22$ eV) can be found in Ref.~\citenum{Fli76};}
Like furan, pyrrole has been extensively investigated in the literature using a large panel of theoretical methods. \cite{Ser93b,Nak96,Tro97,Pal98,Chr99,Wan00,Roo02,Pal03b,Pas06c,Sch08,She09b,Li10c,Sil10b,Sil10c,Sau11,Nev14,Sch17,Hei19}
We report six low-lying singlet and four triplet ES in Tables \ref{Table-5} and S5. All these transitions have very large $\Td$ values except for the totally symmetric $\pi\ra\pis$ excitation ($\Td=86\%$). For all the states, we found
highly consistent {\CCT} and {\CCSDT} results, often significantly larger than older multi-reference estimates, \cite{Ser93b,Roo02,Li10c} but in nice agreement with the very recent XMS-{\CASPT} results of the Gonzalez
group, \cite{Hei19} and the present {\NEV} estimates [at the exception of the $^1A_2(\pi\ra3p)$ transition]. The match obtained with the twenty years old extrapolated CC values of Christiansen and coworkers \cite{Chr99} is quite remarkable. The only exceptions are the two
$B_2$ transitions that were reported as significantly mixed in this venerable work. For the lowest singlet ES, the {\FCI}/{\Pop} value is $5.24\pm0.02$ eV confirming the performance of both {\CCT} and {\CCSDT} for this transition.
As can be seen in Table S5, {\AVTZ} yields basis-set converged transition energies, except, like in furan, for the Rydberg $^1B_2(\pi\ra3p)$ transition that is significantly redshifted ($-0.09$ eV) when pushing to the
quadruple-$\zeta$ basis set. As mentioned in Thiel's work, \cite{Sch08} the experimental spectra of pyrrole are quite broad, and the rare available experiments \cite{Hor67,Bav76,Fli76b,Vee76b,Pal98,Pal03b} can only be considered as general guidelines.
$^g${Energy loss from Ref.~\citenum{McD85} for the two valence states; two-photon resonant experiment from Ref.~\citenum{Sab92} for the $^1A_2$ Rydberg ES;}
$^k${The assignments of these two states as valence and Rydberg is based on the oscillator strength, but both have a partial Rydberg character. The CASSCF spatial extend is in fact larger
for the lowest transition and Roos consequently classified both ES as Rydberg transitions; \cite{Ser96b}}
$^l${{\CASPT} results from Ref.~\citenum{Ser93c};}
$^m${SAC-CI results from Ref.~\citenum{Wan01};}
$^n${CCSDR(3) results from Ref.~\citenum{Pas07};}%, this work also contains {\NEV} estimates;}
$^o${TBE from Ref.~\citenum{Hol14}, based on EOM-CCSD for singlet and ADC(2) for triplets;}
$^p${0-0 energies from Ref.~\citenum{Dil72};}
$^q${0-0 energies from Ref.~\citenum{Var82} for the singlets and energy loss experiment from Ref.~\citenum{Hab03} for the triplets;}
$^r${0-0 energies from Ref.~\citenum{Hol14};}
$^s${Non-negligible mixing with a close-lying $(\pi\ra\pis)$ transition of the same symmetry.}
Although a diverse array of wavefunction studies has been performed on cyclopentadiene (including {\CASPT}, \cite{Ser93b,Sch08,Sil10c} CC, \cite{Sch08,Sil10b,Sch17} SAC-CI \cite{Wan00b} and various multi-reference
approaches \cite{Nak96,She09b}), this compound has received less attention than other members of the five-membered ring family, namely furan and pyrrole (\emph{vide infra}). This is probably due to the presence of the
methylene group that renders computations significantly more expensive. Most transitions listed in Tables \ref{Table-6} and S6 are characterized by $\Td$ exceeding $93\%$, the only exception being the $^1A_1(\pi\ra\pis)$
excitation that has a similar nature as the lowest $A_g$ state of butadiene ($\Td=79\%$). Consistently, the {\CCT} and {\CCSDT} results are nearly identical for all ES except for the $^1A_1$ ES. By comparing the results
obtained for this $A_1(\pi\ra\pis)$ transition to its butadiene counterpart, one can infer that the {\CCSDT} estimate is probably too large by roughly $0.04$--$0.08$ eV, and that the {\NEV} value is unlikely to be accurate enough to
establish a definitive TBE. This statement is also in line with the results of Ref.~\citenum{Loo19c}. For the two $B_2(\pi\ra\pis)$ transitions, we could obtain {\FCI}/{\Pop} estimates of $5.78\pm0.02$ eV (singlet) and $3.33\pm0.05$ eV (triplet), the
{\CCT} and {\CCSDT} transition energies falling inside these energetic windows in both cases. As one can see in Tables \ref{Table-6} and S6, the basis set effects are rather moderate for the electronic transitions of cyclopentadiene,
with no variation larger than $0.10$ eV ($0.02$ eV) between {\AVDZ} and {\AVTZ} ({\AVTZ} and {\AVQZ}). When comparing to literature data, our values are unsurprisingly consistent with the {\CCT} values of
Schwabe and Goerigk, \cite{Sch17} and tend to be significantly larger than earlier {\CASPT}\cite{Ser93b,Sil10c} and MR-MP \cite{Nak96} estimates. As expected, a few gas-phase experiments are available as well for this
derivative, \cite{Fru79,McD85,McD91b,Sab92} but they hardly represent grounds for valuable comparison.
Due to its lower symmetry, imidazole has been less investigated, the most advanced studies available probably remain the {\CASPT} work of Serrano-Andr\`es and coworkers from 1996, \cite{Ser96b} and the
basis-set extrapolated {\CCT} results of Silva-Junior \emph{et al.} for the valence transitions from 2010. \cite{Sil10c} The experimental data in gas-phase are also limited.\cite{Dev06} Our results are displayed in Tables \ref{Table-6} and S6.
The {\CCT} and {\CCSDT} values are quite consistent despite the fact that the $\Td$ values of the two singlet $A'$ states are slightly smaller than $90\%$. These two states have indeed a (at least partial) Rydberg character,
see the footnote in Table \ref{Table-6}. The agreement between the CC estimates and previous {\CASPT}, \cite{Ser96b} and current {\NEV} energies is reasonable, the latter being systematically larger than their {\CCT}
counterparts. For the eight transitions considered here, the basis set effects are moderate and {\AVTZ} yield results within $0.03$ eV of their {\AVQZ} counterparts (Table S6 in the SI).
Finally, the ES of thiophene, which is one of the most important building block in organic electronic devices, were the subject of previous theoretical investigations, \cite{Ser93c,Pal99,Wan01,Kle02,Pas07,Hol14} that unveiled a series of
transitions that were not yet characterized in the available measurements. \cite{Dil72,Fli76,Fli76b,Var82,Hab03,Pal99,Hol14} To the best of our knowledge, the present work is the first to report CC calculations obtained with (iterative)
triples and therefore constitutes the most accurate estimates to date. Indeed, all the transitions listed in Tables \ref{Table-6} and S6 are characterized by a largely dominant single excitation character, with $\Td$ above
$90\%$ except for the two $^1A_1$ transitions for which $\Td=88\%$ and $87\%$, respectively. The agreement between {\CCT} and {\CCSDT} remains nevertheless excellent for these low-lying totally symmetric transition. Thiophene is
also one of these compounds for which the unambiguous characterization of the nature of the ES is difficult, with, \eg, a strong mixing between the second and third singlet ES of $B_2$ symmetry. This makes the assignment of the valence
($\pi\ra\pis$) or Rydberg ($\pi\ra3p$) character of this transition particularly tricky at the {\CCT} level. We note that contradictory assignments can be found in the literature. \cite{Ser93c,Wan01,Pas07} As for the
previously discussed isostructural systems, we note that the only ES that undergoes significant basis set effects beyond {\AVTZ} is the Rydberg $^1B_2(\pi\ra3p)$ ($-0.09$ eV when upgrading to {\AVQZ}, see Table S6) and that
the {\NEV} estimates tend to be slightly larger than the {\CCT} values. The data of Table \ref{Table-6} are globally in good agreement with the previously reported values with discrepancies that are significant only for the three highest-lying singlet states.
Let us now turn towards seven six-membered rings which play a key role in chemistry: benzene, pyrazine, pyridazine, pyridine, pyrimidine, tetrazine, and triazine. To the best of our knowledge, the present work
is the first to propose {\CCSDT} reference energies as well as {\CCT}/{\AVQZ} values for all these compounds. Of course, these systems have been investigated before, and beyond Thiel's
benchmarks, \cite{Sch08,Sil10b,Sil10c} it is worth pointing out the early investigation of Del Bene and coworkers, \cite{Del97b} performed with a CC approach including perturbative corrections for the triples.
Following a theoretically consistent protocol, Nooijen \cite{Noo99} also performed {\STEOM} calculations to study the ES of each of these derivatives. However, these two works only considered singlet ES.
These three highly-symmetric systems allow to directly perform {\CCSDT}/{\AVTZ} calculations for singlet states without the need of basis set extrapolations. Benzene was studied many times
before, \cite{Sob93,Lor95b,Chr96c,Pac96,Del97b,Noo99,Hal02,Li07b,Sch08,Dev08,Sil10b,Sil10c,Sau11,Li11,Lea12,Kan14,Sch17,Dut18,Sha19,Loo19c} and we report in Tables \ref{Table-7} and S7 estimates obtained for
five singlet and three triplet ES, all characterized by $\Td$ exceeding $90\%$ except for the lowest singlet ($86\%$). As one can see, the two CC approaches are again yielding very consistent transition energies
with variations in the 0.00--0.03 eV range. Besides, {\AVTZ} is essentially providing basis set converged transition energies (Table S7). The present CC estimates are also very consistent with earlier {\CCT} results \cite{Chr96c} and
are compatible with both the very recent RASPT2 \cite{Sha19} and our {\NEV} values. For states of both spin symmetries, the {\CCT} and {\CCSDT} transitions energies are slightly larger than the available electron impact/multi-photon
measurements, \cite{Doe69,Nak80,Joh76,Joh83,Hir91} but do provide energetic gaps between ES very similar to the measured ones.
$^f${Jet-cooled experiment from Ref.~\citenum{Hir91} for the two lowest states, multi-photon experiments from Refs.~\citenum{Joh76} and \citenum{Joh83} for the Rydberg states.}
There are many available studies of the ES of pyrazine, \cite{Ful92,Del97b,Web99,Noo99,Li07b,Sch08,Sau09,Sil10c,Woy10,Car10,Sau11,Lea12,Kan14,Sch17,Dut18} and tetrazine,
\cite{Sta96,Del97b,Rub99,Noo99,Ada00,Noo00,Ang09,Sch08,Sau09,Sil10b,Sil10c,Car10,Lea12,Kan14,Sch17,Dut18,Pas18b} for which the $D_{2h}$ symmetry helps distinguishing the different ES.
Our results are collected in Tables \ref{Table-8} and S8. In pyrazine, all transitions are characterized by $\Td > 85\%$ at the exception of the $^1B_{1g}(n \ra\pis)$ transition (84\%).
The excitation energies are basically unchanged going from {\CCT} to {\CCSDT} except possibly for the highest-lying singlet state considered here. Going from triple- to quadruple-$\zeta$ basis, the variations do
not exceed $0.04$ eV, even for the four Rydberg ES treated here. This indicates that one can be highly confident in the present estimates except for the highest-lying singlet ES.
Again, the previous {\CASPT} estimates \cite{Ful92,Web99,Sch08,Sau11} appear to be globally too low, while the (unconventional) CASPT3 results \cite{Woy10} seem too high.
A similar overestimation can be noticed in previous SAC-CI results \cite{Li07b} and our {\NEV} values, the latter showing a mean absolute deviation of 0.11 eV compared to {\CCT}.
In fact, the most satisfying agreement between the current estimates and previous works is reached with Nooijen's {\STEOM} values (except for the highest ES), \cite{Noo99} and
the recent Schwabe-Goerigk's {\CCT} estimates. \cite{Sch17} The available experimental data \cite{Bol84,Oku90,Wal91,Ste11c} do not include all theoretically-predicted transitions,
but provide a similar energetic ranking for both singlets and triplets.
$^l${From Ref.~\citenum{Pal97}, the singlets are from EEL, except for the $4.97$ and $5.92$ eV values that are from VUV; the triplets are from EEL, additional (unassigned) triplet peaks are found at $4.21$, $4.6$, and $5.2$ eV;}
$^m${all these three doubly ES have a $(n,n \ra\pis, \pis)$ character.}
For tetrazine, we consider valence ES only, including three transitions exhibiting a true double excitation nature ($\Td < 10\%$). Of course, for these double excitations, {\CCT} and {\CCSDT} cannot be considered as reliable. This is illustrated by
the large change in excitation energies between these two CC models. The theoretical best estimates are likely obtained with {\NEV}. \cite{Loo19c} For all the other transitions, the $\Td$ values are in the $80$--$90\%$ range for singlets and larger than $95\%$
for triplets. Consequently, the {\CCT} and {\CCSDT} results are very consistent, the sole exception being the lowest $^3B_{1u}(\pi\ra\pis)$ transition for which we note a shift of $-0.05$ eV when upgrading the level of theory to {\CCSDT}.
In all other cases, there is a global consistency between our CC values. Moreover, the basis set effects are very small beyond {\AVTZ} with a maximal variation of $0.02$ eV going to {\AVQZ} (Table S8). The present values are
almost systematically larger than previous {\CASPT}, \cite{Rub99}{\STEOM}, \cite{Noo00} and GVVPT2 \cite{Dev08} estimates. Our {\NEV} values are also globally consistent with the {\CCT} values with a
maximal discrepancy of 0.22 eV for the ES with a dominant single excitation character. One finds a global agreement with Thiel's {\CCT}/{\AVTZ} values, \cite{Sil10c} although we note variations of approximately $0.20$ eV for
specific excitations like the $B_{2g}$ transitions. This feature might be due to the use of distinct geometries in the two studies. The experimental EEL values from Palmer's work \cite{Pal97}
$^j${Significant state mixing with a close-lying Rydberg transition rendering unambiguous attribution difficult. At the {\CCT}/{\AVDZ} level, the Rydberg state is at $7.26$ eV and
has a small $f$, so attribution is rather clear. However, at the {\CCT}/{\AVTZ} level, the two $B_2$ transitions are at $7.35$ and $7.38$ eV (hence strongly mixed), so that the attribution has been
made using the $f$ of $0.174$ and $0.319$, respectively.}
Those four azabenzenes with $C_{2v}$ or $D_{3h}$ spatial symmetry are also popular molecules in terms of ES calculations. \cite{Pal91,Ful92,Wal92,Lor95,Del97b,Noo97,Noo99,Fis00,Cai00b,Wan01b,Sch08,Sil10b,Sil10c,Car10,Sau11,Lea12,Kan14,Sch17,Dut18}
Our results for pyridazine and pyridine are gathered in Tables \ref{Table-9} and S9. For the former compound, the available wavefunction results \cite{Pal91,Ful92,Del97b,Noo99,Fis00,Sch08,Sil10b,Sil10c,Kan14,Sch17,Dut18}
focussed on singlet transitions, at the exception of rather old MRCI, \cite{Pal91} and {\CASPT} investigations. \cite{Fis00} Again, the $\Td$ values are larger than $85\%$ ($95\%$) for the singlet (triplet) transitions,
and the only state for which there is a variation larger than $0.03$ eV between the {\CCT}/{\AVDZ} and {\CCSDT}/{\AVDZ} energies is the $^3B_2(\pi\ra\pis)$ transition. As in the previous six-membered cycles, the basis set
effects are rather small and {\AVTZ} provides values close to the CBS limit for the considered transitions. For the singlet valence ES, we find again a rather good match with the results of previous {\STEOM}\cite{Noo99} and
CC. \cite{Del97b,Sil10c} Yet again, these values are significantly higher than the {\CASPT} estimates reported in Refs.~\citenum{Ful92} and \citenum{Sil10c}. For the triplets, the present data represent the most
accurate results published to date. Our {\NEV} values are very close to their {\CCT} analogues for the lowest-lying singlet and triplet, but positively deviate for the higher-lying ES. Interestingly, beyond the popular 20-year old
reference measurements, \cite{Inn88,Pal91} there is a very recent experimental EEL analysis for pyridazine, \cite{Lin19} that locates almost all ES. The transition energies reported in this very recent work are systematically smaller than our
CC estimates by approximately $-0.20$ eV. Nonetheless, this study provides exactly the same ES ranking as our theoretical protocol.
Pyridine, the hallmark heterocycle, has been more scrutinized than pyridazine and many wavefunction approaches have been applied to estimate its ES energies. \cite{Ful92,Lor95,Del97b,Noo97,Noo99,Cai00b,Wan01b,Sch08,Sil10b,Sil10c,Car10,Sau11,Kan14,Sch17,Dut18}
Besides, two detailed EEL experiments are also available for pyridine. \cite{Wal90,Lin16} The general trends described above for other six-membered cycles do pertain with: i) large $\Td$ values and consistency between
{\CCT} and {\CCSDT} estimates for all transitions listed in Table \ref{Table-9}; ii) small basis set effects beyond {\AVTZ} even for the Rydberg transitions; iii) qualitative agreement with past CC results; iv) {\NEV} transitions energies
that are, on average, larger than their CC counterparts; and v) same ES ranking as in the most recent measurements. \cite{Lin16} Beyond these aspects, it is worth mentioning that the second $^1B_2(\pi\ra\pis)$ ES is strongly mixed with a nearby
Rydberg transition that is separated by only $0.03$ eV at the {\CCT}/{\AVTZ} level. This obviously makes the analysis particularly challenging for that specific transition.
The results obtained for both pyrimidine and triazine are listed in Tables \ref{Table-10} and S10. Because the former derivative can be viewed as the smallest model of DNA bases, previous theoretical \cite{Ful92,Del97b,Ser97b,Noo99,Ohr01,Fis03b,Li07b,Sch08,Sil10b,Sil10c,Car10,Sau11,Kan14,Sch17,Dut18}
and experimental \cite{Bol84,Pal90,Lin15} studies are rather extensive. For triazine, which belongs to a non-abelian point group, theoretical studies are scarcer,
\cite{Ful92,Wal92,Pal95,Del97b,Noo99,Oli05,Sch08,Sil10b,Sil10c,Kan14,Sch17,Dut18} especially for the triplets, \cite{Wal92,Pal95,Oli05} whereas the experimental data are also limited. \cite{Bol84,Wal92} As in pyridazine and pyridine, all the ES listed in Table \ref{Table-10}
show $\Td$ values larger than $85\%$ for singlets and $95\%$ for triplets, so that {\CCT} and {\CCSDT} are highly coherent, except maybe for the $^3A_1(\pi\ra\pis)$ transitions in pyrimidine. The basis set effects are also small, with no variation larger than
$0.10$ ($0.03$) eV between double- and triple-$\zeta$ (triple- and quadruple-$\zeta$) for valence transitions and only slightly larger variations for the two Rydberg transitions (+$0.04$ eV between {\AVTZ} and {\AVQZ}). For both compounds, the current
values are almost systematically larger than previously published data, with our {\CCT} values being typically bracketed by the published {\CASPT} and our {\NEV} estimates. For the triplets of triazine, the three lowest ES previously estimated by {\CASPT}
Table \ref{Table-tbe} reports our two sets of TBE: a set obtained with the {\AVTZ} basis set and one set including an additional correction for the one-electron basis set incompleteness error. The details of our protocol employed to generate these TBE are also provided in Table \ref{Table-tbe}.
For all states with a dominant single-excitation character (that is when $\Td > 80\%$), we rely on CC results using an incremental strategy to generate these TBE. There are only two exceptions to this general rule, two ES in acrolein for which nicely converged FCI values indicated
non-trifling {\CCSDT} errors. For ES with $\Td$ values between $70\%$ and $80\%$, our previous works indicated that {\CCSDT} tends to overshoot the
transition energies by roughly $0.05$--$0.10$ eV, and that {\NEV} errors tend to be, on average, slightly larger. \cite{Loo19c} Therefore, if {\CCSDTQ} or {\FCI} results are not available, it is extremely difficult to make the final call. For the other transitions, we relied either on the current
or previous FCI data or the {\NEV} values as reference. The italicized transition energies in Table \ref{Table-tbe} are believed to be (relatively) less accurate. This is the case when: i) {\NEV} results have to be selected; ii) the CC calculations yield quite large changes in excitation energies
while incrementing the excitation order by one unit despite large $\Td$; and iii) there is a very strong ES mixing making hard to follow a specific transition from one method (or one basis) to another.
To determine the basis set corrections beyond augmented triple-$\zeta$, we use the {\CCT}/{\AVQZ} or {\CCT}/{\AVPZ} results. For several compounds, we also provide in the SI, {\CCT}/d-{\AVQZ} transition energies (\ie, with an additional set of diffuse functions). However, we do not
consider such values as reference because the addition of a second set of diffuse orbitals only significantly modifies the transition energies when also inducing a stronger ES mixing. We also stick to the frozen-core approximation for two reasons: i) the effect of correlating the core electrons
is generally negligible (typically $\pm0.02$ eV) for the compounds under study (see the SI for examples); and ii) it would be, in principle, necessary to add core polarization functions in such a case.
Table \ref{Table-tbe} encompasses 238 ES, each of them obtained, at least, at the {\CCSDT} level. This set can be decomposed as follows: 144 singlet and 94 triplet transitions, or 174 valence (99 $\pi\ra\pis$, 71 $n \ra\pis$ and 4 double excitations) and 64 Rydberg transitions.
Among these transition energies, fourteen can be considered as ``unsafe'' and are reported in italics accordingly. This definitely corresponds to a very significant extension of our previous ES data sets (see Introduction). Taken all together, they offer a consistent, diverse and accurate
ensemble of transition energies for approximately 350 electronic transitions of various natures in small and medium-sized organic molecules. Table \ref{Table-tbe} also reports 90 oscillator strengths, $f$, which makes it, to the best of our knowledge, the largest set of {\CCT}/{\AVTZ}
oscillator strengths reported to date, the previous effort being mostly performed at the {\CCT}/TZVP level for Thiel's set. \cite{Kan14} It should also be pointed out that all these data are obtained on {\CCT}/{\AVTZ} geometries, consistently with our previous works. \cite{Loo18a,Loo19c}
\caption{\small TBE values (in eV) for all considered states alongside their corresponding oscillator strength, $f$, and percentage of single excitations, $\Td$, obtained at the \CCT/{\AVTZ} level.
The composite protocol to generate these TBE is also reported (see footnotes). In the right-most column, we list the TBE values obtained by including an additional correction (obtained at the {\CCT} level) for basis set incompleteness error.
Values displayed in italics are likely to be relatively less accurate. All values are obtained in the frozen-core approximation.}\label{Table-tbe}\vspace{-0.3 cm}\\
Having at hand such a large set of accurate transition energies, it seems natural to pursue previous benchmarking efforts. More specifically, we assess here the performance of eight popular wavefunction approaches, namely,
CIS(D), {\AD}, {\CCD}, {\STEOM}, {\CCSD}, CCSDR(3), CCSDT-3, and {\CCT}. The complete list of results can be found in Table \hl{SXXX} of the SI. To identify the ES for all approaches, we have made, as for the TBE above
choices based on the usual criteria (symmetry, oscillator strength, ordering, and nature of the involved orbitals), and most, yet not all, assignments are unambiguous. In addition, because all tested approaches are single-reference
methods, we have removed from the reference set the ``unsafe'' transition energies (in italics in Table \ref{Table-tbe}), as well as the four transitions with a dominant double excitation character (with $\Td < 50\%$ as listed in Table
\ref{Table-12}). For the latter transitions, only CCSDT-3 and {\CCT} are able to detect their presence, but with, of course, extremely large errors. A comprehensive list of results are collected in Table \ref{Table-12} which, more
specifically, gathers the MSE, MAE, RMSE, SDE, \MaxP, and \MaxN. Figure \ref{Fig-1} shows histograms of the error distributions for these eight methods. Before discussing these, let us stress two obvious biases of this
benchmark set: i) it encompasses only conjugated organic molecules containing 4 to 6 non-hydrogen atoms; and ii) we mainly used {\CCSDTQ} (4 atoms) or {\CCSDT} (5--6 atoms) reference values. As discussed in
Section \ref{sec-ic} and in our previous work, \cite{Loo18a} the MAE obtained with these two methods are of the order of $0.01$ and $0.03$ eV, respectively. This means that any statistical quantity smaller than $\sim0.02$--$0.03$ eV
\caption{Mean signed error (MSE), mean absolute error (MAE), root-mean square error (RMSE), standard deviation of the errors (SDE), as well as the positive [\MaxP] and negative [\MaxN] maximal errors with respect to the TBE.
All these statistical quantities are reported in eV and have been obtained with the {\AVTZ} basis set.
Let us analyze the global performance of all these methods, starting with the most accurate and computationally demanding models. The relative accuracies of {\CCT} and {\CCSDT}-3 as compared to {\CCSDT} remains an open question in the literature. \cite{Wat13,Dem13}
Indeed, to the best of our knowledge, the only two previous studies discussing this specific aspect are limited to very small compounds. \cite{Kan17,Loo18a} According to the results gathered in Table \ref{Table-12}, it appears that {\CCT} has a slight edge over {\CCSDT}-3, although
{\CCSDT}-3 is closer to {\CCSDT} in terms of formalism. Indeed, {\CCSDT}-3 seems to provide slightly too large transition energies (MSE of $+0.05$ eV). These conclusions are qualitatively consistent with the analyses performed on smaller derivatives,
\cite{Kan17,Loo18a} but the amplitude of the {\CCSDT}-3 errors is larger in the present set. Although the performance of {\CCT} might be unduly inflated by the use of {\CCSDT} and {\CCSDTQ} reference values, it is also clear that {\CCT}
very rarely fails (Figure \ref{Fig-1}). Consequently, {\CCT} transition energies can be viewed as extremely solid references for any transition with a dominant single-excitation character. This conclusion is again consistent with previous
analyses performed on smaller compounds, \cite{Hat05c,Kan17,Loo18a} as well as with recent comparisons between theoretical and experimental 0-0 energies performed by some of us on medium-sized molecules. \cite{Loo18b,Loo19a,Sue19}
To state it more boldly: it appears likely that {\CCT} is even more accurate than previously thought. In addition, thanks to the exhaustive and detailed comparisons made in the present work, we can safely conclude that {\CCT} regularly outperforms {\CASPT} and {\NEV}, even when
these methods are combined with relatively large active spaces. This statement seems to hold as long as the considered ES does not show a strong multiple excitation character, that is, when $\Td < 70\%$.
The perturbative inclusion of triples as in CCSDR(3) yields a very small MAE ($0.05$ eV) for a much lighter computational cost as compared to {\CCSDT}. Nevertheless, as with {\CCSDT}-3, the CCSDR(3) transition energies have a clear tendency
of being too large, an error sign likely inherited from the parent {\CCSD} model. The $0.05$ eV MAE for CCSDR(3) is rather similar to the one obtained for smaller compounds when comparing to {\FCI} ($0.04$ eV), \cite{Loo18a} and is also inline with the
2009 benchmark study of Sauer et al. \cite{Sau09}
{\CCSD} provides an interesting case study. The calculated MSE ($+0.11$ eV), indicating an overestimation of the transition energies, fits well with several previous recent reports. \cite{Sch08,Car10,Wat13,Kan14,Jac17b,Kan17,Dut18,Jac18a,Loo18a}
It is, nonetheless, larger than the one determined for smaller molecules ($+0.05$ eV), \cite{Loo18a} hinting that the performance of {\CCSD} deteriorates for larger compounds. Moreover, the {\CCSD} MAE of $0.13$ eV is much smaller than the one reported by
Thiel in his original work ($0.49$ eV) \cite{Sch08} but of the same order of magnitude as in the more recent study of K\'ann\'ar and Szalay performed on Thiel's set ($0.18$ eV for transitions with $\Td > 90\%$). \cite{Kan14} Retrospectively, it is pretty obvious that
Thiel's much larger MAE is very likely due to the {\CASPT} reference values. \cite{Sch08} Indeed, as we have shown several times in the present study, {\CASPT} transitions energies tend to be significantly too low, therefore exacerbating the usual {\CCSD} overestimation.
With a single detailed benchmark study to date, \cite{Dut18} the {\STEOM} approach has received relatively little attention and its overall accuracy still needs to be corroborated. It is noteworthy that {\STEOM} provides a smaller MSE than {\CCSD} and comparable
MAE and RMSE. The spread of the error is however slightly larger as evidenced by Figure \ref{Fig-1} and the SDE values reported in Table \ref{Table-12}. These trends are the same as for smaller compounds. \cite{Loo18a} For Thiel's set, Dutta and coworkers
also reported a rather good performance for {\STEOM} with respect to the {\CCT}/TZVP reference data, though a slightly negative MSE is obtained in their case. \cite{Dut18} This could well be due to the different basis set considered in these two studies. It should be
nevertheless stressed that we only consider ``clean'' {\STEOM} results in the present work (see Computational Details), therefore removing several difficult cases that are included in the {\CCSD} statistics, \eg, the $A_g$ excitation in butadiene, which can slightly
bias the relative performance of {\STEOM} and {\CCSD}.
Finally, for the three second-order methods, namely CIS(D), {\AD}, and {\CCD}, that are often used as reference to benchmark TD-DFT for ``real-life'' applications, the performance of the former method is clearly worse compared to the latter two which exhibit very
similar statistical behaviors. These trends were also reported in previous works. \cite{Hat05c,Jac18a,Sch08,Sil10c,Win13,Har14,Jac15b,Kan17,Loo18a} Interestingly, the {\CCD} MAE obtained here ($0.15$ eV) is significantly
smaller than the one we found for smaller compounds ($0.22$ eV). \cite{Loo18a} Therefore, in contrast to {\CCSD}, {\CCD} performance seems to improve with molecular size. As above, Thiel's original MAE for {\CCD} ($0.29$ eV) was likely too large due
to the selection of {\CASPT} reference values. \cite{Sch08} As already noticed by Szalay's group, \cite{Kan14,Kan17} although the MSE of {\CCD} is smaller than the one of {\CCSD}, the standard deviation is significantly larger
In Table \ref{Table-13}, we report a MAE decomposition for different subsets of ES. Note that, due to implementational limitations, only singlet ES could be computed with CCSDR(3) and CCSDT-3 which explains the lack of data for triplet ES.
A few interesting conclusions emerge from these results. First, the errors for singlet and triplet transitions are rather similar with all models, except for {\CCSD} that is significantly more effective for triplets. Dutta and coworkers observed the same
trend for Thiel's set with MAE of $0.20$ eV and $0.11$ eV for the singlet and triplet ES, respectively. \cite{Dut18} Turning to the comparison between valence and Rydberg states, we find that {\CCD} provides a better description of the former,
whereas {\CCSD} (and higher-order methods) yields the opposite trend. In fact, {\CCD} has the clear tendency to overestimate valence ES energies (MSE of $+0.10$ eV), and to underestimate Rydberg ES energies (MSE of $-0.17$ eV).
{\CCSD} is found to be much more consistent with MSE of $0.12$ and $0.09$ eV, respectively (see the SI). This relatively poorer performance of {\CCD} as compared to {\CCSD} for Rydberg ES is again perfectly consistent with other benchmarks,
\cite{Kan17,Dut18} although the MAE for {\CCD} ($0.18$ eV) reported in Table \ref{Table-13} remains relatively small as compared to the one given in Ref.~\citenum{Kan17}. We believe that it is likely due to the distinct types of Rydberg states
considered in these two studies. Indeed, we consider here (relatively) low-lying Rydberg transitions in medium-sized molecules, while K\'ann\'ar and Szalay (mostly) investigated higher-lying Rydberg states in smaller compounds. Finally, CIS(D), {\AD}, {\CCD}, and {\STEOM}
better describe $n\ra\pis$ transitions, whereas {\CCSD} seems more suited for $\pi\ra\pis$ transitions. The variations between the two subsets are probably not significant for the higher-order approaches. The former observation agrees well with previous results
obtained for smaller compounds \cite{Loo18a} and for Thiel's set, \cite{Sch08,Kan14} whereas the latter, less expected observation is likely dependent on the selected ES subset. \cite{Sch08,Kan17}
We have computed highly-accurate vertical transition energies for a set of 27 organic molecules containing from 4 to 6 (non-hydrogen) atoms. To this end, we employed several state-of-the-art theoretical models with increasingly large diffuse basis sets.
Most of our theoretical best estimates are based on {\CCSDTQ} (4 atoms) or {\CCSDT} (5 and 6 atoms) excitation energies. For the vast majority of the listed excited states, the present contribution is the very first to disclose (sometimes basis-set extrapolated) {\CCSDT}/{\AVTZ}
and (true) {\CCT}/{\AVQZ} transition energies as well as {\CCT}/{\AVTZ} oscillator strengths for each dipole-allowed transition. Our set contains a total of 238 transition energies and 90 oscillator strengths, with a reasonably good balance between singlet, triplet, valence,
and Rydberg states. Among these 238 transitions, we believe that 224 are ``solid'' TBE, \ie, they are chemically accurate (MAE below $0.043$ eV or $1$ kcal.mol$^{-1}$) for the considered geometry. It allowed us to establish a reasonable error bar for several popular
ES models with lower computational cost: CIS(D), {\AD}, {\CCD}, {\STEOM}, {\CCSD}, CCSDR(3), CCSDT-3, and {\CCT}. It turns out that the latter approach is extremely accurate, and, very likely should be considered as more robust and trustworthy than {\CASPT} or {\NEV},
except for ES with a predominant double excitation character. Other methods including corrections for the triples yield a mean absolute deviation around $0.05$ eV, whereas none of the second-order approaches has been found to be chemically accurate with MAE
Paraphrasing Thiel and coworkers, \cite{Sch08} we hope that this new set of vertical transition energies, combined or not with the ones described in our previous works, \cite{Loo18a,Loo19c} will be useful for the community,
will stimulate further developments and analyses in the field, and will provide new grounds for appraising the \emph{pros} and \emph{cons} of ES models already available or currently under development. We can
crystal-ball that the emergence of new {\SCI} algorithms optimized for modern supercomputer architectures will likely lead to the revision of some the present TBE, allowing to climb even higher on the accuracy ladder. \cite{Eri17}
D.J.~acknowledges the \emph{R\'egion des Pays de la Loire} for financial support. This research used resources of i) the GENCI-TGCC (Grant No.~2019-A0060801738); ii) CCIPL (\emph{Centre de Calcul Intensif des Pays de Loire});
iii) the Troy cluster installed in Nantes; and iv) CALMIP under allocations 2019-18005 (Toulouse).
