The template doesn't currently support this, but we will try to incorporate this in the future.
In the meantime, you can easily modify the template. This is the place to look at:
\ifcolmfinal
\lhead{Published as a conference paper at COLM 2024}
\def\And{\end{tabular}\hfil\linebreak[0]\hfil
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}\ignorespaces}%
\def\AND{\end{tabular}\hfil\linebreak[4]\hfil
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}\ignorespaces}%
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}\@author\end{tabular}%
\else
\lhead{Under review as a conference paper at COLM 2024}
\def\And{\end{tabular}\hfil\linebreak[0]\hfil
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}\ignorespaces}%
\def\AND{\end{tabular}\hfil\linebreak[4]\hfil
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}\ignorespaces}%
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}Anonymous authors\\Paper under double-blind review\end{tabular}%
\fi
Change Published as a conference paper at COLM 2024
=> Preprint. Under review.
in the conference sty
Shouldn't the modification be to "Published as a conference paper at COLM 2024" to preserve the author name info?