Although capable of generating creative texts, LLMs are poor judges of what constitutes creativity''. In this paper, we show that we can leverage this knowledge of \textit{how} to write creatively in order to judge \textit{what} is creative. Our mechanistic approach extracts differences in the internal states of the LLM when being prompted to writeboringly'' or ``creatively'' to provide a more robust measure of creativity that corresponds more strongly with human judgment. We also show these internal state differences can be applied to enhance the creativity of generated texts.