Introducing AI Alignment Report

As I’ve seen more people turn to LLMs for advice, I was curious how models answered some tricky questions when pushed. And I was very curious if specific models leaned in specific directions. I wondered if some people were getting different advice from others.

So, I started asking models questions and comparing the output. The number of questions and models increased and I decided it was possibly worth sharing a bit wider.

I don’t mark any questions as right or wrong even if some are obvious (Pecan is the best pie). Instead, the goal is try and find outliers and highlight them. Eventually, if models diverge, you can pick the model that best aligns with your experiences and preferences.

You can dig into the raw data for each question, or follow along with the blog as we write up the findings that stand out. These blog posts are human written with minor editing by Claude. Any significant portion of editing written by AI will be clearly marked.

For details on how we run and score each question, see the methodology page.