'Terrifying': New AI appears to know when humans are testing it

The latest rival to ChatGPT claims the ability to recognize when people are testing it, according to developers at Anthropic, touting what appears to be a new level of awareness for an AI-powered chatbot.

To assess the capabilities of such chatbots, developers typically run a so-called "needle-in-a-haystack" evaluation, which involves asking the software about a longer text into which an unrelated sentence has been artificially inserted.

The aim is to find out how well the software can identify the relevance of information in its context.