Restraint Constraint

“The good news is that Anthropic discovered in the process of developing Claude Mythos that the A.I. could not only write software code more easily and with greater complexity than any model currently available, but as a byproduct of that capability, it could also find vulnerabilities in virtually all of the world’s most popular software systems more easily than before. The bad news is that if this tool falls into the hands of bad actors, they could hack pretty much every major software system in the world, including all those made by the companies in the consortium.” Tom Friedman: Anthropic’s Restraint Is a Terrifying Warning Sign.

+ Anthropic may have gotten to this point first. They won’t be the last. Casey Newton explains why Anthropic’s new model has cybersecurity experts rattled. “One of the world’s three frontier labs has now created a model it says is too dangerous to release to the general public. These dangers emerged not from any specialized cyber training but from the same general improvements that every other lab is currently pursuing. As a result, models with similar capabilities may soon be accessible to criminals, hackers, and nation states — or even more broadly via open source models.”