Chris Anley, Chief Scientist at NCC Group said:
"The International AI Safety Report is a very thorough and useful document. Although it underlines the present uncertainty, it also illuminates several emerging trends: AI systems are showing clear improvements in capability.
As an organization, we respond to a great many incidents. Although we are seeing the use of GenAI in social engineering, we are currently seeing little evidence of the use of AI to coordinate and carry out attacks, although we expect this to occur at some point. This would be likely to surface in incident data as an increase in attacker speed and competence, but the signal may be challenging to detect at first.
Security is always an arms race. New software vulnerabilities are continually being found and being patched; new classes of attack are discovered and mitigated. AI can slightly accelerate and slightly improve everything, but also changes things in different ways for attackers and defenders. Attackers get GenAI for social engineering, and as technical attacks get slightly faster and better, defenders get better at detection, analysis and automation.
Defenders have several fundamental advantages:
-
Attackers must take great care with their own operational security, whereas defenders can more easily use the most capable frontier models. Broadly, defenders have access to better AI, which can amplify all of the benefits
-
Models are only as good as their training data. Defensive information is shared, but cutting-edge attack tools and techniques are not
-
Improvements in automated vulnerability research apply to both attackers and defenders, but defenders fix and report vulnerabilities, reducing the scope for attack
Time will tell whether these advantages tip the balance for defenders."
“While it’s reassuring that AI systems can’t yet launch fully autonomous cyber attacks, it doesn’t remove the risk that AI systems still present. Attackers are already using AI technology to find vulnerabilities and develop exploits, enabling a greater volume of attacks to be launched more easily, with less technical human expertise required.
“By automating large parts of the attack chain, AI enables criminals and state actors to operate at a volume and pace that would be impossible using human teams alone. Even with people still involved in carrying out attacks, the barriers to executing sophisticated attacks are falling fast.
“Organisations should be aware that AI-enabled attacks are the new normal, so they should invest in faster detection, robust controls and defensive AI to keep pace with the scale and speed of modern threats. While AI is a threat, it is also a powerful tool, enabling organisations to predict and respond to attacks at scale and stay ahead of cyber criminals. So, it’s key that organisations lean in and integrate AI capabilities into their security strategies to safeguard their digital future.”