Google's research tools have made a significant breakthrough, identifying vulnerabilities in one of the world's most widely used database engines. The team recently shared this milestone in an official blog post, marking the first application of AI-driven vulnerability detection in real-world software.

从午睡到长眠：使用大型语言模型捕捉真实代码中的漏洞 https://googleprojectzero.blogspot.com/2024/10/from-naptime-to-big-sleep.html

The discovered vulnerability is a stack buffer underflow in , which could allow malicious actors to manipulate data in a way that compromises database integrity. The vulnerability was discovered and reported in early , and the development team patched it on the same day, preventing any actual impact on users. Researchers stated, "We believe this is the first time a proxy has identified an unknown exploitable memory safety issue in widely used real-world software."

Earlier this year, at the event, the Atlanta team discovered a null pointer dereference in , which inspired us to use it for testing to see if we could find more severe vulnerabilities.

使用基于 LLM 的系统自主发现并修复 SQLite3 中的隐藏漏洞 https://team-atlanta.github.io/blog/post-asc-sqlite/

AI-driven vulnerability research, originating from an early research framework, demonstrates the potential of large language models () in vulnerability research. Unlike traditional testing tools, it focuses on identifying edge cases that traditional fuzz testing methods might overlook. As a result, it acts as an enhanced "variant analysis" system, screening code for complex bugs similar to previously discovered vulnerabilities.

According to the team, using this tool for such variant analysis could be a game-changer. By examining recent changes in the code and matching patterns of past issues, it provides an active defense mechanism that can help turn the tide against cyber attackers. Notably, the tool outperforms existing test frameworks, such as the native testing systems of X and Y.

Using a trained model to fuzz a specific set of code (). Fuzz testing is a testing method where a large amount of input and data is fed into a running software to observe its reactions. The technology extends the existing use of fuzz testers by researchers or developers in their development workflows through its trained model. Today, this approach is fragile and only applicable to a specific codebase, but as it evolves, it will become easier to port to other software, thereby expanding its utility.

Fuzz testing is just one way of leveraging in security research. Another technique currently in use is embedding into the developer's workflow and tools to identify coding flaws that lead to vulnerabilities during software writing and review. These assistants, when combined, are beginning to show promise in reducing the workload of developers and capturing security vulnerabilities before they escape and become vulnerabilities for downstream consumers.

The latest findings from real-world experiments were inspired by auxiliary discoveries made during an event. Researchers identified vulnerabilities in the system during this event. Based on this, the team decided to conduct an in-depth test by examining recent submissions and analyzing changes that could lead to errors. Using a structured approach, the code was analyzed and eventually flagged for a stack buffer underflow issue. This vulnerability revolves around a variable that can accept a marked value -, used to indicate special cases. Due to this unique setup, the system code fails to handle all scenarios, ultimately leading to an exploitable vulnerability. Under specific conditions, this could result in system crashes or unauthorized memory access, posing a potentially severe security risk.

Looking Ahead: The Role of AI in Cybersecurity The success of large language models in transforming cybersecurity highlights their potential. AI models like can address vulnerabilities that traditional methods cannot, helping defenders protect systems faster than criminals can exploit them. For Google and the broader tech industry, this development marks a promising step toward an "asymmetric advantage," where defensive tools can surpass the capabilities of cyber threats. The Google team expressed hope that AI will continue to enhance the resilience of widely-used software and improve global user security.

Integrating into Security Workflows This discovery offers security researchers the potential to leverage generative AI to improve the detection of vulnerabilities in commonly used software components, which are based on pre-trained knowledge and models. Since generative AI is trained on large datasets containing previous vulnerabilities and code patterns, they can identify similar vulnerabilities that traditional testing methods or manual analysis might overlook. The use of AI to discover vulnerabilities presents a new opportunity for cybersecurity practitioners and organizations to consider integrating into their security workflows. While fuzz testing and other automated procedures naturally have weaknesses, AI-assisted vulnerability research can help mitigate these shortcomings. Although issues of hallucination and bias based on training data should be considered, and security teams should review all outputs, the collaboration between human experts and can ensure a robust cybersecurity posture.