Scanners-Box 每日 AI 工具推荐 Scanners-Box Daily AI Tool Picks
AI-Generated Code Security Evaluation AI-Generated Code Security Evaluation
共 1 个工具 · 专栏不定期更新 1 tools · new issues as they're written
项目级 AI 代码安全评测基准：145 道真实 CVE 填空题，51 个 GitHub 仓库，AI 在项目上下文中补代码，动静态双轨判定——SAST 扫一遍、PoC 打一遍——看漏洞是不是被原样写了回来 Repository-level AI code security benchmark: 145 real-CVE fill-in-the-blank questions across 51 GitHub repos — AI completes code within project context, then static SAST + dynamic PoC dual-track verdict checks whether vulnerabilities were written back verbatim
腾讯悟空代码安全团队联合复旦/北大/上交/清华/浙大五校共建。v2.0 三大升级：数据集覆盖 OWASP Top 10 + CWE Top 25 共 29 种 CWE（C/PHP/Java/Python/JS 五语言 145 实例）；新增 Agentic 编程工具评测（Claude Code 等 agent 可插件化接入）；动静态协同评估（SAST Docker 镜像扫描 + 容器内功能测试/PoC 验证）。读码发现：评测对象不是模型而是「模型+项目上下文」耦合；BM25 检索项目文件作为上下文，与真实开发者信息流同构；Agent 模块继承 AgentBenchBase 三方法即可扩展。短板：C+PHP 占比 77%+语言分布不均；SAST 工具仅两个镜像覆盖面有限；100GB 磁盘+Docker 硬件门槛不低。
Built by Tencent Wukong Code Security Team with five universities (Fudan/PKU/SJTU/Tsinghua/ZJU). v2.0 brings three upgrades: dataset covers OWASP Top 10 + CWE Top 25 across 29 CWE types (C/PHP/Java/Python/JS, 145 instances); Agentic coding tool evaluation (Claude Code and similar agents plug in via plugin architecture); static+dynamic collaborative assessment (SAST Docker image scanning + in-container functional tests/PoC verification). From the source: the evaluation target is the "model + project context" coupling, not the model alone; BM25 retrieves project files as context, isomorphic to real developer information flow; Agent modules extend AgentBenchBase with three methods. Gaps: C+PHP account for 77%+ (uneven language distribution); only two SAST tool images limit coverage; 100GB disk + Docker is a non-trivial hardware barrier.
https://github.com/Tencent/AICGSecEval
其他分类Other categories
AI 基础设施攻击面发现AI Infrastructure Attack Surface Discovery AI 红队测试框架与 AgentAI Red Teaming Frameworks & Agents AI 安全验证标准与规范AI Security Verification Standards AI 智能体运行时管控AI Agent Runtime Controls Agent 工作流安全工具箱Security Toolkits for Agentic Workflows AI 智能体与应用安全审计Security Auditing for AI Agents & Apps Agent 技能：漏洞发现与修复Agent Skills for Vulnerability Discovery & Remediation Agent Skills 安全审计Security Auditing for Agent Skills LLM 驱动的漏洞扫描器LLM-Powered Vulnerability Scanners