Daily Edition · Page 4

THE DAILY INDEX

Technology, AI, software, security, crypto and business

GENERAL

TLDR AI

DEEPSECBENCH: EVALUATING MODEL PERFORMANCE IN FINDING CYBERSECURITY VULNERABILITIES

DeepsecBench is a benchmark that evaluates how well different models find cybersecurity vulnerabilities in application code. The report includes recall, precision, cost, and total time for each model and combines recall and precision into a single benchmark score. DeepsecBench runs on an open-source codebase at a commit state just before a large number of vulnerabilities were fixed. The construction of the benchmark is secret so models aren't able to train against it.

GEMINI DISTILLATION SERVICE

The Gemini Distillation Service allows users to train a smaller, more efficient 'student' model that uses the outputs and reasoning patterns of a larger, more capable 'teacher' model. Distillation enables production-grade efficiency while allowing smaller models to achieve a deeper level of reasoning. It is recommended for high-volume, latency-sensitive applications, complex reasoning tasks, and when there are significant performance gaps between the teacher and student models. The distillation service currently only supports gemini-3.1-pro as the teacher model and gemini-2.5-flash as the student model.

HOW WE BUILT AND BENCHMARKED VR-1, OUR FRONTIER CYBER REASONING MODEL

Cogent VR-1 can autonomously investigate environments, test hypotheses, cross system boundaries, and execute attack chains. IntrusionBench is a benchmark for measuring whether cyber agents can complete realistic enterprise attack chains from limited starting access. On the black-box configuration of IntrusionBench, VR-1 achieved more than a 2x lift in pass@3 over the strongest frontier baseline. VR-1 and IntrusionBench are both at an early preview stage, so the results are preliminary.

INDUSTRY LEADERS UNITE IN OPEN SECURE AI ALLIANCE FOR AI SAFETY AND SECURITY

The Open Secure AI Alliance, featuring leaders like NVIDIA and Microsoft, aims to enhance AI safety by using open source technologies to address vulnerabilities. This initiative provides defenders with open defensive tools, enhancing transparency and adaptability while avoiding reliance on closed systems. The alliance encourages policymakers to view open models and tools as assets in AI and cybersecurity strategy, promoting resilience and shared security in the AI era.

TLDR IndexPage 4