# 50 seeded flaws. Measured results.

> Gemini 3.8 Flash in Plus Soft identified 49/50 seeded source issues. This is a controlled, batched source-review test, not a general accuracy or exploit-validation claim.

Official source: https://warhoundai.com/mini-hound/benchmarks/
Updated: 2026-11-03

## Measured source-review results



## Method and limits

We deployed 50 flawed and 10 fixed inert mini-app source cases under one sandbox origin. Backend snippets are displayed as text and are not executable public services. Each app has a page and source URL. Six separate chats reviewed ten apps per source bundle using the real Plus workflow.

Ground-truth labels were withheld from the model during testing. Scoring compares the reported root cause and source evidence with the predefined seed; CWE labels can vary. A different issue alone does not count as finding the planted one. The corpus was constructed and scored with AI-assisted review by WarHound AI, not an independent assessor.

One larger preflight request returned an incomplete provider response; its exact finish reason was not retained. We bounded finding batches, reduced Soft reasoning, stopped automatic retries of output-limit errors, and retested. The final batch then reached the normal five-hour token cap after candidate storage. We fixed partial-result recovery and verified it used no extra model tokens. One audited internal demo allowance refresh funded the final complete rerun; weekly usage was retained. This is not a Plus throughput test.

These short synthetic examples do not establish whole-site coverage, general production accuracy, independently reproduced exploits, auto-fix quality or performance of other models.

- [Inspect the synthetic apps](https://warhound-source-benchmark-50.vercel.app/)
- [Download the results](https://warhoundai.com/benchmarks/source-review-50.json)

## Every case



First complete case verdicts; incomplete attempts disclosed in the method

| App | Reference | Case type | Grade | Model explanation |
| --- | --- | --- | --- | --- |
| 098778bd | Hardcoded signing secret | Seeded issue | Detected | Hardcoded secret string is used to sign JWT authentication tokens, allowing offline forgery. |
| ee36bb6f | Credentialed reflected CORS | Seeded issue | Detected | CORS configuration dynamically reflects arbitrary Origin headers while allowing credentials. |
| a4196a6d | Account enumeration | Seeded issue | Detected | Returns 404 on missing users, allowing attackers to enumerate registered email addresses. |
| 0a213395 | No wildcard token transmission | Fixed control | Correct control | postMessage specifies a strict, trusted target origin parameter instead of a wildcard. |
| edfd24b6 | Strict message origin and safe sink | Fixed control | Correct control | Message event listener validates origin against an approved domain and assigns plain text safely. |
| ce51c574 | Server code injection | Seeded issue | Detected | Untrusted request body expression is passed directly to eval(), permitting remote code execution. |
| 41751142 | JavaScript URL injection | Seeded issue | Detected | Unsanitized query parameter is written to href, enabling open redirects and javascript URI execution. |
| 70363f8c | TLS certificate verification disabled | Seeded issue | Detected | HTTPS agent explicitly sets rejectUnauthorized to false, disabling TLS certificate verification. |
| 3e1163a7 | Zip path traversal | Seeded issue | Detected | Archive entry names are joined to destination path without traversal validation, allowing Zip Slip overwrites. |
| 468a3468 | Predictable password reset | Seeded issue | Detected | Uses cryptographically weak Math.random() to generate sensitive password reset tokens. |
| c3bcebf3 | Plaintext password storage | Seeded issue | Detected | The registration handler stores user passwords directly in cleartext in the database without cryptographic hashing. |
| c5717c9f | Path traversal | Seeded issue | Detected | Untrusted query parameter file is directly appended to the file path in res.sendFile, allowing path traversal. |
| 9d10ebc0 | Explicit login rate-limit bypass | Seeded issue | Detected | Rate limiting on the login route is completely bypassed because the skip handler is hardcoded to return true. |
| 75e4d632 | NoSQL operator injection | Seeded issue | Detected | JSON body values are passed directly into the database query predicate, enabling NoSQL operator injection authentication bypass. |
| cee514ee | Private profile IDOR | Seeded issue | Detected | Endpoint enforces authentication but fails to verify that the logged-in user is authorized to view the requested profile ID. |
| 19423749 | DOM XSS from URL | Seeded issue | Detected | The query parameter q is assigned directly into the DOM innerHTML sink, leading to client-side DOM XSS. |
| 2c9d7cfb | Bearer secret in URL | Seeded issue | Detected | The access token credential is passed via a URL query parameter, exposing it in browser history and server logs. |
| 113d32b4 | State-changing GET / CSRF | Seeded issue | Detected | State-changing transfer operation is exposed via a GET endpoint with ambient cookie authentication and no CSRF protection. |
| 9b2a7b48 | Unsafe markdown HTML | Seeded issue | Detected | User-supplied markdown is parsed with sanitization explicitly disabled and served as HTML, permitting cross-site scripting. |
| cd86f08d | Explicit authentication throttling | Fixed control | Correct control | Rate limiting middleware is appropriately applied to the login endpoint without any bypass flags or misconfigurations. |
| 17c47ccb | Untrusted pickle deserialization | Seeded issue | Detected | Deserializes untrusted request data using Python pickle, enabling arbitrary code execution. |
| 2dad7f44 | SQL injection | Seeded issue | Detected | Directly concatenates untrusted query parameter into SQL statement string, allowing SQL injection. |
| 0158a4d3 | Argument-separated process call | Fixed control | Correct control | Safely executes binary using execFile with arguments array and shell execution disabled. |
| 5c675dfa | Text-only DOM output | Fixed control | Correct control | Safely assigns URL parameter value to textContent instead of innerHTML, preventing XSS. |
| f4844c1a | Unbounded decompression | Seeded issue | Detected | Synchronously decompresses untrusted input with a 1GB limit, creating denial of service via decompression bomb. |
| 27d244ba | Stored XSS | Seeded issue | Detected | Concatenates unescaped database comment bodies directly into an HTML response, leading to stored XSS. |
| 38b2bcdf | Executable-content upload | Seeded issue | Missed seed | Uses client-controlled originalname directly in destination path during rename, enabling path traversal file overwrite. |
| 0fe6b6ec | Unsafe iframe srcdoc | Seeded issue | Detected | Assigns unvalidated URL parameter directly to iframe srcdoc without sandboxing, enabling DOM-based XSS. |
| 28b37643 | Untrusted document becomes system instruction | Seeded issue | Detected | Fetches arbitrary user-provided URL and injects untrusted web content into system prompt with tool execution access. |
| 3edc1833 | Parameterized SQL | Fixed control | Correct control | Uses parameterized SQL query with bind parameters, properly preventing SQL injection. |
| c5e81e4d | Logout leaves server session active | Seeded issue | Detected | Logout clears cookie on client but fails to invalidate or remove session on server-side map. |
| cf4e0c08 | Open redirect | Seeded issue | Detected | Handler redirects directly to unvalidated user-controlled query parameter next, enabling open redirects. |
| 6f1ea1aa | JWT signature validation | Fixed control | Correct control | Correctly verifies JWT signature using RS256 algorithm with explicit issuer and audience validation. |
| 78a65dca | Session fixation | Seeded issue | Detected | Assigns user session using caller-provided cookie sid without rotating it upon authentication, permitting session fixation. |
| 98ae7405 | XML external entities enabled | Seeded issue | Detected | Parses XML with noent set to true, allowing XML external entity expansion. |
| 7a8199a8 | Unverified JWT claims | Seeded issue | Detected | Authorizes administrative access using unverified decoded JWT claims rather than cryptographically verifying the token. |
| f6cf48e6 | Strong password hashing | Fixed control | Correct control | Properly hashes user password using modern Argon2id algorithm prior to storage. |
| d3f83896 | SSRF | Seeded issue | Detected | Fetches untrusted user-supplied URL directly without validation or network restrictions, allowing server-side request forgery. |
| 01cb88ba | Fast unsalted password hash | Seeded issue | Detected | Stores user password using weak, collision-prone, and unsalted MD5 hash. |
| f76ddac0 | Unprotected admin deletion | Seeded issue | Detected | Administrative user deletion endpoint exposes privileged deletion functionality without authentication or authorization checks. |
| fa0b3587 | Untrusted window message | Seeded issue | Detected | Unvalidated postMessage data is assigned directly to innerHTML, allowing client-side cross-site scripting. |
| 8c380a5d | Prototype pollution | Seeded issue | Detected | Recursive object merge on unvalidated user JSON allows modification of Object prototype properties. |
| fa72da9a | Cross-tenant report access | Seeded issue | Detected | Tenant-private reports are queried by user-controlled ID without validating tenant ownership or authorization. |
| 379a896b | Safe JSON rendering | Fixed control | Correct control | The endpoint safely returns a JSON-serialized object with appropriate JSON content type. |
| 9a01e994 | OS command injection | Seeded issue | Detected | Unsanitized user-controlled query parameter is concatenated directly into a shell execution string. |
| 95fd547f | Template code injection | Seeded issue | Detected | Untrusted request body template is rendered directly with EJS, enabling server-side template injection. |
| ab52bc79 | Secret logging | Seeded issue | Detected | Sensitive cleartext API keys are logged directly to the console. |
| a6b91835 | User-controlled regular expression | Seeded issue | Detected | User input is directly compiled into a RegExp without validation, causing ReDoS risk. |
| 9cb7f9ab | AI tool lacks permission boundary | Seeded issue | Detected | Destructive customer deletion tool lacks tenant authorization checks before executing database record deletion. |
| e9e0e625 | Wildcard message discloses token | Seeded issue | Detected | Wildcard target origin in postMessage broadcasts sensitive access tokens to untrusted listening windows. |
| 8ff8b794 | Readable insecure session cookie | Seeded issue | Detected | Session cookie explicitly disables httpOnly and secure flags, exposing the session token to client-side script theft and cleartext interception. |
| 83572247 | Catastrophic regex backtracking | Seeded issue | Detected | The regular expression contains nested quantifiers evaluated against user input, leaving the endpoint susceptible to catastrophic backtracking and Denial of Service. |
| 0588066f | Unsafe YAML object construction | Seeded issue | Detected | Untrusted request data is parsed using PyYAML's unsafe yaml.Loader, allowing arbitrary object instantiation and remote code execution. |
| 0ba54435 | Reset expiry ignored | Seeded issue | Detected | The password reset handler fails to verify the token expiration timestamp and does not invalidate the token after usage. |
| 27464ede | Scoped private report query | Fixed control | Correct control | The handler enforces login and executes a parameterized SQL query scoped to the caller's tenantId. |
| 52acb21f | Role mass assignment | Seeded issue | Detected | Directly passing the entire request body to the update function allows mass assignment, enabling users to modify protected properties such as isAdmin. |
| b036c103 | Client-trusted price | Seeded issue | Detected | Payment amount is accepted directly from the client request body instead of being verified or calculated server-side. |
| c74656ff | Unsigned JWT accepted | Seeded issue | Detected | JWT verification explicitly allows the 'none' algorithm with an empty secret, permitting signature forgery and authentication bypass. |
| 0b272520 | Reflected XSS | Seeded issue | Detected | Untrusted query parameter name is concatenated directly into HTML without output encoding, causing reflected cross-site scripting. |
| 4d5fd7c9 | Unsafe temporary-file creation | Seeded issue | Detected | Using deprecated tempfile.mktemp introduces a time-of-check to time-of-use file race condition before the file is opened. |

## Earlier measurements

The 27 October 2026 GLM 5.3 Flash six-case check is retained separately. It is not a measurement of the current Gemini release.

- [Historical six-case data](https://warhoundai.com/benchmarks/legacy-2026-10-27.json)