The menu system will be implemented in a dedicated configuration/setup module. This could simply be a function that prints prompts and collects input, with sensible defaults if the user presses Enter (e.g., default user-agent to a standard one, default timeout = 10 seconds, no proxy, no file output). This keeps the interaction clear and allows the script to run non-interactively as well (if we later add command-line args, we can bypass the interactive prompts by providing all necessary config via args).

🚀 Concurrency and Performance Enhancements

Performance is crucial for a scanner, especially if many endpoints are found. The script will employ concurrency for speed, either through multi-threading or asyncio (or a combination):

• Multi-threaded Scanning: Since the scanning of discovered paths (sending HTTP requests with headers) is an I/O-bound task, we can safely use Python threads to parallelize it. I/O operations release the Global Interpreter Lock, allowing multiple threads to make progress on network requests concurrently​. Using concurrent.futures.ThreadPoolExecutor, we can have a pool of worker threads each handling a subset of the scanning tasks. This can dramatically speed up the process: for example, running 5 threads in parallel could cut down scanning time roughly by a factor of 5, as shown in other web scraping contexts​. We will allow the number of threads to be configurable or choose a sensible default (like 10 threads) balancing speed and server load. Each thread will take URLs from a shared queue of endpoints to test.

• Asyncio Alternative: Alternatively, an asynchronous approach can be used, especially if using Playwright in async mode or httpx for HTTP requests. We could await multiple requests simultaneously. For instance, httpx.AsyncClient can send many requests concurrently and gather results. This approach avoids thread overhead and can be very efficient for a large number of endpoints. However, mixing asyncio with Playwright (which itself can be used asynchronously) might complicate things. A pragmatic solution is to use threading for the HTTP scan phase (since the crawling with Playwright might be easier to manage in synchronous mode).

• Concurrent Crawling: We should also consider parallelizing the crawl if the site is large. Playwright can open multiple pages at once using an async context. We might implement a limited concurrency (e.g., 2-3 pages at a time) for crawling. For example, as we extract new URLs, we could launch a new Page for each if using asyncio. This can be an advanced optimization if needed. Initially, a single-threaded crawl is simpler and fine for moderate site sizes, but the design can note this as an enhancement point.

• Thread-Safety: We will ensure thread-safe handling of shared data. The list of URLs to scan can be processed with ThreadPoolExecutor.map for simplicity, or we can use a thread-safe queue (Python’s queue.Queue) and have threads pull from it until empty. The visited set for crawling is only accessed by the crawler (single thread, unless we do concurrent crawling). The scanner threads will only read from their list of URLs (no modifying shared structures except perhaps logging results, which we can protect with a lock or just collect in a thread-safe list).

• Rate Limiting and Politeness: Since this is a security testing tool, speed is a priority, but we still may want to avoid overloading the target. The user can be advised to set a reasonable thread count. We can also implement a tiny delay or use semaphores to limit concurrency if needed. For example, we might not launch all threads at once if the user’s network or the server could choke. In an advanced scenario, an asynchronous approach could use a semaphore to allow, say, 5 concurrent requests at a time. These details can be adjusted based on testing the script’s performance.

In summary, concurrency will primarily be applied to the scanning phase to test multiple endpoints in parallel. This makes the scanner much faster without sacrificing much accuracy (since each request is independent). As one reference notes, “Multithreading with concurrent.futures can give a significant boost here. We can execute I/O tasks concurrently across multiple threads and see a big speedup”​. Multi-threading is suitable here because network-bound tasks benefit from it even in Python​.

🚀 Core Scanning Logic: Testing Endpoints for the Vulnerability

The heart of the script is the scanning module, which takes the list of discovered endpoints (paths) and checks each for signs of the CVE-2025-29927 vulnerability. The process for each endpoint will be:

• 1. Baseline Request: Send an HTTP GET request to the endpoint without the special header, simulating a normal user request. Record the status code and response body length (or a hash of body) for comparison. Also note any interesting response headers. In particular, if the response contains any of Next.js’s middleware headers like x-middleware-rewrite, x-middleware-next, or x-middleware-redirect, that suggests this route is protected by middleware​. We also check if the status is not 200 (meaning access was denied or redirected), since those are the ones likely to be bypassed. (If the status is already 200 and content loads normally, it’s either a public page or the vulnerability doesn’t apply; we might still test it, but the real interest is in protected pages.)
• 2. Craft Malicious Requests: Send additional requests to the same endpoint, this time including the X-Middleware-Subrequest header. We will try a variety of header values to ensure detection across Next.js versions:

• A generic value like "1" or "true" (some sources imply that simply setting the header to any value triggers the skip​).

• The specific payload used in public exploits, e.g. "middleware:middleware:middleware:middleware:middleware" (five repetitions of "middleware")​. This is known to induce the bypass for the latest versions (13+). We will include exactly this value.

• The alternate payload for projects using the /src directory, e.g. "src/middleware:src/middleware:src/middleware:src/middleware:src/middleware"​

• Optionally, single-segment values like "middleware" or "src/middleware" for completeness (older Next.js versions might use an _middleware file in pages directory, with a slightly different needed payload​, but the multi-segment payloads above largely cover known cases). Each of these requests will be done with the custom header set. We also ensure to use the same method (GET) and include any headers from baseline that might be needed (like cookies or auth tokens if user provided any for a logged-in scan, though typically we scan as unauthenticated).

• 3. Compare Responses: For each header test, compare the response to the baseline:

• If the baseline was an error or redirect (e.g., 401 Unauthorized, 403 Forbidden, or a redirect to login) and one of the header-injected responses is 200 OK with a significantly larger body (or otherwise indicating the page loaded), that is a strong indicator of vulnerability. For example, if /admin returned 403 normally, but with the header returns 200 and contains the admin dashboard HTML, we flag it.

• In some cases, the difference might be a 302 vs a 200, or a 404 vs 200. We will consider a status code change from a non-200 to 200 as a likely sign. Also, if the status remains 200 but the content length changes drastically, that might indicate the header altered the behavior (less common for this particular bug, but a possibility if the page normally delivered one thing and with header delivered another).

• We will implement checks such as: if base_status_code != 200 and test_status_code == 200: (and maybe also ensure test_body_length > base_body_length or contains some authenticated keyword) then flag as vulnerable. If base was a redirect (e.g., 307 to /login), and test yields 200, also flag. Essentially, “was access previously denied but now allowed?”.

• If the response status with the header is 404 or 500 where baseline was a redirect, this could be the cache poisoning scenario (bypass middleware redirect causing a 404 at the origin​). That scenario is a bit harder to detect by just one request, but the presence of a 404 with header when baseline was a redirect might also be noted (though not an auth bypass, it’s still an effect of the vuln). Our focus however is on detecting an auth bypass (200 OK access).

• 4. Logging Results: For each endpoint tested, the script will log the outcome. If no difference is found (not vulnerable), we may keep it in a verbose log or discard it. If a potential vulnerability is found, we record the endpoint, the baseline status, and which header value caused a 200, etc. These will be printed to the console and saved to results.txt if the user opted to save results. We should format this clearly, e.g.:

• [*] /admin -> baseline 403, with X-Middleware-Subrequest (payload X) got 200 [VULNERABLE]

• We can also print something like the response length or a snippet of the response to confirm (maybe just length for brevity, e.g., “len: 0 -> 10240 bytes”). If multiple payloads were tried, we could list which succeeded.

• If the site appears to be not a Next.js application at all (e.g., we didn’t find any /_next/static/ in the homepage, which is a telltale sign​), we might output a note: “No Next.js indicators found, the target may not be using Next.js – likely not vulnerable.” But we can still proceed generically, as a Next.js check is an optimization rather than a necessity.

This logic will be encapsulated cleanly. For instance, we might have a function scan_endpoint(url, session, header_payloads) that returns a result object or dict with whether it’s vulnerable and details. We will incorporate robust checks to avoid false positives. Specifically, requiring a status code change to 200 (or other clear evidence) helps ensure we only flag actual bypasses. As noted in ProjectDiscovery’s analysis, the scanner checks for response status 200 when the special header is included to confirm the vulnerability​.

🚀 Output Formatting and Reporting

The script’s output should be easy to read and interpret, as well as optionally saved to a file. We will format the console output with clear headings and indentation where appropriate. Some considerations:

• After the scan, print a summary of findings. For example: “Scan Complete: 3 vulnerable endpoints found (out of 45 tested).” Then list the vulnerable endpoints with details.

• Use a consistent format for each result line, as shown above, possibly with [VULNERABLE] tags to draw attention. If using a library like rich, we could even color-code “VULNERABLE” in red or yellow. Even without extra libraries, we can use ANSI codes via colorama to highlight, or just uppercase text.

• If no vulnerabilities are found, say so explicitly: “No vulnerabilities detected for CVE-2025-29927.”

• If the results are to be saved, ensure they are written in a similar format to the file. Possibly in a slightly more verbose way or CSV for programmatic use, but since the user specifically mentioned a text file, we will likely just write the same lines into results.txt.

• Also, any critical errors or exceptions encountered (like unable to load a certain page) can be reported in the output in a graceful manner (instead of a stack trace). We can catch exceptions and print a one-line warning per failed URL: e.g., “Timeout loading /blog (skipped)”. This way, the user knows if some paths weren’t tested.

Throughout the execution, we might show a spinner or progress (for long runs) or at least print which page is being crawled or which endpoint is being tested, if verbose mode is on. For a cleaner output, we might only show discovered vulnerable cases at the end, but a running log (perhaps writing to a separate log file) can help with transparency.

Given the emphasis on a clear format, using bullet points or table layout could help when printing multiple results:

• We could tabulate as: Endpoint | Base Status | Base Length | Bypass Status | Bypass Length | HeaderValueUsed | Result

• However, a simple sentence form might be more readable for a wide range of users. We will ensure each result is on a new line and labeled clearly.

By providing both screen output and an optional file save, the tool is useful for both interactive use and automated scanning (where the user can later review the file or integrate it into reports).

🚀 Modular Code Structure and Best Practices

To make the script maintainable and professional-grade, we will organize the code into modules, each handling a distinct aspect of the functionality. A possible project structure:

• crawler.py: Contains the crawling logic using Playwright. It will have functions like crawl_site(start_url, config) -> List[str] that returns a list of discovered internal paths. This module will handle launching the browser, retrieving pages, extracting links, and enforcing filters (domain, static file exclusion). It could also house helper logic to normalize URLs (e.g., remove URL fragments, handle relative paths via urllib.parse.urljoin).

• scanner.py: Contains the scanning logic for the vulnerability. It will include functions such as scan_paths(url_list, config) -> List[ScanResult]. This will manage creating HTTP requests (using requests.Session or an httpx client), applying headers, comparing responses, and collecting results. If using multithreading, this module would create the ThreadPool and manage tasks. It might define a small ScanResult data class to hold info about each path (path, vulnerable: bool, details).

• config.py (or settings.py): Contains code for the user menu and configuration. For instance, a function get_user_config() that interacts with the user and returns a config object/dictionary with all the chosen settings (user_agent, timeout, proxy, output_file flag, etc.). If using CLI args, this module could alternatively parse argparse.ArgumentParser. Essentially, this part isolates all user input and configuration handling.

• utils.py: Utility functions, e.g., for printing banners, formatting output strings, handling color output, or common helper like is_static_resource(url) (to check if a URL likely points to a static file). Also, could include a function for graceful shutdown (to be called on SIGINT).

• main.py: The entry-point script that ties everything together. It will:

• 1. Parse or gather user configuration (using config.py).
• 2. Call the crawler to get the list of endpoints.
• 3. Call the scanner to test those endpoints.
• 4. Receive results and output them in the requested format.
• 5. Ensure any cleanup (closing the browser, closing files, etc.) is done. If distributing as a single script, main could just be at the bottom of one file, but for cleanliness, separating is better.

Each module will be designed to be modular and reusable. For instance, one could reuse crawler.py to get site links for other purposes, or reuse scanner.py to test this vulnerability on a given list of URLs (even without crawling).

Exception Handling and Graceful Shutdown: We will implement robust exception handling:

• Surround network operations with try/except (catch timeouts, connection errors, etc.). If a crawl of a page fails, log it and continue with others. If a scan request fails (e.g., proxy error), mark that endpoint as error but continue scanning the rest.

• Use finally blocks or context managers to ensure resources are cleaned up. For example, use async_playwright() context or ensure browser.close() is called at the end of crawling. Similarly, ensure file handles are closed after writing.

• Handle KeyboardInterrupt (Ctrl+C): We can trap the KeyboardInterrupt in the main loop and initiate a graceful shutdown – e.g., print “Stopping, cleaning up…”, shut down threads (perhaps by using ThreadPoolExecutor.shutdown(wait=False) to stop launching new tasks), and close the browser. This prevents orphan processes or locked files if the user aborts.

• Use logging for debug messages (perhaps via Python’s logging library). In a professional tool, you’d have logging levels; e.g., debug logs could include each request made, while info level only shows high-level progress. The user could set a verbose flag to toggle this. By default, we might log minimal info to not overwhelm the output.

Code Quality: We will adhere to best coding practices:

• Follow PEP8 style guidelines for readability.

• Use meaningful function and variable names.

• Add docstrings to functions explaining their purpose and usage.

• Use type hints for function signatures (Python 3 type annotations) to make the code easier to understand and to catch type issues early.

• Modularize constants (like the list of header payloads, lists of static file extensions to ignore, etc.) at the top or in a config, so they can be easily updated. For example, HEADER_PAYLOADS = ["middleware:middleware:...","src/middleware:..."] etc., defined in one place.

• Possibly include unit tests for some helper functions (if this were a larger project, though for a single-script tool this might be skipped; still, designing with testability in mind is beneficial).

Professional-Grade Enhancements: To make the script more robust and production-ready, we can further consider:

• Authentication support: Allow user to provide cookies or credentials if they want to scan an authenticated section of the site (even though the vulnerability is about bypassing auth, there may be scenarios where you need to log in first to reach certain links to then test bypass on them – albeit the bypass presumably works without valid auth, but this could help in crawling deep links that aren’t public).

• Configuration file: Instead of (or in addition to) interactive input, allow reading options from a config file or environment variables, which is useful for automated deployments of the scanner.

• Output formats: Provide output in multiple formats such as JSON or CSV for integration with other tools. For example, a --json flag could dump the results as machine-readable JSON.

• Integrate with existing frameworks: The logic could be integrated into a larger scanning framework (for instance, turning it into a module for OWASP ZAP or integrating with ProjectDiscovery’s Nuclei by outputting a compatible report). At minimum, ensure the script’s output clearly identifies the vulnerability and affected URLs so it can be used in reports.

• Parallel browser sessions: If targeting very large apps, consider launching multiple browser contexts in parallel for crawling different sections concurrently. Playwright can handle multiple contexts (each context is isolated, akin to separate browser profiles)​. This could speed up crawling significantly at the cost of higher resource usage.

• Graceful degradation: If Playwright fails (say the environment lacks a display or proper installation), the script could fall back to a simpler requests-based crawl (which might miss some links but is better than nothing). This makes the tool more robust in various environments. Similarly, if concurrency is set too high and causes issues, catch those and suggest the user to lower thread count.

By following a clean structure and these best practices, the script will be easier to maintain and extend. Each component can be worked on independently – for instance, improving the crawler’s ability to parse JavaScript-heavy navigation, or updating the scanner with new header payload variations if future research finds additional exploitation patterns.

In conclusion, this design outlines a comprehensive approach to detect CVE-2025-29927 in web applications. It leverages a headless browser for deep crawling, multi-threading for efficient scanning, and robust coding practices for reliability. By comparing responses with and without the special header, it can reliably identify vulnerable endpoints where Next.js middleware is being bypassed​. The result is a professional-grade tool that helps security engineers and developers quickly find and address this critical vulnerability in their applications.