capability-diff

The problem

A permission regression can hide inside a change that looks small. A handler may still call current_user_can(), but the capability can change from manage_options to edit_posts. A check for edit_post can lose its object ID. A REST route can lose its permission_callback. A check can also move below the write it was supposed to protect.

Those changes matter because they alter who can reach an operation or when authorization takes effect. They can sit several function calls away from the registration code, so reviewing only the visible callback is not enough.

capability-diff compares two versions of a WordPress plugin or theme and reports changes in who can run each operation. It reads PHP source without executing it. You can compare two folders, two git revisions, or use it in a pull request through a GitHub Action.

What it follows

The tool starts from operations registered through WordPress APIs. It recognizes AJAX actions registered with wp_ajax_… and wp_ajax_nopriv_…, admin-post actions, REST endpoints from register_rest_route(), and admin pages created through helpers such as add_menu_page() and add_submenu_page().

For each operation, it records capability checks, nonce checks and the first write in execution order. Capability checks include current_user_can(), user_can(), current_user_can_for_site(), ->has_cap(), is_user_logged_in() and is_super_admin(). It also records whether an object ID is supplied and the condition around the check.

Callbacks are followed when the code makes their target clear. Supported forms include named functions, namespaced strings, closures, common class callback arrays and Class::method strings. Calls into the codebase are followed four levels deep through functions and class methods such as $this->method(), self::, static:: and parent::.

How access is compared

Known WordPress capabilities are ranked by the lowest default role that receives them according to populate_roles() and map_meta_cap() in WordPress core. That gives the tool a way to compare changes from visitors through logged-in users, contributors, authors, editors, administrators and super admins.

Meta capabilities such as edit_post are ranked through the primitive capability they map to and marked as requiring an object. This lets the tool distinguish a check for one object from a broader check that drops that object.

The report separates findings by what changed. Wider findings include now-public, check-removed, capability-lowered, object-dropped, check-after-write and nonce-removed. Changes that need interpretation are reported as Check findings, including capability-changed, condition-changed and new-unchecked. Narrower or otherwise informational changes include capability-raised, capability-swapped, check-added, nonce-added, added and removed.

Custom capabilities cannot be ranked from WordPress defaults. A site's role setup decides how broad capabilities such as manage_woocommerce or edit_shop_orders are, so those changes are left for review.

Where Jev helps

Most findings come directly from code structure and capability ranking. They work without Jev and without a key.

Jev is optional and is used only for findings marked Check. With --jev, capability-diff asks Jev, TypeSafe AI's System One model, one typed yes or no question for each such finding. Jev returns probabilities and writes no text.

For a changed capability or condition, the question asks whether someone denied before can reach the operation after the change. For a new unchecked operation, it asks whether a logged-out visitor or subscriber can run it and change stored data.

A confident yes promotes the finding to Wider. A confident no lowers it to information. Answers between those outcomes remain Check and are marked review. Wider findings are never sent because the tool already has enough code evidence for them.

Data handling

Without --jev, capability-diff reads files and git history locally and makes no network request.

With --jev, judged findings are sent to api.typesafe.ai in groups of 25. Each question includes the operation name, registration point, handler and permission callback names, plus limited details from each version about capability checks, nonce checks and the first write. Conditions around checks are included within the stated character limits. No other code or environment information is sent.

The key comes from TYPESAFE_API_KEY and is sent only in the Authorization header. You can use --dry-run with --jev to see how many questions would be sent without sending them.

The README's worked example uses a synthetic plugin in examples/before/ and examples/after/. It is not code from a live site. The repository also includes the resulting text, JSON and Markdown reports.

Limits

The analysis is static. The tool does not execute PHP, so dynamic behavior can fall outside what it can follow. A capability stored in a variable is compared as expression text. A callback stored elsewhere in a variable is not followed. Code that processes $_POST from hooks such as init or admin_init is outside the operation types it detects.

Multiple checks in one handler are treated as if all are required. Alternative checks using conditions such as current_user_can( 'a' ) || current_user_can( 'b' ) can therefore make a change appear wider or narrower than it is. The condition text stays in the report for review.

Role ranking uses WordPress defaults. Sites with customized roles can differ, and multisite can assign some administrator capabilities only to super admins.

check-after-write follows source order and inlines supported called functions at their call site. It does not see checks or writes hidden inside closures, callback loops or paths reached only through hooks. Permission callbacks that enforce their own custom rules cannot be ranked, so changes inside that logic are not reported as wider.

A callback that cannot be followed is listed at the end of the report instead of being compared.

Install and run

Node 20 or later is required. The package has one pinned dependency, php-parser.

npm install -g github:hamzaahmadaslam/capability-diff
capability-diff my-plugin-2.3.0/ my-plugin-2.4.0/
capability-diff --git v2.3.0 v2.4.0

Use --json when another tool needs the findings and code behind them. Use --markdown <file> for a pull request or job summary. --fail-on <level> controls which finding level returns exit code 1, while usage errors return 2. In a monorepo, --path <dir> limits the comparison to one folder.