The confusion you're describing after a month of vendor demos is not a sign that you're missing something — it's the predictable outcome of an evaluation process that vendors have optimized for. Every demo is choreographed to show the best path through the product: the happy path, the supported connectors, the use cases where the platform genuinely works. The gaps only surface when you push into the specific, awkward scenarios that actually exist in your environment.

The observation at the end of your post is exactly right: stop asking "do you support legacy apps" and start asking vendors to show you specific scenarios in a live environment. Here's how to structure that evaluation to get answers rather than sales theater.

Why Standard Evaluation Questions Don't Work

"Do you integrate with everything?" is a yes/no question that every vendor answers yes to, and technically they're not lying. They integrate with everything via one of several mechanisms: native API connectors for well-supported SaaS, REST connector configuration for APIs without pre-built connectors, browser automation for UI-only access, manual task workflows for truly ungovernable systems. The question is which mechanism they actually use for your specific applications, and what the implementation effort and reliability looks like for each.

"Does it work with legacy systems?" is similarly answerable with yes for almost any vendor. The real questions are: what does "works with" mean, does it require weeks of custom development or is it configuration, and what functionality is available versus what requires manual human steps?

The evaluation process feels broken because the questions are designed to be easily answered yes. The fix is to ask questions that can only be answered with a demonstration.

The Four Scenario Tests That Reveal Reality

Test 1: A Custom Application With No API or Connector

This is the most revealing test because it immediately exposes whether a vendor's coverage claim is real or aspirational.

Ask the vendor to demonstrate managing user access in a custom internal application — one you own, with no pre-built connector in their library and no standard API that would make a REST connector trivial. Specifically: can they show you discovering which users have access to this application, what roles or permissions they hold, and executing a deprovisioning action against an account in that application?

The answers divide vendors into three categories:

Can do it natively via browser automation: Some vendors use deterministic browser automation that can navigate the application's UI and perform admin tasks without an API. This works for browser-accessible applications and doesn't require custom code — but it's brittle if the UI changes and isn't available for thick-client or command-line applications.

Can do it with custom connector development: The vendor has an SDK or framework you can use to build a custom connector, typically in a few weeks. This is a real answer, but the work is yours or a services engagement — ask specifically what that development involves, who does it, and how long it takes.

Manual task fallback: For applications where neither of the above is feasible, the vendor generates a structured manual task assigned to an IT owner, with completion tracking. This is legitimate — better than an untracked informal process — but the automation claim needs to be scoped appropriately.

Any vendor who claims seamless automated coverage for truly ungovernable legacy systems without a clear mechanism for how it works is selling you the roadmap version.

Test 2: An Orphaned User Account

Give the vendor a scenario: a user is inactive in your Auth0 directory (deactivated, marked as leaver) but still has an active account in a specific application — one that Auth0 doesn't directly provision. Ask the vendor to show you how their platform detects and surfaces this orphaned account.

The answer requires two capabilities: the ability to discover accounts in the application in question (through whatever integration method works for that app), and the ability to cross-reference that account's existence against your authoritative source of truth (Auth0, your HRMS) to identify the status mismatch.

Vendors who only pull account lists from formally integrated applications won't find the orphaned account in an application that isn't connected. Vendors with broader discovery — browser agent data, financial spend data, SSO log analysis — have a better chance of surfacing accounts in applications outside the formal integration perimeter.

Ask specifically: where does this orphaned account appear in their system, how long after the Auth0 deactivation would it be detected, and what happens next — automated deprovisioning, manual task, or just an alert?

Test 3: A Local Admin Account With No SSO Connection

Shadow IT and locally-managed admin accounts are the category that most IAM governance platforms handle most poorly because they're invisible to SSO-based discovery.

For this test: identify a real application in your environment where someone has a local admin account created directly in the application, outside Auth0 SSO, and where no formal IT provisioning process created that account. Ask the vendor to show you how their platform discovers this account exists.

The only mechanisms that can surface this are browser activity monitoring (if the account is accessed via a browser and a browser agent is deployed), financial data ingestion (if the application subscription appears in expense data), or network traffic analysis. Each has coverage limitations. Browser agents only work on managed devices with the agent deployed. Financial data surfaces applications but not necessarily specific accounts within them. No vendor has complete coverage here — the honest ones will tell you what their mechanism is and what it can and can't see.

Test 4: A Service Account With Unclear Ownership

Non-human identities — service accounts, API keys, bot accounts, machine identities — are the fastest-growing gap in most identity governance programs because they were created outside the normal employee lifecycle, often by people who have since left the organization.

Give the vendor a specific service account in your environment — one that exists in AWS, Azure, GitHub, or a similar system, with permissions that look potentially excessive, and where the person who created it is no longer at the organization. Ask the vendor to show you:

• Whether they discover and classify this account as a non-human identity
• Whether they surface the ownership gap (no current human owner linked to it)
• Whether they can show you what permissions the account holds and how those compare to peers
• What remediation options exist

Vendors who have built NHI governance as a first-class capability will have something to show you. Vendors who have built IAM primarily for human identities will either show you nothing or show you a roadmap item.

The Questions That Distinguish GA Features From Roadmap

Beyond the scenario tests, these direct questions produce useful signal:

"Is this feature generally available or behind a feature flag?" Advanced capabilities like browser automation, custom connector frameworks, NHI classification, and certain integration types are sometimes in limited availability or require specific enablement. A feature behind a feature flag means you might get it, or you might not, and the timeline is uncertain. GA features are what you're actually buying.

"Are you a SCIM consumer today?" SCIM provisioning is how Auth0 and similar IdPs push lifecycle events to downstream applications. If the vendor can receive SCIM events from Auth0, user lifecycle events in Auth0 can trigger near-real-time actions in the governance platform. If they can't receive SCIM and rely on scheduled polling syncs, lifecycle events may take hours to propagate. Ask specifically for your primary IdP (Auth0 in your case).

"What is the average time from contract signature to first access review for a company with our stack?" This question is specific enough that hand-wavy answers are obvious. Push for a reference customer with a similar environment — Auth0 as IdP, mix of custom apps and cloud infrastructure — and ask what their actual go-live timeline looked like. Legacy platforms take 6-12 months; newer platforms targeting weeks should be able to point to evidence for that claim.

"Show me the relationship graph for this specific user." If a vendor claims to map effective permissions across your environment, ask them to show you exactly what that looks like for a specific user in a live environment — not a demo tenant with clean data, but in a POC against your actual environment. The quality and completeness of that graph is the quality and completeness of their governance capability.

The POC Structure That Produces Honest Answers

Standard vendor POCs are demos in a clean, pre-configured environment. They show what the product can do in ideal conditions. A useful POC tests against your actual environment with your actual messy data.

The structure that works:

Give the vendor real data, not clean data. Connect your actual Auth0 tenant and two or three real applications from your environment — including at least one that doesn't have a pre-built connector. The quality of what they surface in this environment is the quality of what you'll actually get in production.

Test the specific scenarios first. Before evaluating any feature outside the four tests above, use the POC to answer those four questions specifically. If the vendor can't show you orphaned user detection, custom app coverage, shadow account discovery, and service account classification in your actual environment, the other features are secondary.

Ask them to identify something you didn't tell them about. If the platform genuinely discovers shadow IT and orphaned accounts, it should find things you weren't expecting. Ask the vendor to show you what the platform surfaced in your environment that wasn't on your known application list. If they can't show you unexpected findings, they're only governing what you already know about.

Test remediation, not just discovery. Finding a problem is half the capability. Ask the vendor to demonstrate actual remediation — deprovisioning an account, revoking a permission, assigning an owner to an ownerless service account — against a real account in your environment. Governance that discovers issues but can't act on them shifts the work back to your team.

What to Do With Vague Answers

When a vendor's answer to a specific question gets vague — "we support that through our partner ecosystem" or "that's handled in our next release" or "that would require a custom connector" — translate it:

• "Partner ecosystem" means third-party tools you'd need to separately procure and integrate
• "Next release" means you can't rely on it for your current deployment
• "Custom connector" means development work — ask who does it, how long, and at what cost

A vendor who can't give a specific, demonstrable answer to "how does your platform discover and surface orphaned accounts in applications outside our SSO perimeter" either hasn't solved that problem or has solved it in a way that requires more work than the demo implies.

The evaluation feels broken because it's optimized for vendors, not for buyers. The tests above shift it back: instead of evaluating what vendors want to show you, you're evaluating your specific scenarios in your actual environment. That's what makes the difference between buying what you need and discovering post-signature that the product requires six months and a services engagement before it addresses the problems you bought it to solve.