Enterprise search connects people to information spread across document systems, collaboration tools, and internal applications. When an IT or search platform lead evaluates solutions, the key decision points emerge after initial ingestion: how content updates, permission revocations, and deployment boundaries behave under production constraints. A structured comparison evaluates connector coverage, permission freshness, and hosting security before selecting an enterprise AI search platform.

Key takeaways

  • Connector coverage needs testing: A supported source name does not establish coverage for every object type, permission rule, or deletion event.
  • Distinguish indexed and live access: Their freshness, availability, and operating trade-offs differ across architectures.
  • Test access changes end to end: Login and tenant isolation alone do not prove document-level permission enforcement.
  • Separate deployment decisions: Hosting location, network boundaries, and operating responsibilities represent distinct evaluation axes.

Choose between indexed search and live source access

An indexed approach copies selected content and metadata into a search index, with retrieval latency and freshness shaped by index capacity and sync pipeline maintenance. A live (federated) approach queries the source system at request time, bypassing a separate content index while query performance and data availability depend on source API consistency and response times. Teams can combine indexed and live paths, routing queries across indexed repositories and federated sources based on update frequency and system performance constraints.

ApproachData pathFreshness & availability trade-offAuthorization responsibilityPotential fit to validate
Indexed searchContent and metadata extracted to search indexTrade-off between index capacity, sync schedule, and query latencySearch index ACLs or search-time index filteringIndexed workloads that meet quality and capacity needs; source-outage retrieval only where freshness and authorization policies permit
Live (federated) accessDirect query to source system at request timeTrade-off between source API consistency, network latency, and availabilitySource-side authorization for the mapped caller; verify identity mapping and downstream enforcementRapidly changing data, systems with direct query APIs
Mixed modelDual-path: index stable assets, federate dynamic toolsTrade-off dependent on query routing logic across indexed and federated pathsDual enforcement across index ACLs and live authHeterogeneous tool stacks with varied security requirements

Neither model guarantees that every retrieved answer is perfectly fresh or secure. Indexed stores can lag behind source changes, whereas live queries can fail during source downtime or return partial results.

In general architecture terms, operating without a separate content search index does not mean content is never passed to an LLM, excluded from transient logs or caches, or immune to exfiltration risks. Furthermore, while Microsoft implements federated connectors via the Model Context Protocol (MCP), MCP is one specific implementation pattern rather than a universal requirement for all federated search architectures.

Official documentation for Microsoft Copilot connectors and Microsoft federated connectors distinguishes admin-managed, organization-scoped indexing with tenant-configured synced connectors from admin-enabled, user-authenticated, user-scoped indexing with self-serve synced connectors. Federated connectors retrieve live content using the user’s source identity via MCP. Evaluators must verify the supported experience, identity mapping mechanisms, and object/ACL scope for each source connector rather than assuming uniform support across Copilot Search, Microsoft Search, and all Copilot surfaces. The presence of an MCP connection alone does not guarantee permission synchronization or automatic downstream authorization.


Build a connector acceptance matrix

Start with the sources that support core user tasks. For each source, record the exact source edition (cloud vs. self-hosted), connector version, credential scope, identity mapping model, and supported object types (such as file attachments, inline comments, or embedded tables). Content updates and permission modifications must be evaluated as separate test events. Use synthetic or approved data in a controlled environment within agreed credential, rate-limit and recovery test boundaries, and record source identifiers and versions.

Lifecycle eventTest caseKey scenarios to includeWhat to record
Content ingestionImport representative files and nested structuresMove/rename events, complex formats, deep folder hierarchiesMissing text, unparsed structures, unsupported objects
Content updatesModify document body or metadata after initial importIncremental changes, file renames, multi-user editsTime elapsed until changed passage is retrievable
Permission changesModify group membership or explicit object ACLsGroup membership removal, inherited restriction changes, group de-provisioningDelay before the revoked identity can no longer retrieve or reuse the affected content
DeletionsRemove a synthetic source documentSoft vs. hard deletion, trash bin purge eventsBehavior of search indices, generated answers, context windows, and caches
Authentication & limitsTest credential expiry and throttling within agreed limitsCredential expiration, API throttling limitsError visibility, retries/checkpoints, partial-progress recovery
RecoveryInterrupt and resume an import in a controlled testCheckpoint restart, duplicate and skipped-object handlingRecovered progress, duplicates, skipped objects and operator effort

Inspect how the system handles document deletion across search results, answer generation, cached responses, and active model context windows. Purging a document from search indices or context caches is distinct from retaining system audit logs or compliance metadata under approved retention policies. Crucially, a system cannot retroactively recall or un-see an answer previously displayed to a user.


Verify permissions through the complete retrieval path

Test users with access, without access, and with changed access. Where a shared service identity is used, verify that the application still enforces each end user’s authorization. Test authorization across the full data flow:

Source Permission Change -> Connector / Identity Mapping -> Index ACL or Live Authorization -> Retrieval Step -> Answer / Citation / Cache / Downstream Tools

Measure permission revocation lag starting from the exact timestamp of the source permission change. During the propagation window, test whether the system enforces explicit re-validation or deny policies, and verify that access for revoked identities to affected content remains restricted until re-authorized once propagation completes.

Evaluation layerTarget verificationOperational safeguard
User authenticationUser identity verified at application boundaryDisambiguate login/tenant isolation from document ACLs
Identity mappingUser ID mapped to source directory identityPrevent privilege escalation or unmapped fallback
Index / Live authIndex ACL filtering or live authorization executionEnforce fail-closed policy when authorization cannot be established under approved policy
Output & cacheAnswers, citations, cached blocks, downstream toolsPrevent leaked snippets in summary or citation previews

System authentication and tenant isolation do not equate to document-level authorization. A successful login test does not confirm that an end user is authorized to view a specific document snippet.

When a source system becomes temporarily unavailable during query execution, allowing stale cached answers depends on data criticality and governance rules. If authorization cannot be established under approved policy, enterprise acceptance criteria should mandate a fail-closed policy that blocks retrieval rather than risking unauthorized disclosure.


Compare deployment boundaries and operational ownership

Hosting location, network boundaries, and operational ownership are distinct technical decisions. Evaluating deployment options requires tracing the complete data path—including search indices, parser/OCR modules, embedding models, LLM endpoints, log/backup storage, telemetry, support access, and identity endpoints.

Deployment patternHosting boundaryExternal dependenciesOperational ownershipVerification questions
SaaSVendor-managed cloud infrastructureVendor API endpoints, public identity providersVendor operates contracted service layers; customer retains authorization, connector, and governance responsibilitiesWhere are embeddings and logs stored? Who has telemetry/support access?
Customer-managed Private CloudCustomer VPC / dedicated cloud accountCloud provider services, potential external LLM APIsCustomer operates agreed stack layers; vendor role varies by contractAre data boundaries enforced within the VPC? Is outbound traffic restricted?
On-premisesCustomer data center / internal networkLocal network endpoints; may depend on external model, identity, licensing, or telemetry endpointsOperational ownership follows agreement among customer, vendor, or partnerCan the system operate without external outbound calls? Are models hosted locally?

These patterns represent evaluative deployment models rather than uniform SKUs offered by every software vendor. Private cloud hosting is not synonymous with on-premises or air-gapped environments; air-gapped operation is a separate network isolation requirement that must be verified for the complete deployment.

According to its public product page, Seahorse Cloud is an integrated option providing S3-compatible object storage, document parsing with semantic chunking, vector database synchronization, and managed agent frameworks with MCP tool-calling support, available via on-premises installation or SaaS subscription. When assessing public product descriptions, separate general platform capabilities from target deployment requirements: vector DB synchronization by itself is not evidence of source ACL synchronization, and unverified source connectors or permission sync SLAs must be formally tested against enterprise requirements.


Choose a pilot that exposes operating work

A pilot should evaluate real operational effort alongside retrieval quality. Select a test set containing frequently updated documents, access-restricted files, and questions designed to test negative boundaries (unsupported or out-of-scope queries). Set acceptance thresholds from workflow requirements and a measured baseline, using a fixed corpus and query benchmark.

Evaluation categoryMetric / Checklist itemMethodology & Scope
Update propagationContent update lagMeasure elapsed time from source save to observable updated content on the configured indexed or live retrieval path
Security propagationPermission revocation lagMeasure elapsed time from source ACL or group membership change to denial of the affected identity’s access to and reuse of the affected content
Deletion propagationDocument deletion propagation lagMeasure elapsed time from source deletion until the deleted content is no longer retrievable or reused under the deletion policy; other authorized content may still support an answer
Query latencyp50 / p95 end-to-end response latencyRecord sample size, concurrency level, and measurement scope
Retrieval qualityRecall@k and answer support accuracyTest against fixed query set and fixed source version
Boundary handlingAbstention rate on unsupported questionsVerify system declines unanswerable or restricted queries
Operating overheadIngestion failure recovery & credential maintenance effortTrack operator hours required for sync error resolution
Financial costQuery, index, model, and connector infrastructure costsCalculate total cost per 1,000 queries and per GB ingested

As checked on September 21, 2026, Microsoft's prebuilt connectors documentation lists over 100 Microsoft and partner connectors; catalog size does not establish object coverage, ACL fidelity or retrieval quality.

Record source, corpus and query-set versions, event sample counts and timestamp definitions. Report unapplied changes and timeouts alongside completed-event timings; interpret p50/p95 only with an adequate sample and disclose the sample size. The outcome of the pilot should be a structured decision record detailing observed latency, permission propagation windows, failure recovery procedures, and assigned ownership for any identified capability gaps.


FAQ

Do citations prevent hallucinations?

No. Citations help users inspect supporting material, but a generated claim can still misrepresent or exceed what the cited source text states.

Does connector support include source ACLs and deletion handling?

Not automatically. Connector integration must be verified per object type for permission mapping, group inheritance, soft/hard deletion handling, and index/cache purge schedules.

How should permission synchronization be tested?

Measure the exact elapsed time between a source ACL change (or group removal) and the denial of that identity’s access to the affected content across search, generated answers, citations, and caches. Test deny policies during propagation windows and enforce fail-closed behavior when authorization cannot be established under approved policy.

Does an on-premises source mean content stays on premises?

No. Trace the full data path, including parsers, embedding models, LLM endpoints, log storage, telemetry, and vendor support access channels.

Is live source access always preferable to indexing?

No. Compare freshness requirements, source system availability, query latency, retrieval quality, and supported authorization models for each specific workflow.