Enterprise search connects people to information spread across document systems, collaboration tools, and internal applications. When an IT or search platform lead evaluates solutions, the key decision points emerge after initial ingestion: how content updates, permission revocations, and deployment boundaries behave under production constraints. A structured comparison evaluates connector coverage, permission freshness, and hosting security before selecting an enterprise AI search platform.
Key takeaways
- Connector coverage needs testing: A supported source name does not establish coverage for every object type, permission rule, or deletion event.
- Distinguish indexed and live access: Their freshness, availability, and operating trade-offs differ across architectures.
- Test access changes end to end: Login and tenant isolation alone do not prove document-level permission enforcement.
- Separate deployment decisions: Hosting location, network boundaries, and operating responsibilities represent distinct evaluation axes.
Choose between indexed search and live source access
An indexed approach copies selected content and metadata into a search index, with retrieval latency and freshness shaped by index capacity and sync pipeline maintenance. A live (federated) approach queries the source system at request time, bypassing a separate content index while query performance and data availability depend on source API consistency and response times. Teams can combine indexed and live paths, routing queries across indexed repositories and federated sources based on update frequency and system performance constraints.
Comparison table Scroll to view every column ↔
| Approach | Data path | Freshness & availability trade-off | Authorization responsibility | Potential fit to validate |
|---|---|---|---|---|
| Indexed search | Content and metadata extracted to search index | Trade-off between index capacity, sync schedule, and query latency | Search index ACLs or search-time index filtering | Indexed workloads that meet quality and capacity needs; source-outage retrieval only where freshness and authorization policies permit |
| Live (federated) access | Direct query to source system at request time | Trade-off between source API consistency, network latency, and availability | Source-side authorization for the mapped caller; verify identity mapping and downstream enforcement | Rapidly changing data, systems with direct query APIs |
| Mixed model | Dual-path: index stable assets, federate dynamic tools | Trade-off dependent on query routing logic across indexed and federated paths | Dual enforcement across index ACLs and live auth | Heterogeneous tool stacks with varied security requirements |
Neither model guarantees that every retrieved answer is perfectly fresh or secure. Indexed stores can lag behind source changes, whereas live queries can fail during source downtime or return partial results.
In general architecture terms, operating without a separate content search index does not mean content is never passed to an LLM, excluded from transient logs or caches, or immune to exfiltration risks. Furthermore, while Microsoft implements federated connectors via the Model Context Protocol (MCP), MCP is one specific implementation pattern rather than a universal requirement for all federated search architectures.
Official documentation for Microsoft Copilot connectors and Microsoft federated connectors distinguishes admin-managed, organization-scoped indexing with tenant-configured synced connectors from admin-enabled, user-authenticated, user-scoped indexing with self-serve synced connectors. Federated connectors retrieve live content using the user’s source identity via MCP. Evaluators must verify the supported experience, identity mapping mechanisms, and object/ACL scope for each source connector rather than assuming uniform support across Copilot Search, Microsoft Search, and all Copilot surfaces. The presence of an MCP connection alone does not guarantee permission synchronization or automatic downstream authorization.
Build a connector acceptance matrix
Start with the sources that support core user tasks. For each source, record the exact source edition (cloud vs. self-hosted), connector version, credential scope, identity mapping model, and supported object types (such as file attachments, inline comments, or embedded tables). Content updates and permission modifications must be evaluated as separate test events. Use synthetic or approved data in a controlled environment within agreed credential, rate-limit and recovery test boundaries, and record source identifiers and versions.
Comparison table Scroll to view every column ↔
| Lifecycle event | Test case | Key scenarios to include | What to record |
|---|---|---|---|
| Content ingestion | Import representative files and nested structures | Move/rename events, complex formats, deep folder hierarchies | Missing text, unparsed structures, unsupported objects |
| Content updates | Modify document body or metadata after initial import | Incremental changes, file renames, multi-user edits | Time elapsed until changed passage is retrievable |
| Permission changes | Modify group membership or explicit object ACLs | Group membership removal, inherited restriction changes, group de-provisioning | Delay before the revoked identity can no longer retrieve or reuse the affected content |
| Deletions | Remove a synthetic source document | Soft vs. hard deletion, trash bin purge events | Behavior of search indices, generated answers, context windows, and caches |
| Authentication & limits | Test credential expiry and throttling within agreed limits | Credential expiration, API throttling limits | Error visibility, retries/checkpoints, partial-progress recovery |
| Recovery | Interrupt and resume an import in a controlled test | Checkpoint restart, duplicate and skipped-object handling | Recovered progress, duplicates, skipped objects and operator effort |
Inspect how the system handles document deletion across search results, answer generation, cached responses, and active model context windows. Purging a document from search indices or context caches is distinct from retaining system audit logs or compliance metadata under approved retention policies. Crucially, a system cannot retroactively recall or un-see an answer previously displayed to a user.
Verify permissions through the complete retrieval path
Test users with access, without access, and with changed access. Where a shared service identity is used, verify that the application still enforces each end user’s authorization. Test authorization across the full data flow:
Source Permission Change -> Connector / Identity Mapping -> Index ACL or Live Authorization -> Retrieval Step -> Answer / Citation / Cache / Downstream Tools
Measure permission revocation lag starting from the exact timestamp of the source permission change. During the propagation window, test whether the system enforces explicit re-validation or deny policies, and verify that access for revoked identities to affected content remains restricted until re-authorized once propagation completes.
Comparison table Scroll to view every column ↔
| Evaluation layer | Target verification | Operational safeguard |
|---|---|---|
| User authentication | User identity verified at application boundary | Disambiguate login/tenant isolation from document ACLs |
| Identity mapping | User ID mapped to source directory identity | Prevent privilege escalation or unmapped fallback |
| Index / Live auth | Index ACL filtering or live authorization execution | Enforce fail-closed policy when authorization cannot be established under approved policy |
| Output & cache | Answers, citations, cached blocks, downstream tools | Prevent leaked snippets in summary or citation previews |
System authentication and tenant isolation do not equate to document-level authorization. A successful login test does not confirm that an end user is authorized to view a specific document snippet.
When a source system becomes temporarily unavailable during query execution, allowing stale cached answers depends on data criticality and governance rules. If authorization cannot be established under approved policy, enterprise acceptance criteria should mandate a fail-closed policy that blocks retrieval rather than risking unauthorized disclosure.
Compare deployment boundaries and operational ownership
Hosting location, network boundaries, and operational ownership are distinct technical decisions. Evaluating deployment options requires tracing the complete data path—including search indices, parser/OCR modules, embedding models, LLM endpoints, log/backup storage, telemetry, support access, and identity endpoints.
Comparison table Scroll to view every column ↔
| Deployment pattern | Hosting boundary | External dependencies | Operational ownership | Verification questions |
|---|---|---|---|---|
| SaaS | Vendor-managed cloud infrastructure | Vendor API endpoints, public identity providers | Vendor operates contracted service layers; customer retains authorization, connector, and governance responsibilities | Where are embeddings and logs stored? Who has telemetry/support access? |
| Customer-managed Private Cloud | Customer VPC / dedicated cloud account | Cloud provider services, potential external LLM APIs | Customer operates agreed stack layers; vendor role varies by contract | Are data boundaries enforced within the VPC? Is outbound traffic restricted? |
| On-premises | Customer data center / internal network | Local network endpoints; may depend on external model, identity, licensing, or telemetry endpoints | Operational ownership follows agreement among customer, vendor, or partner | Can the system operate without external outbound calls? Are models hosted locally? |
These patterns represent evaluative deployment models rather than uniform SKUs offered by every software vendor. Private cloud hosting is not synonymous with on-premises or air-gapped environments; air-gapped operation is a separate network isolation requirement that must be verified for the complete deployment.
According to its public product page, Seahorse Cloud is an integrated option providing S3-compatible object storage, document parsing with semantic chunking, vector database synchronization, and managed agent frameworks with MCP tool-calling support, available via on-premises installation or SaaS subscription. When assessing public product descriptions, separate general platform capabilities from target deployment requirements: vector DB synchronization by itself is not evidence of source ACL synchronization, and unverified source connectors or permission sync SLAs must be formally tested against enterprise requirements.
Choose a pilot that exposes operating work
A pilot should evaluate real operational effort alongside retrieval quality. Select a test set containing frequently updated documents, access-restricted files, and questions designed to test negative boundaries (unsupported or out-of-scope queries). Set acceptance thresholds from workflow requirements and a measured baseline, using a fixed corpus and query benchmark.
Comparison table Scroll to view every column ↔
| Evaluation category | Metric / Checklist item | Methodology & Scope |
|---|---|---|
| Update propagation | Content update lag | Measure elapsed time from source save to observable updated content on the configured indexed or live retrieval path |
| Security propagation | Permission revocation lag | Measure elapsed time from source ACL or group membership change to denial of the affected identity’s access to and reuse of the affected content |
| Deletion propagation | Document deletion propagation lag | Measure elapsed time from source deletion until the deleted content is no longer retrievable or reused under the deletion policy; other authorized content may still support an answer |
| Query latency | p50 / p95 end-to-end response latency | Record sample size, concurrency level, and measurement scope |
| Retrieval quality | Recall@k and answer support accuracy | Test against fixed query set and fixed source version |
| Boundary handling | Abstention rate on unsupported questions | Verify system declines unanswerable or restricted queries |
| Operating overhead | Ingestion failure recovery & credential maintenance effort | Track operator hours required for sync error resolution |
| Financial cost | Query, index, model, and connector infrastructure costs | Calculate total cost per 1,000 queries and per GB ingested |
As checked on September 21, 2026, Microsoft's prebuilt connectors documentation lists over 100 Microsoft and partner connectors; catalog size does not establish object coverage, ACL fidelity or retrieval quality.
Record source, corpus and query-set versions, event sample counts and timestamp definitions. Report unapplied changes and timeouts alongside completed-event timings; interpret p50/p95 only with an adequate sample and disclose the sample size. The outcome of the pilot should be a structured decision record detailing observed latency, permission propagation windows, failure recovery procedures, and assigned ownership for any identified capability gaps.
FAQ
Do citations prevent hallucinations?
No. Citations help users inspect supporting material, but a generated claim can still misrepresent or exceed what the cited source text states.
Does connector support include source ACLs and deletion handling?
Not automatically. Connector integration must be verified per object type for permission mapping, group inheritance, soft/hard deletion handling, and index/cache purge schedules.
How should permission synchronization be tested?
Measure the exact elapsed time between a source ACL change (or group removal) and the denial of that identity’s access to the affected content across search, generated answers, citations, and caches. Test deny policies during propagation windows and enforce fail-closed behavior when authorization cannot be established under approved policy.
Does an on-premises source mean content stays on premises?
No. Trace the full data path, including parsers, embedding models, LLM endpoints, log storage, telemetry, and vendor support access channels.
Is live source access always preferable to indexing?
No. Compare freshness requirements, source system availability, query latency, retrieval quality, and supported authorization models for each specific workflow.