> ## Documentation Index > Fetch the complete documentation index at: https://agentheya.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # data_internal > Private knowledge the agent retrieves from but never exposes verbatim: uploaded files, indexed URLs, and optional external vector stores. # `data_internal` — Internal data Private knowledge the agent retrieves from but never exposes verbatim: uploaded files, indexed URLs, and optional external vector stores. **Section key** `data_internal` · **Shape** object · **Tier** All plans\ **Edit in the dashboard:** [Internal Data page](/sidebar-menu/data-internal)\ **Related:** [external\_stores backends](/sections/data/external-stores) ## Fields | Field | Required | Type | Allowed values / constraints | Default | Description | | ----------------- | -------- | --------------- | ---------------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `files` | no | array of object | — | — | Uploaded documents processed for retrieval-augmented generation. Item fields below. | | `urls` | no | array of object | — | — | URL sources fetched and indexed for RAG. Item fields below. | | `external_stores` | no | array of object | — | `[]` | External knowledge sources the agent should query at chat time alongside the agent's built-in knowledge base. Each entry is a connection to YOUR existing store — a Pinecone or Weaviate cluster, Bedrock KB, Azure AI Search, Vertex AI Search, OpenAI Vector Store, or a GitHub-hosted Obsidian vault — Agentheya does not ingest into it, only queries. Hits are merged with built-in retrieval results. Credentials (api\_key fields) are encrypted at rest. See [external\_stores backends](/sections/data/external-stores) for the full sub-schema. | | `service_sources` | no | array of object | **Pro+** | — | Configured SaaS service data sources (Services card). Each pulls documents/records through a connector added on the /connectors page and ingests them into this scope's retrieval index. Sync-state fields (last\_synced\_at, last\_sync\_status, last\_sync\_error, item\_count) are server-stamped and never trusted from clients. Item fields below. | ### `files` items Uploaded documents processed for retrieval-augmented generation. | Field | Required | Type | Allowed values / constraints | Default | Description | | -------------------- | -------- | ---------------- | --------------------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `name` | **yes** | string | — | — | Original file name. | | `path` | **yes** | string | — | — | Stored file path relative to the agent's files directory. | | `doc_id` | no | string | — | — | Stable short identifier for this ingested object (e.g. 'doc\_k7m2p9q4'). Minted once when the entry first enters the manifest and preserved across renames, re-syncs and re-ingests. Shown on the data page and on the matching Activity ingest row. | | `size` | **yes** | string | — | — | Human-readable file size (e.g. '1.2 MB'). | | `chunk_count` | no | integer | — | — | Number of chunks produced by the ingest pipeline (set after processing). | | `total_tokens` | no | integer | — | — | Total token count across all chunks (set after processing). | | `chunking_config` | no | object | — | — | How the file is split for retrieval. Item fields below. | | `retrieval_enabled` | no | boolean | — | — | When false, the file is kept on disk and indexed but excluded from retrieval (keyword + vector). Default true. | | `excluded_chunks` | no | array of integer | — | — | Chunk indexes excluded from retrieval at query time (set from the chunk inspector). Chunks stay indexed so re-enabling is instant. | | `content_hash` | no | string | — | — | SHA-256 hex of the stored file content. Set on upload / URL fetch; used for change detection on URL re-sync. | | `source_url` | no | string | — | — | Set when this file was imported from a URL; enables re-sync. | | `url_sync` | no | enum | one of `manual`, `daily`, `weekly`, `monthly` | — | Re-sync cadence for URL-imported files. 'daily' / 'weekly' / 'monthly' automatically re-sync once the period since last\_synced\_at has passed; 'manual' re-syncs only on request. | | `last_synced_at` | no | string | — | — | ISO timestamp of the last successful URL fetch (URL-imported files only). | | `render_mode` | no | enum | one of `auto`, `always` | — | JS render mode this URL was imported with (web-render microservice); re-syncs re-apply it so a rendered page never reverts to its raw shell. | | `site_group` | no | string | — | — | Site host (www-folded) this page was imported under by a whole-website crawl. Pages sharing a site\_group display as ONE subtree entry on the data page and count as ONE file toward the per-plan file-count limits. | | `import_root_url` | no | string | — | — | Start URL of the whole-website crawl that imported this page — used by the subtree entry's re-crawl action. | | `align_ontology` | no | string | max len 200 | — | Per-document ontology alignment override applied when this document's knowledge graph is (re-)extracted. Empty/unset = use the agent's default ontology; 'all' = every imported ontology; 'off' = do not align this document even when alignment is on; otherwise the stored\_name of a specific imported ontology. | | `doc_profile` | no | object | — | — | LLM-generated knowledge profile written by the ingest pipeline (not owner-editable): what the document is about + topic keywords. Feeds the agent's document catalog so it can route to the right document. Item fields below. | | `rank_boost` | no | enum | one of `low`, `high` | — | Owner-declared retrieval priority applied as a post-fusion score multiplier. 'normal' is stored by omission. | | `service_source_id` | no | string | — | — | Id of the service source (service\_sources\[].id) that ingested this file. Files sharing a service\_source\_id render as one expandable Services entry and count as ONE file slot. | | `service_item_key` | no | string | — | — | Stable upstream identity '\:\' — the re-sync upsert key (the way source\_url keys URL imports). | | `service_updated_at` | no | string | — | — | Upstream last-modified timestamp of the item at the last sync. | ### `urls` items URL sources fetched and indexed for RAG. | Field | Required | Type | Allowed values / constraints | Default | Description | | -------------- | -------- | ------ | ---------------------------- | ------- | ------------------------------------------------- | | `url` | **yes** | string | — | — | The URL to fetch. | | `bearer_token` | no | string | — | — | Optional bearer token sent when fetching the URL. | ### `service_sources` items Configured SaaS service data sources (Services card). Each pulls documents/records through a connector added on the /connectors page and ingests them into this scope's retrieval index. Sync-state fields (last\_synced\_at, last\_sync\_status, last\_sync\_error, item\_count) are server-stamped and never trusted from clients. | Field | Required | Type | Allowed values / constraints | Default | Description | | ------------------ | -------- | ------- | -------------------------------------------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `id` | **yes** | string | pattern `^svc_[a-z0-9]+$` | — | Stable source id; ingested files reference it via service\_source\_id. | | `name` | **yes** | string | max len 120 | — | Display name shown on the data page. | | `connector_kind` | **yes** | enum | one of `composio`, `mcp`, `openapi`, `graphql`, `s3`, `native` | — | Which connector mechanism this source pulls through. | | `connector_ref` | **yes** | string | max len 200 | — | Composio record id (stringified number) or connector name from the connectors section. | | `service` | **yes** | string | max len 64 | — | Recipe key in config/service-ingest-catalog.json (e.g. 'googledrive', 'jira'), or 'custom'. | | `object_type` | no | string | max len 64 | — | Object-type key within the recipe (e.g. 'files', 'issues'). | | `query` | no | string | max len 2000 | — | Recipe-specific listing filter (Drive query, JQL, SOQL WHERE, search text…). | | `extra_args_json` | no | string | max len 4000 | — | JSON object string merged into the list tool's arguments — for recipes whose tools need per-source identifiers (Sentry org slug, Bitbucket workspace/repo, BambooHR subdomain…). GraphQL: becomes the query variables. | | `include_terms` | no | string | max len 1000 | — | Comma-separated terms — when set, an item's title must contain one to be ingested. | | `exclude_terms` | no | string | max len 1000 | — | Comma-separated terms — items whose title contains one are skipped. | | `max_items` | no | integer | min 1 | — | Per-source item cap; clamped to the plan's service\_import\_item\_limits. | | `sync` | **yes** | enum | one of `manual`, `daily`, `weekly`, `monthly` | — | Automatic re-sync cadence; next sync is due last\_synced\_at + period. | | `chunking_config` | no | object | — | — | Same shape as files\[].chunking\_config; inherited by every file this source ingests. | | `custom` | no | object | — | — | Owner-authored field mapping when service === 'custom' (build-your-own connector). Item fields below. | | `last_synced_at` | no | string | — | — | Server-stamped ISO time of the last completed sync. | | `last_sync_status` | no | enum | one of `ok`, `error`, \`\` | — | | | `last_sync_error` | no | string | max len 2000 | — | | | `item_count` | no | integer | — | — | Files currently held by this source. | | `s3` | no | object | — | — | Direct S3 / S3-compatible bucket config (connector\_kind 's3'). secret\_access\_key is encrypted at rest and read back as '***REDACTED***'. Item fields below. | ### `files.chunking_config` fields How the file is split for retrieval. | Field | Required | Type | Allowed values / constraints | Default | Description | | ----------------------- | -------- | ------- | ------------------------------------------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `chunk_size` | no | integer | — | — | Target chunk size in tokens. Default 512. | | `chunk_overlap_pct` | no | integer | — | — | Overlap between adjacent chunks as a percentage of chunk\_size. Default 0. | | `split_strategy` | no | enum | one of `auto`, `fixed_token`, `recursive`, `semantic`, `structural` | — | Chunking strategy. 'auto' is recommended. | | `metadata` | no | object | — | — | Source metadata stamped on each chunk. Item fields below. | | `contextual_enrichment` | no | object | — | — | Prepend auto-generated context to each chunk to improve retrieval. Item fields below. | | `preview_before_ingest` | no | boolean | — | — | When true, the file is converted and chunked but PAUSES before indexing so the owner can preview and optionally edit chunks on the preview page, then clicks 'Ingest these chunks' to finish. Default false. | ### `files.doc_profile` fields LLM-generated knowledge profile written by the ingest pipeline (not owner-editable): what the document is about + topic keywords. Feeds the agent's document catalog so it can route to the right document. | Field | Required | Type | Allowed values / constraints | Default | Description | | ------------- | -------- | --------------- | ---------------------------- | ------- | ------------------------------------------------------------------------------------------ | | `summary` | no | string | max len 500 | — | 2–3 sentence summary of what the document is about and what kinds of questions it answers. | | `keywords` | no | array of string | max 8 items | — | Up to 8 topic keywords / entity names, most distinctive first. | | `profiled_at` | no | string | — | — | ISO timestamp of the profiling run. | ### `service_sources.custom` fields Owner-authored field mapping when service === 'custom' (build-your-own connector). | Field | Required | Type | Allowed values / constraints | Default | Description | | ---------------- | -------- | ------ | ---------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------- | | `list_tool` | **yes** | string | max len 4000 | — | Connector tool (Composio/MCP name, OpenAPI operationId) that lists items — or, for GraphQL connectors, the full query document. | | `list_args_json` | no | string | max len 4000 | — | JSON object string of extra arguments for the list tool. | | `items_path` | no | string | max len 200 | — | Dot-path to the items array in the result; empty = auto-discover. | | `id_path` | no | string | max len 200 | — | | | `title_path` | no | string | max len 200 | — | | | `body_path` | no | string | max len 200 | — | | | `url_path` | no | string | max len 200 | — | | | `updated_path` | no | string | max len 200 | — | | ### `service_sources.s3` fields Direct S3 / S3-compatible bucket config (connector\_kind 's3'). secret\_access\_key is encrypted at rest and read back as '***REDACTED***'. | Field | Required | Type | Allowed values / constraints | Default | Description | | ------------------- | -------- | ------ | ---------------------------- | ------- | ----------------------------------------------------------- | | `bucket` | **yes** | string | max len 255 | — | | | `region` | no | string | max len 64 | — | | | `endpoint` | no | string | max len 400 | — | Custom https endpoint for R2 / Spaces / MinIO; empty = AWS. | | `prefix` | no | string | max len 512 | — | Only objects under this key prefix are listed. | | `access_key_id` | no | string | max len 256 | — | | | `secret_access_key` | no | string | max len 2048 | — | | ### `files.chunking_config.metadata` fields Source metadata stamped on each chunk. | Field | Required | Type | Allowed values / constraints | Default | Description | | ----------------------- | -------- | ------- | ---------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `source_type` | no | string | — | — | Free-text tag (max 64 chars) for filtered retrieval, e.g. 'policy\_doc', 'faq', 'product\_spec'. Stamped on every chunk's metadata, enabling queries to pre-filter by source\_type in retrieval. | | `include_source` | no | boolean | — | — | Boolean. When true, attaches the file's source path or URL to each chunk's metadata as the 'source' field for citations and retrieval tracing. | | `include_section_title` | no | boolean | — | — | Boolean. When true, stamps each chunk's metadata with 'section\_title' (the heading/title from the document's structure) and 'section\_path' (the hierarchy of headings) if available. | | `include_page_number` | no | boolean | — | — | Boolean. When true, includes page number in each chunk's metadata if the source format supports it (e.g. PDF, DOCX). Currently unavailable for most formats. | | `include_author` | no | boolean | — | — | Boolean. When true and author field is set, attaches the author name to each chunk's metadata for provenance tracking. | | `author` | no | string | — | — | String (max 128 chars, e.g. 'Legal team'). Stamped on chunks when include\_author is true. Identifies the document's author or responsible party. | | `include_date` | no | boolean | — | — | Boolean. When true and doc\_date field is set, attaches the document date to each chunk's metadata for temporal filtering and versioning. | | `doc_date` | no | string | — | — | String (max 64 chars: YYYY-MM-DD format or free text). Stamped on chunks when include\_date is true. Captures when the document was authored or last updated. | ### `files.chunking_config.contextual_enrichment` fields Prepend auto-generated context to each chunk to improve retrieval. | Field | Required | Type | Allowed values / constraints | Default | Description | | ---------------------- | -------- | ------- | ---------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `enabled` | no | boolean | — | — | Boolean. When true, each chunk is prefixed with auto-generated context (doc title / section path) before embedding, which improves retrieval quality. | | `include_doc_title` | no | boolean | — | — | Boolean. When true (and enrichment enabled), prepends the document's title to each chunk for retrieval context. | | `include_section_path` | no | boolean | — | — | Boolean. When true (and enrichment enabled), prepends the section/heading hierarchy (e.g. 'Chapter 2 › Section 2.3') to each chunk for navigation context. | | `include_summary` | no | boolean | — | — | Boolean. When true (and enrichment enabled), prepends a short extractive document lead summary (the doc's opening sentences) to each chunk via the \{summary} placeholder. Deterministic — no LLM call. Default false. | | `custom_template` | no | string | — | — | String (max 2000 chars). Optional Jinja-like template controlling how context is formatted: \{doc\_title}, \{section\_path}, \{summary}, \{chunk}. Defaults to '\{doc\_title}\n\{section\_path}\n\n\{chunk}' when empty. |