Open Watchtower

Data History and complete-source search

Open Advanced / Investigate → Data History for coverage, stored daily prices, retained evidence and source inventory. Global Markets & Data Search and MCP share the expanded search service.

What the counts mean

The complete public Commodity Atlas index initially covers 5,949,702 source rows: 3,061,936 observations,304,612 assets,2,562,190 trade rows and20,964 derived exposure rows. These are reported, estimated, reference and modeled records—not millions of live market quotes. Retained keyword entries, captures, revisions and bars are different units and can overlap.

Daily prices

Provider history is stored by exact provider, symbol, currency, exchange, interval and bar timestamp. Open, high, low, close, adjusted close and volume remain distinct. Published corporate actions and source metadata accompany historical captures. Valid zero/negative historical values and missing source closes are preserved. Missing dates and fields are not interpolated; a market closure is not a zero-price day.

Five-year history now requests daily bars. Calendar-day limits accommodate crypto. For long maximum history, a provider may initially downsample: Watchtower follows explicit provider date bounds and verifies daily granularity before admission. A dated provider bar is not a live trade or official settlement. A retained fallback is explicitly stale; unknown source currency or price is not manufactured as USD or zero.

Historical backfill follows discovered, reviewed native symbols at a paced rate. Unmapped symbols, provider gaps and access/rate-limit pauses remain visible. A provider-wide instrument count does not establish collected history for every listing. Earlier prices can be backfilled only when supplied by the provider; original retrieval and observation dates stay separate.

Feed evidence and retention

Successful admitted public/shared feed responses and reviewed MCP reads enter a durable spool and an append-preserving evidence store. Repeated content is deduplicated with capture-day sightings; source corrections retain versions. Existing public cache snapshots are imported with their original capture clocks. Article links and available feed text remain source claims, not independent verification or a licensed full-article mirror.

User request text, private workspaces, credentials, other users, financial/account state and user-authored study inputs are excluded. Malformed lone Unicode surrogates are normalized for safe storage/display; valid source text remains intact. Geometry remains available in retained snapshot pointer pages rather than being counted as keyword observations.

The archive does not age-delete admitted history. Its20GiB SQLite budget and2GiB spool bound are explicit; admission pauses rather than silently deleting old evidence when storage bounds are reached. Full-source index generations are separate. Retention checks run on a five-minute-after-completion schedule, with source-specific backfill cadence and provider pauses. This is not storage of every market tick or a guarantee of universal real-time feeds.

MCP examples

{
  "tool": "watchtower.search",
  "arguments": {
    "query": "Canada wheat",
    "source": "commodity-atlas-v4-evidence",
    "limit": 5
  }
}

{
  "tool": "watchtower.read_data_route",
  "arguments": {
    "path": "/agent/api/commodity-atlas-v4-evidence",
    "params": {
      "view": "search",
      "dataset": "trade",
      "period": "2022",
      "q": "*",
      "limit": 100
    }
  }
}

{
  "tool": "watchtower.read_data_route",
  "arguments": {
    "path": "/agent/api/market-history-ledger",
    "params": {
      "view": "bars",
      "symbol": "AAPL",
      "interval": "1d",
      "from": "2025-01-01",
      "to": "2025-12-31",
      "limit": 100
    }
  }
}

{
  "tool": "watchtower.read_data_route",
  "arguments": {
    "path": "/agent/api/evidence-history",
    "params": {
      "view": "search",
      "q": "inflation",
      "timeBasis": "observed",
      "limit": 50
    }
  }
}

Use watchtower.get_data_inventory, watchtower.get_price_history and watchtower.get_evidence_history where your client has refreshed its tool list. Existing list_data_routes and read_data_route can discover and read the same sources.

Native source pagination and MCP JSON-pointer pagination are separate. Follow returned nextCursor/nextOffset values rather than assuming a requested page size. Complete Atlas cursors are source/query-bound; archive search cursors fix an upper ingestion watermark. Snapshot string values support text continuation. Read sources and quality fields before interpreting prices, models or historical estimates.

Search scope

Search combines the existing canonical retained corpus, complete public Atlas tables and retained archive revisions. It prioritizes the retained corpus, then complete-source records, then historical versions; scores are not a global statistical ranking. Use source/category filters and native cursor readers for large traversals. Other specialist views retain their own source coverage and supported filters; dataset discovery is not proof every upstream feed is fresh or licensed.

Ancient Map now has a reviewed MCP reader for its actual installed catalogue and source geometry. Its historical/reference qualifications and image/GPL licences remain intact.