Monitor the performance upgrade preview

InfluxDB 3 Enterprise provides system tables and a query telemetry endpoint to monitor file status, query execution, and overall performance of these upgrades.

System tables

The upgraded storage exposes internal state through system tables that you can query with SQL.

system.pt_ingest_wal

View WAL files and their partitions:

SELECT * FROM system.pt_ingest_wal;

Example output:

Use this table to monitor:

• WAL accumulation: Track the number and size of unmerged WAL files
• Partition distribution: See how data is distributed across partitions
• Time coverage: Verify data time ranges

Monitor WAL backlog

Check for WAL accumulation that may indicate merging is falling behind:

SELECT
  COUNT(*) as wal_file_count,
  SUM(size_bytes) / 1024 / 1024 as total_size_mb,
  MIN(min_time) as oldest_data,
  MAX(max_time) as newest_data
FROM system.pt_ingest_wal;

system.pt_ingest_files

View Gen0 files with metadata:

SELECT * FROM system.pt_ingest_files;

Example output:

Use this table to monitor:

• File counts per generation: Track compaction progress
• File sizes: Verify files are within configured limits
• Time ranges: Identify time-unbounded Gen0 files

Monitor file distribution

Check file distribution and compaction status:

SELECT
  generation,
  COUNT(*) as file_count,
  SUM(row_count) as total_rows,
  SUM(size_bytes) / 1024 / 1024 as total_size_mb,
  AVG(size_bytes) / 1024 / 1024 as avg_file_size_mb
FROM system.pt_ingest_files
GROUP BY generation
ORDER BY generation;

Query telemetry

The query telemetry endpoint provides detailed execution statistics for analyzing query performance.

Enable query telemetry

Query the telemetry endpoint after executing a query:

curl -X GET "http://localhost:8181/api/v3/query_sql_telemetry" \
  -H "Authorization: Bearer 
AUTH_TOKEN
"

Replace AUTH_TOKEN with your token.

Telemetry response

The response includes:

Example telemetry output

{
  "query_id": "q_12345",
  "execution_time_us": 4523,
  "chunks": [
    {
      "chunk_id": "c_1",
      "files_scanned": 3,
      "blocks_processed": 12,
      "rows_read": 24000,
      "rows_returned": 150,
      "bytes_read": 1234567
    }
  ],
  "cache_stats": {
    "gen0_hits": 5,
    "gen0_misses": 1,
    "compacted_hits": 8,
    "compacted_misses": 2
  }
}

Performance analysis

Query performance metrics

Track these key metrics for query performance:

Ingest performance metrics

Monitor these metrics for write performance:

Monitor with SQL

Create a performance summary query:

-- File generation summary
SELECT
  'Gen0 files' as metric,
  COUNT(*) as count,
  SUM(size_bytes) / 1024 / 1024 as size_mb
FROM system.pt_ingest_files
WHERE generation = 0

UNION ALL

SELECT
  'Compacted files' as metric,
  COUNT(*) as count,
  SUM(size_bytes) / 1024 / 1024 as size_mb
FROM system.pt_ingest_files
WHERE generation > 0

UNION ALL

SELECT
  'WAL files' as metric,
  COUNT(*) as count,
  SUM(size_bytes) / 1024 / 1024 as size_mb
FROM system.pt_ingest_wal;

Troubleshooting

High WAL file count

Symptom: system.pt_ingest_wal shows many accumulated files.

Possible causes:

• Merge operations falling behind write rate
• Insufficient merge concurrency
• Object storage latency

Solutions:

1. Increase merge concurrency:

   --pt-snapshot-merge-concurrency 8

2. Increase WAL flush interval to create larger, fewer files:

   --pt-wal-flush-interval 5s

3. Check object storage performance and connectivity

High cache miss rate

Symptom: cache_stats shows >40% miss rate.

Possible causes:

• Cache size too small for working set
• Cache recency window too narrow
• Random access patterns across time ranges

Solutions:

1. Increase cache size:

   --pt-file-cache-size 16GB

2. Extend cache recency window:

   --pt-file-cache-recency 24h

3. Extend eviction timeout:

   --pt-file-cache-evict-after 48h

Slow compaction

Symptom: Gen0 file count continues to grow.

Possible causes:

• Compaction concurrency too low
• Compaction cache undersized
• High write rate overwhelming compaction

Solutions:

1. Increase compaction concurrency:

   --pt-compactor-concurrency 8

2. Increase compaction cache:

   --pt-compactor-cache 4GB

3. For distributed deployments, add dedicated compactor nodes:

   influxdb3 serve \
     # ...
     --use-pacha-tree \
     --mode compact

Query node lag

Symptom: Query nodes return stale data.

Possible causes:

• Replication falling behind
• Network latency to object storage
• Insufficient replica concurrency

Solutions:

1. Increase replication concurrency:

   --pt-wal-replica-steady-concurrency 8

2. Reduce merge interval:

   --pt-replica-merge-interval 50ms

3. Increase replica queue size:

   --pt-wal-replica-queue-size 200