Skip to main content
Prometheus collects and stores time-series metrics emitted by applications, services, and system exporters. In bioinformatics and HPC environments, it is commonly used to record system- and application-reported metrics for compute-intensive pipelines. Tracer complements Prometheus by observing how tasks, tools, and processes actually execute at runtime and organizing this behavior by pipeline, run, and execution unit.
If you’re new to Tracer or want a conceptual overview, see How Tracer fits in your stack.

​
What Prometheus does well

Prometheus is designed for metric collection and querying. It provides:
  • Time-series storage for reported metrics
  • Pull-based scraping from exporters and instrumented services
  • Label-based querying and aggregation
  • Integration with alerting systems
These capabilities make Prometheus effective for recording reported metrics over time across many systems.

​
What Prometheus does not observe

Prometheus records metrics that are emitted and scraped at intervals. It does not observe execution directly and does not have inherent awareness of pipeline or task structure. It does not show:
  • Execution behavior inside processes or containers
  • CPU vs I/O vs memory contention during task execution
  • Short-lived processes that start and finish between scrapes
  • Idle time hidden by aggregated utilization
  • How metrics relate to pipeline runs, tasks, or tools
  • How cost maps to actual execution rather than to resource uptime
This information is not present in scraped metrics unless inferred through instrumentation and labels.

​
Why this gap matters

Scientific pipelines often involve heterogeneous tools, nested execution, and short-lived subprocesses. When relying on scraped metrics alone, teams can see that resources were used, but not how work progressed. As a result:
  • Performance bottlenecks must be inferred from aggregates
  • Idle or blocked execution may appear as normal utilization
  • Cost is attributed to infrastructure time, not execution units
  • Custom metrics and labels accumulate assumptions about runtime behavior
These approaches are useful, but they have limits when execution is not directly observed.

​
What Tracer adds

Tracer observes execution directly from the host and container runtime and adds:
  • Observed CPU, memory, disk, and network behavior
  • Visibility into short-lived processes and nested tools
  • Attribution by pipeline, run, task, or execution unit
  • Cost mapping aligned with observed runtime activity
These insights are derived from observed execution, not from reported metrics.

​
Example: scraped metrics versus observed execution

Prometheus shows sustained CPU utilization during a pipeline run. Tracer reveals that:
  • CPU usage remains low for most tasks
  • Execution is dominated by disk I/O wait
  • Short-lived helper processes consume significant time
This indicates an I/O-bound workload rather than insufficient compute. The distinction is visible only when execution behavior is observed directly.

​
Observability comparison

This comparison highlights the difference between scraped metrics and execution-level observation.

​
What Tracer does not replace

Tracer is not a general-purpose metrics backend.
  • It does not replace Prometheus for collecting arbitrary application metrics
  • It does not replace long-term storage for unrelated or user-defined metrics
  • It does not replace metric-based alerting outside execution behavior
  • It does not replace dashboards built from non-execution telemetry
Tracer stores and analyzes execution-derived metrics it observes, scoped to pipeline workloads.

​
When to use Tracer with Prometheus

Tracer is most useful alongside Prometheus when teams need to:
  • Understand pipeline behavior beyond scraped metrics
  • Diagnose performance issues involving short-lived tasks
  • Attribute resource usage and cost to workflows or tools
  • Reduce manual metric instrumentation and correlation
Tracer focuses on execution behavior. Prometheus continues to provide metric collection and querying across broader systems.

​
Summary

Prometheus records metrics that systems and applications report. Tracer adds execution-level visibility that shows how pipelines, tasks, and tools actually behave at runtime and how resource usage and cost map to real work. Together, they provide complementary perspectives without overlap.