Automation Engine

Write a Rule, Let the Cluster Right Itself

When a metric threshold or health event fires, drain the node, reboot it, cut its power or send word — through a traceable execution pipeline that passes every safety gate.

Dry run

Execute a rule with no side effects and see exactly what it would do before it does it.

Every run leaves a trace

The triggering signal, the chosen action and its result are written to a detailed execution record.

What's Inside

From Trigger to Action, Under Control

The automation engine isn't a pile of scripts — it's an executor that runs every rule through the same fail-safe pipeline.

Three Trigger Types

Measurable thresholds, health state and events — you choose which signal a rule watches.

Ready-Made Actions

Drain, reboot, power off, power cycle, image update, notify and script — the daily reflexes of HPC operations.

Fail-Safe Pipeline

Disabled check → approval → rate limit → dry run → real execution; every rule passes these gates in order.

Execution Records

Each run produces an execution record: when, on which signal, with what result — all of it queryable.

Category Targeting

Rules apply to node categories rather than single nodes; as the cluster grows, the rule count doesn't.

Config Drift Repair

Node configurations converge declaratively; manual deviations are corrected on the next pass.

Staying on the Safe Side of Automation

Automation is the most feared thing in a cluster — TULPAR makes it provable first and automatic second.

A new rule is trialled with a forced dry run before it goes live; the output has the same shape as a real execution, so comparison is easy.

A rule cannot fire back-to-back during a signal storm; the rate limit cuts cascading damage off at the source.

Destructive actions can be defined to require approval — automation doesn't take the human out, it takes the load off.

Every execution step lands in the audit log; the answer to "who powered this node off" stays in the panel.

%