Troubleshooting

This guide provides recovery steps and environment-specific caveats for HyperPipe.

Operational Context

The HyperPipe workflow is split into two primary functional roles:

  1. Worker MARG Jobs: These are the high-volume physics evaluations. If these are failing, check your marginal driver’s CLI contract or the input grid format.

  2. Fitting & Posterior Jobs: These are the single-node tasks that compute the posterior and generate the next grid. Failures here typically indicate issues with MC integration or util_ConstructEOSPosterior.py.

For RIFT Users

HyperPipe is a generalized version of the RIFT gravitational-wave PE pipeline. The mapping between the two:

The key difference is that HyperPipe is coordinate-free. The coordinate system is declared in the YAML and passed to the post stage via the same flags that CIP uses.

Diagnostic Commands

Use these commands to monitor and recover your pipeline:

Check DAG status:

condor_q

Tail a worker’s log (Essential for debugging driver crashes):

condor_tail -f <job-id> <log-file>

Re-submit a stalled DAG (Use after fixing a driver bug):

condor_submit_dag marginalize_hyperparameters.dag

Inspect the generated args files (Verify that the DAG created the correct paths):

cat rundir/args_marg_eos.txt
cat rundir/args_puff.txt
cat rundir/args_eos_post.txt

Dry run validation:

general:
  dry-run: true # Validates config -> DAG translation without submitting to Condor

util_RIFT_hyperpipe.py --config my_conf.yaml

# prints the create_eos_posterior_pipeline command without running it