A first look at grid job output

Marc Paterno

2020-08-11

Introduction

Yuanyuan has run several grid jobs using the Spack build of CosmoSIS from CVMFS. This document contains some preliminary analysis of the output. She ran using several samplers: emcee, multinest, metropolis and snake. I have run a single job, using the “metropolis” sampler.

Snake output

Snake is a grid-based sampler, not a MCMC sampler, and so only provides a view of the likelihood of the data, given the cosmological model parameters.

The emcee output

This run used 60 walkers, each producing 5000 samples, for a total of 300000 samples.

First, we look to see if the walkers have converged. We look at only a few of the walkers. Each walker is plotted in a different color.

Both \(\Omega_m\) and \(\sigma_8\) appear to have converged after a few hundred samples. Concentration seems not to have converged at all. I believe the reason is that this parameter is not strongly contained by the data, but rather by the prior that was chosen.

Posterior density plots For the concentration \(C\), the posterior probability density is clearly truncated by the prior. Since there is no fundamental physical law stating \(C \lt 10\), it would seem a wider prior should be used. What should be the shape of this prior? For \(\Omega_m\), the posterior density seems remarkably sharp. Are these data expected to constrain this parameter so strongly? Is \(\sigma_8\) also expected to be so sharply constrained?

The important plot is the analysis of the two parameter correlation between \(\Omega_m\) and \(\sigma_8\):

This plot seems to show come correlation between \(\Omega_m\) and \(\sigma_8\). Is this expected from the physics? Is it understood?

Metropolis-Hastings

Yuanyuan’s run

Yuanyuan ran a fairly long set of chains, each with 8000 samples. She ran a total of 6 chains.

The concentration does not seem to be constrained by these data. The values cover the full range of the prior.

With this run, it appears that all the chains started with very similar values of \(\Omega_m\) and \(\sigma_8\). It is difficult to tell whether we have reached convergences, or if instead the chains had not yet started to cover the full range of variation.

Lacking guidance from the plots, we use the common technique of dropping the first half of the samples as the burn-in period.

Posterior density plots. None of the parameters, except perhaps \(\Omega_m\), is showing a smooth posterior density. The distribution of \(\Omega_m\) is remarkably sharp.

Because the distribution is not very smooth, the contours are not well-defined.

Marc’s run

To see if a much wider range for the prior of concentration, and a randomized starting point for the chains, made a difference, Marc did a much shorter run with different initial conditions, including a much wider posterior for \(C\). This run used 6 chains, and 2000 samples per chain.

This time, it appears that \(\Omega_m\) and \(\sigma_8\) have reached convergence in about 500 samples, but concentration still shows no real convergence or constraint from the data.

Posterior density plots. The concentrtation \(C\) seems now to be more constrained by the data than by the prior. However, there are not enough samples (or chains) to produce a smooth posterior density estimate. Both \(\Omega_m\) and \(\sigma_8\) are significantly less constrained, and also have very different central values and ranges, than in the previous runs. Is this due to the much wider prior chosen for \(C\), or the randomized starting locations for the chains, or by some other thing?

This time, the posterior density contours are well-defined. However, the contours are very different from those produced from the output of the output of emcee sampler. Even the trend of the correlation is very different. Is this because off the very different range of concentraion that was used?

The multinest output

My software for producing posterior density plots from Multinest output is not yet working correctly.

And the plot…