Cosmological perturbations originating from quantum vacuum fluctuations form an open quantum system due to ubiquitous interactions with unobserved degrees of freedom, with decoherence explaining the quantum-to-classical transition in cosmological observations. However, even in minimal single-field inflation, decoherence from gravitational nonlinearity is often underestimated due to some neglected boundary terms from the standard integration-by-parts procedure. Furthermore, quadratic boundary terms, arising naturally from the well-defined variational principle with gravity, significantly affect the momentum-space entanglement of these perturbations. We explore these subtleties in a proposed cosmological Bell test, showing a possible window for Bell violation.