Compute gcd of values of a quadratic polynomial

Let $m$ be a positive integer divisible by $6$ and let $f(x)=\frac{x^2-1}{24}$. Denote by $C_m$ the set of all $x$'s in $[|1,m|]$ that are coprime to $m$. Note that any $x\in C_m$ satisfies $x\equiv 1\mod{2}$ and hence $x^2\equiv 1\mod {8}$, and also $x\equiv \pm 1\mod 3$ and hence $x^2\equiv 1\mod 3$. So the set $D_m=\lbrace f(x) \ | \ x \in C_m \rbrace$ consists of integers.

Conjecture : $\gcd(D_m)=1$.

Any ideas on how to prove this or find a counterexample ?

My thoughts : with the help of a computer, I have checked this up to $m\leq 10^6$. An easy remark is that we are done if $5$ does not divide $m$, because of $f(5)=1$. Numerical data seems to suggest that there are infinitely many pairs of distinct primes $p,q$ with $\gcd(f(p),f(q))=1$, and perhaps Dirichlet's theorem on primes in arithmetic progressions may help here, although I would like to avoid such an advanced tool if possible.

UPDATE 11/24/2021 To such that $g=\gcd(D_m)$ is $1$, it suffices to show that no prime $q$ divides $g$. For the prime $q=2$, we have $2\not\mid f(x)$ iff $x\not\equiv \pm 1 \mod {8}$, so that $2\not\mid g$ iff $C_m$ contains an element that is $\not\equiv \pm 1 \mod {8}$.

For the prime $q=3$, we have $3\not\mid f(x)$ iff $x\not\equiv \pm 1 \mod {9}$, so that $3\not\mid g$ iff $C_m$ contains an element that is $\not\equiv \pm 1 \mod {9}$.

Finally for a prime $q>3$, $q\not\mid f(x)$ iff $x\not\equiv \pm 1 \mod q$, so that $q\not\mid g$ iff $C_m$ contains an element that is $\not\equiv \pm 1 \mod q$.

In all those cases, using the Chinese remainder theorem, one can find a $y_q$ satisfying the congruence requirement and coprime to $m$, but the problem is that $y_q$ might not be in $[|1,m|]$, and if we take the remainder of $y_q$ when divided by $m$ we might lose the congruence properties.

Cross-posted to MO.

Unless I misunderstood, you just want the following claim. Let $\mathcal{Q} = \{8,9,5,7,11,13,17,19,23,\dots\}$.

Claim: For each $n \in 6\mathbb{N}$ and $q \in \mathcal{Q}$, there is $k \le n$ with $\gcd(k,n) = 1$ and $k \not \equiv \pm 1 \pmod{q}$.

Proof: One can check $n=6$, so assume $n\ge 12$. For $q \in \{8,9\}$ choose $k=5$. For $q \in \mathcal{Q}, q > n$, choose any $k \in [2,n-1]$ relatively prime to $n$ (such a $k$ exists because $n \ge 12$). For $q \in \mathcal{Q}$ with $q \in [5,n]$ and $q \nmid n$, choose $k=q$.

The following is a sketch for the $q \in [5,n]$ with $q \mid n$. Suppose there's no $k \le n$ with $\gcd(k,n) = 1$ and $k \not \equiv \pm 1 \pmod{q}$. Then all primes $p \le q$ must divide $n$ (since none are $\pm 1 \pmod{q}$), so $\prod_{p \le q} p \mid n$; in particular, $n \ge \prod_{p \le q} p$. Now any prime $r \le \prod_{p \le q} p$ with $r \not \equiv \pm 1 \pmod{q}$ must divide $n$. There will be many such $r$, so that the product of all such $r$ divides $n$ means $n$ will have to be even bigger. We can iterate.

For the record, I reproduce here Fedor Petrov's very elegant one-line answer from MO: Denote $n=2^ab$ where $b$ is odd. One of numbers $b\pm 2$ is not congruent to $\pm 1$ modulo $q$.