Proof of $\sum_{n=1,3,5,\ldots}^{\infty}\frac{1}{n^4}=\frac{\pi^4}{96}$
$$\frac{\pi^4}{90}=\sum_{n=1}^\infty\frac1{n^4}=\sum_{n=1}^\infty\frac1{(2n)^4}+\sum_{n=1}^\infty\frac1{(2n-1)^4}=\frac1{16}\sum_{n=1}^\infty\frac1{n^4}+\sum_{n=1}^\infty\frac1{(2n-1)^4}\implies$$
$$\implies\sum_{n=1}^\infty\frac1{(2n-1)^4}=\left(1-\frac1{16}\right)\frac{\pi^4}{90}=\frac{\pi^4}{96} $$
You can also use the well known summation formula $$\sum_{n\in\mathbb{Z}}f\left(n\right)=-\sum\left\{ \textrm{residue of }\pi\cot\left(\pi z\right)f\left(z\right)\textrm{ at }f\left(z\right)\textrm{ poles}\right\} $$ which is a consequence of the residue theorem. So it is sufficient to note that $$ \sum_{n\geq1}\frac{1}{\left(2n-1\right)^{4}}=\frac{1}{2}\sum_{n\in\mathbb{Z}}\frac{1}{\left(2n-1\right)^{4}}=-\frac{1}{2}\left\{ \textrm{residue of }\frac{\pi\cot\left(\pi z\right)}{\left(2z-1\right)^{4}}\textrm{ at }\frac{1}{2}\right\} =\color{red}{\frac{\pi^{4}}{96}}.$$
Here is another proof I came up with. Start with the quadruple integral:
$$J=\int_{0}^{1}\int_{0}^{1}\int_{0}^{1}\int_{0}^{1} \frac{1}{1-x^2_1x^2_2x^2_3x^2_4}dx_1dx_2dx_3dx_4.$$
Replace the integrand with a geometric series as such: $$\frac{1}{1-x^2_1x^2_2x^2_3x^2_4}=\sum_{n=0}^{\infty}(x_1x_2x_3x_4)^{2n}.$$ Replace the integrand with the geometric series and integrate term by term to get:
$$J=\sum_{n=0}^{\infty} \frac{1}{(2n+1)^4}.$$
We now try to evaluate: $$J=\int_{0}^{1}\int_{0}^{1}\int_{0}^{1}\int_{0}^{1} \frac{1}{1-x^2_1x^2_2x^2_3x^2_4}dx_1dx_2dx_3dx_4.$$ Make the change of variables $$x_i=\frac{\sin(u_i)}{\cos(u_{i+1})},x_4=\frac{\sin(u_4)}{\cos(u_{1})}, 1\leq i \leq 3.$$ The Jacobian determinant turns out to be: $$\frac{\partial(x_1,...x_4)}{\partial(u_1,...u_4)}=1-x^2_1x^2_2x^2_3x^2_4$$ which cancels with the integrand. The region of integration is the (open) polytope $P$ defined by the constraints $$0<u_{i}+u_{i+1}<\frac{\pi}{2},0<u_{1}+u_{4}<\frac{\pi}{2}, u_{i}>0.$$ So this means $$J=\sum_{n=0}^{\infty} \frac{1}{(2n+1)^4}=\text{Vol}(P)$$ where $\text{Vol}$ means volume. I will consider the scaled polytope $V$ defined by constraints $$0<v_{i}+v_{i+1}<1,0<v_{1}+v_{4}<1, u_{i}>0.$$ You can show through change of variables $$u_i=\frac{\pi v_i}{2}, 1\leq i \leq 4$$ that $$\text{Vol}(P)=\left(\frac{\pi}{2} \right)^4 \text{Vol}(V).$$ So we will be done once we find $\text{Vol}(V).$ We will find $\text{Vol}(V)$ through probability.
We first find $$\text{Pr}\left(v\in V \cap v_1,...v_4<\frac{1}{2}\right),$$ which is the probability we pick an arbitrary point $v$ in $V$ where all coordinates are less than $\frac{1}{2}.$
$$\text{Pr}\left(v\in V \cap v_1,...v_4<\frac{1}{2}\right)=\left(\frac{1}{2}\right)^4=\frac{1}{16}.$$ This is the case because the open hypercube $I=(0,\frac{1}{2})^4 \subset V,$ which you can verify.
Now we find $$\text{Pr}\left(v\in V \cap \text{exactly one } v_i \geq \frac{1}{2}\right),$$ which is the probability we pick an arbitrary point $v$ in $V$ where exactly one coordinate is greater than or equal to $\frac{1}{2}.$ It turns out:
$$\text{Pr}\left(v\in V \cap \text{exactly one } v_i \geq \frac{1}{2}\right)=4\int_{\frac{1}{2}}^{1}\int_{0}^{1-v_1}\int_{0}^{1-v_1}\int_{0}^{\frac{1}{2}} 1 dv_3dv_4dv_2dv_1=\frac{1}{12}.$$ What I did here was compute $$\text{Pr}\left(v\in V \cap \text{only } v_1 \geq \frac{1}{2}\right)$$ and multiplied this answer by $4$ to account for all $4$ cases of this condition.
Now we find $$\text{Pr}\left(v\in V \cap \text{exactly two } v_i \geq \frac{1}{2}\right),$$ which is the probability we pick an arbitrary point $v$ in $V$ where exactly two coordinates are greater than or equal to $\frac{1}{2}.$ It turns out: $$\text{Pr}\left(v\in V \cap \text{exactly two } v_i \geq \frac{1}{2}\right)=4\int_{\frac{1}{2}}^{1}\int_{\frac{1}{2}}^{v_1}\int_{0}^{1-v_1}\int_{0}^{1-v_1} 1 dv_2dv_4dv_3dv_1=\frac{1}{48}.$$ What I did here was compute $$\text{Pr}\left(v\in V \cap \text{only } v_1,v_3 \geq \frac{1}{2}\cap v_3<v_1\right)$$ and multiplied this answer by $4$ to account for all $4$ cases of this condition. Keep in mind here that only two nonconsecutive coordinates can be greater than or equal $\frac{1}{2}$ at the same time. Thus, if we have more than two greater than or equal to $\frac{1}{2}$ at the same time, we will violate the constraints of $V$. Thus, we now just add up the computed probabilities to get: $$\text{Vol}(V)=\frac{1}{16}+\frac{1}{12}+\frac{1}{48}=\frac{1}{6},$$ which means $$\sum_{n=0}^{\infty} \frac{1}{(2n+1)^4}=\text{Vol}(U)=\left(\frac{\pi}{2}\right)^4\text{Vol}(V)=\left(\frac{\pi^4}{16}\right)\frac{1}{6}=\frac{\pi^4}{96}.$$
$\newcommand{\angles}[1]{\left\langle\,{#1}\,\right\rangle} \newcommand{\braces}[1]{\left\lbrace\,{#1}\,\right\rbrace} \newcommand{\bracks}[1]{\left\lbrack\,{#1}\,\right\rbrack} \newcommand{\dd}{\mathrm{d}} \newcommand{\ds}[1]{\displaystyle{#1}} \newcommand{\expo}[1]{\,\mathrm{e}^{#1}\,} \newcommand{\half}{{1 \over 2}} \newcommand{\ic}{\mathrm{i}} \newcommand{\iff}{\Longleftrightarrow} \newcommand{\imp}{\Longrightarrow} \newcommand{\Li}[1]{\,\mathrm{Li}_{#1}} \newcommand{\mrm}[1]{\,\mathrm{#1}} \newcommand{\ol}[1]{\overline{#1}} \newcommand{\pars}[1]{\left(\,{#1}\,\right)} \newcommand{\partiald}[3][]{\frac{\partial^{#1} #2}{\partial #3^{#1}}} \newcommand{\root}[2][]{\,\sqrt[#1]{\,{#2}\,}\,} \newcommand{\totald}[3][]{\frac{\mathrm{d}^{#1} #2}{\mathrm{d} #3^{#1}}} \newcommand{\ul}[1]{\underline{#1}} \newcommand{\verts}[1]{\left\vert\,{#1}\,\right\vert}$ As shown by $\texttt{@Timbuc}$, you just need to know the sum $\ds{\sum_{n = 1}^{\infty}{1 \over n^{4}}}$. It has an interesting evaluation by starting from the $\ds{\cot\pars{z}}$ expansion $\ds{\pars{~\mbox{with}\ z \not= 0, \pm\pi,\pm 2\pi,\ldots~}}$:
\begin{align} \cot\pars{z} & = {1 \over z} + 2z\sum_{n = 1}^{\infty}{1 \over z^{2} - n^{2}\pi^{2}} = {1 \over z} - {2z \over \pi^{2}}\sum_{n = 1}^{\infty}{1 \over n^{2}} {1 \over 1 - z^{2}/ \pars{n^{2}\pi^{2}}} \\[5mm] & = {1 \over z} - {2z \over \pi^{2}}\sum_{n = 1}^{\infty}{1 \over n^{2}} \pars{1 + {z^{2} \over n^{2}\pi^{2}} + \cdots} = {1 \over z} - {2z \over \pi^{2}}\sum_{n = 1}^{\infty}{1 \over n^{2}} -{2z^{3} \over \pi^{4}}\color{#f00}{\sum_{n = 1}^{\infty}{1 \over n^{4}}} - \cdots \end{align}
\begin{align} z\cot\pars{z} & = 1 - {2z^{2} \over \pi^{2}}\sum_{n = 1}^{\infty}{1 \over n^{2}} -\bracks{{48 \over \pi^{4}}\color{#f00}{\sum_{n = 1}^{\infty}{1 \over n^{4}}}}{z^{4} \over 4!} - \cdots \end{align}
$$ \color{#f00}{\sum_{n = 1}^{\infty}{1 \over n^{4}}} = \left.-\,{\pi^{4} \over 48}\,\totald[4]{\bracks{z\cot\pars{z}}}{z} \right\vert_{\ z\ \to\ 0} = \pars{-\,{\pi^{4} \over 48}}\pars{-\,{8 \over 15}} = \color{#f00}{\pi^{4} \over 90} $$