How would you state categorically that $\mathcal{P}(X)$ and $2^X$ contains the same information?

Solution 1:

One possibility is stating that $2$ is the subobject classifier of the category of sets. You can have a look at the wiki or the nlab. Indeed you can see the statement as establishing a certain (natural) isomorphism of the functors $\mathrm{Sub}_{\mathrm{Set}}$ and $\mathrm{Hom}(-,2)$. (I was going to post this as a comment but I guess it's more like an answer?).

Solution 2:

Your intuition is correct. I would phrase this as saying there is a natural isomorphism from the functor $$\operatorname{Hom}({-}, 2) : \mathbf{Set}^\mathrm{op} \to \mathbf{Set}$$ to the functor $$\mathcal{P} : \mathbf{Set}^\mathrm{op} \to \mathbf{Set}.$$ Note that there is more than one way to make the assignment $X \mapsto \mathcal{P}(X)$ into a functor. Here, I'm using the way that sends a function $f : X\to Y$ to the inverse image operation $\mathcal{P}(f) = f^{-1} : \mathcal{P}(Y) \to \mathcal{P}(X)$.

Exercise: What's another way to make the assignment $X\mapsto \mathcal{P}(X)$ into a functor?

Exercise (if you are familiar with Yoneda lemma): The Yoneda lemma tells us that natural transformations $\operatorname{Hom}({-}, 2) \to \mathcal{P}$ correspond to naturally to elements of $\mathcal{P}(2)$. Identify which subset of $2$ witnesses the natural isomorphism you want.