For which categories we can solve $\text{Aut}(X) \cong G$ for every group $G$?

It is usually said that groups can (or should) be thought of as "symmetries of things". The reason is that the "things" which we study in mathematics usually form a category and for every object $X$ of a (locally small) category $\mathcal{C}$, the set of automorphisms (symmetries) of $X$, denoted by $\text{Aut}_{\mathcal{C}}(X)$, forms a group.

My question is: Which categories that occur naturally in mathematics admit all kinds of symmetries? More precisely, for which categories we can solve the equation (of course up to isomorphism) $$\text{Aut}_{\mathcal{C}}(X) = G$$ for every group $G$?

I will write what I could find myself about this, which also hopefully illustrates what kind of answers that would interest me:

  • Negative for $\mathsf{Set}$: Infinite sets have infinite symmetry groups and for finite sets we get $S_n$'s. So if we let $G$ to be any finite group which is not isomorphic to some $S_n$, the equation has no solution.

  • Negative for $\mathsf{Grp}$: No group can have its automorphism group a cyclic group of odd order.
  • Positive for $\mathsf{Grph}$ (category of graphs): Frucht's theorem settles this for finite groups. Also according to the wikipedia page, the general situation was solved independently by de Groot and Sabidussi.

  • An obvious necessary condition is that $\mathcal{C}$ should be a large category.
  • This paper shows that the equation can be solved if $\mathcal{C}$ is the category of Riemann surfaces with holomorphic mappings and $G$ is countable.

  • If we take $\mathcal{C}$ to be the category of fields with zero characteristic, I guess the equation relates to the inverse Galois problem. Edit: This may be much easier than the inverse Galois problem, as Martin Brandenburg commented.

  • Solution 1:

    Every finite group arises as the automorphism group of a finite poset. This is the subject of Automorphism groups of finite posets by J. A. Barmak and E. G. Minian, available online.

    Since the category of finite posets is isomorphic(!) to the category of finite $T_0$ spaces, in particular every finite group is the automorphism group of a topological space. More generally, it is proven in Spaces with given homeomorphism groups by Thornton, also available online, that every finitely generated group is the automorphism group of a topological space. I wonder if we can realize every group?

    The paper Representing a Profinite Group as the Homeomorphism Group of a Continuum by K. H. Hofmann and S. A. Morris, available online, deals with the question whether every profinite group is isomorphic to the automorphism group of a compact connected space (endowed with the compact open topology).