Can I attach a .dmg without mounting the drive that it is sitting in?

If you mount a drive one of the things which happens is that the file system type gets identified so macOS afterwards knows how to access the content of the drive (and find the DMG for instance). So operating systems can't access files on a drive without mounting it first.

Your actual problem seems to come from the fact that TestDisk is a tool to recover data from physical drives. So it could be used to work with the original disk your DMG represents, but it won't be able to recover data from a DMG.

The reason testdisk fails is because it needs to write to the device, but your dmg is probably stored in read-only format, so it is not able to write to the virtual device that represents the dmg.

Because it's not smart enough to check if the device is a read-only media (like read-only dmg), it assumes that it cannot write to the disk because a partition is mounted. When a partition is mounted, it's locked/write protected so that you don't lose data.

To solve this, you can either "restore" the dmg to an actual physical drive then run testdisk on it, or you can mount this dmg as read-write using a shadow file. See man hdiutil for more details.

-shadow [shadowfile]
                     Use a shadow file in conjunction with the data in the pri-
                     mary image file.  This option prevents modification of the
                     original image and allows read-only images to be attached
                     read/write.  When blocks are being read from the image,
                     blocks present in the shadow file override blocks in the
                     base image.  All data written to an attached device will
                     be redirected to the shadow file.  If not specified,
                     shadowfile defaults to image.shadow. If the shadow file
                     does not exist, it is created.  hdiutil verbs taking
                     images as input accept -shadow, -cacert, and
                     -insecurehttp.

So your command would probably look something like: hdiutil attach -nomount -noverify /path/to/dmg -shadow (/path/to/shadow/file (optional))