How do I set the at command shell to bash?

You can't change the default shell of at, it is hardcoded as /bin/sh in the source.

The source code of at clarifies this, from at.c:

    /* POSIX.2 allows the shell specified by the user's SHELL environment                                                           
       variable, the login shell from the user's password database entry,                                                                
       or /bin/sh to be the command interpreter that processes the at-job.                                                               
       It also alows a warning diagnostic to be printed.  Because of the                                                                
       possible variance, we always output the diagnostic. */

    fprintf(stderr, "warning: commands will be executed using /bin/sh\n");

Implementation, from atd.c:

if (execle("/bin/sh", "sh", (char *) NULL, nenvp) != 0)

The man page conforms accordingly:

at and batch read commands from standard input or a specified file which are to be executed at a later time, using /bin/sh.

That leaves you to recompile your own copy as the only solution to meet your need.

Looking at man at, it seems like you can't change the shell.

However, you could just launch your commands inside a Bash shell inside at's SH shell, like this:

at 01:23 <<< "bash -c 'notify-send \"running from $SHELL\"'"