dig +trace: no servers could be reached for my domain?

Solution 1:

frontend.myapp.com.  14400 IN  NS  d-123sdf1234asd.execute-api.ap-northeast-1.amazonaws.com.

The NS record type is for name servers.

That current record is essentially saying: if you want to know anything DNS related about the frontend.myapp.com subdomain or about any record below that subdomain, ask the name server running on d-123sdf1234asd.execute-api.ap-northeast-1.amazonaws.com

And d-123sdf1234asd.execute-api.ap-northeast-1.amazonaws.com is not a name server at all, so there will not be any response to DNS queries sent there.
That is the connection timed out; no servers could be reached error you see.

I think that what you intended was to set up a DNS CNAME record, which basically says, if you want to reach frontend.myapp.com. use the IP-address of d-123sdf1234asd.execute-api.ap-northeast-1.amazonaws.com.

Solution 2:

You need to create a CNAME record pointing to AWS API gateway d-123sdf1234asd.execute-api.ap-northeast-1.amazonaws.com., not an NS record.