Is "yay or nay" an acceptable alternative to "yea or nay"?

The words yea and yay are homophones, meaning they are pronounced the same. Yea is a somewhat specialized word (“yes—used in oral voting”) most often used in a spoken context, so I would expect that many people would not realize there are two spellings for two different meanings.

Using the spelling yay for yea is therefore an eggcorn. Here's a discussion of it at the Eggcorn Database forum.

There were no examples of yay being used for yea at the Corpus of Contemporary American English, so it doesn't seem to be an emerging usage that is getting into the kinds of edited texts used to populate that corpus. On the other hand, there are plenty of Google results, including some on major, presumably professionally-edited sites, like ABC News and New York Magazine. Interestingly, a large portion of the top Google results were related to fashion.

How embarrassing for them: anyone who knows that there are two spellings for the two words would instantly identify yay for yea as an error.

Even the normally quite liberal and descriptivist Merriam-Webster dictionary has no entry at all for ‘yay’. Only Collins English Dictionary has an entry for yay, and it’s only for the exclamatory sense.

So, yes, it does appear that this substitution is an emerging usage, but no authorities at all countenance it just yet, and given that there is an unimpeachable substitute, I don’t recommend that anyone use it.

I've never seen that spelling of "yea" in that context. I wouldn't use it personally - it looks too much like the exclamative.