Period placement in a sentence containing both parantheses and quotes

Which of the following is correct:

1. (This is a "test").
2. (This is a "test.")
3. (This is a "test".)

Let's first look at the parentheses (Example #1). As is brought out in this answer (only the relevant point has been included):

Examine the material enclosed by parentheses. Is it an entire sentence? If so, place the period inside the closing parenthesis.

Since "This is a test" is a complete sentence, the period belongs inside of the parentheses, so the first example is incorrect (though, people will definitely still understand you even if you are "slightly incorrect" in that case).

The other two examples you posted will vary depending on if you are reading American English or British English. Example #2 is preferred in Britain, Example #3 is preferred in the United States. See the following answer for more details: How should I punctuate around quotes?