If I can say "not that good a review," does that mean I can say "not that good reviews"?

I'm new to the template, so please forgive my ignorance of this community's parlance, formalities.

I'd imagine that many here have seen the construction:

"Adjective + Article + Noun," as in "so fine a person," or "that fine a person."

My question pertains to the possibility of a "Adjective + Plural Noun" construction, as in "They weren't THAT GOOD REVIEWS," or, "They weren't THAT GOOD PEOPLE." This sounds stilted at best and at worst, wrong. Do these sentences require the addition of an "Article + Singular Noun + Of" between the adjective and plural noun, as in "They weren't that good A GROUP OF people"?

Thank you to all who can shed light on the matter. I'm still a high school student and am acquiring the tools necessary to analyze problems like this one.

I would agree with your judgement that *"They weren't that good reviews " and *"They weren't that good people" are both ungrammatical.

This judgment seems to be consistent with Frank Van Eynde's description of "The Big Mess Construction":

This construction, for which Berman (1974) coined the term Big Mess Construction, only ocurs in nominals with an indefinite article. It does not occur in nominals with another kind of determiner, as in (5a), nor in nominals without determiner, as in (5b).

 (5) a. * How serious some problem is it?
     b. * They are so good bargains I can’t resist buying them.

Van Eynde does refer to so as an adverb, and I would infer that he classifies that this way as well:

It is worth adding that some of the degree denoting adverbs license the addition of another dependent: so, for instance, licenses a that-clause, as in (33), and too a gapped VP[to], as in too complex a problem to solve here and now (p. 428).

However, it wouldn't surprise me to learn that another linguist used different terminology. John Lawler left a comment describing another point of view:

It's a required lexical part of an idiomatic construction. POS is not a useful category for lexical items like so, such, than, etc; mostly they fall into some grammatical category like complementizer – a marker for the construction or construction type without any particular status outside it.

On a more technical level, Van Eynde refers to "so" and like words in this construction as filling the grammatical role of "functor of the adjective." I had not encountered the term "functor" before reading this paper, and I'm not sure how commonly it is used. Just knowing the name doesn't really tell you anything important about it, though: if you read the paper, you'll see how he defines it and what he thinks is the grammatical structure of this construction.

I don't know if your example sentences might be grammatical for some speakers. This construction seems to be somewhat prone to re-interpretation, perhaps because of its odd structure. In particular, the article "a" is often preceded by "of," and sometimes "of" is used before a noun without the indefinite article. I found the Van Eynde article linked in a comment left by Russell at April 20, 2016 on the following Language Log article, "Bad of shape," which describes one innovative usage.

I also found an earlier Language Log article that describes another non-standard usage with a mass noun, "It doesn't seem like that painful of work": "Not that adjective of (a) noun"

You can find results on the web for "that good of reviews," such as the following GameFAQs thread: "I knew this game wouldn't get that good of reviews"