"What memory!" or "What a memory!"?

Solution 1:

Neither is wrong. They just mean different things.

“What memory!” refers to "memory" as a general concept.

If someone remembered things very well, you might say "What memory!" to mean "What (good) memory (you have)!"

“What a memory!” refers to a specific memory.

If someone told you a story, you might say "What a memory!" to mean "What a (good) memory (you have just talked about)!"

Edit: As noted in the comments, you could say "What a memory!" to refer to the general concept of memory as well. The example would be similar to "What a (good) memory (you have)!"

Solution 2:

The second is correct if referring to a singular instance of memory. If you were impressed by someone's ability to remember, that would be singular and again the second form is correct there.

If you were referring to many memories rather than a specific memory, you would use the plural "what memories!" (for example, when referring to an entire year "1986, ah the memories!").

The first version ("what memory!") could be taken as sarcasm, even with the exclamation mark. It could be used colloquially if someone referred to a memory you had of something, and you wanted to convey that you didn't actually remember the occasion that they were referring to.