"I didn't realize"

In your example, both are grammatical and natural, and they mean the same thing.