"can remind" or "can to remind"?

They do what they can // to remind us.

Can is the verb of the object clause what they can. It does not form a verb with what follows. What follows is an infinitive, to remind, which stands on its own and expresses purpose. You could rewrite your sentence in this way

They do what they can [in order/so as] to remind us.

ADDITION: Can is a modal verb, and one of its meanings is

to be able to:

• Can you drive?
• She can speak four languages.
• The doctors are doing all that they can, but she's still not breathing properly.
• Do the best you can - I realize the circumstances are not ideal. (Cambridge)

As you can see, can can be used on its own, but there is always a verb implied. In the last two examples I quoted from Cambridge, the clauses could read:

The doctors are doing all that they can [DO]...

and

Do the best you can [DO]...

So yes, your intuition that can must modify every verb that's next to it bears some truth, but the mere presence of TO between CAN and the VERB following tells you that CAN does not modify that verb. It modifies the verb that is implied and was omitted to avoid repetition. I will write your sentence again for clarity

They do what they can [DO] [in order/so as] to remind us.

In this sentence, they do what they can to remind the rest of us, what they can (wh- clause) is the object of the verb (They) DO, whereas to remind the rest of us is an adjunct of purpose modifying the verb (They) do.

Great poets are expressly aware of this, and they do what they can to remind the rest of us.

You have not parsed the sentence correctly.

The full sentence is

Great poets are expressly aware of this, and they do what (= those things that) they can do in order to remind the rest of us.