Can we use to-infinitive after 'have trouble'?
No, you may not use the to-infinitive after to have trouble. You may only use the -ing form. You can think of it as an abbreviated version of this:
I have trouble with speaking English.
And for the most part, to-infinitives make poor objects of prepositions; for that were gerunds invented.