Progressive infinitive

Could anyone please explain why I can't use "progressive infinitive" in the sentence below:

▪It was my fault not to be making you laugh.


The sentence reads oddly. 'Not to be making someone laugh' at a particular moment isn't exactly a fault. Perhaps you mean It was my fault that I could not make you laugh?