Forcing a faction to revolt
As it was the biggest faction, I realized any action that angered all vassal's would push this faction to an ultimatum (which is what I want) before any other faction, thus I just went over my demesne limit until their ultimatum fired, took the ultimatum, and immediately gave away extra lands before other factions became too powerful.
EDIT: In hindsight, I could have used the Court Jester title as well, which is more or less the point of it.
Another alternative is to raise all the troops of the target vassal(s) and let them sit until the vassal(s) rebel. This is usually used if you just want to remove a leader (of a faction or otherwise) without instigating a full civil-war, but it could be applicable here.
Other things one could try is to revoke the leader's honorary titles or make him or her the court jester.