Solution 1:

While, according to the OED, penitent (first found in English in 1341) and repentant (from 1230) have separate roots, they have similar meanings, and are partially defined in terms of each other.

The OED has for the pair:

Penitent: A. adj. 1.a. That repents with sincere desire to amend the sin or wrongdoing; repentant, contrite.

B. noun 1. A person who repents; a repentant sinner.

Repentant: A. adj. 1. Experiencing repentance; feeling contrition or regret for past sins or actions; penitent.
B. n. 1. A person who repents, a penitent.

In both cases, the majority of citations given imply a religious context.

The definitions in Merriam-Webster are even more circular:

Penitent (adj.): feeling or expressing humble or regretful pain or sorrow for sins or offenses : repentant

Repentant (adj.): 1 : experiencing repentance : penitent 2 : expressive of repentance

From the definitions found in these two dictionaries, it's difficult to draw any distinction between the terms. Both are predominantly found in a religious context, but neither one is, from the definitions given, necessarily more religious than the other.

Solution 2:

James Fernald, English Synonyms and Antonyms (1914) bundles repentance and penitence with attrition, compunction, contriteness, contrition, regret, remorse, self-condemnation, and sorrow, and then offers this succinct distinction between the two:

Repentance is sorrow for sin with self-condemnation, and complete turning from the sin. Penitence is transient, and may involve no change of character or conduct.

The 1947 edition of Fernald's book, now titled Funk & Wagnalls Standard Handbook of Synonyms, Antonyms, and Prepositions, rearranges the order of the sentences in the larger block of text, but leaves the wording of the remarks about repentance and penitence unchanged.

Meanwhile, Webster's Dictionary of Synonyms (1942) offers this commentary on penitence and repentance, which it places in a group with contrition, attrition, compunction, and remorse:

Penitence, repentance, contrition, attrition, compunction, and remorse agree in denoting sorrow or regret for sin or wrongdoing. Penitence implies little more than such sorrow or regret; as, the outward signs of penitence; his penitence is only skin-deep; he showed his penitence in many ways. Repentance is richer in its implications, for it also implies a change of heart, an awareness of one's shortcomings morally or or spiritually, or of the evil of one's actions or life as a whole. "I came not to call the righteous, but sinners to repentance" (Luke v. 32) "God of his mercy give/ You patience to endure, and true repentance" (Shak[espeare]).

Merriam-Webster's Dictionary of Synonyms (1984) updates the examples of repentance and penitence, but not the language of the distinction that its predecessor drew 42 years earlier.

I find the insistence by both of these resources on the relative superficiality of penitence surprising. Perhaps in the first half of the twentieth century, there was a general sense that penitence was mere lip service and repentance was heartfelt—but if so, I think that distinction has very nearly vanished in current usage.

With regard to the adjectives penitent and repentant, I acknowledge that penitent seems to me rather more likely to appear in connection with conduct such as enduring physical chastisement as the price of forgiveness, whereas repentant doesn't seem especially concerned with the scourging of one's sins by some third party. But in general today, I don't detect a difference in sincerity between a person who is penitent and one who is repentant.