Which of the two sounds more natural: corestrict or correstrict?

In mathematics, one uses the prefix co- to denote something that's dual to an already known object, for instance: limit -> colimit, basis -> cobasis, cycle -> cocycle, tangent -> cotagent, domain -> codomain.

On the other hand, words that arose by adding co- usually have a doubled 'r', for instance: responds -> correspond, relate -> correlate, radiation -> corradiation.

Similarly there's a notion of a restriction of a function, when one restricts its domain. This would naturally be called a corestriction/correstriction. Which of these two spellings sounds more natural and should be preferred? (if you're a native speaker, please state the country you're from)

Both spellings are used in the literature.

Corestriction:

  • https://projecteuclid.org/download/pdf_1/euclid.pjm/1102371109
  • https://www.tandfonline.com/doi/full/10.1080/00927870802103636
  • https://arxiv.org/abs/1409.3166

Correstriction:

  • https://www.tandfonline.com/doi/full/10.1080/00927870802103636
  • https://projecteuclid.org/journals/kodai-mathematical-journal/volume-9/issue-3/The-cohomology-of-rm-SL_2F_p-and-the-Hecke-algebra-actions/10.2996/kmj/1138037273.pdf
  • http://inveniov1.uvci.edu.ci/record/10082?ln=en

According to the New Oxford English Dictionary on my computer both co- and cor- (as well as the following: com- col- con-) are forms of a prefix originating from Latin cum meaning 'with'.

com- (also co-, col-, con-, or cor-)
prefix
with; together; jointly; altogether.
examples: combine | command | collude.


The dictionary encourages selection between the various forms of this prefix in the following manner:

USAGE
Com- is used before b, m, p, also occasionally before vowels and f. The following variant forms occur: co- especially before vowels, h, and gn; col- before l; cor- before r; and con- before other consonants.

However, my impression is that with more modern usage of this prefix, co- is preferred over its other forms since this form is more likely to be understood by audiences. For example, according to the formula which the dictionary prescribes in selecting between these various co- prefix forms, words such as coproduce would become comproduce, which I am inclined to believe would fail to be understood by more than just the average reader. Additionally, costar would be constar, which could easily be associated with
con
1. The use of deception to persuade someone to do or believe something.
2. A disadvantage.
The dictionary's further demonstration of how co- has been used also seems to indicate that the previous formula is generally now ignored in favor of co-.

USAGE
In modern American English, the tendency increasingly is to write compound words beginning with co- without hyphenation, as in costar, cosignatory, and coproduce. British usage generally tends more often to show a preference for the older, hyphenated spelling, but even in Britain the trend seems to be in favor of less hyphenation than in the past. In both the US and the UK, for example, the spellings of coordinate and coed are encountered with or without hyphenation, but the more common choice for either word in either country is without the hyphen. Co- with the hyphen is often used in compounds that are not yet standard (co-golfer), or to prevent ambiguity (co-driver—because codriver could be mistaken for cod river), or simply to avoid an awkward spelling (co-own is clearly preferable to coown). There are also some relatively less common terms, such as co-respondent (in a divorce suit), where the hyphenated spelling distinguishes the word's meaning and pronunciation from that of the more common correspondent.

NOTE: If the original formula had been adhered to, instead of the above examples we would have
constar vs co-star
consignatory vs co-signatory
comproduce vs co-produce
congolfer vs co-golfer
condriver vs co-driver
correspondent vs co-respondent

The last example also demonstrates how the form of the prefix that is chosen influences the way the word is read and interpreted—correspondent and corespondent both being valid words yet with completely different meanings.

So regarding your question co- vs cor-:
corestrict reads like |ˈkōrəˌstrikt| or |ˈkōrēˌstrikt|
correstrict reads like |ˌkôrəˈstrikt|
and the audience will be encouraged to interpret the words accordingly.

Furthermore there already seems to be a preference in mathematics for the co- form of the prefix.

co-
4. Mathematics of the complement of an angle: cosine. • the complement of: colatitude | coset.