How suffixes like -ness and -ship are chosen when forming abstract nouns?

Solution 1:

There is no system in wordformation with suffixes. It is mostly a matter from which language the suffixed words are taken. A great percentage of these words were taken from Latin, and Latin already had a highly developed system of prefixes and suffixes. The other words where taken from French wtih the French suffixes. So it is almost impossible to predict which suffix a word will have. Of course you will find subsystems,eg the nouns with -ness are almost all formed from adjectives, but it is not warm *warmness but warmth, and it is not cold *coldness, but cold adj and cold noun. Wordformation is a vast and unsystematic area. I have a reference work on this matter - Hans Marchand, The Categories and Types of Present-Day English Word-Formation, Beck München 1969-2, 545 pages. I must say I seldom use it, now and then I look up a special affix, but the book is not really important for me.

Solution 2:

English is a very irregular language, certainly made more so by the number of words that come from other languages (e.g loanwords and Calques)

I would suggest using a English dictionary (or word list, if your program is doing the parsing), to look up the root word, to see if there is an established word that matches your pattern. This will take care of the normal expressions.

For neologisms, (i.e.the ones that don't exist in the dictionary or word list), I do not know of a way to generate them automatically so they sound valid. I am sure some linguist can uncover some arcane grammatical rule which can be applied.