When and why did the N-word and "negro" go apart?
Both the terms nigger and negro come from the Spanish and Portuguese Negro which denotes "black". But today they have widely different connotations, the former is considered a horrible racial slur, while the latter was the prefered way to refer to a person of black ancestry or appearance until the 1960s–70s, was used by MLK in his "I have a dream" speech and is still used in the full name of the UNCF.
When did the terms go apart, and why does one of them have a strong racist connotation while the other doesn't?
Solution 1:
Tone partly comes from origin, and partly from use.
If someone coins a word based on things already held in low regard, or start applying such a word to a group, then that word is going to start out with negative connotations. It could perhaps lose them, but the inverse is also true.
If someone coins a word that is not based on things held in low regard, it could still indicate a pointed distinction; that is, one bothers to point out that some people are X and treat those who are not X as "normal".
And with either of those, and even words coined with distinctly positive connotations, the word can acquire negative connotations through use.
And so it is with the words you mention. They are mostly used by racists, which makes them acquire the associations racists apply to them, which means that they are mostly used by racists.
Two particular influences were:
The abolitionists beginning to favour coloured. The more they avoided the terms n----r (and to a lesser extent, negro), the more that term was only used by racists, and hence the more often it would be associated with racist views, and hence the more non- or less-racist people would avoid it, and so on.
Objections to the extension of the word. In Britain it was sometimes used to refer to any foreign person who wasn't white. In the US the term "inside-out n----r" was applied to white people whose white privilege other white people wanted to remove or reduce (particularly those relatively new white immigrants to the US, particularly from mainly-Catholic countries, such as Ireland, Poland and Italy). Such people didn't object "well, technically that word applies to a different group to me", they objected to being associated with black people, who they too were mostly bigoted against, and that was indeed the intended insult. In using a word associated with black people as an insult to others, it increased the degree of insult not only to the target, but also to black people.
Some words flip, black (which outside of matters of skin tone has some negative connotations already) was once mostly used by racists, while coloured or negro were not. Negro started to be avoided because of its place in some phrases particularly associated with history (in particular house negro and field negro), and because it's got a madey-uppy othering quality that white does not. Meanwhile black was argued to be perfectly accurate (or at least as accurate as white is for white people) and not innately any more negative than white. As African-Americans started to favour black, the more racist people deliberately avoided it, and so the more so black lost its derogatory sense, and negro gained one, leading it to be further restricted only to racists, and so on.
There was also a lot of anthropological use of negroid, which was mostly pseudo-anthropological nonsense, and mostly racist. These days, while it retains some value in forensic and physical anthropology, the study by anthropologists of how different peoples define race has made any claim to objective distinctions between so-called "races" infeasible. See for example the American Anthropological Association Statement on "Race". The more that sort of anthropology was discredited, the more the term negroid and the related negro were associated with racist views, the more it would be avoided by all but racists. It hence also became a word that, especially when used by someone who isn't black themselves, doesn't mean "black person", but means "black person who, as such, I think less of". The shift is partly passive (racists just don't bother to avoid such words) and partly active (racists deliberately use words found offensive, because well racist people are the sort of scum that would do that). It is also particularly favoured by the worse racists, because they want to suggest their bigotry is based on some sort of scientific understanding.
We can find a similar shift in gay (originally coined as an insult based on the sense of gay of "sexually immoral") becoming the preferred term, while homosexual (originally coined as a neutral scientific term) is very often used by homophobes who want to portray their bigotry as neutral and objective.
Negress is particularly othering, in being a term for a black woman for which there is no comparable term (what's the word for a white woman?) and so combining sexual and racial discrimination in the very idea that we need treat black women as such a special case (and often, point of curiosity and target of sexual predation). As that became more conciously analysed with the growth of both the civil rights movement and the women's movement, and the (often fraught) intersection of the two (e.g. the Combahee River Collective), that led to both negress and negro being more disfavoured.
One possible response to all this is to think "well, this is all very arbitrary, if we just use all the words, they'll all stop being hurtful", as in e.g. Lenny Bruce's famous piece. In reality though, that's not very far removed from saying "gosh, it would be great if we'd all just stop being racist"; it's predicated on (a) bigotry being so rare among the more privileged group that they'd wash out the remaining pointedly bigoted uses and (b) bigotry being so rarely experienced by the less privileged group that such an experiment wouldn't be hurtful. If we lived in such a world, there'd be no need for such an exercise to begin with.
Solution 2:
The brief answer is that both terms became somewhat offensive in the 19th century, but negro (and later Negro with a capital) was deliberately rehabilitated by American black intellectuals from the 1890s through the 1930s to become the preferred polite racial term, before it fell into disuse in the 1970s.
[EDITORIAL NOTE: In the past year or so (2020-21), we appear to have lost the use-mention distinction for the mere quotation or citation of the six-letter word discussed in the question. Mere quotation of the word from another source or use of it to clarify linguistic context is now viewed as racist by many and apparently in some cases can be viewed as grounds for dismissal from employment. I therefore have removed all actual spelled out versions of the word in this answer. Apologies if this decreases the clarity of the answer. Please note that some of the links are to sources that use the unexpurgated version of the word.]
It's unclear whether n_____ ever had a neutral meaning. Prior to 1800 or so, the spelling with gg was extremely rare. Instead, the word niger was occasionally used, but (as the OED notes) this may often have been a learned "direct reborrowing" of the Latin word niger, meaning the color "black." Niger appeared as a neutral term before 1800, even in some Abolitionist treatises. However, the early pronunciation of this niger is uncertain, and thus it may or may not have a direct connection to the later offensive slur n_____, which only really begins to appear with that spelling (as well as other new variants, such as niggur and niggah) in the 19th century.
In contrast, the OED notes that "In standard English usage the word Negro had already become the usual neutral term by the end of the 17th cent."
According to Randall Kennedy's detailed book N[_____]: The Strange Career of a Troublesome Word (2002): [Editorial underscores introduced, see EDIT note above.]
No one knows precisely when or how niger turned derisively into n[_____] and attained a pejorative meaning. We do know, however, that by the end of the first third of the nineteenth century, n[_____] had already become a familiar and influential insult.
In A Treatise on the Intellectual Character and Civil and Political Condition of the Colored People of the United States and the Prejudice Exercised Towards Them (1837), Hosea Easton wrote that n[_____] "is an opprobrious term, employed to impose contempt upon [blacks] as an inferior race..."
Kennedy's quotations go on to describe in detail how the term had already acquired a strong derogatory association well before the American Civil War. The OED has its earliest clear citations of the word appearing in abusive senses in the 1810s.
Negro, on the other hand, never acquired this level of derogatory meaning. Nevertheless, as alternative terms came to be used among free blacks and by the abolitionist movement (notably colored), the word negro remained the more generic term associated with slavery (e.g., used in phrases like the Negro Question in politics). Perhaps due to its similar sound to n_____, some came to question its use by the mid-1800s as well. For example, black businessman James Forten asked in 1831: "Why do our friends as well as our enemies call us negroes? We feel it a term of reproach, and could wish our friends would call us by some other name."
By the end of the Civil War, with the abolitionist cause victorious, many simply wanted to leave the older terms behind. Others embraced the term negro, preferring it to words like colored which could also refer to other non-white races. A 1967 article from Ebony summarizes the conflicts of this time:
We are told, for example, that Blanche Kelso Bruce, the first black man to serve a full term in the U. S. Senate, refused to use the word "colored," saying: "I am a Negro, and proud of my race." Bruce's example was not followed by all Reconstruction leaders. In the North Carolina constitutional convention of 1888, James Walker Hood, one of 15 black delegates, denied that "there was a Negro on the floor of the Convention." Outraged and insulted, he insisted "that the word Negro had no significance as to color, but could only be used in a reproachful or degrading sense, and he further declared that no man on that floor knew where the term originated, since it was not found in ancient history, inspired or profane." In the South Carolina constitutional convention of the same year, T. J. Coghlan, a radical white Southerner, offered a resolution which urged that steps be taken to "expunge forever from the vocabulary of South Carolina, the epithets 'n[_____],' 'negro,' and 'yankee' . . . and to punish this insult by fine and imprisonment."
The general sentiment during Reconstruction seemed to be away from negro, except among some groups of newly emancipated slaves. It began its long rehabilitation in the 1890s and early 1900s, led by prominent black intellectuals, particularly Booker T. Washington and W.E.B. DuBois. The long history of the term and its "strength" as a word (both in general sound and implications, as well as not needing to be paired with other words, as in colored people) seemed to be primary motivations, as DuBois argued in 1928:
Negro is a fine word. Etymologically and phonetically it is much better and more logical than "African" or "colored" or any of the various hyphenated circumlocutions. Of course, it is not "historically" accurate. No name was historically accurate: neither "English," "French," "German," "White," "Jew," "Nordic," nor Anglo-Saxon." They were all at first nicknames, misnomers, accidents, grown eventually to conventional habits and achieving accuracy because, and simply because, wide and continued usage rendered them accurate. In this sense, "Negro" is quite as accurate, quite as old, and quite as definite as any name of any great group of people.
The sentiment had been growing throughout the early 1900s, with advocates also arguing for capitalization, to put the term on the same footing as other terms for groups of various heritage (French, Irish, Jewish, etc.). Again from the Ebony article:
By 1919, the Negro Year Book could report: "There is an increasing use of the word 'Negro' and a decreasing use of the word 'colored' and 'Afro-American' to designate us as a people. The result is that the word 'Negro' is, more and more, acquiring a dignity that it did not have in the past." During this same period, there was an aggressive campaign for capitalization of the word "Negro." This campaign, which was led by the NAACP, peaked in 1930 when the New York Times announced that it would print the word "Negro" with a capital letter. In an editorial (March 7, 1930), the newspaper said: "In our 'style book' 'Negro' is now added to the list of words to be capitalized. It is not merely a typographical change; it is an act of recognition of racial self-respect for those who have been for generations in 'the lower case.'"
By the 1930s, academics and intellectuals commonly discussed Negro art, Negro poetry, and Negro music, and it supplanted colored as the preferred polite term in most contexts. It was used on official forms (like the census), in the names of organizations (like the United Negro College Fund), and most blacks began to self-identify with it.
The fall of Negro happened in the 1960s, with another group of black intellectuals and activists arguing that the term simply could not be separated from its use during the period of slavery. Richard Moore's book The Name "Negro": Its Origin and Evil Use (1960) was probably the first significant blow, arguing: "Its origin is vile and infamous. It began in indignity. It began in immorality, and the consciousness and the dignity of man must now rise and dispense with it forever."
The sentiment began to build among the Black Power movement and the Nation of Islam. Negro was more frequently accused of being an invention of oppressors (and suspiciously similar to the most offensive slur n_____), while black -- previously viewed as an insult, or a technical term denoting full black ancestry in the previous era that differentiated a full black from a mulatto or a quadroon or an octaroon -- was redeemed by the militant anti-establishment Black Power leaders.
The replacement of Negro was rapid and decisive. Polls from the mid-1960s show 50-70% of blacks preferring the racial term Negro, with significant numbers also holding onto colored; Black was a term mostly for younger people and radicals. But, by the mid-1970s, Black had by far become the dominant term, with 10% or less of blacks retaining a preference for Negro.
While there were arguments in certain circles to similarly rehabilitate n_____ from its status as a slur, as black had been recovered from its disrespectful status, these never really caught on outside of particular uses among blacks (often still containing a somewhat derogatory or ironic sense and frequently with the variant spellings nigga or niggah).
Meanwhile, at least in the U.S., Negro now has a status varying from antiquated or old-fashioned to offensive and taboo. By the mid-1980s, it ceased to be a common term even among the old black intellectuals who had championed it; Thurgood Marshall was the last Supreme Court justice to use it. Organizations founded when the term was popular have either changed their names or sometimes avoid referencing Negro (e.g., the United Negro College Fund has stated a preference for being called U.N.C.F. to avoid the term). Nowadays, public figures who use it risk strong negative reactions, and it caused quite a sensation when it appeared on the 2010 census (and will henceforth be removed). While a small group of people apparently still prefer the term, and many others find the term unproblematic, its old associations with slavery have ultimately proved too strong for the early twentieth-century cause in favor of it to stick.
(Some other sources used here and here.)