What purpose does an '-o' serve?
The OED (1st edition—another answer supplies a more recent treatment) regards this as a (usually Scots) variant of older -a, both being common tags on the rhyming words in popular ballads (-o from 1727, -a from 1567). See this and this for examples. Note that the convention is only to record the extra syllable in the first stanza, no doubt to save the printer effort.
At A, inter., 4 OED conjectures that it arose in "the necessary retention of ME. final -e where wanted for measure" (that is, meter). I can adduce no example, nor is it likely that any could be found, since would have occurred in the spoken (or sung) language's evolution—and certainly before printing and a rising middle-class market provided an incentive to record such vulgarities.
Uniting arguments at both A and O, we find it implied that over the course of time this extra syllable came to be regarded as an interjection, Ah! or Oh!, evolved thence into a stock ornament of popular song, and eventually became so identified with the genre that it became a 'signature' of burlesque balladry.
This seems very plausible to me. -o marks the genre—and incidentally provides the balladmaker additional melodic opportunity.
EDIT: I believe this answers your question with respect to your first two quotations. In the third o more likely represents a weary interjection. @tchrist's answer seems to me to address your penultimate paragraph.
I can't compete with @tchrist's Tolkien quotation, but I can provide an instance of 'burlesque', from W.S.Gilbert's Yeomen of the Guard:
I have a song to sing, O!
Sing me your song, O!
It is sung to the moon
by a love-lorn loon
Who fled from the mocking throng, O!
It's the song of a merryman, moping mum,
Whose soul was sad, and whose glance was glum
Who sipped no sup and who craved no crumb
As he sighed for the love of a ladye.
Sullivan's tune's pretty nifty, too.
Edit
I should have checked the OED3; it has significantly expanded its treatment.
They seem much more certain now, citing three distinct origins, then following that up with significant discussion in their finest print, and then their four distinct senses. Skip the fine print the first time you read this.
Etymology: < -o, of various origins:
- (i) as the final syllable of words of chiefly Romance origin;
- (ii) as the vowel that became final after the shortening of a word by dropping the syllables following a medial o, especially in compounds truncated after a prefix or combining form ending in -o ; and
- (iii) < ho int.1, O int., and oh int., occurring as a second element in various exclamatory phrases. The suffix is attached both to full words and to truncated forms of words and phrases.
(Here is where the fine print occurs.)
Forming slang and colloquial nouns, adjectives, and interjections. The use of the suffix is widespread in English-speaking countries and is especially associated with Australia.
Forming interjections, as whacko int., whammo int.
Forming familiar, informal equivalents of nouns and adjectives, as (from truncated word-forms) aggro n. and adj., combo n., metho n.1; (from complete words) bucko n., kiddo n.; cheapo adj., neato adj.
Forming personal nouns from non-personal nouns, as milko n., wino n.1
Forming nouns from adjectives, as pinko n., weirdo n.
Fine Print
Borrowing of words ending in -o from Italian, Portuguese, and Spanish, with retention of the final vowel, began in the early 16th cent. (e.g. Morisco adj., frizado n.). Previously, such forms had usually been assimilated to native phonological structure by the removal of the final vowel. Since unstressed final long vowels had not hitherto been part of English phonology, there was a tendency for the vowels o and a in this position to be confused; in spelling, final o frequently replaced a and other syllables (e.g. in camisado n. (Spanish camisada), calico n. (compare the place name Calicut), both from before 1550), and this tendency to substitute o continued into the 17th cent. and later (compare mango n.1 (Portuguese manga ), lingo n.1 (Portuguese lingua)); there are also English formations which appear to have been made in imitation of Romance loanwords (e.g. stingo n. and perhaps the nonce-word twango int.). By the late 17th cent., final -o seems to have become assimilated into English phonology and appears in new words of undetermined origin, e.g. bingo n.1, rhino n.1
The shortening of a word immediately after a medial o, and in particular where this occurs at the end of a prefix or combining form, first appears in the late 17th cent. and early 18th centuries, e.g. plenipo n., memo n., and hypo n.1 This probably established an association of the ending -o with casual or light-hearted use which it has retained ever since. Further examples are attested in the early 19th cent., e.g. (combining forms) Anglo n.1, mezzo n.1, typo n.; (other words) compo n.2, loco n.1 After 1851 this type of clipping becomes, and has remained, extremely common. Truncation after a written o not pronounced as such may be the explanation of tambo n.1 (mid 19th cent.), which, if not the first example of the use of the suffix -o, seems to anticipate its later pattern of use with truncated word-forms.
The attachment of ho int.1, O int., and oh int. to other words to form conventional cries and refrains is attested from late Middle English, e.g. in heave ho int. and n., hey-ho int.; alive ho! is attested from the early 18th cent. or earlier (see (all) alive, (alive), oh! at alive adj. Phrases 2); and expressions such as righto int. and n., billy-o n., and cheerio int. are attested from the later 19th and early 20th cent. A number of words occur in the second half of the 19th cent. which appear to have their origin in the attachment of one of these interjections to a noun or adjective to form a refrain-like or vocative expression (such as a form of address or a public announcement) which was subsequently used as an ordinary noun or adjective (compare spell oh! (or ho!) at spell n.3 3c). Some of these are normally or frequently written as hyphenated compounds of -ho , -O or -oh, e.g. smoke-ho n., dead-oh adj. (late 19th cent.), bottle-o(h n., rabbit-o n. and int. (early 20th cent.), daddy-o n. (mid 20th cent.), while others (see below) are treated in spelling as having the suffix -o; their classification as cases or non-cases of the present suffix is rather arbitrary. The earliest example which shows a clear transition from one use to the other is milko int. and n.; other examples of the use of the suffix in similar formations are kiddo n. and Relievo n.2 (a game named after the cry used in it) (late 19th cent.), whizzo int. and adj., and socko int., adj., and n. (first half of 20th cent.).
From the early 20th cent. the addition of the suffix to complete words to form nouns, adjectives, and interjections of all kinds becomes very common, e.g. wino n.1, whacko int., and cheapo adj. The earliest example of the addition of the suffix to a truncated word is probably beano n. (second half of 19th cent.), followed by Salvo n.3, an Australian formation; another (uncertain) example from the same time and place is robbo n. Since the beginning of the 20th cent. formations of this kind have become numerous, e.g. ammo n., arvo n. (one of several later Australian formations), and wacko adj.
The suffix is not infrequently used to create product names, among the earliest being blanco n. and Oxo n. in the late 19th cent.
Examples
It’s not surprising you heard it in songs; I think it happens there a lot. Consider the words the children’s song The Farmer in the Dell:
The farmer in the dell
The farmer in the dell
Heigh-ho, the derry-o
The farmer in the dell.
This kind of thing is used a great deal by the “nonsense” of Tolkien’s Tom Bombadil character when he sings — which he pretty much does always, even when it’s written as prose.
Here is a bit from when he rescues the Hobbits from Old Man Willow:
He turned round and listened, and soon there could be no doubt: someone was singing a song; a deep glad voice was singing carelessly and happily, but it was singing nonsense:
Hey dol! merry dol! ring a dong dillo!
Ring a dong! hop along! fal lal the willow!
Tom Bom, jolly Tom, Tom Bombadillo!Half hopeful and half afraid of some new danger, Frodo and Sam now both stood still. Suddenly out of a long string of nonsense-words (or so they seemed) the voice rose up loud and clear and burst into this song:
Hey! Come merry dot! derry dol! My darling!
Light goes the weather-wind and the feathered starling.
Down along under Hill, shining in the sunlight,
Waiting on the doorstep for the cold starlight,
There my pretty lady is. River-woman’s daughter,
Slender as the willow-wand, clearer than the water.
Old Tom Bombadil water-lilies bringing
Comes hopping home again. Can you hear him singing?
Hey! Come merry dol! deny dol! and merry-o,
Goldberry, Goldberry, merry yellow berry-o!
Poor old Willow-man, you tuck your roots away!
Tom’s in a hurry now. Evening will follow day.
Tom’s going home again water-lilies bringing.
Hey! Come derry dol! Can you hear me singing?
I imagine uncountably many similar examples could be pulled from literature and nursery rhymes alike.
In and of itself, the "-o" suffix has no real "meaning" or "etymology".
I think one could easily over-analyse this suffix. Even if they've never been taught it in school, all English speakers know that the sounds they use to form words are basically divided into "vowels" and "consonants". In relaxed/familiar contexts, we habitually discard trailing consonants from words, and sometimes add vowels.
The name Steven/Stephen is routinely shortened to "Steve", and may then be extended to "Stevie", for example. As it happens, not many Stevens / Stephens are called "Steve-o", but Danno, Richo, Robbo etc. are standard "diminutive" forms.
Appended to names of individuals, the suffix often conveys familiarity/inclusiveness/approval. The speaker (but not necessarily everyone else) can use this special "pet name". As with "Dad/Daddie", it's only used by certain people, of certain others.
Added to identifiers that aren't actually names, those "positive" associations are usually swamped by the more dismissive/condemnatory connotations of "familiarity" (that which breeds contempt). We know of something, and wish to convey that fact through the way we refer to it. But we want to place some "distance" between ourselves and the referent, by using a variant of the standard identifier. Thus, psycho, wino, fatso, whacko, weirdo, dumbo, etc.
In another common context, it's just an interjection (based on "Oh!") that adds an element of informality (good-o, right[y]-[h]o, run like billy-o, etc.).
Apparently other languages use the suffix with "negative" connotations - examples from Swedish: "fetto" 'fat person' (< "fet" 'fat'), "neggo" 'negative person' (< "negative" 'negative'), "dummo" 'stupid person' (< "dum" 'stupid'), "nordo" 'neird' (< "nord" 'neird')
TL;DR: "-o" is just a generic "diminutive" where the "meaning" (if any) depends on context.