Should the word after "e.g." be capital

There is no need to capitalise the word after "e.g." because it's an acronym; it's short for exempli gratia, basically Latin for "for example". The full stop doesn't make it the end of a sentence. The confusion might arise from the fact that unlike other acronyms the letters are not usually capitalised. I believe that this is because the acronym has been around long enough that it is treated a bit like it's a word itself.

As far as capitalising at the start of a sentence: I might if I ever were to use it, but I can't think of a case where I ever would. Remember that it stands for "for example". Starting a sentence with "for example" seems fairly nonsensical, as it's a conjunctive adverb.

"Should I always plug in electrical items? e.g. Electric hairdryer, phone charger.."

is much better written:

"Should I always plug in electrical items, e.g. electric hairdryer, phone charger?"