How to go to the n'th character, not byte, of a file?

You can start from the beginning of the buffer, and use search() to match N characters. The only pitfall is considering the newline characters, too. Here's a custom gco mapping that does this:

function! s:GoToCharacter( count )
    let l:save_view = winsaveview()
    " We need to include the newline position in the searches, too. The
    " newline is a character, too, and should be counted.
    let l:save_virtualedit = &virtualedit
    try
        let [l:fixPointMotion, l:searchExpr, l:searchFlags] = ['gg0', '\%#\_.\{' . (a:count + 1) . '}', 'ceW']
        silent! execute 'normal!' l:fixPointMotion

        if search(l:searchExpr, l:searchFlags) == 0
            " We couldn't reach the final destination.
            execute "normal! \<C-\>\<C-n>\<Esc>" | " Beep.
            call winrestview(l:save_view)
            return 0
        else
            return 1
        endif
    finally
        let &virtualedit = l:save_virtualedit
    endtry
endfunction
" We start at the beginning, on character number 1.
nnoremap <silent> gco :<C-u>if ! <SID>GoToCharacter(v:count1 - 1)<Bar>echoerr 'No such position'<Bar>endif<Bar><CR>

Note that this counts just one character for the CR-LF combination in buffers that have 'fileformat' set to dos.

What about just gg0 to go to the first character in the file, then 4l to go work your way 5 characters in? I suppose this may fail if you have empty lines or if you want to count linebreaks.