How to automatically add / remove scroll-bars as needed by text height

If you just want to toggle scroll bars on/off interactively, or on a hook, or from your code, then scroll-bar-mode should be all you need.

You can also use menu-bar-no-scroll-bar, menu-bar-left-scroll-bar, and menu-bar-right-scroll-bar. Or just do what each of those commands does: (customize-set-variable 'scroll-bar-mode WHATEVER). Or use set-scroll-bar-mode or set-window-scroll-bars, similarly. It depends on what behavior you are looking for.

I recommend M-x apropos scroll-bar. (Or if you use Icicles, just C-h f scroll-bar S-TAB, then repeat C-M-down...)

You can add it to mode-line-position, so that update of the mode line automatically triggers turning scroll bars on/off. This pretty much works, for instance:

(setq-default
 mode-line-position
 '(:eval
   (progn
     (if (> (count-lines (point-min) (point-max)) (window-height))
         (set-window-scroll-bars nil 20 t)
       (set-window-scroll-bars nil 0 t))
     `((-3 ,(propertize
             "%p"
             'local-map mode-line-column-line-number-mode-map
             'mouse-face 'mode-line-highlight
             'help-echo "Buffer position, mouse-1: Line/col menu"))
       (line-number-mode
        ((column-number-mode
          (10 ,(propertize
                " (%l,%c)"
                'face (and (> (current-column)
                              modelinepos-column-limit)
                           'modelinepos-column-warning)
                'local-map mode-line-column-line-number-mode-map
                'mouse-face 'mode-line-highlight
                'help-echo "Line and column, mouse-1: Line/col menu"))
          (6 ,(propertize
               " L%l"
               'local-map mode-line-column-line-number-mode-map
               'mouse-face 'mode-line-highlight
               'help-echo "Line number, mouse-1: Line/col menu"))))
        ((column-number-mode
          (5 ,(propertize
               " C%c"
               'face (and (> (current-column)
                             modelinepos-column-limit)
                          'modelinepos-column-warning)
               'local-map mode-line-column-line-number-mode-map
               'mouse-face 'mode-line-highlight
               'help-echo "Column number, mouse-1: Line/col menu")))))))))

You can alternatively use the following, which employs Stefan's suggestion to make it work better with scaled text, visual-line-mode, images, etc. However, in that case, scroll bars kick in whenever some text is outside the window because of scrolling, regardless of whether that text would fit in the window. Whether that is a feature or not is for you to decide. ;-)

(setq-default
 mode-line-position
 '(:eval
   (let ((scroll-bars  (nth 2 (window-scroll-bars))))
     (if (or (> (point-max) (window-end))  (< (point-min) (window-start)))
         (unless scroll-bars (set-window-scroll-bars nil 20 t))
       (when scroll-bars (set-window-scroll-bars nil 0 t)))
     (unless (equal scroll-bars (nth 2 (window-scroll-bars))) (redraw-frame))
     `((-3 ,(propertize
             "%p"
             'local-map mode-line-column-line-number-mode-map
             'mouse-face 'mode-line-highlight
             'help-echo "Buffer position, mouse-1: Line/col menu"))
       (line-number-mode
        ((column-number-mode
          (10 ,(propertize
                " (%l,%c)"
                'face (and (> (current-column)
                              modelinepos-column-limit)
                           'modelinepos-column-warning)
                'local-map mode-line-column-line-number-mode-map
                'mouse-face 'mode-line-highlight
                'help-echo "Line and column, mouse-1: Line/col menu"))
          (6 ,(propertize
               " L%l"
               'local-map mode-line-column-line-number-mode-map
               'mouse-face 'mode-line-highlight
               'help-echo "Line number, mouse-1: Line/col menu"))))
        ((column-number-mode
          (5 ,(propertize
               " C%c"
               'face (and (> (current-column)
                             modelinepos-column-limit)
                          'modelinepos-column-warning)
               'local-map mode-line-column-line-number-mode-map
               'mouse-face 'mode-line-highlight
               'help-echo "Column number, mouse-1: Line/col menu")))))))))

If you are used to using scroll bars only as visual indicator of where you are in the buffer (rather than looking at the modeline for that), you might like a package like yascroll. It uses the fringe for displaying the scroll bar, and it does not show it if lines of text in the document are less than the visible window, as you want.

Advantage is that there is no widgets taking up real state from your screen.

Disadvantage is that you cannot use the scroll bars with the mouse (some users never do that, anyways)