How to tell emacs to open .h file in C++ mode?

Try this:

(add-to-list 'auto-mode-alist '("\\.h\\'" . c++-mode))

Whenever you open .h files, C++-mode will be used.

Another approach for using both c-mode and c++-mode as appropriate, is to use directory local variables to set the mode.

Directory variables are evaluated after the mode has been set¹, so you can actually write a .dir-locals.el file for your C++ project containing this:

((c-mode . ((mode . c++))))

And Emacs will change the mode to c++-mode whenever it had initially set it to c-mode.

If you work with a mix of C and C++ projects, this makes for a pretty trivial solution on a per-project basis.

Of course, if the majority of your projects are C++, you might set c++-mode as the default², and you could then use this approach in reverse to switch to c-mode where appropriate.

¹normal-mode calls (set-auto-mode) and (hack-local-variables) in that order. See also: How can I access directory-local variables in my major mode hooks?

² To do so, add

(add-to-list 'auto-mode-alist '("\\.h\\'" . c++-mode))

to your .emacs file which open .h files in C++ mode by default.

If you don't want this to apply to every .h file, you can add the following to the bottom of your C++ header files.

// Local Variables:
// mode: c++
// End:

This will work for any Emacs variables that you want to set on a per file basis. Emacs ignores the leading characters, so use whatever comment characters are appropriate for the file type.