Ubuntu 16.04 headphones are not working

After searching for a solution for 2 months, I was able to solve my own problem. The problem also exists in Linux Mint, hence it's likely this is an issue which exists in Debian.

This post has useful information about the issue:

[SOLVED]Front panel headphones not working with onboard Intel HD Audio

At some point it mentions a package hda-jack-retask. I have searched for it a little and found out that it's under alsa-tools-gui. Run it with,

sudo hdajackretask

In this GUI, I have overridden my front jack to headphones and installed boot override.

Now I have crystal clear sounds from my headphones and also it can detect whether they are plugged in or not! It mutes the speakers when they are plugged in.

This may not be a common issue. Other solutions I found on the internet just did not work.

I was not able to get it working with above solution. Below one worked. Open terminal, install and open pavucontrol

$ sudo apt-get install pavucontrol
$ pavucontrol

This will open a 'volume control' window. In 'output device' section scroll down to 'Build-in Audio Analog Sterio'. Here change the port to "Headphones(unplugged)"

My answer was the same as BcK's answer, except I had to override "headphone" to the "line out" port, see the image below. Everything works perfectly now, thanks Bck!