what is the idiom for doing something covertly

Solution 1:

You named it, do something on the sly.

on the sly: (Fig) secretly and deceptively: Martin was having an affair with the maid on the sly. (McGraw-Hill Dictionary of American Idioms and Phrasal Verbs)

Alternately, consider sneak around (with someone) and [do sonething] on the down-low (or DL).

sneak around: do something without someone's knowledge, especially to engage in romantic relationships: I suspect her husband has been sneaking around. I think his wife was sneaking around on him. (The American Heritage Dictionary of Phrasal Verbs)

You snuck around with him all summer and never told me. Not once. Out with some stupid, womanizing football player who would've broken your heart in two if given the chance. (Game for Trouble)

on the down-low: (AmEng, idiomatic, slang, euphemistic) Secretly sleeping with someone other than one's partner.Your Dictionary)

Reportedly, Eddie agreed to end the double dates with LeAnn, but that didn’t stop him from seeing her on the down-low. (Your Tango)

Solution 2:

A bit on the side

a ​sexual ​relationship with someone who is not ​married to you, or the ​person you have the ​relationship with

http://dictionary.cambridge.org/dictionary/english/a-bit-on-the-side

Solution 3:

To cheat on someone is perhaps the commonest expression. As in: The guys is cheating on his wife.

You can also say "messing around" or "fooling around". The above are phrasal verbs; a more idiomatic expression would be "working late at office".

Solution 4:

Perhaps behind closed doors: in secret, privately; away from observers, reporters, or intruders, usually in a closed room.