How to use a dataset to fill in a missing column in another set

If you have all possible pairs of "categ_var_1" and "categ_var_2", you can map category values to target values and use the mapper on df2:

Something like:

mapper = df1.set_index(['categ_var_1','categ_var_2'])['binary_target'].to_dict()
out = df2.apply(tuple, axis=1).map(mapper)

Then it gives:

0    NaN
1    NaN
2    NaN
3    NaN
4    1.0

As you can see there are lots of NaN values that because those combinations are not in mapper.

Then what you can do is to create dummy variables from each category and use classification algorithm, maybe like logistic regression (scikit-learn has a class for it). Use df1 as a training set and df2 as a target set. This will obviously give you a prediction, not an exact value. Also note that depending on the number of options in each category, the size of the independent variables might become enormous, so the size of df1 must be big as well, otherwise the prediction won't make much sense.