How to maintain only a small set of non-default kernel configs?

You can add all the config options you want to override in a simple Bash script that exports them as environment variables before running make. For options that you want to disable (i.e. set from y to n) you will need to directly modify the .config file as simply exporting them as =n will not work.

#!/bin/bash

make defconfig

# Options you want to modify
export CONFIG_FOO=123
export CONFIG_BAR=456

# Options which you want to disable
sed -ie '/CONFIG_BAZ=y/ s/=y/=n/' .config

make all -j

Beware though that you should check for config dependencies (with make menuconfig or make gconfig) to see whether the config options you are dealing with also depend on others or have others as dependencies. In such case you also want to include those in your Bash script as needed.