Calling the "source" command from subprocess.Popen

source is not an executable command, it's a shell builtin.

The most usual case for using source is to run a shell script that changes the environment and to retain that environment in the current shell. That's exactly how virtualenv works to modify the default python environment.

Creating a sub-process and using source in the subprocess probably won't do anything useful, it won't modify the environment of the parent process, none of the side-effects of using the sourced script will take place.

Python has an analogous command, execfile, which runs the specified file using the current python global namespace (or another, if you supply one), that you could use in a similar way as the bash command source.

You could just run the command in a subshell and use the results to update the current environment.

def shell_source(script):
    """Sometime you want to emulate the action of "source" in bash,
    settings some environment variables. Here is a way to do it."""
    import subprocess, os
    pipe = subprocess.Popen(". %s; env" % script, stdout=subprocess.PIPE, shell=True)
    output = pipe.communicate()[0]
    env = dict((line.split("=", 1) for line in output.splitlines()))
    os.environ.update(env)

Broken Popen("source the_script.sh") is equivalent to Popen(["source the_script.sh"]) that tries unsuccessfully to launch 'source the_script.sh' program. It can't find it, hence "No such file or directory" error.

Broken Popen("source the_script.sh", shell=True) fails because source is a bash builtin command (type help source in bash) but the default shell is /bin/sh that doesn't understand it (/bin/sh uses .). Assuming there could be other bash-ism in the_script.sh, it should be run using bash:

foo = Popen("source the_script.sh", shell=True, executable="/bin/bash")

As @IfLoop said, it is not very useful to execute source in a subprocess because it can't affect parent's environment.

os.environ.update(env) -based methods fail if the_script.sh executes unset for some variables. os.environ.clear() could be called to reset the environment:

#!/usr/bin/env python2
import os
from pprint import pprint
from subprocess import check_output

os.environ['a'] = 'a'*100
# POSIX: name shall not contain '=', value doesn't contain '\0'
output = check_output("source the_script.sh; env -0",   shell=True,
                      executable="/bin/bash")
# replace env
os.environ.clear() 
os.environ.update(line.partition('=')[::2] for line in output.split('\0'))
pprint(dict(os.environ)) #NOTE: only `export`ed envvars here

It uses env -0 and .split('\0') suggested by @unutbu

To support arbitrary bytes in os.environb, json module could be used (assuming we use Python version where "json.dumps not parsable by json.loads" issue is fixed):

To avoid passing the environment via pipes, the Python code could be changed to invoke itself in the subprocess environment e.g.:

#!/usr/bin/env python2
import os
import sys
from pipes import quote
from pprint import pprint

if "--child" in sys.argv: # executed in the child environment
    pprint(dict(os.environ))
else:
    python, script = quote(sys.executable), quote(sys.argv[0])
    os.execl("/bin/bash", "/bin/bash", "-c",
        "source the_script.sh; %s %s --child" % (python, script))