Oozie: Launch Map-Reduce from Oozie <java> action?

Solution 1:

> how do I get the output of the MapReduce...

Back to the basics.

Since you don't care to mention which version of Hadoop and which version of Oozie you are using, I will assume a "recent" setup (e.g. Hadoop 2.7 w/ TimelineServer and Oozie 4.2). And since you don't mention which kind of interface you use (command-line? native Oozie/Yarn UI? Hue?) I will give a few examples using good'old'CLI.

> oozie jobs -localtime -len 10 -filter name=CrazyExperiment

Shows the last 10 executions of "CrazyExperiment" workflow, so that you can inject the appropriate "Job ID" in next commands.

> oozie job -info 0000005-151217173344062-oozie-oozi-W

Shows the status of that execution, from Oozie point of view. If your Java action is stuck in PREP mode, then Oozie failed to submit it to YARN; otherwise you will find something like job_1449681681381_5858 under "External ID". But beware! The job prefix is a legacy thing; the actual YARN ID is application_1449681681381_5858.

> oozie job -log 0000005-151217173344062-oozie-oozi-W

Shows the Oozie log, as could be expected.

> yarn logs -applicationId application_1449681681381_5858

Shows the consolidated logs for AppMaster (container #1) and Java action Launcher (container #2) -- after execution is over. The stdout log for Launcher contains a whole shitload of Oozie debug stuff, the real stdout is at the very bottom.

In case your Java action successfully spawned another YARN job, and you were careful to display the child "application ID", you should be able to retrieve it there and run another yarn logs command against it.

Enjoy your next 5 days of debugging ;-)