Why is sed -E find/replace showing unexpected output

I'm trying to extract sha256:13b918c5a5eadfed53597146332889dc5e10d1a8edbcdc42f7a872531766aab8 from the following output. The output is in a file called d2.txt.

d2.txt:

The push refers to repository [...]
331ebf1e6bb7: Layer already exists 
9bb0b3c0e55b: Layer already exists 
9f59b9615f5e: Layer already exists 
82621df65774: Layer already exists 
3e123f0af898: Layer already exists 
93defbb4091e: Layer already exists 
bc21254008da: Layer already exists 
53619ba80b4a: Layer already exists 
18eb03bf3058: Layer already exists 
daf4ddfb16e5: Layer already exists 
b5639327d5be: Layer already exists 
30ccd09e6f92: Layer already exists 
167efff21776: Layer already exists 
fee20f1b745d: Layer already exists 
d0fe97fa8b8c: Layer already exists 
v1.0: digest: sha256:13b918c5a5eadfed53597146332889dc5e10d1a8edbcdc42f7a872531766aab8 size: 3470

Using grep I can use the following to identify the line with the digest:

grep -E '^.*(sha256:[a-z0-9]{64}).*' d2.txt

which returns:

v1.0: digest: sha256:13b918c5a5eadfed53597146332889dc5e10d1a8edbcdc42f7a872531766aab8 size: 3470

Using the parenthesis from this regex to define capture group 1 (the sha256:hash), I attempt to run this in sed. But Instead getting just the line with the digest, I get all the lines in d2.txt. (With the correct capture group on the sha256 line!).

sed -E  s/'^.*(sha256:[a-z0-9]{64}).*'/'\1'/g d2.txt

returns:

The push refers to repository [...]
331ebf1e6bb7: Layer already exists 
9bb0b3c0e55b: Layer already exists 
9f59b9615f5e: Layer already exists 
82621df65774: Layer already exists 
3e123f0af898: Layer already exists 
93defbb4091e: Layer already exists 
bc21254008da: Layer already exists 
53619ba80b4a: Layer already exists 
18eb03bf3058: Layer already exists 
daf4ddfb16e5: Layer already exists 
b5639327d5be: Layer already exists 
30ccd09e6f92: Layer already exists 
167efff21776: Layer already exists 
fee20f1b745d: Layer already exists 
d0fe97fa8b8c: Layer already exists 
sha256:13b918c5a5eadfed53597146332889dc5e10d1a8edbcdc42f7a872531766aab8

So why does sed return the full text all lines where there is no regex match?


You can use grep with -o to print only the matched part.

grep -Eo 'sha256:[[:alnum:]]{64}' d2.txt

Or with sed you can prevent the default printing with -n and use p to only print the line with the substitution.

sed -En  's/^.*(sha256:[a-z0-9]{64}).*/\1/p' d2.txt

Both will output

sha256:13b918c5a5eadfed53597146332889dc5e10d1a8edbcdc42f7a872531766aab8

With your shown samples, in awk with awk's match function you can try following code:

awk 'match($0,/sha256:[a-z0-9]{64}/){print substr($0,RSTART,RLENGTH)}' d2.txt

In case you have ONLY ONE MATCH in whole file and you want to print it then use exit also with above code to make this faster and we need not to read whole file then,

awk 'match($0,/sha256:[a-z0-9]{64}/){print substr($0,RSTART,RLENGTH);exit}' d2.txt