Copy every 512 bytes and skip next 8 bytes from Input file to a Output file

I have an input file which is always a multiple of 520 bytes . i,e

Input file 1- 1040 bytes
Input file 2- 5200 bytes 

I need to copy every 512 bytes from input file and skip the next 8 bytes from input files to output file i.e

Output file data = Copy first 512 bytes of input file - Skip 8 bytes - Copy second 512 bytes of input file- Skip 8 bytes --..

Therefore, my output file should always be multiple of 512 bytes

Input file 1- 1040 bytes        Output file 1 - 1024 bytes
Input file 2- 5200 bytes        Output file 2 - 5120 bytes

I need to do this in shell script, any help here please.


The right tool

bbe is "a sed-like editor for binary files". In Debian it's in a package named bbe. What you want can be done like this:

<"Input file 1" bbe -b :520 -e 'd 512 *' >"Output file 1"

bbe is the right tool. If you can use it then use it. The below script is in case you cannot use bbe.


Portable script

Not the fastest code but here it is:

#!/bin/sh

useful=512
useless=8

while :; do
   dd bs=1 count="$useful" 2>/dev/null
   [ "$(dd bs=1 count="$useless" 2>/dev/null | wc -c)" -lt "$useless" ] && exit
done

Save the code as my_filter, then invoke:

<"Input file 1" ./my_filter >"Output file 1"

Notes:

  • I think the code is portable.
  • bbe is way faster (although with a file of 5200 bytes or less you probably won't notice the difference).
  • It may seem bs="$useful" count=1 should be better, but only if your dd supports iflag=fullblock (it may not) and you use it. Read What's the POSIX way to read an exact number of bytes from a file?
  • The code exits when the second dd reads less than it should. This way EOF is detected. For your files ("always a multiple of 520 bytes") the first dd will read less than it should before the second one does, but detecting this situation with the first dd in a portable way while still saving the output is not as straightforward as using wc -c with the second dd whose output should be discarded anyway.
  • Because of how the script detects EOF, it won't discard a partial useful block if it ever happens (i.e. if EOF happens in the middle of it). For your files ("always a multiple of 520 bytes") this should never happen though. With arbitrary sizes it can happen. Note the provided bbe command generates the same output as the script even in these circumstances.
  • To customize the script and use values other than 512 and 8, it's enough to modify the useful=512 and useless=8 lines.
  • Because of how the script detects EOF, it will never exit if you set useless=0 (this edge case is trivially solved by cp "Input file 1" "Output file 1").
  • dd supports skip=. Using it to skip $useless bytes seems quite elegant, but then detecting EOF gets complicated.
  • You can easily modify the code to take useful and useless from positional parameters. $1 expands to the value of the first positional parameter, $2 expands to the value of the second. So instead of useful=512 and useless=8 you can use useful="$1" and useless="$2" respectively. Then instead of ./my_filter you need to run ./my_filter 512 8. This will allow you to pass arbitrary values at runtime.