Best way to read structured binary files with Java

I have to read a binary file in a legacy format with Java.

In a nutshell the file has a header consisting of several integers, bytes and fixed-length char arrays, followed by a list of records which also consist of integers and chars.

In any other language I would create structs (C/C++) or records (Pascal/Delphi) which are byte-by-byte representations of the header and the record. Then I'd read sizeof(header) bytes into a header variable and do the same for the records.

Something like this: (Delphi)

type
  THeader = record
    Version: Integer;
    Type: Byte;
    BeginOfData: Integer;
    ID: array[0..15] of Char;
  end;

...

procedure ReadData(S: TStream);
var
  Header: THeader;
begin
  S.ReadBuffer(Header, SizeOf(THeader));
  ...
end;

What is the best way to do something similar with Java? Do I have to read every single value on its own or is there any other way to do this kind of "block-read"?

Solution 1:

To my knowledge, Java forces you to read a file as bytes rather than being able to block read. If you were serializing Java objects, it'd be a different story.

The other examples shown use the DataInputStream class with a File, but you can also use a shortcut: The RandomAccessFile class:

RandomAccessFile in = new RandomAccessFile("filename", "r");
int version = in.readInt();
byte type = in.readByte();
int beginOfData = in.readInt();
byte[] tempId;
in.read(tempId, 0, 16);
String id = new String(tempId);

Note that you could turn the responce objects into a class, if that would make it easier.

Solution 2:

If you would be using Preon, then all you would have to do is this:

public class Header {
    @BoundNumber int version;
    @BoundNumber byte type;
    @BoundNumber int beginOfData;
    @BoundString(size="15") String id;
}

Once you have this, you create Codec using a single line:

Codec<Header> codec = Codecs.create(Header.class);

And you use the Codec like this:

Header header = Codecs.decode(codec, file);

Solution 3:

You could use the DataInputStream class as follows:

DataInputStream in = new DataInputStream(new BufferedInputStream(
                         new FileInputStream("filename")));
int x = in.readInt();
double y = in.readDouble();

etc.

Once you get these values you can do with them as you please. Look up the java.io.DataInputStream class in the API for more info.

Solution 4:

I may have misunderstood you, but it seems to me you're creating in-memory structures you hope will be a byte-per-byte accurate representation of what you want to read from hard-disk, then copy the whole stuff onto memory and manipulate thence?

If that's indeed the case, you're playing a very dangerous game. At least in C, the standard doesn't enforce things like padding or aligning of members of a struct. Not to mention things like big/small endianness or parity bits... So even if your code happens to run it's very non-portable and risky - you depend on the compiler's creator not changing its mind on future versions.

Better to create an automaton to both validate the structure being read (byte per byte) from HD is valid, and filling an in-memory structure if it's indeed OK. You may loose some milliseconds (not so much as it may seem for modern OSes do a lot of disk read caching) though you gain platform and compiler independence. Plus, your code will be easily ported to another language.

Post Edit: In a way I sympathize with you. In the good-ol' days of DOS/Win3.11, I once created a C program to read BMP files. And used exactly the same technique. Everything was nice until I tried to compile it for Windows - oops!! Int was now 32 bits long, rather than 16! When I tried to compile on Linux, discovered gcc had very different rules for bit fields allocation than Microsoft C (6.0!). I had to resort to macro tricks to make it portable...

Solution 5:

I used Javolution and javastruct, both handles the conversion between bytes and objects.

Javolution provides classes that represent C types. All you need to do is to write a class that describes the C structure. For example, from the C header file,

struct Date {
    unsigned short year;
    unsigned byte month;
    unsigned byte day;
};

should be translated into:

public static class Date extends Struct {
    public final Unsigned16 year = new Unsigned16();
    public final Unsigned8 month = new Unsigned8();
    public final Unsigned8 day   = new Unsigned8();
}

Then call setByteBuffer to initialize the object:

Date date = new Date();
date.setByteBuffer(ByteBuffer.wrap(bytes), 0);

javastruct uses annotation to define fields in a C structure.

@StructClass
public class Foo{

    @StructField(order = 0)
    public byte b;

    @StructField(order = 1)
    public int i;
}

To initialize an object:

Foo f2 = new Foo();
JavaStruct.unpack(f2, b);