Loading and displaying large text files

In a Swing application, I sometimes need to support read-only access to large, line-oriented text files that are slow to load: logs, dumps, traces, etc. For small amounts of data, a suitable Document and JTextComponent are fine, as shown here. I understand the human limitations of browsing large amounts of data, but the problematic stuff seems like it's always in the biggest file. Is there any practical alternative for larger amounts of text in the 10-100 megabyte, million-line range?

Because of the size, you'll surely want to load the file in the background to avoid blocking the event dispatch thread; SwingWorker is a common choice. Instead of using a Document, consider updating a TableModel and displaying the lines of text in the rows of a JTable. This offers several advantages:

• Results will begin appearing immediately, and there will be reduced perceived latency.

• JTable uses the flyweight pattern for rendering, which scales well into the multi-megabyte, million-line range.

• You can parse the input as it is being read to create an arbitrary column structure.

• You can leverage the sorting and filtering features of JTable, for example.

• You can use TablePopupEditor to focus on a single line.

Addendum: The example below uses DefaultTableModel for convenience. To reduce overhead, extend AbstractTableModel and manage a List<String> or List<RowData>, as shown here. The example displays indeterminate progress; changes to display intermediate progress are shown here.

Code:

import java.awt.BorderLayout;
import java.awt.EventQueue;
import java.beans.PropertyChangeEvent;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.util.List;
import javax.swing.JFrame;
import javax.swing.JProgressBar;
import javax.swing.JScrollPane;
import javax.swing.JTable;
import javax.swing.SwingWorker;
import javax.swing.table.DefaultTableModel;
import javax.swing.table.TableModel;

/**
 * @see https://stackoverflow.com/a/25526869/230513
 */
public class DisplayLog {

    private static final String NAME = "/var/log/install.log";

    private static class LogWorker extends SwingWorker<TableModel, String> {

        private final File file;
        private final DefaultTableModel model;

        private LogWorker(File file, DefaultTableModel model) {
            this.file = file;
            this.model = model;
            model.setColumnIdentifiers(new Object[]{file.getAbsolutePath()});
        }

        @Override
        protected TableModel doInBackground() throws Exception {
            BufferedReader br = new BufferedReader(new FileReader(file));
            String s;
            while ((s = br.readLine()) != null) {
                publish(s);
            }
            return model;
        }

        @Override
        protected void process(List<String> chunks) {
            for (String s : chunks) {
                model.addRow(new Object[]{s});
            }
        }
    }

    private void display() {
        JFrame f = new JFrame("DisplayLog");
        f.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        DefaultTableModel model = new DefaultTableModel();
        JTable table = new JTable(model);
        JProgressBar jpb = new JProgressBar();
        f.add(jpb, BorderLayout.NORTH);
        f.add(new JScrollPane(table));
        f.pack();
        f.setLocationRelativeTo(null);
        f.setVisible(true);
        LogWorker lw = new LogWorker(new File(NAME), model);
        lw.addPropertyChangeListener((PropertyChangeEvent e) -> {
            SwingWorker.StateValue s = (SwingWorker.StateValue) e.getNewValue();
            jpb.setIndeterminate(s.equals(SwingWorker.StateValue.STARTED));
        });
        lw.execute();
    }

    public static void main(String[] args) {
        EventQueue.invokeLater(() -> {
            new DisplayLog().display();
        });
    }
}

I would separate the problem.

The first one is model - Document building speed

The second is the Document rendering - building tree of views to represent the Document.

A question is whether you need font effects like keywords colorizing?

I would start from Document building part. IMHO reading the file via EditorKit.read() should be fast even for big files. I would use the PainDocument for the purpose and check whether the pure model is built fast enough for your application. If yes it's fine just use the Document as model. If not implement your own Document interface because AbstractDocument has plenty of methods for update processing (e.g. writeLock).

When we have the Document loading fast enough we have to solve the Document rendering. BY default the views used in javax.swing.text are really flexible. They are designed as base classes to be extended - thus has a lot of code we don't need. E.g. measuring.

For the feature I would use Monospaced font, we don't need wrap so measurements of the view widht is fast = longest row char count * char widht.

The height is also char height * amount of lines.

So our PLainTextViewReplacement is really fast. Also we don't have to render the whole view but just a fragment visible in our scroll pane. Thus rendering could be mmuch much faster.

Of course there should be a lot of work to provide correct caret navigation, selection etc.