The problem is that you are printing tabulators and new lines at the wrong places within the loops but the major problem with your code are the array indices and matrix size. You can easily solve that by using variables for number of rows and columns.

public class BusSeatReservation {
    public static void main(String[] args) {
        System.out.println("\t\t\tBUS SEAT RESERVATION");
        int rows = 10;
        int cols = 4;

        char[][] matrix = new char[rows][cols];

        System.out.print("\t\t");
        for (int col = 0; col < cols; col++) {
            System.out.print("\tCol " + (col+1));

        }
        System.out.println();
        for (int row = 0; row < rows; row++) {
            System.out.print("Row " + (row+1) + "\t|\t");

            for (int seat = 0; seat < cols; seat++) {
                matrix[row][seat] = '*';
                System.out.print(matrix[row][seat] + "\t\t");

            }
            System.out.println();
        }

    }
}