Entropy of matrix
Solution 1:
First of all, you shall keep in mind that there are actually many entropy definitions. The most common one is so called Shannon information entropy
$$H(X) = -\sum_{i=1}^{n}p_i\log p_i$$
where $p_i$ is the probability of seeing the $i$th possible outcome of $X$.
Therefore
- Entropy is defined close-related to the probability distribution of random variable $X$
- Entropy does not care about correlation or independence, because only the probability distribution matters.
- Yes we do have conditional entropy, see wiki pages for details.
I am not sure in what context you want to find the entropy for a matrix, but in image processing, where images are represented by matrices. The way that we measure the entropy of an image is to
- Find the distribution of pixel intensities (i.e. the distribution of element values)
- Compute entropy using the information entropy formula
Solution 2:
You may be interested in the Von Neumann entropy of a matrix, which is defined as the sum of the entropies of the eigenvalues. Ie, for $$A = P \begin{bmatrix}\lambda_1 \\ & \lambda_2 \\ && \ldots \\ &&& \lambda_n \end{bmatrix} P^{-1}$$ with positive $\lambda_i$, the entropy is, $$H(A):=-\sum_i \lambda_i \log \lambda_i.$$
For more on the definition of the von Neumann entropy you might look here on wikipedia, and for how to maximize it numerically you could look at my answer on this Computer Science stack exchange thread.
For rectangular matrices, you could extend the definition by replacing the eigenvalues with singular values in the SVD, though it's not clear what this would mean.