TensorFlow: questions regarding tf.argmax() and tf.equal()

I am learning the TensorFlow, building a multilayer_perceptron model. I am looking into some examples like the one at: https://github.com/aymericdamien/TensorFlow-Examples/blob/master/notebooks/3_NeuralNetworks/multilayer_perceptron.ipynb

I then have some questions in the code below:

def multilayer_perceptron(x, weights, biases):
    :
    :

pred = multilayer_perceptron(x, weights, biases)
    :
    :

with tf.Session() as sess:
    sess.run(init)
         :
    correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))

    accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
    print ("Accuracy:", accuracy.eval({x: X_test, y: y_test_onehot}))

I am wondering what do tf.argmax(prod,1) and tf.argmax(y,1) mean and return (type and value) exactly? And is correct_prediction a variable instead of real values?

Finally, how do we get the y_test_prediction array (the prediction result when the input data is X_test) from the tf session? Thanks a lot!

Solution 1:

tf.argmax(input, axis=None, name=None, dimension=None)

Returns the index with the largest value across axis of a tensor.

input is a Tensor and axis describes which axis of the input Tensor to reduce across. For vectors, use axis = 0.

For your specific case let's use two arrays and demonstrate this

pred = np.array([[31, 23,  4, 24, 27, 34],
                [18,  3, 25,  0,  6, 35],
                [28, 14, 33, 22, 20,  8],
                [13, 30, 21, 19,  7,  9],
                [16,  1, 26, 32,  2, 29],
                [17, 12,  5, 11, 10, 15]])

y = np.array([[31, 23,  4, 24, 27, 34],
                [18,  3, 25,  0,  6, 35],
                [28, 14, 33, 22, 20,  8],
                [13, 30, 21, 19,  7,  9],
                [16,  1, 26, 32,  2, 29],
                [17, 12,  5, 11, 10, 15]])

Evaluating tf.argmax(pred, 1) gives a tensor whose evaluation will give array([5, 5, 2, 1, 3, 0])

Evaluating tf.argmax(y, 1) gives a tensor whose evaluation will give array([5, 5, 2, 1, 3, 0])

tf.equal(x, y, name=None) takes two tensors(x and y) as inputs and returns the truth value of (x == y) element-wise. 

Following our example, tf.equal(tf.argmax(pred, 1),tf.argmax(y, 1)) returns a tensor whose evaluation will givearray(1,1,1,1,1,1).

correct_prediction is a tensor whose evaluation will give a 1-D array of 0's and 1's

y_test_prediction can be obtained by executing pred = tf.argmax(logits, 1)

The documentation for tf.argmax and tf.equal can be accessed by following the links below.

tf.argmax() https://www.tensorflow.org/api_docs/python/math_ops/sequence_comparison_and_indexing#argmax

tf.equal() https://www.tensorflow.org/versions/master/api_docs/python/control_flow_ops/comparison_operators#equal

Solution 2:

Reading the documentation:

tf.argmax

Returns the index with the largest value across axes of a tensor.

tf.equal

Returns the truth value of (x == y) element-wise.

tf.cast

Casts a tensor to a new type.

tf.reduce_mean

Computes the mean of elements across dimensions of a tensor.

Now you can easily explain what it does. Your y is one-hot encoded, so it has one 1 and all other are zero. Your pred represents probabilities of classes. So argmax finds the positions of best prediction and correct value. After that you check whether they are the same.

So now your correct_prediction is a vector of True/False values with the size equal to the number of instances you want to predict. You convert it to floats and take the average.

Actually this part is nicely explained in TF tutorial in the Evaluate the Model part

Solution 3:

tf.argmax(input, axis=None, name=None, dimension=None)

Returns the index with the largest value across axis of a tensor.

For the case in specific, it receives pred as argument for it's input and 1 as axis. The axis describes which axis of the input Tensor to reduce across. For vectors, use axis = 0.

Example: Given the list [2.11,1.0021,3.99,4.32] argmax will return 3 which is the index of the highest value.

correct_prediction is a tensor that will be evaluated later. It is not a regular python variable. It contains the necessary information to compute the value later. For this specific case, it will be part of another tensor accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float")) and will be evaluated by eval on accuracy.eval({x: X_test, y: y_test_onehot}).

y_test_prediction should be your correct_prediction tensor.