Batch prediction Input

I have a tensorflow model deployed on Vertex AI of Google Cloud. The model definition is:

item_model = tf.keras.Sequential([
  tf.keras.layers.StringLookup(
      vocabulary=item_vocab, mask_token=None),
  tf.keras.layers.Embedding(len(item_vocab) + 1, embedding_dim)
])

user_model = tf.keras.Sequential([
  tf.keras.layers.StringLookup(
      vocabulary=user_vocab, mask_token=None),
  # We add an additional embedding to account for unknown tokens.
  tf.keras.layers.Embedding(len(user_vocab) + 1, embedding_dim)
])


class NCF_model(tf.keras.Model):
    def __init__(self,user_model, item_model):
        super(NCF_model, self).__init__()
        # define all layers in init
        
        self.user_model = user_model
        self.item_model  = item_model
        self.concat_layer   = tf.keras.layers.Concatenate()
        self.feed_forward_1 = tf.keras.layers.Dense(32,activation= 'relu')
        self.feed_forward_2 = tf.keras.layers.Dense(64,activation= 'relu')
        self.final = tf.keras.layers.Dense(1,activation= 'sigmoid')


    def call(self, inputs ,training=False):
        user_id , item_id = inputs[:,0], inputs[:,1]
        x = self.user_model(user_id)
        y = self.item_model(item_id)

        x = self.concat_layer([x,y])
        x = self.feed_forward_1(x)
        x = self.feed_forward_2(x)
        x = self.final(x)


        return x

The model has two string inputs and it outputs a probability value. When I use the following input in the batch prediction file, I get an empty prediction file. Sample of csv input file:

userid,itemid
yuu,190767
yuu,364
yuu,154828
yuu,72998
yuu,130618
yuu,183979
yuu,588

When I use a jsonl file with the following input.

{"input":["yuu", "190767"]}

I get the following error.

('Post request fails. Cannot get predictions. Error: Exceeded retries: Non-OK result 400 ({\n    "error": "Failed to process element: 0 key: input of \'instances\' list. Error: INVALID_ARGUMENT: JSON object: does not have named input: input"\n}) from server, retry=3.', 1)

What seems to be going wrong with these inputs?

Solution 1:

After a bit of experimenting, I found out what was wrong with the batch prediction input. In the csv file, the item column was being interpreted as an integer whereas the model has a string as an input. I'm not sure why there was no output at all in that case and I couldn't find the logs for the batch prediction.

The correct format for jsonlines was:

["user1", "item1"]
["user2", "item2"]
["user3", "item3"]

The one I used assumed the input was a named layer, 'input'. In all of this, I found the documentation of google cloud to be lacking.