question regarding the prediction stage

[fbiying87]

Hi there,

first of all, very nice tutorials you have written. But I have a short question regarding the prediction stage at the end.
lstm_out, indices = F.max_pool1d(lstm_out, lstm_out.size(2), return_indices=True) # Figure 9
Why do you need max_pooling over the entire sequence again and not just take the last output stage of the lstm model? I have also seen some model, where people just use the last output stage to do further predictions.

Thanks! :-)

[ngarneau]

Hey there,
You are totally right, you can take the last hidden, and I think it is the thing you should do actually.
I wrote this tutorial beginning using Pytorch and at that time it made sense to me to max pool over the set of hidden states.
Thanks for pointing out!