There is no doubt that it's hard to remember all of the words heared!!l
In my opinion you should merely try to memorize or write some key words and main phrases not all of them!
It's better to believe in your mind and be more self confident.
By the way ! you should say "I don't have" instead of "I have not" .