Instead of asking does memory matters, you should ask how to use the memory as effectively as possible. Yesterday I read a book about "memory hacking" and I learned something very valuable. It said memory masters don't have a better memory, but they know how to concentrate better. That was something I have not realized earlier.
I am one of the worst types to be a poker player, very impatient, and can't focus for a long period (that's why I don't play long tournaments that much), that's why I struggle a lot memorizing complex stuff. I saw this as something to overcome, and I have done that partly.
There are many different memory techniques like memory palace, loci method, PEG system etc. just google them. By these techniques, you can memorize for example any possible ranges you need to use, and actually very easily. I have been very bad at math also, then I saw a video about "mathemagician", and found a book that teaches mental math secrets. It gave me a lot of hope when I saw it is very easy to calculate super complex math problems in a few seconds with not a lot of effort.
If you want to learn memory techniques, I recommend you to google "art of memory". In that site, there is many helpful techniques and forum where you can practice improving memory.
And as a bonus tip: Anki (flashcard
App). I add there almost everything I study. It is a super-effective way to learn as fast as possible.
Hope this answer helped.
