How much you try to remember is a matter of personal choice and willingness. Richard Vaughan, quite a famous teacher of English here in Spain, always stresses that learning common words pays dividends over learning less common ones. The example I've heard him use more than once is between the verbs to sleep and to be. To sleep isn't exactly an obscure verb but in comparison to the verb to be it is. The trouble with that theory is that certain words are common under certain circumstances. You hardly ever know when the circumstances will arise when you will need more words. The verb to fry and the nouns egg and chip aren't particularly common words (In the Richard Vaughan sense) but in a greasy spoon, when you want fried egg and chips, they are.
Use and repetition is important too. Once upon a time I used Excel spreadsheets and Access databases. I was never good with them but I knew the basics. I haven't used them for years now and I wouldn't have the faintest idea where to begin with designing a simple database. You may think that living in Spain I would use the language all the time, and I do, but most of my conversations are very simple transactions. In the supermarket, in the bar, where a couple of stock phrases will suffice. I often greet people in the street and exchange a few words about their family or the weather but it's very seldom that the conversation strays to the movement of refugees or US Foreign Policy or even a bit of gossip about some event in the area. In this sort of case Richard Vaughan's common phrases and words theory works well. It's like the Spanish waiter or waitress on the coast. They speak to their British customers in English but most of those waiters and waitresses don't really speak English, they speak the menu.
All this said my Spanish isn't too bad nowadays. I can nearly always get what I want though there may be a lot of fumbling and stumbling along the way. I can read a newspaper, listen to the radio, watch the TV and even read the car handbook. With the online conversation I can even practise real conversations. But my Spanish is still far from good. If I'm watching a film at the cinema I can lose the thread completely. Understanding the lyrics of songs is usually beyond me unless I see them written down and even in something as commonplace as watching the TV news my understanding lets me down from time to time. While I can overhear, and understand, something said in English through all sorts of extraneous sounds and in all sorts of unfavourable circumstances I need a following wind to not lose the thread in Spanish.
After all this time and all the effort it is frustrating beyond belief.