My score is starting to drop because I stopped using credit

The big one on my list is “Age of credit history” – F
The only reason I even have THAT card is I needed one in April to rent a vehicle in Reno. And I rented a vehicle in October as well. But it had never even been used for a purchase until last week, when I bought something from a website I was unsure of. I figured it was better to risk a credit card than my debit card. 🙂 That card will NEVER carry a balance, so the age of it probably won’t matter in the long run.
I’ll have a mortgage for a few years, so I can’t get to zero FICO for a long time. I’d consider THAT a measure of success… 🙂