Why is paying with CryptoCurrency still not mainstream

Looking at how Cryptocurrency has blown up over the last 5 - 6 years, it makes me wonder why we are still at a point where I can't pay for really anything I use in my day to day life with cryptocurrency without going out of my way to use middlemen to allow me to do that.

I would think that the biggest thing holding crypto CURRENCY back would be fees and transaction times, yet there's already coins like Nano that are.. don't make me say it. "Instant and Feeless TM" and before you assume it's just a nano shill post, realize that every single person on /r/CryptoCurrency has heard of Nano at this point so that would accomplish nothing.

But really what is holding crypto back from being actually used as a currency? Is it just a waiting game at this point? Or are there actually major problems that need addressed?