I go to Aldi and BigLots for discount shopping, and Wegmans for premium stuff. I generally alternate weeks. because they're on different sides of town. Aldi is faaaar cheaper than Walmart, and none of the irritation. It might be due to my area, but Walmart isn't even cheap. Their prices are inline with the upper middle end stores. I can get groceries cheaper at other stores near me.
Aldi is definitely worth a stop. If it were closer, I'd go every week. They get limited runs of stuff, so it's fun seeing if there's anything new. You probably won't get a full shopping order there, but it has everything essential. The quality is on par with with the big national brands, and the lack of choice is oddly satisfying. You take what they have, or not. No decision making over brand, or size of container :^D
Definitely check out the cheese section. They frequently get interesting stuff there. My most recent acquisition was a trio of cheddar cheeses from Scotland, Ireland, and Wales. Haven't tried them yet, but I'm sure they're good.