Thrift stores
Goodwill and Salvation Army (near me) tend to lean towards the 'miss' side of hit or miss: somewhat minimal stock that changes slowly, and it seems like they get picked over instantly, or before the CDs even go out. Missing discs, wrong discs and scratched discs are common, so check for all of that. Still fast to check with the LLM project photo method, and cheap.
The next step up are thrift stores with a narrower purpose: hospice shops, humane societies, and there's also a local Friends of the Library that has just books and some CDs. Same issues as Goodwill, but fewer missing or mixed-up discs, and fewer totally scratched-up ones. Likely based on who is donating to those places where I live (YMMV).
Big-box thrift
One step up from those are the 'big box' thrift places, like Savers. More selection, though not great, and prices are more like 4 bucks. I don't live near one so I don't go much, but I usually find something interesting.
Half Price Books
Past that you get into places like Half Price Books, which has a pretty good selection, but prices start being more in the $6-15 range depending on what it is. They do have sales, though, and the condition has always been great: no missing discs, no wrong disc in the case or anything. Actually organized by artist.
Record stores
Then you have actual music media stores; Rasputin and Amoeba are the local-ish examples. They have seemingly everything. Prices are generally higher, though places like Rasputin have the 50 cent bin, which is massive. It just isn't oriented to using LLMs super well. Still super cheap, and you could probably spend 45 minutes finding some good cheap stuff.
Shopping online
But this is all digging around and physically going to stores, which takes time and energy, and while I like to do that, sometimes I'm just after a particular album. For that, searching online is where it's at: eBay, Alibris, Half Price Books, Amazon and Discogs are the way to go.
eBay often has the best prices. Discogs has the most exacting selection and standards: you know exactly what you're getting, down to the pressing, and the condition is typically very accurate, but prices are higher. Alibris is pretty good, though shipping has some challenges, as they don't combine shipping unless you're buying from the same seller. HPB seems to have multiple sellers on there, presumably different stores or regions selling their extra stock.
I've only had one CD be 'bad' so far, one that wouldn't play or accurately rip to a lossless file.
Why this site
I got tired of doing all my own comparison shopping, so I wrote a little tooling for it. Thought the results might be fun to share, so I wired up some pages with some 'essential' albums for the genres I've been going through. It will likely evolve as I play around with this more over the months and years.