That's another good question.
Here are some variations on the real-word definition of cullet.
Scraps of broken or waste glass gathered for remelting, especially with new material.
Waste glass for melting down to be reused.
Broken or waste glass suitable for remelting.
Perhaps variant of collet (literally: little neck, referring to the glass neck of newly blown bottles, etc).
http://www.thefreedictionary.com/cullet
So when you see glass companies with piles of glass to be recycled into marbles, that's definitely cullet.
But marble collectors sometimes have slightly different definitions, so we call the big chunks of glass found at marble factory sites "cullet" even if the chunks might never have been intended to be reused.
And I have a lot of jars of out-of-round and broken Jabos which I have labelled as cullet. And now that you ask I'm not sure if I was correct to do so, but that's how I thought of them at the time.