Mode Collapse
How Mode Collapse Works
Your generator trained too hard on the reward and found exactly one sample it likes. Left alone, it will emit that one mode forever — perfect loss, zero diversity, pure slop. Your job is to be the only regularizer in the building.
- The generator offers you a handful of candidate samples each round.
- Tap the rarest, most under-represented one (smallest bar on the histogram).
- Picking the rare sample pushes the distribution back toward uniform and raises entropy.
- Picking a common mode — or hesitating — lets the distribution collapse and entropy drain.
- Keep entropy above the dashed threshold line. The longer you survive, the higher your score.
Why Is This Hard?
The generator is biased. Every round it stuffs the candidate tray with copies of its favorite mode, hoping you'll pick one out of habit. As time passes it ramps the speed, deepens the bias, and quietly bleeds entropy on its own. You are fighting both a clock and a distribution that desperately wants to be boring.
Slop Fact: Real GANs collapse like this all the time — the generator discovers one output that reliably fools the discriminator and just spams it. The standard fix is minibatch discrimination, which is a fancy way of saying "please, for the love of god, vary it up." You are the minibatch.