One thing I love about learning math is that, contrary to popular belief, you can generalize it to apply to many aspects of life. Yes, the real life you live daily.

Recently, I was learning the problem of overfitting in machine learning and how regularization can alleviate it.

Let’s generalize this idea into real life problem using one half-true meme as an example: the A students work for the B students. The C students run the businesses.

The problem with many diligent students is that they focus too much on test marks. Now, this won’t be a problem if getting many A will translate into real world success.

The overfitting problem here is then: trying too hard on training for simulated tests that you forget that real world problems is unlike test problems.

The solution for this is simple. You regularize by taking into account about life outside education institutions. Ask what matters in real life, and then shift some of your focus there instead of focusing too much on test marks.

This model of overfitting regularization declares formally why the meme above is only half true. Not all C students run businesses. A lot of C students get C because they are lazy (and not because they’re focusing on real life matters). This can then be said as a case of underfitting, where the function neither fits the training set, nor does it fit real life data. The lazy C students neither get good test marks, nor run businesses.

I never think that this regularization idea could be generalized before I read a particularly poignant comment (opinion) in RibbonFarm.com’s post titled The Milo Criterion.

What the comment said, in simple terms, is that the problem with LEAN production, in the context of startup, is that entrepreneur can took into account the present feedback of his current customer base too much, that he failed to take into account the potential changes of preferences of his current customer base in the future.

If the entrepreneur implement the LEAN production too efficiently against present preferences (complete with optimizations), he’ll have a hard time pivoting to the new preferences of his customer.

Or succinctly: the danger of LEAN production is overfitting to the current customer dataset.

The commenter proposed alternative is the Apple Inc. approach where they give what the customers need even though the customers don’t know that they need it yet.

Or in math-y terms: we regularize this LEAN overfitting problem by minimizing the weight of current customer preference (and by envisioning customers’ preferences in the future).

There are many examples that you can find of overfitting problems, and one potential solution is simply to regularize (there are other solutions too).

Now, as a final thought/example: what should you do if you overfit to the customs of a decaying institution?