@y0b1byte
In a neighbourhood of 0, the loss is O(eps), while the Lp regularization is O(eps^p)
If p>1, the loss always win (L2 reg)
If p<1, the regul always wins (0 is a local minimum)
If p=1, the two can balance out, creating a local minimum if the loss's gradient is low enough