Scaling laws
2020
Loss falls predictably as a power law in compute, data and parameters. Kaplan 2020, Chinchilla 2022. The reason labs spend billions on training runs.
Sources: AI4U editorial.
2020
Loss falls predictably as a power law in compute, data and parameters. Kaplan 2020, Chinchilla 2022. The reason labs spend billions on training runs.
Sources: AI4U editorial.