This has arguably happened / is happening with llama 3 405B right?

Is this the company's self-reported cost (i.e. cost of electricity)? Or is it the public cost for the corresponding G/TPUs days at market rate on e.g. AWS?

Cost of compute

Does this refer only to spending on compute for the model's final training run?

@StevenK Ooo, good point. I did assume that that's what it meant when I bought in. I thing it would already be true if they included all costs (institutional overhead, researcher labor, development pipeline costs, etc.)