nostr:nprofile1qy2hwumn8ghj7un9d3shjtnyd968gmewwp6kyqpqqcrqlugrsf9tljl4m5pvz6uwr95gjsh04cy9zd8rlkrljkxtdjhs26sve2 nostr:nprofile1qy2hwumn8ghj7un9d3shjtnyd968gmewwp6kyqpqqmshcslmwan3erj3ulczepe8q8a8xdy...
@nprofile1q...
This does not seem right. The first paper you link to states
"We observe that the average unitary energy cost of a pipeline is relatively low, at 10 Wh. However, due to repeated invocations of these pipelines in real settings, the aggregated energy consumption cost per project is high, averaging 22 kWh"
I believe that the aggregated energy consumption cost per project refers to the historical total energy usage, though I have to admit that I do not fully understand what they mean by it.
I cannot find your 37kW number in the paper (nor is it the right unit), and I think that it is off by a few of orders of magnitude. Using 10 Wh for a pipeline means that it would be equivalent to about 100-200 LLM tokens. Additionally you refer that CIs do not improve in efficiency, but the paper notes that their results likely overestimate Wh spent:
"Data centers often use hardware optimized for running such intensive pipelines and are known to be more energy-efficient, at scale, than regular machines [29]. Indeed, it is common to execute pipelines using preemptible instances on underutilized cloud servers, such as AWS Spot Instances. Power and energy management within data centers cannot be directly compared to an end-user machine, so we do not claim that our results are representative of the energy that these pipelines would consume in real-world settings."
Finally, the 22 kWh figure is heavily influenced by high outliers, the 75th percentile is 7,5kWh. That's not to say that CI doesn't matter, but one CI run does not come close to ~500.000 LLM tokens.
This does not seem right. The first paper you link to states
"We observe that the average unitary energy cost of a pipeline is relatively low, at 10 Wh. However, due to repeated invocations of these pipelines in real settings, the aggregated energy consumption cost per project is high, averaging 22 kWh"
I believe that the aggregated energy consumption cost per project refers to the historical total energy usage, though I have to admit that I do not fully understand what they mean by it.
I cannot find your 37kW number in the paper (nor is it the right unit), and I think that it is off by a few of orders of magnitude. Using 10 Wh for a pipeline means that it would be equivalent to about 100-200 LLM tokens. Additionally you refer that CIs do not improve in efficiency, but the paper notes that their results likely overestimate Wh spent:
"Data centers often use hardware optimized for running such intensive pipelines and are known to be more energy-efficient, at scale, than regular machines [29]. Indeed, it is common to execute pipelines using preemptible instances on underutilized cloud servers, such as AWS Spot Instances. Power and energy management within data centers cannot be directly compared to an end-user machine, so we do not claim that our results are representative of the energy that these pipelines would consume in real-world settings."
Finally, the 22 kWh figure is heavily influenced by high outliers, the 75th percentile is 7,5kWh. That's not to say that CI doesn't matter, but one CI run does not come close to ~500.000 LLM tokens.
1