Considering incorporate into huggingface
Thank you for your excellent work. Do you consider incorporating the new optimizer into huggingface so that the whole community can train LLM using it? I just checked their implementation and find it very straightforward, it's basically naming a new class and incorporating some names into the init function.
kyegomez / Sophia
Effortless plugin and play Optimizer to cut model training costs by 50%. New optimizer that is 2x faster than Adam on LLMs.
378
Contributors get 40% of received funds after fees
How does funding with Polar work?
1
Pay now to fund the work behind this issue.
2
Get updates on progress being made.
3
Maintainer is rewarded once the issue is completed.
FAQ
Backer
You're funding impactful open source efforts
Contributor
You want to contribute to this effort
Maintainer
You want to get funding like this too