some questions about DifferentialTransformer
1
👍
0The variable λ in the forward function of the DifferentialTransformer class is not being used. Is it necessary to pass it? Regarding the initialization of λ, should the following function be used:
def lambda_init_fn(depth):
return 0.8 - 0.6 * math.exp(-0.3 * depth)
kyegomez / DifferentialTransformer
An open source community implementation of the model from "DIFFERENTIAL TRANSFORMER" paper by Microsoft.
15
Contributors get 40% of received funds after fees
How does funding with Polar work?
1
Pay now to fund the work behind this issue.
2
Get updates on progress being made.
3
Maintainer is rewarded once the issue is completed.
FAQ
Backer
You're funding impactful open source efforts
Contributor
You want to contribute to this effort
Maintainer
You want to get funding like this too