Trying to train any of the Vision models that you have around, but im not being lucky on finding one that is workable both in terms of training and prediction.
File /data/conda/envs/eqbench/lib/python3.10/site-packages/zeta/structs/auto_regressive_wrapper.py:186, in AutoregressiveWrapper.generate(self, start_tokens, seq_len, eos_token, strategy, temperature, filter_logits_fn, filter_thres, min_p_pow, min_p_ratio, gamma, **kwargs)
182 b, t = start_tokens.shape
184 out = start_tokens
--> 186 if self.speculative:
187 for _ in range(seq_len):
188 x = out[:, -self.max_seq_len]
File /data/conda/envs/eqbench/lib/python3.10/site-packages/torch/nn/modules/module.py:1695, in Module.__getattr__(self, name)
1693 if name in modules:
1694 return modules[name]
-> 1695 raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
AttributeError: 'AutoregressiveWrapper' object has no attribute 'speculative'
Im keen on pretraining a model, which one u think will give good performance and that the code is reproducible at your repo? i tried kosmos-x also, having as well some issue to run it.
kyegomez / CM3Leon
An open source implementation of "Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning", an all-new multi modal AI that uses just a decoder to generate both text and images
362
Contributors get 40% of received funds after fees
How does funding with Polar work?
1
Pay now to fund the work behind this issue.
2
Get updates on progress being made.
3
Maintainer is rewarded once the issue is completed.
FAQ
Backer
You're funding impactful open source efforts
Contributor
You want to contribute to this effort
Maintainer
You want to get funding like this too