Hello, I tested the code in the **Basic Usage** section of the README.md and `result = model.forward(video, instructions)` results in the following error. ``` Traceback (most recent call last): File "", line 1, in File "/home/robocat/miniforge3/envs/robocat/lib/python3.11/site-packages/robocat/model.py", line 762, in forward logits = self.model(video, text) ^^^^^^^^^^^^^^^^^^^^^^^ File "/home/robocat/miniforge3/envs/robocat/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/robocat/miniforge3/envs/robocat/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<@beartype(robocat.model.Gato.forward) at 0x7f4bb9d9cf40>", line 46, in forward File "/home/robocat/miniforge3/envs/robocat/lib/python3.11/site-packages/classifier_free_guidance_pytorch/classifier_free_guidance_pytorch.py", line 137, in inner return fn_maybe_with_text(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/robocat/miniforge3/envs/robocat/lib/python3.11/site-packages/classifier_free_guidance_pytorch/classifier_free_guidance_pytorch.py", line 130, in fn_maybe_with_text return fn(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/robocat/miniforge3/envs/robocat/lib/python3.11/site-packages/robocat/model.py", line 691, in forward learned_tokens = self.token_learner(tokens) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/robocat/miniforge3/envs/robocat/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/robocat/miniforge3/envs/robocat/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/robocat/miniforge3/envs/robocat/lib/python3.11/site-packages/robocat/model.py", line 509, in forward attn = self.net(x) ^^^^^^^^^^^ File "/home/robocat/miniforge3/envs/robocat/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/robocat/miniforge3/envs/robocat/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/robocat/miniforge3/envs/robocat/lib/python3.11/site-packages/torch/nn/modules/container.py", line 215, in forward input = module(input) ^^^^^^^^^^^^^ File "/home/robocat/miniforge3/envs/robocat/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/robocat/miniforge3/envs/robocat/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/robocat/miniforge3/envs/robocat/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 460, in forward return self._conv_forward(input, self.weight, self.bias) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/robocat/miniforge3/envs/robocat/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 456, in _conv_forward return F.conv2d(input, weight, bias, self.stride, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: Given groups=8, weight of size [12288, 768, 1, 1], expected input[12, 512, 112, 112] to have 6144 channels, but got 512 channels instead ``` Also there are a bunch of robotcat instead of robocat in the readme. ## Upvote & Fund - We're using [Polar.sh](https://polar.sh/kyegomez) so you can upvote and help fund this issue. - We receive the funding once the issue is completed & confirmed by you. - Thank you in advance for helping prioritize & fund our backlog.

Kye Gomez/RoboCAT

Issues looking for funding