: /root/.cache/pip/wheels/20/7b/3f/2807682bad2fba40ed888e6309597a5fda545ab30964c835aa
Successfully built deepspeed
Installing collected packages: tokenizers, SentencePiece, safetensors, ninja, hjson, bitsandbytes, xxhash, rouge, einops, dill, multiprocess, huggingface-hub, transformers, datasets, lion-pytorch, deepspeed, accelerate
Successfully installed SentencePiece-0.1.99 accelerate-0.21.0 bitsandbytes-0.40.2 datasets-2.13.1 deepspeed-0.10.0 dill-0.3.6 einops-0.6.1 hjson-3.1.0 huggingface-hub-0.16.4 lion-pytorch-0.1.2 multiprocess-0.70.14 ninja-1.11.1 rouge-1.0.1 safetensors-0.3.1 tokenizers-0.13.3 transformers-4.30.2 xxhash-3.2.0
[2023-07-17 22:42:48,068] [INFO] [real_accelerator.py:133:get_accelerator] Setting ds_accelerator to cuda (auto detect)
2023-07-17 22:42:50.272490: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
A100 GPU detected, using flash attention if input tensor is on cuda
/content/Andromeda/Andromeda/optimus_prime/attend.py:168: UserWarning: Memory efficient kernel not used because: (Triggered internally at ../aten/src/ATen/native/transformers/cuda/sdp_utils.h:545.)
out = F.scaled_dot_product_attention(
/content/Andromeda/Andromeda/optimus_prime/attend.py:168: UserWarning: Memory Efficient attention has been runtime disabled. (Triggered internally at ../aten/src/ATen/native/transformers/cuda/sdp_utils.h:338.)
out = F.scaled_dot_product_attention(
/content/Andromeda/Andromeda/optimus_prime/attend.py:168: UserWarning: Flash attention kernel not used because: (Triggered internally at ../aten/src/ATen/native/transformers/cuda/sdp_utils.h:547.)
out = F.scaled_dot_product_attention(
/content/Andromeda/Andromeda/optimus_prime/attend.py:168: UserWarning: Both fused kernels do not support non-null attn_mask. (Triggered internally at ../aten/src/ATen/native/transformers/cuda/sdp_utils.h:191.)
out = F.scaled_dot_product_attention(
Traceback (most recent call last):
File "/content/Andromeda/benchmarking.py", line 237, in <module>
forward_pass_time = speed_metrics.forward_pass_time()
File "/content/Andromeda/benchmarking.py", line 66, in forward_pass_time
model_input = self.model.decoder.forward(torch.randint(0, 50304, (1, 8192), device=device, dtype=torch.long))[0]
File "/content/Andromeda/Andromeda/optimus_prime/autoregressive_wrapper.py", line 141, in forward
logits = self.net(inp, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/content/Andromeda/Andromeda/optimus_prime/x_transformers.py", line 1422, in forward
x = self.attn_layers(x, mask = mask, mems = mems, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/content/Andromeda/Andromeda/optimus_prime/x_transformers.py", line 1155, in forward
out, inter = block(x, mask = mask, context_mask = self_attn_context_mask, attn_mask = attn_mask, rel_pos = self.rel_pos, rotary_pos_emb = rotary_pos_emb, prev_attn = prev_attn, mem = layer_mem)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/content/Andromeda/Andromeda/optimus_prime/x_transformers.py", line 581, in forward
return self.fn(x, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/content/Andromeda/Andromeda/optimus_prime/x_transformers.py", line 863, in forward
out, intermediates = self.attend(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/content/Andromeda/Andromeda/optimus_prime/attend.py", line 198, in forward
return self.flash_attn(q, k, v, mask = mask, attn_bias = attn_bias)
File "/content/Andromeda/Andromeda/optimus_prime/attend.py", line 168, in flash_attn
out = F.scaled_dot_product_attention(
RuntimeError: No available kernel. Aborting execution.
Pay now to fund the work behind this issue.
Get updates on progress being made.
Maintainer is rewarded once the issue is completed.
You're funding impactful open source efforts
You want to contribute to this effort
You want to get funding like this too