Hi. I have used Captum for LLaMA-2 successfully before. But now I wanted to use it again and it generates the error below. I also copy and pasted the example from Understanding Llama2 with Captum LLM Attribution same problem.
What is the problem. Somebody help!
Traceback (most recent call last):
Cell In[6], line 9
attr_res = llm_attr.attribute(inp, target=target)
File ~/Documents/Research/January 2024/captum/captum/attr/_core/llm_attr.py:375 in attribute
cur_attr = self.attr_method.attribute(
File ~/Documents/Research/January 2024/captum/captum/log/__init__.py:42 in wrapper
return func(*args, **kwargs)
File ~/Documents/Research/January 2024/captum/captum/attr/_core/feature_ablation.py:289 in attribute
initial_eval = self._strict_run_forward(
File ~/Documents/Research/January 2024/captum/captum/attr/_core/feature_ablation.py:599 in _strict_run_forward
forward_output = _run_forward(*args, **kwargs)
File ~/Documents/Research/January 2024/captum/captum/_utils/common.py:531 in _run_forward
output = forward_func(
File ~/Documents/Research/January 2024/captum/captum/attr/_core/llm_attr.py:249 in _forward_func
model_kwargs = self.model._update_model_kwargs_for_generation(
File ~/anaconda3/envs/llmft/lib/python3.11/site-packages/transformers/generation/utils.py:681 in _update_model_kwargs_for_generation
model_kwargs["cache_position"] = model_inputs.get("cache_position", None)
AttributeError: 'NoneType' object has no attribute 'get'