evaluation error

[GhadaAlfattni]

Hi

I was trying to run your code on another task. I have processed the data to be similar to semeval data. I have also amended the number of relations and the 2 files for evaluation with my relation types. However, I keep getting this error, any idea why?

[Epoch: 1, 3216/ 32185 points] total loss, accuracy per batch: 0.703, 0.733
[Epoch: 1, 6432/ 32185 points] total loss, accuracy per batch: 0.610, 0.774
[Epoch: 1, 9648/ 32185 points] total loss, accuracy per batch: 0.567, 0.794
[Epoch: 1, 12864/ 32185 points] total loss, accuracy per batch: 0.577, 0.780
[Epoch: 1, 16080/ 32185 points] total loss, accuracy per batch: 0.561, 0.785
[Epoch: 1, 19296/ 32185 points] total loss, accuracy per batch: 0.536, 0.793
[Epoch: 1, 22512/ 32185 points] total loss, accuracy per batch: 0.545, 0.799
[Epoch: 1, 25728/ 32185 points] total loss, accuracy per batch: 0.540, 0.792
[Epoch: 1, 28944/ 32185 points] total loss, accuracy per batch: 0.522, 0.799
[Epoch: 1, 32160/ 32185 points] total loss, accuracy per batch: 0.489, 0.810
06/21/2020 12:24:46 AM [INFO]: Evaluating test samples...
0%| | 1/1644 [00:15<7:00:27, 15.35s/it]
Traceback (most recent call last):
File "main_task.py", line 48, in
net = train_and_fit(args)
File "/Volumes/ghada/nlp/BERT-Relation-Extraction-master/src/tasks/trainer.py", line 159, in train_and_fit
results = evaluate_results(net, test_loader, pad_id, cuda)
File "/Volumes/ghada/nlp/BERT-Relation-Extraction-master/src/tasks/train_funcs.py", line 93, in evaluate_results
e1_e2_start=e1_e2_start)
File "/Users/ghada/opt/anaconda3/envs/relationBert/lib/python3.6/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/Volumes/ghada/nlp/BERT-Relation-Extraction-master/src/model/BERT/modeling_bert.py", line 734, in forward
embedding_output = self.embeddings(input_ids=input_ids, position_ids=position_ids, token_type_ids=token_type_ids, inputs_embeds=inputs_embeds)
File "/Users/ghada/opt/anaconda3/envs/relationBert/lib/python3.6/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/Volumes/ghada/nlp/BERT-Relation-Extraction-master/src/model/BERT/modeling_bert.py", line 177, in forward
position_embeddings = self.position_embeddings(position_ids)
File "/Users/ghada/opt/anaconda3/envs/relationBert/lib/python3.6/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/Users/ghada/opt/anaconda3/envs/relationBert/lib/python3.6/site-packages/torch/nn/modules/sparse.py", line 114, in forward
self.norm_type, self.scale_grad_by_freq, self.sparse)
File "/Users/ghada/opt/anaconda3/envs/relationBert/lib/python3.6/site-packages/torch/nn/functional.py", line 1724, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
IndexError: index out of range in self

[plkmo]

probably got to do with your test_loader outputs

[VahidehReshadat]

Hi

I was trying to run your code on another task. I have processed the data to be similar to semeval data. I have also amended the number of relations and the 2 files for evaluation with my relation types. However, I keep getting this error, any idea why?

[Epoch: 1, 3216/ 32185 points] total loss, accuracy per batch: 0.703, 0.733
[Epoch: 1, 6432/ 32185 points] total loss, accuracy per batch: 0.610, 0.774
[Epoch: 1, 9648/ 32185 points] total loss, accuracy per batch: 0.567, 0.794
[Epoch: 1, 12864/ 32185 points] total loss, accuracy per batch: 0.577, 0.780
[Epoch: 1, 16080/ 32185 points] total loss, accuracy per batch: 0.561, 0.785
[Epoch: 1, 19296/ 32185 points] total loss, accuracy per batch: 0.536, 0.793
[Epoch: 1, 22512/ 32185 points] total loss, accuracy per batch: 0.545, 0.799
[Epoch: 1, 25728/ 32185 points] total loss, accuracy per batch: 0.540, 0.792
[Epoch: 1, 28944/ 32185 points] total loss, accuracy per batch: 0.522, 0.799
[Epoch: 1, 32160/ 32185 points] total loss, accuracy per batch: 0.489, 0.810
06/21/2020 12:24:46 AM [INFO]: Evaluating test samples...
0%| | 1/1644 [00:15<7:00:27, 15.35s/it]
Traceback (most recent call last):
File "main_task.py", line 48, in
net = train_and_fit(args)
File "/Volumes/ghada/nlp/BERT-Relation-Extraction-master/src/tasks/trainer.py", line 159, in train_and_fit
results = evaluate_results(net, test_loader, pad_id, cuda)
File "/Volumes/ghada/nlp/BERT-Relation-Extraction-master/src/tasks/train_funcs.py", line 93, in evaluate_results
e1_e2_start=e1_e2_start)
File "/Users/ghada/opt/anaconda3/envs/relationBert/lib/python3.6/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/Volumes/ghada/nlp/BERT-Relation-Extraction-master/src/model/BERT/modeling_bert.py", line 734, in forward
embedding_output = self.embeddings(input_ids=input_ids, position_ids=position_ids, token_type_ids=token_type_ids, inputs_embeds=inputs_embeds)
File "/Users/ghada/opt/anaconda3/envs/relationBert/lib/python3.6/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/Volumes/ghada/nlp/BERT-Relation-Extraction-master/src/model/BERT/modeling_bert.py", line 177, in forward
position_embeddings = self.position_embeddings(position_ids)
File "/Users/ghada/opt/anaconda3/envs/relationBert/lib/python3.6/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/Users/ghada/opt/anaconda3/envs/relationBert/lib/python3.6/site-packages/torch/nn/modules/sparse.py", line 114, in forward
self.norm_type, self.scale_grad_by_freq, self.sparse)
File "/Users/ghada/opt/anaconda3/envs/relationBert/lib/python3.6/site-packages/torch/nn/functional.py", line 1724, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
IndexError: index out of range in self

It seems the number of relation types in your data is different from the default! notice that the sequence of the name entities for the same relation is important and is counted as a separate relation type as well

[plkmo]

If you want to run the task on your own data, do note that in preprocessing_funcs.py line 70:
rm = Relations_Mapper(df_train['relations']),
the relation classes are mapped using the train set only. So either you will need to ensure all relations classes are captured in the train dataset, or modify the code to ensure all relation classes in both train + test sets are captured. This may be causing your errors.