-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Description
您好,想请问一下关于evaluation中代码的问题,我看到在SFT和RL训练的时候,都是直接取special token对应index的hidden state,为什么在evaluation的时候这里有个+1
def get_embedding_idx(generated_ids_trimmed, EMBEDDING_TOKEN_ID):
embedding_idx = []
# Search from the last token forward
for i, out_ids in enumerate(generated_ids_trimmed):
embed_exist = False
for j in range(len(out_ids) - 1, -1, -1):
if out_ids[j] == EMBEDDING_TOKEN_ID:
if j + 1 >= len(out_ids) - 1:
embedding_idx.append(-1)
else:
embedding_idx.append(j + 1)
embed_exist = True
break
if not embed_exist:
embedding_idx.append(-1)
# embedding_idx.append(-1)
return embedding_idx
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels