Why use answerdotai/ModernBERT-base as embedding model?

[bmaz]

Hi AXLMRIN!

Thank you for creating this tutorial!
Here is my first question about your approach: ModernBERT is not finetuned for document embeddings. From this discussion : https://huggingface.co/answerdotai/ModernBERT-base/discussions/9 I would rather try with joe32140/ModernBERT-base-msmarco
Or do you consider ModernBERT without finetuning good enough?

Best,
Béatrice

[bmaz]

Sorry, I didn't add the link to the line I was referring to:

bertopic-tutorial/tutorial.qmd

Line 377 in fe295d6

embedding_model = "answerdotai/ModernBERT-base"

I see that in the next cell you use gte-multilingual-base, so maybe the introduction to ModernBERT is not required?

[AXLMRIN]

Thank you for your remark! The thing is that I created the pipeline using Alibaba-NLP/gte-multilingual-base because out of habit, but it requires to set the trust_remote parameter to True meaning you can't use it for the BERtopic pipeline¹. When writing the tutorial I replaced "Alibaba-NLP/gte-multilingual-base" with "answerdotai/ModernBERT-base" ; but it was just a facade that I forgot to fix.

I will try and use "joe32140/ModernBERT-base-msmarco"! Thanks!

Footnotes

1. `BERTopic(model_name="Alibaba-NLP/gte-multilingual-base").fit_transform()" won't work; you have to generate your embeddings first. ↩