Embedding Models: from Architecture to Implementation
3 Contextualized token embeddings
单词嵌入：word2vec/glove无法捕获上下文，BERT可以捕获上下文
BERT（an encoder only transformer model）被大量用作 sentence embedding model 的组件
4 Token vs. sentence embedding
dual encoder 架构做 Q/A时的 sentence embedding
5 Training a dual encoder
6 Using embeddings in RAG
7
用embeddings快速获取文档，再用更准确的Cross-encoder重排序
Understanding and Applying Text Embeddings
6
先Top K，再Top P，再Temperature
