Row 70417
Content Data
This page contains data entry 70417 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.
Learning to binarize and rank with CLIP to reduce storage by 32x for text or multimodal search and recommendations.
Article: [https://www.marqo.ai/blog/learn-to-binarize-clip-for-multimodal-retrieval-and-ranking](https://www.marqo.ai/blog/learn-to-binarize-clip-for-multimodal-retrieval-and-ranking)
* Binary embeddings during CLIP rank-tuning preserve between 87-93% of fp32 embeddings. * Pseudo-quantization with sigmoid with 4x scaled temperature is (almost) universally better than tanh (see next point). * Cosine similarity on 0/1 (sigmoid) is better than -1, 1 (tanh) - pretty sure this is because cosine has better degeneracy (D vs DxN) as it penalises embeddings that are not on the same hyper-sphere (it also biases for fewer non-zero elements). * Use L1 to approximate hamming distance during training which is marginally better than cosine (for 0/1). * Evaluated using [GS-10M](https://github.com/marqo-ai/GCL) for multimodal retrieval using exact KNN. * Fp32 embeddings retain full fidelity when auxiliary binary loss is added . * Evaluated across in-domain, novel query, novel document and zero-shot settings. * Can be combined with [Matryoshka](https://www.marqo.ai/blog/matryoshka-representation-learning-with-clip-for-multimodal-retrieval-and-ranking) if really necessary but fidelity does suffer (not shown).
| Field | Value |
|---|---|
| text | Learning to binarize and rank with CLIP to reduce storage by 32x for text or multimodal search and recommendations. Article: [https://www.marqo.ai/blog/learn-to-binarize-clip-for-multimodal-retrieval-and-ranking](https://www.marqo.ai/blog/learn-to-binarize-clip-for-multimodal-retrieval-and-ranking) * Binary embeddings during CLIP rank-tuning preserve between 87-93% of fp32 embeddings. * Pseudo-quantization with sigmoid with 4x scaled temperature is (almost) universally better than tanh (see… |
| label | r/machinelearning |
| dataType | post |
| communityName | r/MachineLearning |
| datetime | 2024-05-23 |
| username_encoded | Z0FBQUFBQm5Lak1mcTFTN1ZRTEpCZ216Q0ZBeERxa3pnWHZvYjN4Qlc3LUVvX1gydWZabnpjcXh2Y0QtU3NmbHdRVG5tNDE0OEZIeGQ1TTlCZWwyUkYtMXFickxuT0VET1E9PQ== |
| url_encoded | Z0FBQUFBQm5Lak92TURfUG51YmhrLWxReFlybXlOVm9KSUdKdExuYkFFOTdMQUNsdmJFWkdkZlJVaVlndEFyN29aaUxyVFRrd3plOVVRUXY0TTB6OW1HZTNBS1RDVEZ6dG1PRTJNdG16SDIzbGtlWDhweGo2am5ORnA1QkM0NU94TEtmemJpWmhId0lnMU52OGZTc1B6REVLTzN5V3JHU3R0ZGdtNXhlalk1WnlzVWpMY255cHJQelpfeVNpUkFOdHdOMVhsU3czby00RmtDT1VxRHM1YjQ2LXRsYkx6dFZwQT09 |
Raw Record
{
"text": "Learning to binarize and rank with CLIP to reduce storage by 32x for text or multimodal search and recommendations.\n\n \nArticle: [https://www.marqo.ai/blog/learn-to-binarize-clip-for-multimodal-retrieval-and-ranking](https://www.marqo.ai/blog/learn-to-binarize-clip-for-multimodal-retrieval-and-ranking)\n\n* Binary embeddings during CLIP rank-tuning preserve between 87-93% of fp32 embeddings.\n* Pseudo-quantization with sigmoid with 4x scaled temperature is (almost) universally better than tanh (see next point).\n* Cosine similarity on 0/1 (sigmoid) is better than -1, 1 (tanh) - pretty sure this is because cosine has better degeneracy (D vs DxN) as it penalises embeddings that are not on the same hyper-sphere (it also biases for fewer non-zero elements).\n* Use L1 to approximate hamming distance during training which is marginally better than cosine (for 0/1).\n* Evaluated using [GS-10M](https://github.com/marqo-ai/GCL) for multimodal retrieval using exact KNN.\n* Fp32 embeddings retain full fidelity when auxiliary binary loss is added .\n* Evaluated across in-domain, novel query, novel document and zero-shot settings.\n* Can be combined with [Matryoshka](https://www.marqo.ai/blog/matryoshka-representation-learning-with-clip-for-multimodal-retrieval-and-ranking) if really necessary but fidelity does suffer (not shown).",
"label": "r/machinelearning",
"dataType": "post",
"communityName": "r/MachineLearning",
"datetime": "2024-05-23",
"username_encoded": "Z0FBQUFBQm5Lak1mcTFTN1ZRTEpCZ216Q0ZBeERxa3pnWHZvYjN4Qlc3LUVvX1gydWZabnpjcXh2Y0QtU3NmbHdRVG5tNDE0OEZIeGQ1TTlCZWwyUkYtMXFickxuT0VET1E9PQ==",
"url_encoded": "Z0FBQUFBQm5Lak92TURfUG51YmhrLWxReFlybXlOVm9KSUdKdExuYkFFOTdMQUNsdmJFWkdkZlJVaVlndEFyN29aaUxyVFRrd3plOVVRUXY0TTB6OW1HZTNBS1RDVEZ6dG1PRTJNdG16SDIzbGtlWDhweGo2am5ORnA1QkM0NU94TEtmemJpWmhId0lnMU52OGZTc1B6REVLTzN5V3JHU3R0ZGdtNXhlalk1WnlzVWpMY255cHJQelpfeVNpUkFOdHdOMVhsU3czby00RmtDT1VxRHM1YjQ2LXRsYkx6dFZwQT09"
}
Entry Information
- Entry ID: 70417
- Repository: Axioma AXP
- Dataset: arrmlet/reddit_dataset_36
- Total Entries: 100,000