The MTEB benchmark has been deemed ineffective for evaluating embedding models, particularly due to its inability to accurately reflect performance across different domains. Observations indicate that embedding model performance varies significantly in-domain versus out-of-domain, making the benchmark less reliable. Alternatives such as NV-Embed are recommended for better performance metrics, specifically for domain-specific tasks. Moreover, the community is pushing for the abandonment of traditional training sets, proposing a move towards out-of-domain testing as a way to mitigate data leakage issues that compromise benchmark integrity. The discussion emphasizes the need for rigorous standards and methodologies in evaluating machine learning models, focusing on real-world applicability and the prevention of data leakage during training and testing processes.