As the first cloud service provider to launch Elasticsearch 8.9, Alibaba Cloud is not just offering the cutting-edge Elasticsearch Relevance Engine™ (ESRE™); it’s blending enhanced AI capabilities with Elasticsearch’s native search functionalities to offer users unprecedented innovation and exploration opportunities.
The rapid advancement and widespread application of artificial intelligence have yielded significant outcomes across various industries. As a potent search engine, Alibaba Cloud Elasticsearch has consistently provided enterprises with efficient and precise search services. Now, as the first provider to introduce version 8.9 domestically, Alibaba Cloud offers an integrated search solution that combines the best AI practices with Elastic’s native capabilities, opening new frontiers for user creativity and inquiry.
The latest upgrade from version 8.5 to 8.9 introduces key features that notably augment Alibaba Cloud Elasticsearch’s capabilities in vector search and hybrid search, significantly enhancing the accuracy and relevance of search results.
📋 New Features at a Glance
- 🔀 Support for mixed ranking of text and vector retrieval results through Reciprocal Rank Fusion (RRF).
- 🧠 Extended vector dimensionality up to 2048.
- 🚀 Improved brute force search performance.
- 🔍 Support for KNN queries spanning multiple fields.
- 🤖 Inclusion of built-in ELSER models.
- 📂 Reliable support for distributed management of NLP models.
- ➕ And more…
Learn More:
🚀 Vector Search: Giving Search the Wings of Progress
Vector search, a key addition to the 8.x series, transcends traditional keyword-based search. Utilizing the power of machine learning and artificial intelligence, it converts textual content into vector representations. Words within textual data are represented as vectors, and search retrieval is based on calculating the distances between these vectors to determine textual similarity. This translates into more effective and high-fidelity text retrieval. Compared to classic text searches, vector search enhances semantic relationships between words and documents, bringing a notable boost in search relevance. Not only limited to text, this approach extends to images, voice, and other data types, broadening the application spectrum. Most importantly, vector search allows customization of search results to cater to user preferences, offering a truly personalized search experience.
🔍 Hybrid Search with RRF: Doubling Down on Results and Performance
Hybrid Search RRF allows for the comprehensive re-ranking of result sets retrieved in varying ways, culminating in finely-tuned final rankings. By fusing results from BM25 relevance ranking and vector similarity recall through RRF, the overall precision of rankings receives a marked improvement. The superiority of hybrid search compared to single-faceted techniques is clear—it combines multiple search technologies to yield integrated and more accurate results. In terms of adaptability, enterprises can craft search solutions tailored to their unique business needs, enjoying increased flexibility. For empirical evidence of RRF’s impact on search result precision and relevance, one can view tests conducted on Alibaba Cloud Elasticsearch with semantic query results optimized via RRF.
| RRF Mixed Arrangement Query | Vector search | Text Search |
|---|---|---|
| Paragraph ID | Accuracy | Paragraph ID |
| 8588222 | 0 | 8588222 |
| 8588219 | 3 | 8588219 |
| 7911557 | 3 | 6080460 |
| 128984 | 3 | 128984 |
| 6080460 | 3 | 4254815 |
| 2697752 | 2 | 6343521 |
| 4254815 | 1 | 1020793 |
| 1721142 | 0 | 4254811 |
| 6343521 | 1 | 1959030 |
| 8588227 | 0 | 4254813 |
The three query types yielded varying results in terms of accuracy—scaled from ’not relevant’ to ‘completely relevant.’ It’s evident from the rankings that RRF’s ability to synthesize vector and text query results pushes relevant documents - such as “7911557”, previously absent from vector results, to the forefront. Simultaneously, RRF spotlighted the importance of documents like “6080460”, which the text query originally overlooked, thereby sharpening recall precision.
With the release of the new version, Alibaba Cloud Elasticsearch reaffirms its commitment to advancing search capabilities, bringing users a smarter and more profound search experience. Moving forward, Alibaba Cloud Elasticsearch continues to innovate, breaking new ground in search technology and expanding possibilities for users.
Learn More: Enhancing Search Accuracy with RRF(Reciprocal Rank Fusion) in Alibaba Cloud Elasticsearch 8.x
🔓 30-Day Free Trial: Implement the Latest Version of Elasticsearch
Alibaba Cloud Elasticsearch is a fully managed Elasticsearch cloud service built on the open-source Elasticsearch, supporting out-of-the-box functionality and pay-as-you-go while being 100% compatible with open-source features. Not only does it provide the cloud-ready components of the Elastic Stack, including Elasticsearch, Logstash, Kibana, and Beats, but it also partners with Elastic to offer the free X-Pack (Platinum level advanced features) commercial plugin. This integration includes advanced features such as security, SQL, machine learning, alerting, and monitoring, and is widely used in scenarios such as real-time log analysis, information retrieval, and multi-dimensional data querying and statistical analysis.
For more information about Elasticsearch, please visit https://www.alibabacloud.com/en/product/elasticsearch.
