Exa AI Research Blog
Discover the latest in AI research and semantic search technology on the Exa blog. Learn how our neural network search engine provides high-quality web data for AI applications
TL;DR
- A new benchmark has been created for company search, focusing on retrieval of structured data over memorized knowledge.
- The benchmark includes an ~800-query dataset and an open-sourced evaluation harness.
- It distinguishes between static facts (e.g., founding year) and dynamic facts (e.g., employee count, funding).
- Two evaluation tracks are included: Retrieval (testing company retrieval for queries) and RAG (testing fact extraction from retrieved content).
- The dataset was designed to avoid well-known large companies and instead focus on regional players, smaller companies, and niche verticals to ensure retrieval is necessary.
- The benchmark is part of a larger effort to build an evaluation ecosystem for various entity types and search domains.