Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models ...
Hugging Face has launched Community Evals, a feature that enables benchmark datasets on the Hub to host their own leaderboards and automatically collect evaluation results from model repositories.