Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models.
Brian Beers is a digital editor, writer, Emmy-nominated producer, and content expert with 15+ years of experience writing about corporate finance & accounting, fundamental analysis, and investing.