evaluation.py 281 B

123456789101112131415
  1. from pydantic import BaseModel
  2. class EvalConfig(BaseModel):
  3. job_id: str
  4. test_split_ratio: float = 0.1
  5. batch_size: int = 4
  6. metrics: list[str] = ["perplexity", "loss"]
  7. class EvalResult(BaseModel):
  8. id: str
  9. job_id: str
  10. metrics: dict
  11. created_at: str