Installation

This guide covers the installation process for OpenEvals.

Requirements

Hardware

Model Size	Minimum VRAM	Recommended	With Quantization
1-3B	4 GB	8 GB	2 GB
7-9B	8 GB	16 GB	5 GB
13-14B	16 GB	24 GB	8 GB
70B+	40 GB	80 GB+	20 GB

Software

Python 3.10+
CUDA 11.8+ (for GPU acceleration)
50 GB+ disk space (models and datasets)

Basic Installation

Clone the Repository

git clone https://github.com/heilcheng/openevals.git
cd openevals

Create Virtual Environment

python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

Install Dependencies

pip install -r requirements.txt

Development Installation

For contributors who want to modify the codebase:

pip install -e .
pip install pre-commit
pre-commit install

Optional Dependencies

For additional features:

# Visualization support
pip install matplotlib seaborn plotly

# Statistical analysis
pip install scipy statsmodels

# Web platform
pip install fastapi uvicorn

Configuration

Authentication

OpenEvals requires a HuggingFace token to download models and datasets.

Option 1: Environment Variable

export HF_TOKEN=your_huggingface_token

Option 2: HuggingFace CLI

huggingface-cli login

To get a token, visit https://huggingface.co/settings/tokens.

Docker Installation

For containerized deployment:

docker build -t openevals .
docker run --gpus all -it openevals

Troubleshooting

Common Issues

CUDA not available

Ensure CUDA drivers are installed and nvidia-smi shows your GPU.

Out of memory

Enable quantization in your configuration:

hardware:
  quantization: true

Model download fails

Verify your HuggingFace token has the necessary permissions for gated models.