Pillar Service

简体中文

Project Overview

Pillar Service is a multifunctional service platform based on FastAPI, supporting image caption generation, text translation, and more. It is suitable for rapid integration and extension of AI inference services.

Key Features

• FastAPI-based, fast response, and easy to extend
• Supports image captioning (JoyCaption) and text translation
• Model cache and memory management
• Detailed health check and logging

Environment Requirements

• Python 3.11
• Conda (Anaconda or Miniconda recommended)
• NVIDIA GPU and driver (for GPU acceleration)
• Dependencies listed in requirements.txt

Installation & Running

# 1. Create and activate conda environment
conda create -n pillar_env python=3.11
conda activate pillar_env

# 2. Configure Tsinghua conda-forge mirror (recommended for China)
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
conda config --set show_channel_urls yes

# 3. Install CUDA dependencies
conda install -y cudatoolkit cudnn 

# 4. Install PyTorch (example: CUDA 12.4)
conda install -y pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia

# 5. Install Python dependencies
pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

# 6. Install bitsandbytes and accelerate
pip install bitsandbytes --prefer-binary -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install accelerate -i https://pypi.tuna.tsinghua.edu.cn/simple

# 7. Check CUDA availability
python -c "import torch; print('[INFO] torch.cuda.is_available():', torch.cuda.is_available())"

# 8. Start the service
run_server.bat

Main Directory Structure

Pillar_Service/
├── server/
│   ├── main.py              # FastAPI entry point
│   ├── api/                 # API routes and service management
│   └── ...
├── Pillar/
│   ├── dto/                 # Data Transfer Objects (DTO)
│   ├── service/             # Business service implementations
│   └── util/                # Utilities and constants
├── requirements.txt         # Python dependencies
├── run_server.bat           # Startup script (Windows)
└── README.md                # Project documentation

API Overview

• /translate
  Text translation API, POST request, parameter: text.
• /joycaption/generate
  Image caption generation API, POST request, parameters: image and related generation options.
• /health
  Health check API, GET request.

For detailed parameters and return formats, please refer to code comments or API documentation.

FAQ & Support

• Slow or failed dependency installation: Use Tsinghua, USTC, or other China mirrors for pip/conda.
• bitsandbytes device not supported: Make sure you have an NVIDIA GPU, CUDA driver, and PyTorch CUDA version.
• For other issues, please refer to the documentation or submit an issue.

---

Feel free to open an issue or contact us if you have any questions!