Getting Started

1. git clone

   git clone git@github.com:andrewhsugithub/SynthHead-Fusion.git --recursive
   git submodule update --recursive
   pnpm i

2. copy env files

   cp ./scripts/.env.sh.template ./scripts/.env.sh
   cp ./scripts/services/.env.server ./scripts/services/.env.server

   Enter the env values

3. update user db

   cd packages/user
   docker compose up
   pnpm user:db
   docker compose down

4. copy env and paste your pubic key(.pem file) into keys folder for each microservice

   • auth service

   cd packages/auth/
   cp .env.template .env

   • bucket service

   cd packages/bucket/
   cp .env.template .env
   cp .env.server.template .env.server

   • GPT-Audio

   cd packages/GPT-Audio/
   cp .env.template .env
   cp .env.server.template .env.server

   • GPT-SoVITS-Inference

   cd packages/GPT-SoVITS-Inference/
   cp .env.server.template .env.server

   • LivePortrait

   cd packages/LivePortrait/
   cp .env.server.template .env.server

   • MuseTalk

   cd packages/MuseTalk/
   cp .env.template .env
   cp .env.server.template .env.server

   • Real3DPortrait

   cd packages/Real3DPortrait/
   cp .env.server.template .env.server

   Enter the env values

5. spin up servers

   docker compose up
   pnpm dev:remote

Note: all env values can be found in discord

Spinning up the servers for each microservice

Run each service in an independent shell.

1. GPT-SoVITS-Inference

bash ./scripts/services/GPT-SoVITS-Inference.sh

Port: 5000

2. GPT-Audio (depends on GPT-SoVITS-Inference)

bash ./scripts/services/GPT-Audio.sh

Port: 7998

3. MuseTalk

bash ./scripts/services/MuseTalk.sh

Port:7999

4. Real3DPortrait

bash ./scripts/services/Real3DPortrait.sh

Port: 8001

5. LivePortrait

bash ./scripts/services/LivePortrait.sh

Port: 8000

6. Bucket

bash ./scripts/services/bucket.sh

Port: 3002

Adding submodules

Note: to avoid sync errors I've temporarily put the repos that are not submodules in .gitignore, so if you've decided to add your repo to submodules remember to remove them from .gitignore

cd ml/
git submodule add <remote_url> <repo_name>

git status # (optional): see your repo is untracked or not
git add <repo_name>
git commit -m <commit_message>
git push

Updating repo

git switch main
git pull
git submodule update --recursive

or (WIP: need to wait for all submodules to be added)

cd ml/scripts/
bash sync_repo.sh <branch_name>

Contributing on same server

To see who's name is it now:

cd <the_repo_contributing>
git config --local --list

Remember to change your name when contributing

cd <the_repo_contributing>
git config --local user.email <your_email>
git config --local user.name <your_username>

example:

cd LivePortrait/
git config --local user.email andrew1mail@gmail.com
git config --local user.name andrewhsugithub

git config --local user.email h0920185003@gmail.com
git config --local user.name h44343880

How to run each model respectively

Real3DPortrait:

cd Real3DPortrait/
bash ./start.sh

GPT-SoVITS-Inference:

cd GPT-SoVITS-Inference/
bash ./start.sh

LivePortrait:

cd LivePortrait/
bash ./start.sh

GPT-Audio:

cd GPT-Audio
bash ./start.sh

How to run all

1. add env variables

cp ./scripts/.env.sh.template ./scripts/.env.sh

bash ./scripts/run_all.sh

TODO

• add logger, openapi, docs to every service
• add Read3D-Interface into submodule
• add videoCutting into submodule
• turn into web app
• make realtime
• use gfpgan to make higher resolution (can this be done in realtime though?)
• use frame interpolation methods to make video more smooth (for demo purposes, can't do this in realtime)