Wiki-Quellcode von Whisper-AI
Zuletzt geändert von René Schmidt am 2025/03/06 22:38
Zeige letzte Bearbeiter
| author | version | line-number | content |
|---|---|---|---|
| 1 | = Whisper-WebUI = | ||
| 2 | |||
| 3 | |||
| 4 | A Gradio-based browser interface for [[Whisper>>url:https://github.com/openai/whisper]]. You can use it as an Easy Subtitle Generator! | ||
| 5 | |||
| 6 | [[~[~[image:https://private-user-images.githubusercontent.com/97279763/396437782-caea3afd-a73c-40af-a347-8d57914b1d0f.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDEzMDA1MTQsIm5iZiI6MTc0MTMwMDIxNCwicGF0aCI6Ii85NzI3OTc2My8zOTY0Mzc3ODItY2FlYTNhZmQtYTczYy00MGFmLWEzNDctOGQ1NzkxNGIxZDBmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAzMDYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMzA2VDIyMzAxNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTY2MjEzMGY2YWEyMzM4ODAzZTk3YmFiY2EyNWMzOGIwMjdhM2JhMmZiNDZjZTQzNDY5MDFiODBiYmMwNDk2NWImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.uBtCRmRlYf3zF2aULRd--eajIi4EtuumFyaFSjH5O4s~|~|alt="screen"~]~]>>url:https://private-user-images.githubusercontent.com/97279763/396437782-caea3afd-a73c-40af-a347-8d57914b1d0f.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDEzMDA1MTQsIm5iZiI6MTc0MTMwMDIxNCwicGF0aCI6Ii85NzI3OTc2My8zOTY0Mzc3ODItY2FlYTNhZmQtYTczYy00MGFmLWEzNDctOGQ1NzkxNGIxZDBmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAzMDYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMzA2VDIyMzAxNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTY2MjEzMGY2YWEyMzM4ODAzZTk3YmFiY2EyNWMzOGIwMjdhM2JhMmZiNDZjZTQzNDY5MDFiODBiYmMwNDk2NWImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.uBtCRmRlYf3zF2aULRd--eajIi4EtuumFyaFSjH5O4s]] | ||
| 7 | |||
| 8 | == Notebook == | ||
| 9 | |||
| 10 | |||
| 11 | If you wish to try this on Colab, you can do it in [[here>>url:https://colab.research.google.com/github/jhj0517/Whisper-WebUI/blob/master/notebook/whisper-webui.ipynb]]! | ||
| 12 | |||
| 13 | = Feature = | ||
| 14 | |||
| 15 | |||
| 16 | * Select the Whisper implementation you want to use between : | ||
| 17 | ** [[openai/whisper>>url:https://github.com/openai/whisper]] | ||
| 18 | ** [[SYSTRAN/faster-whisper>>url:https://github.com/SYSTRAN/faster-whisper]] (used by default) | ||
| 19 | ** [[Vaibhavs10/insanely-fast-whisper>>url:https://github.com/Vaibhavs10/insanely-fast-whisper]] | ||
| 20 | * Generate subtitles from various sources, including : | ||
| 21 | ** Files | ||
| 22 | ** Youtube | ||
| 23 | ** Microphone | ||
| 24 | * Currently supported subtitle formats : | ||
| 25 | ** SRT | ||
| 26 | ** WebVTT | ||
| 27 | ** txt ( only text file without timeline ) | ||
| 28 | * Speech to Text Translation | ||
| 29 | ** From other languages to English. ( This is Whisper's end-to-end speech-to-text translation feature ) | ||
| 30 | * Text to Text Translation | ||
| 31 | ** Translate subtitle files using Facebook NLLB models | ||
| 32 | ** Translate subtitle files using DeepL API | ||
| 33 | * Pre-processing audio input with [[Silero VAD>>url:https://github.com/snakers4/silero-vad]]. | ||
| 34 | * Pre-processing audio input to separate BGM with [[UVR>>url:https://github.com/Anjok07/ultimatevocalremovergui]]. | ||
| 35 | * Post-processing with speaker diarization using the [[pyannote>>url:https://huggingface.co/pyannote/speaker-diarization-3.1]] model. | ||
| 36 | ** To download the pyannote model, you need to have a Huggingface token and manually accept their terms in the pages below. | ||
| 37 | **1. [[https:~~/~~/huggingface.co/pyannote/speaker-diarization-3.1>>url:https://huggingface.co/pyannote/speaker-diarization-3.1]] | ||
| 38 | **1. [[https:~~/~~/huggingface.co/pyannote/segmentation-3.0>>url:https://huggingface.co/pyannote/segmentation-3.0]] | ||
| 39 | |||
| 40 | === Pipeline Diagram === | ||
| 41 | |||
| 42 | |||
| 43 | [[~[~[image:https://private-user-images.githubusercontent.com/97279763/389952408-1d8c63ac-72a4-4a0b-9db0-e03695dcf088.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDEzMDA1MTQsIm5iZiI6MTc0MTMwMDIxNCwicGF0aCI6Ii85NzI3OTc2My8zODk5NTI0MDgtMWQ4YzYzYWMtNzJhNC00YTBiLTlkYjAtZTAzNjk1ZGNmMDg4LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAzMDYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMzA2VDIyMzAxNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWViOTYzZTdlYzdiYTRkMzYwY2YxZmZhOGM4Njc4OGNkMDFjYmIzZjMzNTdlYjA0ODUyZTZkNWZiNDQwZjlmMGQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.-7Zyn-HpREZdCWVjZ_hExH50vL-tSiCkxQmuBHgaJvE~|~|alt="Transcription Pipeline"~]~]>>url:https://private-user-images.githubusercontent.com/97279763/389952408-1d8c63ac-72a4-4a0b-9db0-e03695dcf088.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDEzMDA1MTQsIm5iZiI6MTc0MTMwMDIxNCwicGF0aCI6Ii85NzI3OTc2My8zODk5NTI0MDgtMWQ4YzYzYWMtNzJhNC00YTBiLTlkYjAtZTAzNjk1ZGNmMDg4LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAzMDYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMzA2VDIyMzAxNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWViOTYzZTdlYzdiYTRkMzYwY2YxZmZhOGM4Njc4OGNkMDFjYmIzZjMzNTdlYjA0ODUyZTZkNWZiNDQwZjlmMGQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.-7Zyn-HpREZdCWVjZ_hExH50vL-tSiCkxQmuBHgaJvE]] | ||
| 44 | |||
| 45 | = Installation and Running = | ||
| 46 | |||
| 47 | |||
| 48 | * ((( | ||
| 49 | == Running with Pinokio == | ||
| 50 | ))) | ||
| 51 | |||
| 52 | The app is able to run with [[Pinokio>>url:https://github.com/pinokiocomputer/pinokio]]. | ||
| 53 | |||
| 54 | 1. Install [[Pinokio Software>>url:https://program.pinokio.computer/#/?id=install]]. | ||
| 55 | 1. Open the software and search for Whisper-WebUI and install it. | ||
| 56 | 1. Start the Whisper-WebUI and connect to the http:~/~/localhost:7860. | ||
| 57 | |||
| 58 | * ((( | ||
| 59 | == Running with Docker == | ||
| 60 | ))) | ||
| 61 | |||
| 62 | 1. Install and launch [[Docker-Desktop>>url:https://www.docker.com/products/docker-desktop/]]. | ||
| 63 | 1. Git clone the repository | ||
| 64 | |||
| 65 | {{{git clone https://github.com/jhj0517/Whisper-WebUI.git}}} | ||
| 66 | |||
| 67 | 1. Build the image ( Image is about 7GB~~ ) | ||
| 68 | |||
| 69 | {{{docker compose build }}} | ||
| 70 | |||
| 71 | 1. Run the container | ||
| 72 | |||
| 73 | {{{docker compose up}}} | ||
| 74 | |||
| 75 | 1. Connect to the WebUI with your browser at http:~/~/localhost:7860 | ||
| 76 | |||
| 77 | If needed, update the [[docker-compose.yaml>>url:https://github.com/jhj0517/Whisper-WebUI/blob/master/docker-compose.yaml]] to match your environment. | ||
| 78 | |||
| 79 | * ((( | ||
| 80 | == Run Locally == | ||
| 81 | ))) | ||
| 82 | |||
| 83 | === Prerequisite === | ||
| 84 | |||
| 85 | |||
| 86 | To run this WebUI, you need to have git, 3.10 <= python <= 3.12, FFmpeg. | ||
| 87 | And if you're not using an Nvida GPU, or using a different CUDA version than 12.4, edit the [[requirements.txt>>url:https://github.com/jhj0517/Whisper-WebUI/blob/master/requirements.txt]] to match your environment. | ||
| 88 | |||
| 89 | Please follow the links below to install the necessary software: | ||
| 90 | |||
| 91 | * git : [[https:~~/~~/git-scm.com/downloads>>url:https://git-scm.com/downloads]] | ||
| 92 | * python : [[https:~~/~~/www.python.org/downloads/>>url:https://www.python.org/downloads/]] **3.10 ~~ 3.12 is recommended.** | ||
| 93 | * FFmpeg : [[https:~~/~~/ffmpeg.org/download.html>>url:https://ffmpeg.org/download.html]] | ||
| 94 | * CUDA : [[https:~~/~~/developer.nvidia.com/cuda-downloads>>url:https://developer.nvidia.com/cuda-downloads]] | ||
| 95 | |||
| 96 | After installing FFmpeg, **make sure to add the FFmpeg/bin folder to your system PATH!** | ||
| 97 | |||
| 98 | === Installation Using the Script Files === | ||
| 99 | |||
| 100 | |||
| 101 | 1. git clone this repository | ||
| 102 | |||
| 103 | {{{git clone https://github.com/jhj0517/Whisper-WebUI.git}}} | ||
| 104 | |||
| 105 | 1. Run install.bat or install.sh to install dependencies. (It will create a venv directory and install dependencies there.) | ||
| 106 | 1. Start WebUI with start-webui.bat or start-webui.sh (It will run python app.py after activating the venv) | ||
| 107 | |||
| 108 | And you can also run the project with command line arguments if you like to, see [[wiki>>url:https://github.com/jhj0517/Whisper-WebUI/wiki/Command-Line-Arguments]] for a guide to arguments. | ||
| 109 | |||
| 110 | = VRAM Usages = | ||
| 111 | |||
| 112 | |||
| 113 | This project is integrated with [[faster-whisper>>url:https://github.com/guillaumekln/faster-whisper]] by default for better VRAM usage and transcription speed. | ||
| 114 | |||
| 115 | According to faster-whisper, the efficiency of the optimized whisper model is as follows: | ||
| 116 | |||
| 117 | |=Implementation|=Precision|=Beam size|=Time|=Max. GPU memory|=Max. CPU memory | ||
| 118 | |openai/whisper|fp16|5|4m30s|11325MB|9439MB | ||
| 119 | |faster-whisper|fp16|5|54s|4755MB|3244MB | ||
| 120 | |||
| 121 | If you want to use an implementation other than faster-whisper, use ~-~-whisper_type arg and the repository name. | ||
| 122 | Read [[wiki>>url:https://github.com/jhj0517/Whisper-WebUI/wiki/Command-Line-Arguments]] for more info about CLI args. | ||
| 123 | |||
| 124 | If you want to use a fine-tuned model, manually place the models in models/Whisper/ corresponding to the implementation. | ||
| 125 | |||
| 126 | Alternatively, if you enter the huggingface repo id (e.g, [[deepdml/faster-whisper-large-v3-turbo-ct2>>url:https://huggingface.co/deepdml/faster-whisper-large-v3-turbo-ct2]]) in the "Model" dropdown, it will be automatically downloaded in the directory. | ||
| 127 | |||
| 128 | [[~[~[image:https://private-user-images.githubusercontent.com/97279763/389192679-76487a46-b0a5-4154-b735-ded73b2d83d4.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDEzMDA1MTQsIm5iZiI6MTc0MTMwMDIxNCwicGF0aCI6Ii85NzI3OTc2My8zODkxOTI2NzktNzY0ODdhNDYtYjBhNS00MTU0LWI3MzUtZGVkNzNiMmQ4M2Q0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAzMDYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMzA2VDIyMzAxNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTBlMmJkM2E2YWUyZWViMzliMmJlMTE4MjNjYzU3ZTUzOTI4Y2Y0N2YzNWNiYWE4ZjNkYWVlNzA5NjYzMzhiYzMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.uG4-xbp8-hrJGsuzEBqzvsdK5iAm1KcSatFSUhDJ2GI~|~|alt="image"~]~]>>url:https://private-user-images.githubusercontent.com/97279763/389192679-76487a46-b0a5-4154-b735-ded73b2d83d4.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDEzMDA1MTQsIm5iZiI6MTc0MTMwMDIxNCwicGF0aCI6Ii85NzI3OTc2My8zODkxOTI2NzktNzY0ODdhNDYtYjBhNS00MTU0LWI3MzUtZGVkNzNiMmQ4M2Q0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAzMDYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMzA2VDIyMzAxNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTBlMmJkM2E2YWUyZWViMzliMmJlMTE4MjNjYzU3ZTUzOTI4Y2Y0N2YzNWNiYWE4ZjNkYWVlNzA5NjYzMzhiYzMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.uG4-xbp8-hrJGsuzEBqzvsdK5iAm1KcSatFSUhDJ2GI]] |