Converting Speech to Text in Python Using OpenAI Whisper#

Prerequisites:#

• A Python installation (3.7.1 or later).
• An OpenAI API key. Obtain this from the OpenAI platform website after creating an account. API usage incurs costs based on the amount of audio processed.
• The openai Python library installed.

Step-by-Step Guide:#

1. Install the OpenAI library: Use pip, Python’s package installer:

   1
   pip install openai
   

2. Set up your OpenAI API key: It is recommended to set your API key as an environment variable (OPENAI_API_KEY). This prevents hardcoding the key directly in scripts.

   1
   # On Linux/macOS
   2
   export OPENAI_API_KEY='YOUR_API_KEY'
   3
   
   4
   # On Windows (Command Prompt)
   5
   set OPENAI_API_KEY='YOUR_API_KEY'
   6
   
   7
   # On Windows (PowerShell)
   8
   $env:OPENAI_API_KEY='YOUR_API_KEY'
   

   The openai library automatically reads this environment variable. Alternatively, the key can be passed directly in the code, though this is less secure for production environments.

3. Prepare the audio file: The OpenAI API supports several audio formats, including flac, mp3, mp4, mpeg, mpga, m4a, ogg, wav, and webm. The maximum file size is 25 MB. For larger files, audio segmentation is required before sending them to the API.

4. Write the Python code: Import the library, open the audio file, and make the API call using openai.audio.transcriptions.create.

   1
   import os
   2
   from openai import OpenAI
   3
   
   4
   # The library automatically reads OPENAI_API_KEY from environment variables
   5
   client = OpenAI()
   6
   
   7
   # Path to your audio file
   8
   audio_file_path = "path/to/your/audiofile.mp3"
   9
   
   10
   # Ensure the file exists
   11
   if not os.path.exists(audio_file_path):
   12
       print(f"Error: File not found at {audio_file_path}")
   13
   else:
   14
       try:
   15
           with open(audio_file_path, "rb") as audio_file:
   16
               transcript = client.audio.transcriptions.create(
   17
                   model="whisper-1", # Currently the standard Whisper model offered by OpenAI API
   18
                   file=audio_file
   19
               )
   20
   
   21
           print("Transcription:")
   22
           print(transcript.text)
   23
   
   24
       except Exception as e:
   25
           print(f"An error occurred: {e}")
   

5. Run the script: Execute the Python file from your terminal.

   1
   python your_script_name.py
   

API Call Options:#

The client.audio.transcriptions.create method accepts optional parameters:

• response_format: Specify the output format (e.g., ‘json’, ‘text’, ‘srt’, ‘vtt’). Default is ‘json’.
• language: Specify the input language using a two-letter ISO-639-1 code (e.g., ‘en’, ‘es’, ‘fr’). Providing the language can improve accuracy and speed.
• prompt: Provide a short piece of text likely to appear in the audio. This can help guide the model, especially for specific terminology or names.
• temperature: Controls the randomness of the output (0 to 1). Lower values (closer to 0) make the output more deterministic.

Example with Language and Prompt:#

1
import os
2
from openai import OpenAI
3

4
client = OpenAI()
5

6
audio_file_path = "path/to/your/spanish_audio.wav" # Assuming a Spanish audio file
7

8
if not os.path.exists(audio_file_path):
9
    print(f"Error: File not found at {audio_file_path}")
10
else:
11
    try:
12
        with open(audio_file_path, "rb") as audio_file:
13
            transcript = client.audio.transcriptions.create(
14
                model="whisper-1",
15
                file=audio_file,
16
                language="es", # Specify Spanish
17
                prompt="Keywords related to technology and artificial intelligence." # Example prompt
18
            )
19

20
        print("Transcription (Spanish):")
21
        print(transcript.text)
22

23
    except Exception as e:
24
        print(f"An error occurred: {e}")

Pros of Using the OpenAI API:#

• Ease of Use: Minimal setup; no need to manage model weights or significant local dependencies.
• Scalability: Handles varying workloads without local infrastructure changes.
• Managed Service: OpenAI handles the underlying infrastructure, model updates, and performance optimization.
• Supports Many Formats: Handles a wide range of common audio file types directly.

Cons of Using the OpenAI API:#

• Cost: API usage incurs per-minute costs.
• Latency: Transcription requires sending data over the internet and waiting for the response.
• File Size Limit: Restricted to 25 MB per file, requiring manual segmentation for longer audio.
• Internet Dependency: Requires an active internet connection.
• Data Privacy: Audio data is sent to OpenAI’s servers (though OpenAI has data usage policies).

Prerequisites:#

• A Python installation (3.8 or later recommended).
• The whisper Python library installed.
• torch and torchaudio for handling the model and audio data.
• ffmpeg installed on the system (often required by torchaudio to load various audio formats).
• Sufficient computational resources, especially RAM and potentially a GPU for faster processing, particularly with larger models.

Step-by-Step Guide:#

1. Install the necessary libraries: Use pip. Installing torch with CUDA support (for GPU acceleration) is highly recommended if a compatible GPU is available. Refer to the PyTorch installation guide for specific commands based on the CUDA version.

   1
   # Basic installation (CPU-only for torch)
   2
   pip install openai-whisper torch torchaudio ffmpeg-python
   3
   
   4
   # For GPU support (example for CUDA 11.8) - check PyTorch documentation for correct command
   5
   # pip install openai-whisper torch torchaudio --index-url https://download.pytorch.org/whl/cu118
   6
   # pip install ffmpeg-python
   

   Note: The openai-whisper package name is slightly different from the openai API library.

2. Ensure ffmpeg is installed: Download and install ffmpeg from ffmpeg.org. Ensure the ffmpeg executable is available in your system’s PATH. The ffmpeg-python library provides Python bindings but relies on the underlying ffmpeg executable.

3. Write the Python code: Import the whisper library, load a Whisper model, load the audio file using whisper.load_audio, and transcribe it using model.transcribe.

   1
   import whisper
   2
   import os
   3
   
   4
   # Path to your audio file
   5
   audio_file_path = "path/to/your/audiofile.wav" # .wav is a common format, but ffmpeg allows others
   6
   
   7
   # Ensure the file exists
   8
   if not os.path.exists(audio_file_path):
   9
       print(f"Error: File not found at {audio_file_path}")
   10
   else:
   11
       try:
   12
           # Load a Whisper model
   13
           # Available models: tiny, base, small, medium, large
   14
           # Append '.en' to model name for English-only versions (e.g., 'base.en')
   15
           # English-only models are slightly smaller and faster for English audio
   16
           print("Loading Whisper model...")
   17
           model = whisper.load_model("base") # Choose a model size (e.g., 'base', 'medium', 'large')
   18
           print("Model loaded.")
   19
   
   20
           # Load audio and pad/trim it to fit 30 seconds
   21
           print("Loading audio...")
   22
           audio = whisper.load_audio(audio_file_path)
   23
           print("Audio loaded.")
   24
   
   25
           # Transcribe the audio
   26
           print("Transcribing audio...")
   27
           result = model.transcribe(audio)
   28
           print("Transcription complete.")
   29
   
   30
           print("Transcription:")
   31
           print(result["text"])
   32
   
   33
       except Exception as e:
   34
           print(f"An error occurred: {e}")
   

4. Run the script: Execute the Python file. The first time a model size is used, the weights will be downloaded (typically to ~/.cache/whisper).

   1
   python your_local_whisper_script.py
   

Model Sizes:#

The whisper.load_model() function accepts different model sizes:

• tiny: Fastest, smallest, lowest accuracy.
• base: Faster, smaller than medium/large, moderate accuracy.
• small: Good balance of speed and accuracy.
• medium: Higher accuracy, slower, larger.
• large: Highest accuracy, slowest, largest.

English-only versions (tiny.en, base.en, small.en, medium.en) are available. They are slightly more accurate for English audio and have smaller download sizes.

Pros of Using the Local Library:#

• Offline Capability: Does not require an internet connection after the model is downloaded.
• No Per-Use Cost: Once setup, there are no recurring costs per transcription.
• No File Size Limits: Can process audio files of any length (though large files benefit from segmentation for performance).
• Full Control: Data remains on local infrastructure.
• Customization: Potential for more advanced usage, like accessing intermediate outputs or exploring fine-tuning (though fine-tuning Whisper is complex).

Cons of Using the Local Library:#

• Setup Complexity: Requires installing multiple dependencies, including ffmpeg, and potentially configuring GPU support.
• Hardware Requirements: Demands significant CPU resources and RAM; GPU is strongly recommended for acceptable performance, especially with larger models.
• Performance Variability: Transcription speed depends heavily on local hardware capabilities.
• Maintenance: Requires managing library versions and model weights.
• Initial Download: Requires downloading potentially large model files (e.g., large model is several GB).