# AI-Crash-Course
AI Crash Course to help busy builders catch up to the public frontier of AI research in 2 weeks

**Intro:** I’m [Henry Shi](https://www.linkedin.com/in/henrythe9th/) and I started Super.com in 2016 and grew it to $150MM+ in annual revenues and recently exited. As a traditional software founder, I needed to quickly catch up to the frontier of AI research to figure out where the next opportunities and gaps were. I compiled a list of resources that were essential for me and should get you caught up within 2 weeks.

For more context, checkout the [original twitter thread](https://x.com/henrythe9ths/status/1877056425454719336)

**Start Here:**
[Neural Network \-\> LLM Series](https://www.youtube.com/watch?v=aircAruvnKk&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi)

**Then get up to speed via Survey papers:**

- Follow the ideas in the survey paper that interest you and dig deeper

[LLM Survey](https://arxiv.org/pdf/2402.06196v2) \- 2024
[Agent Survey](https://arxiv.org/pdf/2308.11432) \- 2023
[Prompt Engineering Survey](https://arxiv.org/pdf/2406.06608) \- 2024

**AI Papers:** (prioritize ones with star \*)

**Foundational Modelling:**
[**Transformers**\*](https://arxiv.org/pdf/1706.03762) (foundation, self-attention) \- 2017
[Scaling Laws](https://arxiv.org/pdf/2001.08361)/[**GPT3**\*](https://arxiv.org/pdf/2005.14165) (conviction to scale up GPT2/3/4) \- 2020
[LoRA](https://arxiv.org/abs/2106.09685) (Fine tuning) \- 2021
[Training Compute-Optimal LLMs](https://arxiv.org/pdf/2203.15556) \- 2022
[**RLHF**\*](https://arxiv.org/pdf/2203.02155) (InstructGPT-\>ChatGPT) \- 2022
[DPO](https://arxiv.org/pdf/2305.18290) (No need for RL/Reward model) \- 2023
[LLM-as-Judge](https://arxiv.org/pdf/2306.05685) (On par with human evaluations) \- 2023
[MoE](https://arxiv.org/pdf/2401.04088) (MIxture of Experts) \- 2024

**Planning/Reasoning:**
[AlphaZero](https://arxiv.org/pdf/1712.01815)/[**MuZero**\*](https://arxiv.org/pdf/1911.08265) (RL without prior knowledge of game or rules) \- 2017/2019
[**CoT**\* (Chain of Thought)](https://arxiv.org/pdf/2201.11903)/[ToT (Tree of Thoughts)](https://arxiv.org/pdf/2305.10601)/[GoT (Graph of Thoughts)](https://arxiv.org/pdf/2308.09687)/[Meta CoT](https://arxiv.org/pdf/2501.04682) \- 2022/2023/2023/2025
[ReACT](https://arxiv.org/pdf/2210.03629) (Generate reasoning traces and task-specific actions in interleaved manner) \- 2022
[Let’s Verify Step by Step](https://arxiv.org/pdf/2305.20050) (Process \> Outcome) \- 2023
[**ARC-Prize**\*](https://arxiv.org/pdf/2412.04604) (Latest methods for solving ARC-AGI problems) \- 2024
[**DeepSeek R1**\*](https://arxiv.org/pdf/2501.12948v1) (Building OSS o1-level reasoning model with pure RL, no SFT, no RM) \- 2025

**Applications:**
[Toolformer](https://arxiv.org/pdf/2302.04761) (LLMs to use tools) \- 2023
[GPT4](https://arxiv.org/pdf/2303.08774) (Overview of GPT4, but fairly high level) \- 2023
[**Llama3**\*](https://arxiv.org/pdf/2407.21783) (In depth details of how Meta built Llama3 and the various configurations and hyperparameters) \- 2024
[Gemini1.5](https://arxiv.org/pdf/2403.05530) (Multimodal across 10MM context window) \- 2024
[Deepseekv3](https://github.com/deepseek-ai/DeepSeek-V3/blob/main/DeepSeek_V3.pdf) (Building a frontier OSS model at a fraction of the cost of everyone else) \- 2024
[SWE-Agent](https://arxiv.org/pdf/2405.15793)/[OpenHands](https://arxiv.org/pdf/2407.16741) (OpenSource software development agents) \- 2024

**Benchmarks:**
[BIG-Bench](https://arxiv.org/pdf/2206.04615) (First broad & diverse collaborative OSS benchmark) \- 2022
[SWE-Bench](https://arxiv.org/pdf/2310.06770) (Real world software development) \- 2023
[Chatbot Arena](https://arxiv.org/pdf/2403.04132) (Live human preference Elo ratings) \- 2024

<hr />

**Videos/Lectures:**
[3Blue1Brown on Foundational Math/Concepts](https://www.youtube.com/@3blue1brown)
[Build a Large Language Model (from Scratch) \#1 Bestseller](https://www.amazon.com/Build-Large-Language-Model-Scratch/dp/1633437167)
[Andrej Kaparthy: Zero to Hero Series](https://www.youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ)
[Yannic Kilcher Paper Explanations](https://www.youtube.com/@YannicKilcher)
[Noam Brown (o1 founder) on Planning in AI](https://www.youtube.com/watch?v=eaAonE58sLU)
[Stanford: Building LLMs](https://www.youtube.com/watch?v=9vM4p9NN0Ts)
[Foundations of LLMs](https://arxiv.org/pdf/2501.09223)
[Why You’re Not Too Old to Pivot Into AI](https://www.latent.space/p/not-old) (motivation)

**Helpful Websites:**
[History of Deep Learning](https://github.com/adam-maj/deep-learning?tab=readme-ov-file) \- summary timeline of deeplearning with major breakthroughs and key concepts
[Full Stack Deep Learning](https://fullstackdeeplearning.com/) \- courses for building AI products
[Prompting Guide](https://www.promptingguide.ai/) \- extensive list of prompting techniques and examples
[a16z AI Cannon](https://a16z.com/ai-canon/) \- similar list of resources, but longer (slightly dated)
[2025 AI Engineer Reading List](https://www.latent.space/p/2025-papers) \- longer reading list, broken out by focus area
[State of Generative Models 2024](https://nrehiew.github.io/blog/2024/) \- good simple summary of current state

**Others (non LLMs):**
[Vision Transformer](https://arxiv.org/pdf/2010.11929) (no need for CNNs) \- 2021
[Latent Diffusion](https://arxiv.org/pdf/2112.10752) (Text-to-Image) \- 2021

**Obvious/easy papers (to get your feet wet if you're new to papers):**
[CoT (Chain of Thought)](https://arxiv.org/pdf/2201.11903) \- 2022
[SELF-REFINE: Iterative Refinement with Self-Feedback](https://arxiv.org/pdf/2303.17651) \- 2023

README.md


# Changelog

## [0.4.0] - 2024-11-16

### Added
- Add Google Singlespeaker (Journey) and Multispeaker TTS models
- Fixed limitations of Google Multispeaker TTS model: 5000 bytes input limite and 500 bytes per turn limit.
- Updated tests and docs accordingly

## [0.3.6] - 2024-11-13

### Added
- Add longform podcast generation support
  - Users can now generate longer podcasts (20-30+ minutes) using the `--longform` flag in CLI or `longform=True` in Python API
  - Implements "Content Chunking with Contextual Linking" technique for coherent long-form content
  - Configurable via `max_num_chunks` and `min_chunk_size` parameters in conversation config
  - `word_count` parameter removed from conversation config as it's no longer used

## [0.3.3] - 2024-11-08

### Breaking Changes
- Loading images from 'path' has been removed for security reasons. Please specify images by passing an 'url'.

### Added
- Add podcast generation from topic "Latest News in U.S. Politics"
- Integrate with 100+ LLM models (OpenAI, Anthropic, Google etc) for transcript generation
- Integrate with Google's Multispeaker TTS model for high-quality audio generation
- Deploy [REST API](https://github.com/souzatharsis/podcastfy/blob/main/usage/api.md) with FastAPI
- Support for raw text as input
- Add PRIVACY_POLICY.md
- Start TESTIMONIALS.md
- Add apps using Podcastfy to README.md

### Fixed
- #165 Fixed audio generation in Windows OS issue: Normalize path separators for cross-platform compatibility

## [0.2.3] - 2024-10-15

### Added
- Add local llm option by @souzatharsis
- Enable running podcastfy with no API KEYs thanks to solving #18 #58 #65 by @souzatharsis and @ChinoUkaegbu
- Add user-provided TSS config such as voices #10 #6 #27 by @souzatharsis
- Add open in collab and setting python version to 3.11 by @Devparihar5 #57
- Add edge tts support by @ChinoUkaegbu
- Update pypdf with pymupdf(10x faster then pypdf) #56 check by @Devparihar5
- Replace r.jina.ai with simple BeautifulSoap #18 by @souzatharsis

### Fixed
- Fixed CLI for user-provided config #69 @souzatharsis

## [0.2.2] - 2024-10-13

### Added
- Added API reference docs and published it to https://podcastfy.readthedocs.io/en/latest/

### Fixed
- ([#52](https://github.com/user/podcastfy/issues/37)) Fixed simple bug introduced in 0.2.1 that broke the ability to generate podcasts from text inputs!
- Fixed one example in the documentation that was not working.

## [0.2.1] - 2024-10-12

### Added
- ([#8](https://github.com/user/podcastfy/issues/8)) Podcastfy is now multi-modal! Users can now generate audio from images by simply providing the paths to the image files.

### Fixed
- ([#40](https://github.com/user/podcastfy/issues/37)) Updated default ElevenLabs voice from `BrittneyHart` to `Jessica`. The latter was a non-default voice I used from my account, which caused error for users who don't have it.

## [0.2.0] - 2024-10-10

### Added
- Parameterized podcast generation with Conversation Configuration ([#11](https://github.com/user/podcastfy/issues/11), [#3](https://github.com/user/podcastfy/issues/3), [#4](https://github.com/user/podcastfy/issues/4))
  - Users can now customize podcast style, structure, and content
  - See [Conversation Customization](usage/conversation_custom.md) for detailed options
  - Updated demo in [podcastfy.ipynb](podcastfy.ipynb)
- LangChain integration for improved LLM interface and observability ([#29](https://github.com/user/podcastfy/issues/29))
- Changelog to track version updates ([#22](https://github.com/user/podcastfy/issues/22))
- Tests for Customized conversation scenarios

### Fixed
- CLI now correctly reads from user-provided local .env file ([#37](https://github.com/user/podcastfy/issues/37))

CHANGELOG.md

An Open Source Python alternative to NotebookLM's podcast feature: Transforming Multimodal Content into Captivating Multilingual Audio Conversations with GenAI


# Contributor Covenant Code of Conduct

## Our Pledge

In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation.

## Our Standards

Examples of behavior that contributes to creating a positive environment include:

* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members

Examples of unacceptable behavior by participants include:

* The use of sexualized language or imagery and unwelcome sexual attention or advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a professional setting

## Our Responsibilities

Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.

Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.

## Scope

This Code

CODE_OF_CONDUCT.md


# Contributor Guidelines

Thank you for your interest in contributing to Podcastfy! We welcome contributions from the community to help improve and expand this project. Please follow these guidelines to ensure a smooth collaboration process.

## Getting Started

1. Fork the repository on GitHub.
2. Clone your fork locally: `git clone https://github.com/your-username/podcastfy.git`
3. Create a new branch for your feature or bug fix: `git checkout -b feature/your-feature-name`

## Code Style

- Follow PEP 8 style guidelines for Python code.
- Use tabs for indentation instead of spaces.
- Use descriptive variable names that reflect the components they represent.
- Include docstrings for all functions, classes, and modules.

## Development

- Poetry is the preferred but not mandatory dependency manager. Install it with `pip install poetry`.
    - Contributors can opt to use `uv` instead and generate and push updated requirements.txt from it.
- Sphinx is used as the documentation generator. Install it with `pip install sphinx`.
    - `make doc-gen` to generate the documentation.

## Submitting Changes

1. Commit your changes with clear, descriptive commit messages.
2. Push your changes to your fork on GitHub.
3. Submit a pull request to the main repository.

## Pre-Pull Request Checklist

1. Managing dependencies
    - Add new dependencies with `poetry add <new-dependency>`
    - Remove a dependency with `poetry remove <dependency-name>`.
    - Then generate requirements.txt with `poetry export -f requirements.txt --output requirements.txt --without-hashes`
2. Testing
    - Consider adding new tests at test/*.py, particularly if implementing user facing change.
    - Test locally: `poetry run pytest`
    - Tests (tests/*.py) are run automatically by GitHub Actions, double check that they pass.
3. Docs
    - Update any documentation if required README.md, usage/*.md, *.ipynb etc.
    - Regenerate documentation (/docs) if there are any changes in docstrings or modules' interface (`make doc-gen`)

## Reporting Issues

- Use the GitHub issue tracker to report bugs or suggest enhancements.
- Provide a clear and detailed description of the issue or suggestion.
- Include steps to reproduce the bug, if applicable.

## Code of Conduct

Please note that this project is released with a [Contributor Code of Conduct](CODE_OF_CONDUCT.md). By participating in this project, you agree to abide by its terms.

## Questions?

If you have any questions or need further clarification, please don't hesitate to ask in the GitHub issues section.

Thank you for contributing to Podcastfy!

GUIDELINES.md


# Privacy Policy

**Effective Date:** 11/03/2024

Podcastfy is an open-source project that does not collect, store, or transmit any personal user data. All processing occurs locally on your machine or through third-party services that you configure.

## Use of Third-Party Services

When you use Podcastfy with third-party services (such as APIs for text-to-speech or language models), any data transmitted to these services is subject to their respective privacy policies. You are responsible for reviewing and agreeing to the terms and policies of these third-party providers.

## Data Processing

- **Local Processing:** All content transformation and processing are performed locally unless explicitly configured to use external services.
- **No Data Collection:** Podcastfy does not collect or send any user data to the developers or any third parties without your consent.

## User Responsibility

Users are responsible for:

- Ensuring compliance with all applicable laws and regulations regarding data privacy.
- Protecting any personal or sensitive data processed through the application.
- Reviewing the privacy policies of any third-party services used in conjunction with Podcastfy.

## Contact Information

If you have any questions or concerns about this Privacy Policy, please open an issue on our [GitHub repository](https://github.com/souzatharsis/podcastfy/issues).

PRIVACY_POLICY.md


# Podcastfy.ai 🎙🤖
An Open Source API alternative to NotebookLM's podcast feature: Transforming Multimodal Content into Captivating Multilingual Audio Conversations with GenAI

https://github.com/user-attachments/assets/5d42c106-aabe-44c1-8498-e9c53545ba40

[Paper](https://github.com/souzatharsis/podcastfy/blob/main/paper/paper.pdf) |
[Python Package](https://github.com/souzatharsis/podcastfy/blob/59563ee105a0d1dbb46744e0ff084471670dd725/podcastfy.ipynb) |
[CLI](https://github.com/souzatharsis/podcastfy/blob/59563ee105a0d1dbb46744e0ff084471670dd725/usage/cli.md) |
[Web App](https://openpod.fly.dev/) |
[Feedback](https://github.com/souzatharsis/podcastfy/issues)

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/souzatharsis/podcastfy/blob/main/podcastfy.ipynb)
[![PyPi Status](https://img.shields.io/pypi/v/podcastfy)](https://pypi.org/project/podcastfy/)
![PyPI Downloads](https://static.pepy.tech/badge/podcastfy)
[![Issues](https://img.shields.io/github/issues-raw/souzatharsis/podcastfy)](https://github.com/souzatharsis/podcastfy/issues)
[![Pytest](https://github.com/souzatharsis/podcastfy/actions/workflows/python-app.yml/badge.svg)](https://github.com/souzatharsis/podcastfy/actions/workflows/python-app.yml)
[![Docker](https://github.com/souzatharsis/podcastfy/actions/workflows/docker-publish.yml/badge.svg)](https://github.com/souzatharsis/podcastfy/actions/workflows/docker-publish.yml)
[![Documentation Status](https://readthedocs.org/projects/podcastfy/badge/?version=latest)](https://podcastfy.readthedocs.io/en/latest/?badge=latest)
[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
![GitHub Repo stars](https://img.shields.io/github/stars/souzatharsis/podcastfy)

Podcastfy is an open-source Python package that transforms multi-modal content (text, images) into engaging, multi-lingual audio conversations using GenAI. Input content includes websites, PDFs, images, YouTube videos, as well as user provided topics.

Unlike closed-source UI-based tools focused primarily on research synthesis (e.g. NotebookLM ❤), Podcastfy focuses on open source, programmatic and bespoke generation of engaging, conversational content from a multitude of multi-modal sources, enabling customization and scale.

## Testimonials 💬

> "Love that you casually built an open source version of the most popular product Google built in the last decade"

> "Loving this initiative and the best I have seen so far especially for a 'non-techie' user."

> "Your library was very straightforward to work with. You did Amazing work brother 🙏"

> "I think it's awesome that you were inspired/recognize how hard it is to beat NotebookLM's quality, but you did an *incredible* job with this! It sounds incredible, and it's open-source! Thank you for being amazing!"

[![Star History Chart](https://api.star-history.com/svg?repos=souzatharsis/podcastfy&type=Date&theme=dark)](https://api.star-history.com/svg?repos=souzatharsis/podcastfy&type=Date&theme=dark)

## Audio Examples 🔊
This sample collection was generated using this [Python Notebook](usage/examples.ipynb).

### Images
Sample 1: Senecio, 1922 (Paul Klee) and Connection of Civilizations (2017) by Gheorghe Virtosu
***

<video src="https://github.com/user-attachments/assets/a4134a0d-138c-4ab4-bc70-0f53b3507e6b"></video>
***
Sample 2: The Great Wave off Kanagawa, 1831 (Hokusai) and Takiyasha the Witch and the Skeleton Spectre, c. 1844 (Kuniyoshi)
***

<video src="https://github.com/user-attachments/assets/f6aaaeeb-39d2-4dde-afaf-e2cd212e9fed"></video>
***
Sample 3: Pop culture icon Taylor Swift and Mona Lisa, 1503 (Leonardo da Vinci)
***

<video src="https://github.com/user-attachments/assets/3b6f7075-159b-4540-946f-3f3907dffbca"></video>

### Text
| Audio | Description  | Source |
|-------|--|--------|
| <video src="https://github.com/user-attachments/assets/ef41a207-a204-4b60-a11e-06d66a0fbf06"></video>  | Personal Website | [Website](https://www.souzatharsis.com) |
| [Audio](https://soundcloud.com/high-lander123/amodei?in=high-lander123/sets/podcastfy-sample-audio-longform&si=b8dfaf4e3ddc4651835e277500384156) (`longform=True`) | Lex Fridman Podcast: 5h interview with Dario Amodei Anthropic's CEO |  [Youtube](https://www.youtube.com/watch?v=ugvHCXCOmm4) |
| [Audio](https://soundcloud.com/high-lander123/benjamin?in=high-lander123/sets/podcastfy-sample-audio-longform&si=dca7e2eec1c94252be18b8794499959a&utm_source=clipboard&utm_medium=text&utm_campaign=social_sharing) (`longform=True`)| Benjamin Franklin's Autobiography | [Book](https://www.gutenberg.org/cache/epub/148/pg148.txt) |

### Multi-Lingual Text
| Language | Content Type | Description | Audio | Source |
|----------|--------------|-------------|-------|--------|
| French | Website | Agroclimate research information | [Audio](https://audio.com/thatupiso/audio/podcast-fr-agro) | [Website](https://agroclim.inrae.fr/) |
| Portuguese-BR | News Article | Election polls in São Paulo | [Audio](https://audio.com/thatupiso/audio/podcast-thatupiso-br) | [Website](https://noticias.uol.com.br/eleicoes/2024/10/03/nova-pesquisa-datafolha-quem-subiu-e-quem-caiu-na-disputa-de-sp-03-10.htm) |

## Quickstart 💻

### Prerequisites
- Python 3.11 or higher
- `$ pip install ffmpeg` (for audio processing)

### Setup
1. Install from PyPI
  `$ pip install podcastfy`

2. Set up your [API keys](usage/config.md)

### Python
```python
from podcastfy.client import generate_podcast

audio_file = generate_podcast(urls=["<url1>", "<url2>"])
```
### CLI
```
python -m podcastfy.client --url <url1> --url <url2>
```

## Usage 💻

- [Python Package Quickstart](podcastfy.ipynb)

- [How to](usage/how-to.md)

- [Python Package Reference Manual](https://podcastfy.readthedocs.io/en/latest/podcastfy.html)

- [CLI](usage/cli.md)

## Customization 🔧

Podcastfy offers a range of customization options to tailor your AI-generated podcasts:
- Customize podcast [conversation](usage/conversation_custom.md) (e.g. format, style, voices)
- Choose to run [Local LLMs](usage/local_llm.md) (156+ HuggingFace models)
- Set other [Configuration Settings](usage/config.md)

## Features ✨

- Generate conversational content from multiple sources and formats (images, text, websites, YouTube, and PDFs).
- Generate shorts (2-5 minutes) or longform (30+ minutes) podcasts.
- Customize transcript and audio generation (e.g., style, language, structure).
- Generate transcripts using 100+ LLM models (OpenAI, Anthropic, Google etc).
- Leverage local LLMs for transcript generation for increased privacy and control.
- Integrate with advanced text-to-speech models (OpenAI, Google, ElevenLabs, and Microsoft Edge).
- Provide multi-language support for global content creation.
- Integrate seamlessly with CLI and Python packages for automated workflows.

## Built with Podcastfy 🚀

- [OpenNotebook](https://www.open-notebook.ai/)
- [SurfSense](https://www.surfsense.net/)
- [OpenPod](https://openpod.fly.dev/)
- [Podcast-llm](https://github.com/evandempsey/podcast-llm)
- [Podcastfy-HuggingFace App](https://huggingface.co/spaces/thatupiso/Podcastfy.ai_demo)

## Updates 🚀🚀

### v0.4.0+ release
- Released new Multi-Speaker TTS model (is it the one NotebookLM uses?!?)
- Generate short or longform podcasts
- Generate podcasts from input topic using grounded real-time web search
- Integrate with 100+ LLM models (OpenAI, Anthropic, Google etc) for transcript generation

See [CHANGELOG](CHANGELOG.md) for more details.

## License

This software is licensed under [Apache 2.0](LICENSE). See [instructions](usage/license-guide.md) if you would like to use podcastfy in your software.

## Contributing 🤝

We welcome contributions! See [Guidelines](GUIDELINES.md) for more details.

## Example Use Cases 🎧🎶

- **Content Creators** can use `Podcastfy` to convert blog posts, articles, or multimedia content into podcast-style audio, enabling them to reach broader audiences. By transforming content into an audio format, creators can cater to users who prefer listening over reading.

- **Educators** can transform lecture notes, presentations, and visual materials into audio conversations, making educational content more accessible to students with different learning preferences. This is particularly beneficial for students with visual impairments or those who have difficulty processing written information.

- **Researchers** can convert research papers, visual data, and technical content into conversational audio. This makes it easier for a wider audience, including those with disabilities, to consume and understand complex scientific information. Researchers can also create audio summaries of their work to enhance accessibility.

- **Accessibility Advocates** can use `Podcastfy` to promote digital accessibility by providing a tool that converts multimodal content into auditory formats. This helps individuals with visual impairments, dyslexia, or other disabilities that make it challenging to consume written or visual content.

## Contributors

        ↑ Back to Top ↑

README.md


- "Love that you casually built an open source version of the most popular product Google built in the last decade"
- "Your library was very straightforward to work with. You did Amazing work brother 🙏"
- "I think it's awesome that you were inspired/recognize how hard it is to beat NotebookLM's quality, but you did an *incredible* job with this! It sounds incredible, and it's open-source! Thank you for being amazing!"
- "Discovered your work last night. Stunning accomplishment. Well done."
- "Loving this initiative and the best I have seen so far especially for a "non-techie" user."

TESTIMONIALS.md


---
title: 'When Content Speaks Volumes: Podcastfy — An Open Source Python Package Bridging Multimodal Data and Conversational Audio with GenAI'
tags:
  - Python
  - generative AI
  - GenAI
  - text-to-speech
  - large language models
  - content transformation
  - accessibility
authors:
  - name: Tharsis T. P. Souza
    orcid: 0000-0003-3260-9526
    affiliation: "1, 2"
affiliations:
  - name: Columbia University in the City of New York
    index: 1
  - name: Instituto Federal de Educacao, Ciencia e Tecnologia do Sul de Minas (IFSULDEMINAS)
    index: 2
date: 11/03/2024
bibliography: paper.bib
---

# Abstract

`Podcastfy` is an open-source Python framework that programmatically transforms multisourced, multimodal content into multilingual, natural-sounding audio conversations using generative AI. By converting various types of digital content - including images, websites, YouTube videos, and PDFs - into conversational audio formats, `Podcastfy` enhances accessibility, engagement, and usability for a wide range of users. As an open-source project, `Podcastfy` benefits from continuous community-driven improvements, enhancing its adaptability to evolving user requirements and accessibility standards.

# Statement of Need

The rapid expansion of digital content across various formats has intensified the need for tools capable of converting diverse information into accessible and digestible forms [@johnson2023adaptive; @chen2023digital; @mccune2023accessibility]. Existing solutions often fall short due to their proprietary nature, limited multimodal support, or inadequate accessibility features [@marcus2019design; @peterson2023web; @gupta2023advances].

`Podcastfy` addresses this gap with an open-source solution that supports multimodal input processing and generates natural-sounding, summarized conversational content. Leveraging advances in large language models (LLMs) and text-to-speech (TTS) synthesis, `Podcastfy` aims to benefit a diverse group of users — including content creators, educators, researchers, and accessibility advocates — by providing a customizable solution that transforms digital content into multilingual textual and auditory formats, enhancing accessibility and engagement.

# Features

- Generate conversational content from multiple sources and formats (images, websites, YouTube, and PDFs).
- Customize transcript and audio generation (e.g., style, language, structure, length).
- Create podcasts from pre-existing or edited transcripts.
- Leverage cloud-based and local LLMs for transcript generation (increased privacy and control).
- Integrate with advanced text-to-speech models (OpenAI, ElevenLabs, and Microsoft Edge).
- Provide multi-language support for global content creation and enhanced accessibility.
- Integrate seamlessly with CLI and Python packages for automated workflows.

See [audio samples](https://github.com/souzatharsis/podcastfy?tab=readme-ov-file#audio-examples-).

# Use Cases

`Podcastfy` is designed to serve a wide range of applications, including:

- **Content Creators** can use `Podcastfy` to convert blog posts, articles, or multimedia content into podcast-style audio, enabling them to reach broader audiences. By transforming content into an audio format, creators can cater to users who prefer listening over reading.

- **Educators** can transform lecture notes, presentations, and visual materials into audio conversations, making educational content more accessible to students with different learning preferences. This is particularly beneficial for students with visual impairments or those who have difficulty processing written information.

- **Researchers** can convert research papers, visual data, and technical content into conversational audio. This makes it easier for a wider audience, including those with disabilities, to consume and understand complex scientific information. Researchers can also create audio summaries of their work to enhance accessibility.

- **Accessibility Advocates** can use `Podcastfy` to promote digital accessibility by providing a tool that converts multimodal content into auditory formats. This helps individuals with visual impairments, dyslexia, or other disabilities that make it challenging to consume written or visual content.

# Implementation and Architecture

`Podcastfy` implements a modular architecture designed for flexibility and extensibility through five main components, as shown in Figure 1.

1. **Client Interface**
   - Provides both CLI (Command-Line Interface) and API interfaces.
   - Coordinates the workflow between processing layers.
   - Implements a unified interface for podcast generation through the `generate_podcast()` method.

2. **Configuration Management**
   - Offers extensive customization options through a dedicated module.
   - Manages system settings and user preferences, such as podcast name, language, style, and structure.
   - Controls the behavior of all processing layers.

3. **Content Extraction Layer**
   - Extracts content from various sources, including websites, PDFs, and YouTube videos.
   - The `ContentExtractor` class coordinates three specialized extractors:
     - `PDFExtractor`: Handles PDF document processing.
     - `WebsiteExtractor`: Manages website content extraction.
     - `YouTubeTranscriber`: Processes YouTube video content.
   - Serves as the entry point for all input types, providing standardized text output to the transcript generator.

4. **LLM-based Transcript Generation Layer**
   - Uses large language models to generate natural-sounding conversations from extracted content.
   - The `ContentGenerator` class manages conversation generation using different LLM backends:
     - Integrates with LangChain to implement prompt management and common LLM access through the `BaseChatModel` interface.
     - Supports both local (`LlamaFile`) and cloud-based models.
     - Uses `ChatGoogleGenerativeAI` for cloud-based LLM services.
   - Allows customization of conversation style, roles, and dialogue structure.
   - Outputs structured conversations in text format.

5. **Text-to-Speech (TTS) Layer**
   - Converts input transcripts into audio using various TTS models.
   - The `TextToSpeech` class implements a factory pattern:
     - The `TTSFactory` creates appropriate providers based on configuration.
     - Supports multiple backends (OpenAI, ElevenLabs, and Microsoft Edge) through the `TTSProvider` interface.
   - Produces the final podcast audio output.

![Podcastfy's simplified architecture and workflow diagram showing the main components and their interactions.](podcastfy.png){width=80%}

The modular architecture enables independent development and maintenance of each component. This pipeline design ensures a clean separation of concerns while maintaining seamless data transformation between stages. This modular approach also facilitates easy updates and extensions to individual components without affecting the rest of the system.

The framework is offered as a Python package, with a command-line interface as well as a REST API, making it accessible to users with different technical backgrounds and requirements.

# Quick Start

## Prerequisites
- Python 3.11 or higher
- `$ pip install ffmpeg` (for audio processing)

## Setup
1. Install from PyPI
  `$ pip install podcastfy`

2. Set up [API keys](usage/config.md)

## Python
```python
from podcastfy.client import generate_podcast

audio_file = generate_podcast(urls=["<url1>", "<url2>"])
```
## CLI
```
python -m podcastfy.client --url <url1> --url <url2>
```

# Customization Examples

`Podcastfy` offers various customization options that make it versatile for different types of content transformation. To accomplish that, we leverage LangChain's [@langchain2024] prompt management capabilities to dynamically construct prompts for the LLM, adjusting conversation characteristics such as style, roles, and dialogue structure. Below are some examples that demonstrate its capabilities.

## Academic Debate

The following Python code demonstrates how to configure `Podcastfy` for an academic debate:

```python
from podcastfy import generate_podcast

debate_config = {
    "conversation_style": ["formal", "debate"],
    "roles_person1": "main presenter",
    "roles_person2": "opposing viewpoint",
    "dialogue_structure": ["Introduction", "Argument Presentation", "Counterarguments", "Conclusion"]
}

generate_podcast(
    urls=["PATH/TO/academic-article.pdf"],
    conversation_config=debate_config
)
```

In this example, the roles are set to "main presenter" and "opposing viewpoint" to simulate an academic debate between two speakers on a chosen topic. This approach is especially useful for educational content that aims to present multiple perspectives on a topic. The output is structured with clear sections such as introduction, argument presentation, counterarguments, and conclusion, allowing listeners to follow complex ideas easily.

## Technical Tutorial

In this example, the configuration is optimized for creating technical tutorial content.

```python
tutorial_config = {
    "word_count": 2500,
    "conversation_style": ["instructional", "step-by-step"],
    "roles_person1": "expert developer",
    "roles_person2": "learning developer",
    "dialogue_structure": [
        "Concept Introduction",
        "Technical Background",
        "Implementation Steps",
        "Common Pitfalls",
        "Best Practices"
    ],
    "engagement_techniques": [
        "code examples",
        "real-world applications",
        "troubleshooting tips"
    ],
    "creativity": 0.4
}

generate_podcast(
    urls=["https://tech-blog.com/tutorial"],
    conversation_config=tutorial_config
)
```

The roles are set to "expert developer" and "learning developer" to create a natural teaching dynamic. The dialogue structure follows a logical progression from concept introduction through implementation and best practices. The engagement_techniques parameter ensures the content remains practical and applicable by incorporating code examples, real-world applications, and troubleshooting guidance. A moderate creativity setting (0.4) maintains technical accuracy while allowing for engaging explanations and examples.

## Storytelling Adventure

The following Python code demonstrates how to generate a storytelling podcast:

```python
from podcastfy import generate_podcast

story_config = {
    "conversation_style": ["adventurous", "narrative"],
    "creativity": 1.0,
    "roles_person1": "narrator",
    "roles_person2": "character",
    "dialogue_structure": ["Introduction", "Adventure Begins", "Challenges", "Resolution"]
}

generate_podcast(
    urls=["SAMPLE/WWW.URL.COM"],
    conversation_config=story_config
)
```

In this example, `Podcastfy` creates an engaging story by assigning roles like "narrator" and "character" and adjusting the creativity parameter for richer descriptions. Using this configuration, `Podcastfy` can generate engaging narrative content. By adjusting the creativity parameter, `Podcastfy` can create a story involving multiple characters, unexpected plot twists, and rich descriptions.

## Additional Examples

### Daily News Briefing
```python
news_config = {
    "word_count": 1500,
    "conversation_style": ["concise", "informative"],
    "podcast_name": "Morning Briefing",
    "dialogue_structure": [
        "Headlines",
        "Key Stories",
        "Market Update",
        "Weather"
    ],
    "roles_person1": "news anchor",
    "roles_person2": "field reporter",
    "creativity": 0.3
}

generate_podcast(
    urls=[
        "https://news-source.com/headlines",
        "https://market-updates.com/today"
    ],
    conversation_config=news_config
)
```

### Language Learning Content
```python
language_config = {
    "output_language": "Spanish",
    "word_count": 1000,
    "conversation_style": ["educational", "casual"],
    "engagement_techniques": [
        "vocabulary explanations",
        "cultural context",
        "pronunciation tips"
    ],
    "roles_person1": "language teacher",
    "roles_person2": "curious student",
    "creativity": 0.6
}

generate_podcast(
    urls=["https://spanish-content.com/article"],
    conversation_config=language_config
)
```

## Working with Podcastfy Modules

`Podcastfy`'s components are designed to work independently, allowing flexibility in updating or extending each module. The data flows from the `ContentExtractor` module to `ContentGenerator` and finally to the `TexttoSpeech` converter, ensuring a seamless transformation of multimodal content into audio. In this section, we provide some examples of how to use each module.

## Content Extraction
Podcastfy's `content_extractor.py` module allows users to extract content from a given URL, which can be processed further to generate a podcast. Below is an example of how to use the content extraction component:

```python
from podcastfy.content_extractor import ContentExtractor

# Initialize the content extractor
extractor = ContentExtractor()

# Extract content from a URL
url = "https://example.com/article"
extracted_content = extractor.extract_content(url)

print("Extracted Content:")
print(extracted_content)
```

This example demonstrates how to extract text from a given URL. The extracted content is then passed to the next stages of processing.

## Content Generation

The `content_generator.py` module is responsible for generating conversational content based on textual input. Below is an example of how to use the content generation component:

```python
from podcastfy.content_generator import ContentGenerator

# Initialize the content generator
generator = ContentGenerator(api_key="<GEMINI_API_KEY>")

# Generate conversational content
input_text = "This is a sample input text about artificial intelligence."
generated_conversation = generator.generate_conversation(input_text)

print("Generated Conversation:")
print(generated_conversation)
```

 Users can opt to run a cloud-based LLM (Gemini) or run a local (potentially Open Source) LLM model ([see local llm configuration](https://github.com/souzatharsis/podcastfy/blob/main/usage/local_llm.md)).

## Text-to-Speech Conversion

The `text_to_speech.py` module allows the generated transcript to be converted into audio. Below is an example of how to use the text-to-speech component:

```python
from podcastfy.text_to_speech import TextToSpeech

# Initialize the text-to-speech converter
tts = TextToSpeech(model='elevenlabs', api_key="<ELEVENLABS_API_KEY>")

# Convert the generated conversation to speech
input_text = "<Person1>This is a sample conversation generated by Podcastfy.</Person1><Person2>That's great!</Person2>"
output_audio_file = "output_podcast.mp3"
tts.convert_to_speech(input_text, output_audio_file)

print(f"Audio saved to {output_audio_file}")
```

This example demonstrates how to use the `TextToSpeech` class to convert generated text into an audio file. Users can specify different models for TTS, such as `elevenlabs`, `openai`, or `edge` (free to use).

# Limitations

`Podcastfy` has several limitations, including:

- **Content Accuracy and Quality**
  - The accuracy of generated conversations depends heavily on the capabilities of the underlying LLMs.
  - Complex technical or domain-specific content may not always be accurately interpreted or summarized.
  - The framework cannot guarantee the factual correctness of generated content, requiring human verification for critical applications.

- **Language Support Constraints**
  - While multilingual support is available, performance may vary significantly across different languages.
  - Less common languages may have limited TTS voice options and lower-quality speech synthesis.
  - Nuanced cultural contexts and idioms may not translate effectively across languages.

- **Technical Dependencies**
  - Reliance on third-party APIs (OpenAI, ElevenLabs, Google) introduces potential service availability risks.
  - Local LLM options, while providing independence, require significant computational resources.
  - Network connectivity is required for cloud-based services, limiting offline usage.

- **Content Extraction Challenges**
  - Complex webpage layouts or dynamic content may not be accurately extracted.
  - PDF extraction quality depends on document formatting and structure.
  - YouTube video processing depends on the availability of transcripts.

- **Accessibility Considerations**
  - Generated audio may not fully meet all accessibility standards.
  - Limited support for real-time content processing.
  - May require additional processing for users with specific accessibility needs.

These limitations highlight areas for future development and improvement of the framework. Users should carefully consider these constraints when implementing `Podcastfy` for their specific use cases and requirements.

# Limitations

`Podcastfy` faces several key limitations in its current implementation. The accuracy and quality of generated content heavily depends on the underlying LLMs, with complex technical content potentially being misinterpreted. Additionally, while multilingual support is available, performance varies across languages, with less common languages having limited TTS voice options. The framework also relies on third-party APIs which introduces service availability risks, and local LLM options require significant computational resources.

These limitations highlight areas for future development and improvement of the framework. Users should carefully consider these constraints when implementing `Podcastfy` for their specific use cases and requirements.

# Conclusion

`Podcastfy` contributes to multimodal content accessibility by enabling the programmatic transformation of digital content into conversational audio. The framework addresses accessibility needs through automated content summarization and natural-sounding speech synthesis. Its modular design and configurable options allow for flexible content processing and audio generation workflows that can be adapted for different use cases and requirements.

We invite contributions from the community to further enhance the capabilities of `Podcastfy`. Whether it's by adding support for new input modalities, improving the quality of conversation generation, or optimizing the TTS synthesis, we welcome collaboration to make `Podcastfy` more powerful and versatile.

# Acknowledgements

We acknowledge the open-source community and the developers of the various libraries and tools that make `Podcastfy` possible. Special thanks to the developers of LangChain, Llamafile and HuggingFace. We are particularly grateful to all our [contributors](https://github.com/souzatharsis/podcastfy/graphs/contributors) who have helped improve this project.

# References

paper/paper.md


* NotebookLM by Google
* Storm by Stanford University
* Open Notebook by @lf
* Open NotebookLM
* podlm.ai
* notebooklm.ai

paper/related-work.md


# Podcastfy REST API Documentation

## Overview

The Podcastfy API allows you to programmatically generate AI podcasts from various input sources. This document outlines the API endpoints and their usage.

## Using cURL with Podcastfy API

### Prerequisites
1. Confirm cURL installation:
```bash
curl --version
```

### API Request Flow
Making a prediction requires two sequential requests:
1. POST request to initiate processing - returns an `EVENT_ID`
2. GET request to fetch results - uses the `EVENT_ID` to fetch results

Between step 1 and 2, there is a delay of 1-3 minutes. We are working on reducing this delay and implementing a way to notify the user when the podcast is ready. Thanks for your patience!

### Basic Request Structure
```bash
# Step 1: POST request to initiate processing
# Make sure to include http:// or https:// in the URL
curl -X POST https://thatupiso-podcastfy-ai-demo.hf.space/gradio_api/call/process_inputs \
  -H "Content-Type: application/json" \
  -d '{
    "data": [
      "text_input",
      "https://yourwebsite.com",
      [],  # pdf_files
      [],  # image_files
      "gemini_key",
      "openai_key",
      "elevenlabs_key",
      2000,  # word_count
      "engaging,fast-paced",  # conversation_style
      "main summarizer",  # roles_person1
      "questioner",  # roles_person2
      "Introduction,Content,Conclusion",  # dialogue_structure
      "PODCASTFY",  # podcast_name
      "YOUR PODCAST",  # podcast_tagline
      "openai",  # tts_model
      0.7,  # creativity_level
      ""  # user_instructions
    ]
  }'

# Step 2: GET request to fetch results
curl -N https://thatupiso-podcastfy-ai-demo.hf.space/gradio_api/call/process_inputs/$EVENT_ID

# Example output result
event: complete
data: [{"path": "/tmp/gradio/bcb143f492b1c9a6dbde512557541e62f090bca083356be0f82c2e12b59af100/podcast_81106b4ca62542f1b209889832a421df.mp3", "url": "https://thatupiso-podcastfy-ai-demo.hf.space/gradio_a/gradio_api/file=/tmp/gradio/bcb143f492b1c9a6dbde512557541e62f090bca083356be0f82c2e12b59af100/podcast_81106b4ca62542f1b209889832a421df.mp3", "size": null, "orig_name": "podcast_81106b4ca62542f1b209889832a421df.mp3", "mime_type": null, "is_stream": false, "meta": {"_type": "gradio.FileData"}}]

```

You can download the file by extending the URL prefix "https://thatupiso-podcastfy-ai-demo.hf.space/gradio_a/gradio_api/file=" with the path to the file in variable `path`. (Note: The variable "url" above has a bug introduced by Gradio, so please ignore it.)

### Parameter Details
| Index | Parameter | Type | Description |
|-------|-----------|------|-------------|
| 0 | text_input | string | Direct text input for podcast generation |
| 1 | urls_input | string | URLs to process (include http:// or https://) |
| 2 | pdf_files | array | List of PDF files to process |
| 3 | image_files | array | List of image files to process |
| 4 | gemini_key | string | Google Gemini API key |
| 5 | openai_key | string | OpenAI API key |
| 6 | elevenlabs_key | string | ElevenLabs API key |
| 7 | word_count | number | Target word count for podcast |
| 8 | conversation_style | string | Conversation style descriptors (e.g. "engaging,fast-paced") |
| 9 | roles_person1 | string | Role of first speaker |
| 10 | roles_person2 | string | Role of second speaker |
| 11 | dialogue_structure | string | Structure of dialogue (e.g. "Introduction,Content,Conclusion") |
| 12 | podcast_name | string | Name of the podcast |
| 13 | podcast_tagline | string | Podcast tagline |
| 14 | tts_model | string | Text-to-speech model ("gemini", "openai", "elevenlabs", or "edge") |
| 15 | creativity_level | number | Level of creativity (0-1) |
| 16 | user_instructions | string | Custom instructions for generation |

## Using Python

### Installation

```bash
pip install gradio_client
```

### Quick Start

```python
from gradio_client import Client, handle_file

client = Client("thatupiso/Podcastfy.ai_demo")
```

### API Endpoints

#### Generate Podcast (`/process_inputs`)

Generates a podcast from provided text, URLs, PDFs, or images.

##### Parameters

| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| text_input | str | Yes | - | Raw text input for podcast generation |
| urls_input | str | Yes | - | Comma-separated URLs to process |
| pdf_files | List[filepath] | Yes | None | List of PDF files to process |
| image_files | List[filepath] | Yes | None | List of image files to process |
| gemini_key | str | No | "" | Google Gemini API key |
| openai_key | str | No | "" | OpenAI API key |
| elevenlabs_key | str | No | "" | ElevenLabs API key |
| word_count | float | No | 2000 | Target word count for podcast |
| conversation_style | str | No | "engaging,fast-paced,enthusiastic" | Conversation style descriptors |
| roles_person1 | str | No | "main summarizer" | Role of first speaker |
| roles_person2 | str | No | "questioner/clarifier" | Role of second speaker |
| dialogue_structure | str | No | "Introduction,Main Content Summary,Conclusion" | Structure of dialogue |
| podcast_name | str | No | "PODCASTFY" | Name of the podcast |
| podcast_tagline | str | No | "YOUR PERSONAL GenAI PODCAST" | Podcast tagline |
| tts_model | Literal['openai', 'elevenlabs', 'edge'] | No | "openai" | Text-to-speech model |
| creativity_level | float | No | 0.7 | Level of creativity (0-1) |
| user_instructions | str | No | "" | Custom instructions for generation |

##### Returns

| Type | Description |
|------|-------------|
| filepath | Path to generated audio file |

##### Example Usage

```python
from gradio_client import Client, handle_file

client = Client("thatupiso/Podcastfy.ai_demo")

# Generate podcast from URL
result = client.predict(
    text_input="",
    urls_input="https://example.com/article",
    pdf_files=[],
    image_files=[],
    gemini_key="your-gemini-key",
    openai_key="your-openai-key",
    word_count=1500,
    conversation_style="casual,informative",
    podcast_name="Tech Talk",
    tts_model="openai",
    creativity_level=0.8
)

print(f"Generated podcast: {result}")
```

### Error Handling

The API will return appropriate error messages for:
- Invalid API keys
- Malformed input
- Failed file processing
- TTS generation errors

### Rate Limits

Please be aware of the rate limits for the underlying services:
- Gemini API
- OpenAI API
- ElevenLabs API

## Notes

- At least one input source (text, URL, PDF, or image) must be provided
- API keys are required for corresponding services
- The generated audio file format is MP3

usage/api.md


## CLI

Podcastfy can be used as a command-line interface (CLI) tool. See below some usage examples.
Please make sure you follow configuration instructions first - [See Setup](README.md#setup).

1. Generate a podcast from URLs (using OpenAI TTS by default):
   ```
   python -m podcastfy.client --url https://example.com/article1 --url https://example.com/article2
   ```

2. Generate a podcast from URLs using ElevenLabs TTS:
   ```
   python -m podcastfy.client --url https://example.com/article1 --url https://example.com/article2 --tts-model elevenlabs
   ```

3. Generate a podcast from a file containing URLs:
   ```
   python -m podcastfy.client --file path/to/urls.txt
   ```

4. Generate a podcast from an existing transcript file:
   ```
   python -m podcastfy.client --transcript path/to/transcript.txt
   ```

5. Generate only a transcript (without audio) from URLs:
   ```
   python -m podcastfy.client --url https://example.com/article1 --transcript-only
   ```

6. Generate a podcast using a combination of URLs and a file:
   ```
   python -m podcastfy.client --url https://example.com/article1 --file path/to/urls.txt
   ```

7. Generate a podcast from image files:
   ```
   python -m podcastfy.client --image path/to/image1.jpg --image path/to/image2.png
   ```

8. Generate a podcast with a custom conversation configuration:
   ```
   python -m podcastfy.client --url https://example.com/article1 --conversation-config path/to/custom_config.yaml
   ```

9. Generate a podcast from URLs and images:
   ```
   python -m podcastfy.client --url https://example.com/article1 --image path/to/image1.jpg
   ```

10. Generate a transcript using a local LLM:
    ```
    python -m podcastfy.client --url https://example.com/article1 --transcript-only --local
    ```

11. Generate a podcast from raw text input:
    ```
    python -m podcastfy.client --text "Your raw text content here that you want to convert into a podcast"
    ```

12. Generate a longform podcast from URLs:
    ```
    python -m podcastfy.client --url https://example.com/article1 --url https://example.com/article2 --longform
    ```

For more information on available options, use:
   ```
   python -m podcastfy.client --help
   ```

usage/cli.md


# Podcastfy Configuration

## API keys

The project uses a combination of a `.env` file for managing API keys and sensitive information, and a `config.yaml` file for non-sensitive configuration settings. Follow these steps to set up your configuration:

1. Create a `.env` file in the root directory of the project.
2. Add your API keys and other sensitive information to the `.env` file. For example:

   ```
   GEMINI_API_KEY=your_gemini_api_key_here
   ELEVENLABS_API_KEY=your_elevenlabs_api_key_here
   OPENAI_API_KEY=your_openai_api_key_here
   ```

## API Key Requirements

The API Keys required depend on the model you are using for transcript generation and audio generation.

- Transcript generation (LLMs):

  - By default, Podcastfy uses Google's `gemini-1.5-pro-latest` model. Hence, you need to set `GEMINI_API_KEY`.
  - See how to configure other LLMs [here](how-to.md#custom-llm-support).

- Audio generation (TTS):
  - By default, Podcastfy uses OpenAI TTS. Hence, you need to set `OPENAI_API_KEY`.
  - Additional supported models are ElevenLabs ('elevenlabs'), Microsoft Edge ('edge') and Google TTS ('gemini'). All but Edge require an API key.

> [!Note]
> Never share your `.env` file or commit it to version control. It contains sensitive information that should be kept private. The `config.yaml` file can be shared and version-controlled as it doesn't contain sensitive data.

## Example Configurations

Here's a table showing example configurations:

| Configuration        | Base LLM  | TTS Model              | API Keys Required                 |
| -------------------- | --------- | ---------------------- | --------------------------------- |
| Default              | Gemini    | OpenAI                 | GEMINI_API_KEY and OPENAI_API_KEY |
| No API Keys Required | Local LLM | Edge                   | None                              |
| Recommended          | Gemini    | 'geminimulti' (Google) | GEMINI_API_KEY                    |

In our experience, Google's Multispeaker TTS model ('geminimulti') is the best model in terms of quality followed by ElevenLabs which offers great customization (voice options and multilingual capability). Google's multispeaker TTS model is limited to English only and requires an additional set up step.

## Setting up Google TTS Model

You can use Google's Multispeaker TTS model by setting the `tts_model` parameter to `geminimulti` in `Podcastfy`.

Google's Multispeaker TTS model requires a Google Cloud API key, you can use the same API key you are already using for Gemini or create a new one. After you have secured your API Key there are two additional steps in order to use Google Multispeaker TTS model:

- Step 1: You will need to enable the Cloud Text-to-Speech API on the API key.

  - Go to "https://console.cloud.google.com/apis/dashboard"
  - Select your project (or create one by clicking on project list and then on "new project")
  - Click "+ ENABLE APIS AND SERVICES" at the top of the screen
  - Enter "text-to-speech" into the search box
  - Click on "Cloud Text-to-Speech API" and then on "ENABLE"
  - You should be here: "https://console.cloud.google.com/apis/library/texttospeech.googleapis.com?project=..."

- Step 2: You need to add the Cloud Text-to-Speech API permission to the API KEY you're using on the Google Cloud console.

  - Go to https://console.cloud.google.com/apis/credentials
  - Click on whatever key you're using for Gemini
  - Go down to API Restrictions and add the Cloud Text-to-Speech API

⚠**NOTE :**

By default, **Google Multi-Speaker voices** are only available to **allowlisted projects**. If you wish to use these voices, follow the steps below:

- **Prerequisites:** A **paid Google Cloud support subscription** is required to proceed.
- **Request Access:** You'll need to **contact Google Cloud Support** to get Multi-Speaker voices enabled for your project.
- **Common Error:** If Multi-Speaker voices are not enabled, you will encounter the following runtime error:
  ```bash
  RuntimeError: Failed to generate audio: 403 Multi-speaker voices are only available to allowlisted    projects
  ```
- **How to Proceed:**
  - Navigate to the **Support** section in your **GCP Console**.

  - Open a new case under **"Cases"** and provide the necessary project details.

  - Google Cloud Support should be able to assist you in enabling this feature.

    ![google-multispeaker-support](../data/images/google-multispeaker-support.png)

Phew!!! That was a lot of steps but you only need to do it once and you might be impressed with the quality of the audio. See [Google TTS](https://cloud.google.com/text-to-speech) for more details. Thank you @mobarski and @evandempsey for the help!

## Conversation Configuration

See [conversation_custom.md](conversation_custom.md) for more details.

## Running Local LLMs

See [local_llm.md](local_llm.md) for more details.

## Optional configuration

The `config.yaml` file in the root directory contains non-sensitive configuration settings. You can modify this file to adjust various parameters such as output directories, text-to-speech settings, and content generation options.

The application will automatically load the environment variables from `.env` and the configuration settings from `config.yaml` when it runs.

See [Configuration](config_custom.md) if you would like to further customize settings.

usage/config.md


# Podcastfy Advanced Configuration Guide

Podcastfy uses a `config.yaml` file to manage various settings and parameters. This guide explains each configuration option available in the file.

## Content Generator

- `gemini_model`: "gemini-1.5-pro-latest"
  - The Gemini AI model used for content generation.
- `max_output_tokens`: 8192
  - Maximum number of tokens for the output generated by the AI model.
- `temperature`: 1
  - Controls randomness in the AI's output. 0 means deterministic responses. Range for gemini-1.5-pro: 0.0 - 2.0 (default: 1.0)
- `langchain_tracing_v2`: false
  - Enables LangChain tracing for debugging and monitoring. If true, requires langsmith api key

## Content Extractor

- `youtube_url_patterns`:
  - Patterns to identify YouTube URLs.
  - Current patterns: "youtube.com", "youtu.be"

## Website Extractor

- `markdown_cleaning`:
  - `remove_patterns`:
    - Patterns to remove from extracted markdown content.
    - Current patterns remove image links, hyperlinks, and URLs.

## YouTube Transcriber

- `remove_phrases`:
  - Phrases to remove from YouTube transcriptions.
  - Current phrase: "[music]"

## Logging

- `level`: "INFO"
  - Default logging level.
- `format`: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
  - Format string for log messages.

## Website Extractor

- `markdown_cleaning`:
	- `remove_patterns`:
		- Additional patterns to remove from extracted markdown content:
		- '\[.*?\]': Remove square brackets and their contents
		- '\(.*?\)': Remove parentheses and their contents
		- '^\s*[-*]\s': Remove list item markers
		- '^\s*\d+\.\s': Remove numbered list markers
		- '^\s*#+': Remove markdown headers
- `unwanted_tags`:
	- HTML tags to be removed during extraction:
		- 'script', 'style', 'nav', 'footer', 'header', 'aside', 'noscript'
- `user_agent`: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'
	- User agent string to be used for web requests
- `timeout`: 10
	- Request timeout in seconds for web scraping

usage/config_custom.md


# Podcastfy Conversation Configuration

Podcastfy offers a range of customization options to tailor your AI-generated podcasts. This document outlines how you can adjust parameters such as conversation style, word count, and dialogue structure to suit your specific needs.

## Table of Contents

1. [Parameters](#parameters)
2. [Customization Examples](#customization-examples)
   1. [Academic Debate](#academic-debate)
   2. [Storytelling Adventure](#storytelling-adventure)
3. [Customization Scenarios](#customization-scenarios)
   1. [Using the Python Package](#using-the-python-package)
   2. [Using the CLI](#using-the-cli)
4. [Notes of Caution](#notes-of-caution)

## Conversation Parameters

Podcastfy uses the default conversation configuration stored in [podcastfy/conversation_config.yaml](https://github.com/souzatharsis/podcastfy/blob/main/podcastfy/conversation_config.yaml).

| Parameter | Default Value | Type | Description |
|-----------|---------------|------|-------------|
| conversation_style | ["engaging", "fast-paced", "enthusiastic"] | list[str] | Styles to apply to the conversation |
| roles_person1 | "main summarizer" | str | Role of the first speaker |
| roles_person2 | "questioner/clarifier" | str | Role of the second speaker |
| dialogue_structure | ["Introduction", "Main Content Summary", "Conclusion"] | list[str] | Structure of the dialogue |
| podcast_name | "PODCASTIFY" | str | Name of the podcast |
| podcast_tagline | "Your Personal Generative AI Podcast" | str | Tagline for the podcast |
| output_language | "English" | str | Language of the output |
| engagement_techniques | ["rhetorical questions", "anecdotes", "analogies", "humor"] | list[str] | Techniques to engage the audience |
| creativity | 1 | float | Level of creativity/temperature (0-1) |
| user_instructions | "" | str | Custom instructions to guide the conversation focus and topics |
| max_num_chunks | 7 | int | Maximum number of rounds of discussions in longform |
| min_chunk_size | 600 | int | Minimum number of characters to generate a round of discussion in longform |

## Text-to-Speech (TTS) Settings

Podcastfy uses the default TTS configuration stored in [podcastfy/conversation_config.yaml](https://github.com/souzatharsis/podcastfy/blob/main/podcastfy/conversation_config.yaml).

### ElevenLabs TTS

- `default_voices`:
  - `question`: "Chris"
    - Default voice for questions in the podcast.
  - `answer`: "Jessica"
    - Default voice for answers in the podcast.
- `model`: "eleven_multilingual_v2"
  - The ElevenLabs TTS model to use.

### OpenAI TTS

- `default_voices`:
  - `question`: "echo"
    - Default voice for questions using OpenAI TTS.
  - `answer`: "shimmer"
    - Default voice for answers using OpenAI TTS.
- `model`: "tts-1-hd"
  - The OpenAI TTS model to use.

### Gemini Multi-Speaker TTS
- `default_voices`:
  - `question`: "R"
    - Default voice for questions using Gemini Multi-Speaker TTS.
  - `answer`: "S"
    - Default voice for answers using Gemini Multi-Speaker TTS.
  - `model`: "en-US-Studio-MultiSpeaker"
    - Model to use for Gemini Multi-Speaker TTS.
  - `language`: "en-US"
    - Language of the voices.

### Gemini TTS
- `default_voices`:
  - `question`: "en-US-Journey-D"
    - Default voice for questions using Gemini TTS.
  - `answer`: "en-US-Journey-O"
    - Default voice for answers using Gemini TTS.

### Edge TTS

- `default_voices`:
  - `question`: "en-US-JennyNeural"
    - Default voice for questions using Edge TTS.
  - `answer`: "en-US-EricNeural"
    - Default voice for answers using Edge TTS.

### General TTS Settings

- `default_tts_model`: "openai"
  - Default text-to-speech model to use.
- `output_directories`:
  - `transcripts`: "./data/transcripts"
    - Directory for storing generated transcripts.
  - `audio`: "./data/audio"
    - Directory for storing generated audio files.
- `audio_format`: "mp3"
  - Format of the generated audio files.
- `temp_audio_dir`: "data/audio/tmp/"
  - Temporary directory for audio processing.
- `ending_message`: "Bye Bye!"
  - Message to be appended at the end of the podcast.

## Customization Examples

These examples demonstrate how conversations can be altered to suit different purposes, from academic rigor to creative storytelling. The comments explain the rationale behind each choice, helping users understand how to tailor the configuration to their specific needs.

### Academic Debate

This configuration transforms the podcast into a formal academic debate, encouraging deep analysis and critical thinking. It's designed for educational content or in-depth discussions on complex topics.

```python
{
    "word_count": 3000,  # Longer to allow for detailed arguments
    "conversation_style": ["formal", "analytical", "critical"],  # Appropriate for academic discourse
    "roles_person1": "thesis presenter",  # Presents the main argument
    "roles_person2": "counterargument provider",  # Challenges the thesis
    "dialogue_structure": [
        "Opening Statements",
        "Thesis Presentation",
        "Counterarguments",
        "Rebuttals",
        "Closing Remarks"
    ],  # Mimics a structured debate format
    "podcast_name": "Scholarly Showdown",
    "podcast_tagline": "Where Ideas Clash and Knowledge Emerges",
    "engagement_techniques": [
        "socratic questioning",
        "historical references",
        "thought experiments"
    ],  # Techniques to stimulate critical thinking
    "creativity": 0  # Low creativity to maintain focus on facts and logic
}
```

### Storytelling Adventure

This configuration turns the podcast into an interactive storytelling experience, engaging the audience in a narrative journey. It's ideal for fiction podcasts or creative content marketing.

```yaml
word_count: 1000  # Shorter to maintain pace and suspense
conversation_style:
  - narrative
  - suspenseful
  - descriptive  # Creates an immersive story experience
roles_person1: storyteller
roles_person2: audience participator  # Allows for interactive elements
dialogue_structure:
  - Scene Setting
  - Character Introduction
  - Rising Action
  - Climax
  - Resolution  # Follows classic storytelling structure
podcast_name: Tale Spinners
podcast_tagline: Where Every Episode is an Adventure
engagement_techniques:
  - cliffhangers
  - vivid imagery
  - audience prompts  # Keeps the audience engaged and coming back
creativity: 0.9 # High creativity for unique and captivating stories
```

## Customization Scenarios

### Using the Python Package

When using the Podcastfy Python package, you can customize the conversation by passing a dictionary to the `conversation_config` parameter:

```python
from podcastfy.client import generate_podcast

custom_config = {
    "word_count": 200,
    "conversation_style": ["casual", "humorous"],
    "podcast_name": "Tech Chuckles",
    "creativity": 0.7
}

generate_podcast(
    urls=["https://example.com/tech-news"],
    conversation_config=custom_config
)
```

### Using the CLI

When using the Podcastfy CLI, you can specify a path to a YAML file containing your custom configuration:

```bash
podcastfy --url https://example.com/tech-news --conversation-config path/to/custom_config.yaml
```

The `custom_config.yaml` file should contain your configuration in YAML format:

```yaml
word_count: 200
conversation_style:
  - casual
  - humorous
podcast_name: Tech Chuckles
creativity: 0.7
```

## Notes of Caution

- The `word_count` is a target, and the AI may generate more or less than the specified word count. Low word counts are more likely to generate high-level discussions, while high word counts are more likely to generate detailed discussions.
- The `output_language` defines both the language of the transcript and the language of the audio. Here's some relevant information:
  - Bottom-line: non-English transcripts are good enough but non-English audio is work-in-progress.
  - Transcripts are generated using Google's Gemini 1.5 Pro by default, which supports 100+ languages. Other user-defined models may or may not support non-English languages.
  - Audio is generated using `openai` (default), `elevenlabs`, `gemini`, `geminimulti` or `edge` TTS models.
    - The `gemini`(Google) TTS model supports multiple languages and can be controlled by the `output_language` parameter and respective voice choices. Eg. `output_language="Tamil"`, `question="ta-IN-Standard-A"`, `answer="ta-IN-Standard-B"`. Refer to [Google Cloud Text-to-Speech documentation](https://cloud.google.com/text-to-speech/docs/voices) for more details.
    - The `geminimulti`(Google) TTS model supports only English voices. Also, not every Google Cloud project might have access to multi-speaker voices (Eg. `en-US-Studio-MultiSpeaker`). In case if you get - `"Multi-speaker voices are only available to allowlisted projects."`, you can fallback to `gemini` TTS model.
    - The `openai` TTS model supports multiple languages automatically, however non-English voices still present sub-par quality in my experience.
    - The `elevenlabs` TTS model has English voices by default, in order to use a non-English voice you would need to download a custom voice for the target language in your `elevenlabs` account settings and then set the `text_to_speech.elevenlabs.default_voices` parameters to the voice you want to use in the [config.yaml file](https://github.com/pedroslopez/podcastfy/blob/main/podcastfy/config.yaml) (this config file is only available in the source code of the project, not in the pip package, hence if you are using the pip package you will not be able to change the ElevenLabs voice). For more information on ElevenLabs voices, visit [ElevenLabs Voice Library](https://elevenlabs.io/voice-library)

usage/conversation_custom.md


# Docker Setup Guide for Podcastfy

This guide explains how to use Docker to run Podcastfy in your local environment or for development.

## Prerequisites

- Docker installed on your system [1]
- Docker Compose [1]
- API keys [2]

[1] See Appendix A for detailed installation instructions.
[2] See [config.md](https://github.com/souzatharsis/podcastfy/blob/main/usage/config.md) for more details.

## Available Images

Podcastfy provides pre-built Docker images through GitHub Container Registry (ghcr.io):

1. **Production Image**: `ghcr.io/souzatharsis/podcastfy:latest`
   - Contains the latest PyPI release
   - Recommended for production use

2. **Development Image**: `ghcr.io/souzatharsis/podcastfy:dev`
   - Includes development tools and dependencies
   - Used for contributing and development

## Deployment

### Quick Deployment Steps

1. Create a new directory and navigate to it:
```bash
mkdir -p /path/to/podcastfy
cd /path/to/podcastfy
```

2. Create a `.env` file with your API keys (see [config.md](https://github.com/souzatharsis/podcastfy/blob/main/usage/config.md) for more details):
```plaintext
GEMINI_API_KEY=your_gemini_api_key
OPENAI_API_KEY=your_openai_api_key  # Optional: only needed for OpenAI TTS
```

3. Create a `docker-compose.yml`:
```yaml
version: '3.8'

services:
  podcastfy:
    image: ghcr.io/souzatharsis/podcastfy:latest
    environment:
      - GEMINI_API_KEY=${GEMINI_API_KEY}
      - OPENAI_API_KEY=${OPENAI_API_KEY}
    ports:
      - "8000:8000"
    command: python3 -m podcastfy.server
    healthcheck:
      test: ["CMD", "python3", "-c", "import podcastfy"]
      interval: 30s
      timeout: 10s
      retries: 3
```

4. Pull and start the container:
```bash
docker pull ghcr.io/souzatharsis/podcastfy:latest
docker-compose up podcastfy
```

The service will be available at `http://localhost:8000`

### Directory Structure
```
/path/to/podcastfy/
├── .env                  # Environment variables
└── docker-compose.yml    # Docker Compose configuration
```

## Development Setup

### Using Pre-built Development Image

1. Pull the development image:
```bash
docker pull ghcr.io/souzatharsis/podcastfy:dev
```

2. Clone the repository and start development environment:
```bash
git clone https://github.com/souzatharsis/podcastfy.git
cd podcastfy
docker-compose up podcastfy-dev
```

### Building Locally

Alternatively, you can build the images locally:
```bash
# Build production image
docker-compose build podcastfy

# Build development image
docker-compose build podcastfy-dev
```

## Running Tests

Run the test suite using:
```bash
docker-compose up test
```

This will run tests in parallel using pytest-xdist.

## Environment Variables

Required environment variables:
- `GEMINI_API_KEY` - Your Google Gemini API key
- `OPENAI_API_KEY` - Your OpenAI API key (optional: only needed for OpenAI TTS)

## Container Details

### Production Container
- Based on Ubuntu 24.04
- Installs Podcastfy from PyPI
- Includes FFmpeg for audio processing
- Runs in a Python virtual environment
- Exposed port: 8000

### Development Container
- Based on Ubuntu 24.04
- Includes development tools (flake8, pytest)
- Mounts local code for live development
- Runs in editable mode (`pip install -e .`)
- Exposed port: 8001

## Continuous Integration

The Docker images are automatically:
- Built and tested on every push to main branch
- Built and tested for all pull requests
- Published to GitHub Container Registry
- Tagged with version numbers for releases (v*.*.*)

## Health Checks

All services include health checks that:
- Run every 30 seconds
- Verify Podcastfy can be imported
- Timeout after 10 seconds
- Retry up to 3 times

## Common Commands

```bash
# Pull latest production image
docker pull ghcr.io/souzatharsis/podcastfy:latest

# Pull development image
docker pull ghcr.io/souzatharsis/podcastfy:dev

# Start production service
docker-compose up podcastfy

# Start development environment
docker-compose up podcastfy-dev

# Run tests
docker-compose up test

# Build images locally
docker-compose build

# View logs
docker-compose logs

# Stop all containers
docker-compose down
```

## Troubleshooting

### Common Issues

1. **API Key Errors**
   - Verify your `.env` file exists and contains valid API keys
   - Check if the environment variables are properly passed to the container

2. **Port Conflicts**
   - Ensure ports 8000 (production) and 8001 (development) are available
   - Modify the port mappings in `docker-compose.yml` if needed

3. **Volume Mounting Issues (Development)**
   - Verify the correct path to your local code
   - Check permissions on the mounted directories

4. **Image Pull Issues**
   - Ensure you have access to the GitHub Container Registry
   - If you see "unauthorized" errors, the image might be private
   - Try authenticating with GitHub: `docker login ghcr.io -u YOUR_GITHUB_USERNAME`

### Verifying Installation

You can verify your installation by checking if the package can be imported:
```bash
# Check production version
docker run --rm ghcr.io/souzatharsis/podcastfy:latest python3 -c "import podcastfy"

# Check development setup
docker-compose exec podcastfy-dev python3 -c "import podcastfy"
```

## System Requirements

Minimum requirements:
- Docker Engine 20.10.0 or later
- Docker Compose 2.0.0 or later
- Sufficient disk space for Ubuntu base image (~400MB)
- Additional space for Python packages and FFmpeg

## Support

If you encounter any issues:
1. Check the container logs: `docker-compose logs`
2. Verify all prerequisites are installed
3. Ensure all required environment variables are set
4. Open an issue on the [Podcastfy GitHub repository](https://github.com/souzatharsis/podcastfy/issues)

## Appendix A: Detailed Installation Guide

### Installing Docker

#### Windows
1. Download and install [Docker Desktop for Windows](https://docs.docker.com/desktop/install/windows-install/)
   - For Windows 10/11 Pro, Enterprise, or Education: Enable WSL 2 and Hyper-V
   - For Windows 10 Home: Enable WSL 2
2. After installation, start Docker Desktop
3. Verify installation:
```bash
docker --version
```

#### macOS
1. Download and install [Docker Desktop for Mac](https://docs.docker.com/desktop/install/mac-install/)
   - For Intel chip: Download Intel package
   - For Apple chip: Download Apple Silicon package
2. After installation, start Docker Desktop
3. Verify installation:
```bash
docker --version
```

#### Ubuntu/Debian
```bash
# Remove old versions
sudo apt-get remove docker docker-engine docker.io containerd runc

# Install prerequisites
sudo apt-get update
sudo apt-get install \
    ca-certificates \
    curl \
    gnupg \
    lsb-release

# Add Docker's official GPG key
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

# Set up repository
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install Docker Engine
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin

# Add your user to docker group (optional, to run docker without sudo)
sudo usermod -aG docker $USER
newgrp docker

# Verify installation
docker --version
```

#### Other Linux Distributions
- [CentOS](https://docs.docker.com/engine/install/centos/)
- [Fedora](https://docs.docker.com/engine/install/fedora/)
- [RHEL](https://docs.docker.com/engine/install/rhel/)

### Installing Docker Compose

Docker Compose is included with Docker Desktop for Windows and macOS. For Linux:

```bash
# Download the current stable release
sudo curl -L "https://github.com/docker/compose/releases/download/v2.24.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

# Apply executable permissions
sudo chmod +x /usr/local/bin/docker-compose

# Verify installation
docker-compose --version
```

### Post-Installation Steps

1. Verify Docker is running:
```bash
docker run hello-world
```

2. Configure Docker to start on boot (Linux only):
```bash
sudo systemctl enable docker.service
sudo systemctl enable containerd.service
```

## Appendix B: Getting API Keys

### Google Gemini API Key
1. Visit [Google AI Studio](https://makersuite.google.com/app/apikey)
2. Create or sign in to your Google account
3. Click "Create API Key"
4. Copy and save your API key

### OpenAI API Key
You only need an OpenAI API key if you want to use the OpenAI Text-to-Speech model.
1. Visit [OpenAI API Keys](https://platform.openai.com/api-keys)
2. Create or sign in to your OpenAI account
3. Click "Create new secret key"
4. Copy and save your API key

## Appendix C: Installation Validation

After installing all prerequisites, verify everything is set up correctly:

```bash
# Check Docker version
docker --version

# Check Docker Compose version
docker-compose --version

# Verify Docker daemon is running
docker ps

# Test Docker functionality
docker run hello-world
```

usage/docker.md


# How to

All assume you have podcastfy installed and running.

## Table of Contents

- [Custom LLM Support](#custom-llm-support)
- [Running Local LLMs](#running-local-llms)
- [How to use your own voice in audio podcasts](#how-to-use-your-own-voice-in-audio-podcasts)
- [How to customize the conversation](#how-to-customize-the-conversation)
- [How to generate multilingual content](#how-to-generate-multilingual-content)
- [How to steer the conversation](#how-to-steer-the-conversation)
- [How to generate longform podcasts](#how-to-generate-longform-podcasts)

## Custom LLM Support

Podcastfy offers a range of LLM models for generating transcripts including OpenAI, Anthropic, Google as well as local LLM models.

### Cloud-based LLMs

By default, Podcastfy uses Google's `gemini-1.5-pro-latest` model. To select a particular cloud-based LLM model, users can pass the `llm_model_name` and `api_key_label` parameters to the `generate_podcast` function. See [full list of supported models](https://docs.litellm.ai/docs/providers) for more details.

For example, to use OpenAI's `gpt-4-turbo` model, users can pass `llm_model_name="gpt-4-turbo"` and `api_key_label="OPENAI_API_KEY"`.

```python
audio_file = generate_podcast(
    urls=["https://en.wikipedia.org/wiki/Artificial_intelligence"],
    llm_model_name="gpt-4-turbo",
    api_key_label="OPENAI_API_KEY"
)
```

Remember to have the correct API key label and value in your environment variables (`.env` file).

### Running Local LLMs

See [local_llm.md](local_llm.md) for more details.

## How to use your own voice in audio podcasts

You just need to use ElevenLabs TSS backend and pass a custom config to use your voice instead of podcastfy's default:

1. Create elevenlabs account, get and [set up](https://github.com/souzatharsis/podcastfy/blob/main/usage/config.md) eleven labs API KEY

2. Clone your voice on elevenlabs website (let's say its name is 'Robbert')

4. Create custom conversation config (let's call it custom_config.yaml) to use your voice name instead of the default as described [here](https://github.com/souzatharsis/podcastfy/blob/main/usage/conversation_custom.md#text-to-speech-tts-settings). Set either question or answer voice below to 'Robbert' in elevenlabs > default_voices.

6. Run podcastfy with tts-model param as elevenlabs

CLI
   ```
   python -m podcastfy.client --url https://example.com/article1 --url https://example.com/article2 --tts-model elevenlabs --conversation-config path/to/custom_config.yaml
   ```
For Python example, checkout Customization section at [python notebook](https://github.com/souzatharsis/podcastfy/blob/main/podcastfy.ipynb).

## How to customize the conversation

You can customize the conversation by passing a custom [conversation_config.yaml](https://github.com/souzatharsis/podcastfy/blob/main/podcastfy/conversation_config.yaml) file to the CLI:

```
python -m podcastfy.client --url https://example.com/article1 --url https://example.com/article2 --tts-model elevenlabs --conversation-config path/to/custom_config.yaml
```

You can also pass a dictionary with the custom config to the python interface generate_podcast function:

```python
from podcastfy.client import generate_podcast

custom_config = {
    "word_count": 200,
    "conversation_style": ["casual", "humorous"],
    "podcast_name": "Tech Chuckles",
    "creativity": 0.7
}

generate_podcast(
    urls=["https://example.com/tech-news"],
    conversation_config=custom_config
)
```
For more details, checkout [conversation_custom.md](https://github.com/souzatharsis/podcastfy/blob/main/usage/conversation_custom.md).

## How to generate multilingual content

In order to generate transcripts in a target language, simply set `output_language` = your target language. See [How to customize the conversation](#how-to-customize-the-conversation) on how to pass custom configuration to podcastfy. Set --transcript-only to get only the transcript without audio generation.

In order to generation audio, you can simply use openai TTS model which by default is multilingual. However, in my experience OpenAI's TTS multilingual quality is subpar. Instead, consdier using elevenlabs backend. See [How to use your own voice in audio podcasts](#how-to-use-your-own-voice-in-audio-podcasts) but instead of using your own voice you should download and set a voice in your target language for it to work.

Sample audio:
- [French](https://github.com/souzatharsis/podcastfy/blob/main/data/audio/podcast_FR_AGRO.mp3)
- [Portugue-BR](https://github.com/souzatharsis/podcastfy/blob/main/data/audio/podcast_thatupiso_BR.mp3)

The PT-BR audio actually uses my own cloned voice as AI Host 2.

## How to steer the conversation

You can guide the conversation focus and topics by setting the `user_instructions` parameter in your custom configuration. This allows you to provide specific instructions to the AI hosts about what aspects they should emphasize or explore.

Things to try:
- Focus on a specific topic (e.g. "Focus the discussion on key capabilities and limitations of modern AI models")
- Target a specific audience (e.g. "Explain concepts in a way that's accessible to someone new to Computer Science")

For example, using the CLI with a custom YAML:

```yaml
user_instructions: "Make connections with quantum computing"
```

```
python -m podcastfy.client --url https://en.wikipedia.org/wiki/Artificial_intelligence --conversation-config path/to/custom_config.yaml
```

## How to generate longform podcasts

By default, Podcastfy generates shortform podcasts. However, users can generate longform podcasts by setting the `longform` parameter to `True`.

```python
audio_file = generate_podcast(
    urls=["https://example.com/article1", "https://example.com/article2"],
    longform=True
)
```

LLMs have a limited ability to output long text responses. Most LLMs have a `max_output_tokens` of around 4096 and 8192 tokens. Hence, long-form podcast transcript generation is challeging. We have implemented a technique I call "Content Chunking with Contextual Linking" to enable long-form podcast generation by breaking down the input content into smaller chunks and generating a conversation for each chunk while ensuring the combined transcript is coherent and linked to the original input.

By default, shortform podcasts (default configuration) generate audio of about 2-5 minutes while longform podcasts may reach 20-30 minutes.

Users may adjust lonform podcast length by setting the following parameters in your customization params (conversation_config.yaml):
- `max_num_chunks` (default: 7): Sets maximum number of rounds of discussions.
- `min_chunk_size` (default: 600): Sets minimum number of characters to generate a round of discussion.

A "round of discussion" is the output transcript obtained from a single LLM call. The higher the `max_num_chunks` and the lower the `min_chunk_size`, the longer the generated podcast will be.
Today, this technique allows the user to generate long-form podcasts of any length if input content is long enough. However, the conversation quality may decrease and its length may converge to a maximum if `max_num_chunks`/`min_chunk_size` is to high/low particularly if input content length is limited.

Current implementation limitations:
- Images are not yet supported for longform podcast generation
- Base LLM model is fixed to Gemini

Above limitations are somewhat easily fixable however we chose to make updates in smaller but quick iterations rather than making all-in changes.

usage/how-to.md


## Attribution

1. If you use `Podcastfy` in your software, we kindly ask you to add attribution. "Powered by Podcastfy.ai" would suffice. Please reach out, we would love to learn more how you are using `Podcastfy` and how we can better enable your use case.
2. Feel free to add your product to the the "[Built with Podcastfy](https://github.com/souzatharsis/podcastfy?tab=readme-ov-file#built-with-podcastfy-)" list by submitting a PR to our README.

## License

Additionally, `Podcastfy` is licensed under Apache 2.0. The Apache License 2.0 is a permissive free software license that allows you to use this sotfware for both non-commercial or commercial purposes.
Please review the [License](../LICENSE) in order to know your obligations.
here is a set of steps I will list without any warranty or liability:

1. Include a copy of the license in your project:

In your project root, create a NOTICE.txt or THIRD_PARTY_LICENSES.txt file and include the content from the file [NOTICE](../NOTICE)

2. Add attribution in your README.md:
```markdown
## Acknowledgments

This project includes code from Podcastfy(https://github.com/souzatharsis/podcastfy/), licensed under the Apache License 2.0.
```

3. Keep the original copyright notices in any files you copy/modify

4. If you modified the code, indicate your changes:
```python
# Modified from original source: [Podcastfy](https://github.com/souzatharsis/podcastfy/)
# Changes made:
# - Added feature X
# - Modified function Y
# - Removed component Z
```

Important points:
- You don't need to use the same license for your project
- You must preserve all copyright, patent, trademark notices
- State significant modifications you made
- Include the original Apache 2.0 license text
- Attribution should be clear and reasonable

usage/license-guide.md


# Local LLM Support

Running local LLMs can offer several advantages such as:
- Enhanced privacy and data security
- Cost control and no API rate limits
- Greater customization and fine-tuning options
- Reduced vendor lock-in

We enable serving local LLMs with [llamafile](https://github.com/Mozilla-Ocho/llamafile). In the API, local LLM support is available through the `is_local` parameter. If `is_local=True`, then a local (llamafile) LLM model is used to generate the podcast transcript. Llamafiles of LLM models can be found on [HuggingFace, which today offers 156+ models](https://huggingface.co/models?library=llamafile).

All you need to do is:

1. Download a llamafile from HuggingFace
2. Make the file executable
3. Run the file

Here's a simple bash script that shows all 3 setup steps for running TinyLlama-1.1B locally:

```bash
# Download a llamafile from HuggingFace
wget https://huggingface.co/jartine/TinyLlama-1.1B-Chat-v1.0-GGUF/resolve/main/TinyLlama-1.1B-Chat-v1.0.Q5_K_M.llamafile

# Make the file executable. On Windows, instead just rename the file to end in ".exe".
chmod +x TinyLlama-1.1B-Chat-v1.0.Q5_K_M.llamafile

# Start the model server. Listens at http://localhost:8080 by default.
./TinyLlama-1.1B-Chat-v1.0.Q5_K_M.llamafile --server --nobrowser
```

Now you can use the local LLM to generate a podcast transcript (or audio) by setting the `is_local` parameter to `True`.

## Python API

```python
from podcastfy.client import generate_podcast

# Generate a tech debate podcast about artificial intelligence
generate_podcast(
    urls=["www.souzatharsis.com"],
    is_local=True  # Using a local LLM
)
```

## CLI

To use a local LLM model via the command-line interface, you can use the `--local` or `-l` flag. Here's an example of how to generate a transcript using a local LLM:

```bash
python -m podcastfy.client --url https://example.com/article1 --transcript-only --local
```

## Notes of caution

When using local LLM models versus widely known private large language models:

1. Performance: Local LLMs often have lower performance compared to large private models due to size and training limitations.

2. Resource requirements: Running local LLMs can be computationally intensive, requiring significant CPU/GPU resources.

3. Limited capabilities: Local models may struggle with complex tasks or specialized knowledge that larger models handle well.

5. Reduced multimodal abilities: Local LLMs will be assumed to be text-only capable

6. Potential instability: Local models may produce less consistent or stable outputs compared to well-tested private models oftentimes producing transcripts that cannot be used for podcast generation (TTS) out-of-the-box

7. Limited context window: Local models often have smaller context windows, limiting their ability to process long inputs.

Always evaluate the trade-offs between using local LLMs and private models based on your specific use case and requirements. We highly recommend extensively testing your local LLM before productionizing an end-to-end podcast generation and/or manually checking the transcript before passing to TTS model.

usage/local_llm.md