# embeddings_queue

created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 014f6f8a-3cb5-469c-a787-a9e8d127344e
encoding: FLOAT32
metadata:
{"source": "http://en.wikipedia.org/wiki/Generative_artificial_intelligence", "doc_id": "232f3bfa-45f1-0087-6903-d9d3d42a7d0d", "source_type": "web", "start_index": 88826, "chroma:document": "January 14, 2025. Retrieved January 14, 2025. ^ \"TV channels are using AI-generated presenters to read the news. The question is, will we trust them?\". BBC News. January 26, 2024. Archived from the original on January 26, 2024. Retrieved May 24, 2024. ^ Tait, Amelia (October 20, 2023). \"'Here is the news. You can't stop us': AI anchor Zae-In grants us an interview\". The Guardian. ISSN\u00a00261-3077. Archived from the original on January 28, 2024. Retrieved May 24, 2024. ^ Kuo, Lily (November 9, 2018). \"World's first AI news anchor unveiled in China\". The Guardian. ISSN\u00a00261-3077. Archived from the original on February 20, 2024. Retrieved May 24, 2024. ^ \"These ISIS news anchors are AI fakes. Their propaganda is real\". Washington Post. May 17, 2024. Archived from the original on May 19, 2024. Retrieved May 24, 2024. ^ Mullin, Benjamin; Grant, Nico (July 20, 2023). \"Google Tests A.I. Tool That Is Able to Write News Articles\". The New York Times. ISSN\u00a00362-4331. Archived from the original on"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: c88db6ed-0494-458f-adfe-c2b43c4ebfb0
encoding: FLOAT32
metadata:
{"source": "http://en.wikipedia.org/wiki/Generative_artificial_intelligence", "doc_id": "232f3bfa-45f1-0087-6903-d9d3d42a7d0d", "source_type": "web", "start_index": 89635, "chroma:document": "May 24, 2024. ^ Mullin, Benjamin; Grant, Nico (July 20, 2023). \"Google Tests A.I. Tool That Is Able to Write News Articles\". The New York Times. ISSN\u00a00362-4331. Archived from the original on May 16, 2024. Retrieved May 24, 2024. ^ Stenberg, Mark (February 27, 2024). \"Google Is Paying Publishers Five-Figure Sums to Test an Unreleased Gen AI Platform\". Adweek. Archived from the original on March 9, 2024. Retrieved April 17, 2024. ^ Knibbs, Kate (February 7, 2024). \"Confessions of an AI Clickbait Kingpin\". Wired. ISSN\u00a01059-1028. Archived from the original on May 18, 2024. Retrieved May 24, 2024. ^ Knibbs, Kate (January 26, 2024). \"How Beloved Indie Blog 'The Hairpin' Turned Into an AI Clickbait Farm\". Wired. ISSN\u00a01059-1028. Archived from the original on April 14, 2024. Retrieved May 24, 2024. ^ Koebler, Jason (July 9, 2024). \"A Beloved Tech Blog Is Now Publishing AI Articles Under the Names of Its Old Human Staff\". 404 Media. Archived from the original on July 12, 2024. Retrieved August"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 77554720-02eb-4063-9f08-c49d825d93ae
encoding: FLOAT32
metadata:
{"source": "http://en.wikipedia.org/wiki/Generative_artificial_intelligence", "doc_id": "232f3bfa-45f1-0087-6903-d9d3d42a7d0d", "source_type": "web", "start_index": 90436, "chroma:document": "^ Koebler, Jason (July 9, 2024). \"A Beloved Tech Blog Is Now Publishing AI Articles Under the Names of Its Old Human Staff\". 404 Media. Archived from the original on July 12, 2024. Retrieved August 27, 2024. ^ Hollister, Sean (July 10, 2024). \"Early Apple tech bloggers are shocked to find their name and work have been AI-zombified\". The Verge. Archived from the original on July 12, 2024. Retrieved August 27, 2024. ^ \"AI slop is already invading Oregon's local journalism\". Oregon Public Broadcasting. December 9, 2024. Archived from the original on December 9, 2024. Retrieved December 10, 2024. ^ Knibbs, Kate (February 26, 2024). \"How a Small Iowa Newspaper's Website Became an AI-Generated Clickbait Factory\". Wired. ISSN\u00a01059-1028. Archived from the original on February 26, 2024. Retrieved December 10, 2024. ^ Koebler, Jason; Cole, Samantha; Maiberg, Emanuel; Cox, Joseph (January 26, 2024). \"We Need Your Email Address\". 404 Media. Archived from the original on December 2, 2024."}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: d7bfb9cb-3733-4f3f-9044-203bd906029f
encoding: FLOAT32
metadata:
{"source": "http://en.wikipedia.org/wiki/Generative_artificial_intelligence", "doc_id": "232f3bfa-45f1-0087-6903-d9d3d42a7d0d", "source_type": "web", "start_index": 91235, "chroma:document": "December 10, 2024. ^ Koebler, Jason; Cole, Samantha; Maiberg, Emanuel; Cox, Joseph (January 26, 2024). \"We Need Your Email Address\". 404 Media. Archived from the original on December 2, 2024. Retrieved December 10, 2024. ^ \"Meet the Serbian Businessman/DJ Who Runs the Zombie AI Southwest Journal - Racket\". Racket. February 16, 2024. Archived from the original on November 13, 2024. Retrieved December 10, 2024. ^ Lima-Strong, Cristiano (January 11, 2024). \"Senators warn AI could lead to 'destruction' of local news\". Washington Post. ISSN\u00a00190-8286. Archived from the original on January 11, 2024. Retrieved May 24, 2024. ^ \"OpenAI strikes $5 million-plus local news deal\". Axios. July 18, 2023. Archived from the original on July 19, 2023. Retrieved May 24, 2024. ^ Kelly, Heather (May 22, 2024). \"Meta walked away from news. Now the company's using it for AI content\". Washington Post. ISSN\u00a00190-8286. Archived from the original on May 22, 2024. Retrieved May 24, 2024. ^ \"How WIRED Will Use"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 3f0fe406-301d-49b3-add9-0ecf876fe79f
encoding: FLOAT32
metadata:
{"source": "http://en.wikipedia.org/wiki/Generative_artificial_intelligence", "doc_id": "232f3bfa-45f1-0087-6903-d9d3d42a7d0d", "source_type": "web", "start_index": 92036, "chroma:document": "\"Meta walked away from news. Now the company's using it for AI content\". Washington Post. ISSN\u00a00190-8286. Archived from the original on May 22, 2024. Retrieved May 24, 2024. ^ \"How WIRED Will Use Generative AI Tools\". Wired. Archived from the original on December 30, 2023. Retrieved January 8, 2024. ^ Barrett, Amanda (November 15, 2018). \"Standards around generative AI\". Associated Press. Archived from the original on September 23, 2023. Retrieved January 8, 2024. ^ Viner, Katharine; Bateson, Anna (June 16, 2023). \"The Guardian's approach to generative AI\". The Guardian. ISSN\u00a00261-3077. Archived from the original on January 3, 2024. Retrieved January 8, 2024. ^ Becker, K. B.; Simon, F. M.; Crum, C. (2023). \"Policies in parallel? A comparative study of journalistic AI policies in 52 global news organisations\". pp.\u00a08\u20139. doi:10.31235/osf.io/c4af9. ^ Newman, Nic; Fletcher, Richard; Robertson, Craig T.; Arguedas, Amy Ross; Nielsen, Rasmus Fleis (June 2024). \"Digital News Report 2024\""}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 69ac9df7-608d-4e49-9e0c-42d69c1ba580
encoding: FLOAT32
metadata:
{"source": "http://en.wikipedia.org/wiki/Generative_artificial_intelligence", "doc_id": "232f3bfa-45f1-0087-6903-d9d3d42a7d0d", "source_type": "web", "start_index": 92836, "chroma:document": "news organisations\". pp.\u00a08\u20139. doi:10.31235/osf.io/c4af9. ^ Newman, Nic; Fletcher, Richard; Robertson, Craig T.; Arguedas, Amy Ross; Nielsen, Rasmus Fleis (June 2024). \"Digital News Report 2024\" (PDF). Reuters Institute for the Study of Journalism. p.\u00a020. doi:10.60625/risj-vy6n-4v57. Retrieved June 20, 2024.  vteGenerative AIConcepts Autoencoder Deep learning Generative adversarial network Generative pre-trained transformer Large language model Neural network Prompt engineering Retrieval-augmented generation Reinforcement learning from human feedback Self-supervised learning Transformer Variational autoencoder Vision transformer Word embedding ModelsText Claude DBRX Gemini GPT 1 2 3 J ChatGPT 4 4o o1 o3 Grok Granite Llama Mistral Large PanGu-\u03a3 Qwen Image Aurora DALL-E Firefly Flux Ideogram Midjourney Stable Diffusion Video Dream Machine Gen-3 Alpha Hailuo AI Kling Sora Veo VideoPoet Music Udio Suno AI Companies 01.AI Alibaba Anthropic Baichuan DeepSeek ElevenLabs Google DeepMind"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 0b98c2dc-4f6d-46f3-910c-799f9c58470a
encoding: FLOAT32
metadata:
{"source": "http://en.wikipedia.org/wiki/Generative_artificial_intelligence", "doc_id": "232f3bfa-45f1-0087-6903-d9d3d42a7d0d", "source_type": "web", "start_index": 93636, "chroma:document": "Midjourney Stable Diffusion Video Dream Machine Gen-3 Alpha Hailuo AI Kling Sora Veo VideoPoet Music Udio Suno AI Companies 01.AI Alibaba Anthropic Baichuan DeepSeek ElevenLabs Google DeepMind Hugging Face Kuaishou Meta AI MiniMax Mistral AI Moonshot AI OpenAI Runway Stability AI Synthesia xAI Zhipu AI  Category Commons vteDigital artToolsHardware Computer CGI 2D graphics 2.5D 3D graphics Xerox 3D printer Software Graphic art software Fractal-generating software Animation software Forms Art game Artificial intelligence art ASCII art Computer art scene Computer music Crypto art Cyberarts Digital illustration Digital imaging Digital painting Digital photography Digital poetry Digital architecture Electronic music Evolutionary art Fractal art Generative art Generative artificial intelligence Generative music GIF art Glitch art Immersion Interactive art Internet art Motion graphics Music visualization Photograph manipulation Pixel art Render art Software art Systems art Texture mapping"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 01371aa5-490a-4f8c-b857-e00c441883f0
encoding: FLOAT32
metadata:
{"source": "http://en.wikipedia.org/wiki/Generative_artificial_intelligence", "doc_id": "232f3bfa-45f1-0087-6903-d9d3d42a7d0d", "source_type": "web", "start_index": 94436, "chroma:document": "Generative music GIF art Glitch art Immersion Interactive art Internet art Motion graphics Music visualization Photograph manipulation Pixel art Render art Software art Systems art Texture mapping Virtual art Notable artists Refik Anadol Cory Arcangel Sougwen Chung Harold Cohen Char Davies Stephanie Dinkins Jake Elwes David Em Desmond Paul Henry Mario Klingemann Lynn Hershman Leeson Zachary Lieberman Margot Lovejoy Mauro Martino Eric Millikin Hamid Naderi Yeganeh Trevor Paglen Casey Reas Anna Ridler Ben Rubin (artist) Karl Sims Camille Utterback Pindar Van Arman Notable artworks Edmond de Belamy Barnsley fern Jesus Dress Up Listening Post (artwork) Remember To Rise Organizationsand conferences Artfutura Artmedia Austin Museum of Digital Art Computer Arts Society EVA Conferences Los Angeles Center for Digital Art Lumen Prize onedotzero SIGGRAPH V&A Digital Futures Retrieved from \"https://en.wikipedia.org/w/index.php?title=Generative_artificial_intelligence&oldid=1273153879\" Categories:"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 5f2d54ca-e135-4a0a-90f6-6fb88c5afd72
encoding: FLOAT32
metadata:
{"source": "http://en.wikipedia.org/wiki/Generative_artificial_intelligence", "doc_id": "232f3bfa-45f1-0087-6903-d9d3d42a7d0d", "source_type": "web", "start_index": 95237, "chroma:document": "Center for Digital Art Lumen Prize onedotzero SIGGRAPH V&A Digital Futures Retrieved from \"https://en.wikipedia.org/w/index.php?title=Generative_artificial_intelligence&oldid=1273153879\" Categories: Artificial neural networksDeep learningMachine learningGenerative artificial intelligence2020s in computing2023 in computing2024 in computing2025 in computingHidden categories: CS1 Japanese-language sources (ja)CS1 Italian-language sources (it)Articles with short descriptionShort description is different from WikidataUse mdy dates from October 2023Articles needing cleanup from July 2024All pages needing cleanupWikipedia list cleanup from July 2024  This page was last edited on 1 February 2025, at 01:05\u00a0(UTC). Text is available under the Creative Commons Attribution-ShareAlike 4.0 License; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia\u00ae is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization."}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 3a69d3b9-5a0c-4652-ba0c-307e2f195834
encoding: FLOAT32
metadata:
{"source": "http://en.wikipedia.org/wiki/Generative_artificial_intelligence", "doc_id": "232f3bfa-45f1-0087-6903-d9d3d42a7d0d", "source_type": "web", "start_index": 96032, "chroma:document": "additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia\u00ae is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.  Privacy policy About Wikipedia Disclaimers Contact Wikipedia Code of Conduct Developers Statistics Cookie statement Mobile view    Search   Search   Toggle the table of contents  Generative artificial intelligence     45 languages  Add topic"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 3afe9897-e8dd-49da-9d8a-99353aae8631
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 0, "chroma:document": "Chroma | \ud83e\udd9c\ufe0f\ud83d\udd17 LangChain  Skip to main contentJoin us at Interrupt: The Agent AI Conference by LangChain on May 13 & 14 in San Francisco!IntegrationsAPI ReferenceMoreContributingPeopleError referenceLangSmithLangGraphLangChain HubLangChain JS/TSv0.3v0.3v0.2v0.1\ud83d\udcacSearchProvidersAnthropicAWSGoogleHugging FaceMicrosoftOpenAIMoreProvidersAcreomActiveloop Deep LakeAerospikeAI21 LabsAimAINetworkAirbyteAirtableAlchemyAleph AlphaAlibaba CloudAnalyticDBAnnoyAnthropicAnyscaleApache Software FoundationApache DorisApifyAppleArangoDBArceeArcGISArgillaArizeArthurArxivAscendAskNewsAssemblyAIAstra DBAtlasAwaDBAWSAZLyricsBAAIBagelBagelDBBaichuanBaiduBananaBasetenBeamBeautiful SoupBibTeXBiliBiliBittensorBlackboardbookend.aiBoxBrave SearchBreebs (Open Knowledge)BrowserbaseBrowserlessByteDanceCassandraCerebrasCerebriumAIChaindeskChromaClarifaiClearMLClickHouseClickUpCloudflareClovaCnosDBCogniSwitchCohereCollege ConfidentialCometConfident AIConfluenceConneryContextCouchbaseCozeCrateDBC"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 788d7793-bf42-4135-9552-7f8912ddcc19
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 902, "chroma:document": "ConfidentialCometConfident AIConfluenceConneryContextCouchbaseCozeCrateDBC TransformersCTranslate2CubeDappierDashVectorDatabricksDatadog TracingDatadog LogsDataForSEODataheraldDedocDeepInfraDeepSeekDeepSparseDiffbotDingoDBDiscordDocArrayDoclingDoctranDocugamiDocusaurusDriaDropboxDSPyDuckDBDuckDuckGo SearchE2BEden AIElasticsearchElevenLabsEmbedchainEpsillaEtherscanEverly AIEverNoteExaFacebook - MetaFalkorDBFalkorDBFaunaFiddlerFigmaFireCrawlFireworks AIFlyteFMP Data (Financial Data Prep)Forefront AIFriendli AIGeopandasGitGitBookGitHubGitLabGoldenGoodfireGoogleSerper - Google Search APIGooseAIGPT4AllGradientGraphsignalGrobidGroqGutenbergHacker NewsHazy ResearchHeliconeHologresHTML to textHuaweiHugging FaceHyperbrowserIBMIEIT SystemsiFixitiFlytekIMSDbInfinispan VSInfinityInfinoIntelIuguJaguarJavelin AI GatewayJenkinsJina AIJohnsnowlabsJoplinKDB.AIKineticaKoboldAIKonkoKoNLPYK\u00f9zuLabel StudiolakeFSLanceDBLangChain Decorators"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: c7bd652e-b5d0-420e-b159-6ab35f0a7e5b
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 1634, "chroma:document": "SystemsiFixitiFlytekIMSDbInfinispan VSInfinityInfinoIntelIuguJaguarJavelin AI GatewayJenkinsJina AIJohnsnowlabsJoplinKDB.AIKineticaKoboldAIKonkoKoNLPYK\u00f9zuLabel StudiolakeFSLanceDBLangChain Decorators \u2728LanternLindormLinkupLiteLLMLlamaIndexLlama.cppLlamaEdgellamafileLLMonitorLocalAILog10MariTalkMarqoMediaWikiDumpMeilisearchMemcachedMetalMicrosoftMilvusMindsDBMinimaxMistralAIMLflow AI Gateway for LLMsMLflowMLXModalModelScopeModern TreasuryMomentoMongoDBMongoDB AtlasMotherduckMot\u00f6rheadMyScaleNAVERNeo4jNLPCloudNomicNotion DBNucliaNVIDIAObsidianOceanBaseOracle Cloud Infrastructure (OCI)OctoAIOllamaOntotext GraphDBOpenAIOpenLLMOpenSearchOpenWeatherMapOracleAI Vector SearchOutlineOutlinesPandasPaymanAIPebbloPerplexityPetalsPostgres EmbeddingPGVectorPineconePipelineAIPipeshiftPortkeyPredibasePrediction GuardPremAIPromptLayerPsychicPubMedPullMd LoaderPygmalionAIQdrantRAGatouillerank_bm25Ray ServeRebuffRedditRedisRemembrallReplicateRoamSema4 (fka Robocorp)RocksetRunhouseRWKV-4Salute"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: f55f6059-ee90-4533-a86a-db0407b12cba
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 2439, "chroma:document": "GuardPremAIPromptLayerPsychicPubMedPullMd LoaderPygmalionAIQdrantRAGatouillerank_bm25Ray ServeRebuffRedditRedisRemembrallReplicateRoamSema4 (fka Robocorp)RocksetRunhouseRWKV-4Salute DevicesSambaNovaSAPScrapeGraph AISearchApiSearxNG Search APISemaDBSerpAPIShale ProtocolSingleStoreDBscikit-learnSlackSnowflakespaCySparkSparkLLMSpreedlySQLiteStack ExchangeStarRocksStochasticAIStreamlitStripeSupabase (Postgres)NebulaTairTelegramTencentTensorFlow DatasetsTiDBTigerGraphTigrisTiloresTogether AI2MarkdownTranswarpTrelloTrubricsTruLensTwitterTypesenseUnstructuredUpstageupstashUpTrainUSearchVDMSVearchVectaraVespavliteVoyageAIWeights & BiasesWeights & Biases tracingWeights & Biases trackingWeatherWeaviateWhatsAppWhyLabsWikipediaWolfram AlphaWriterxAIXataXorbits Inference (Xinference)YahooYandexYeager.aiYellowbrick01.AIYouYouTubeZepZhipu AIZillizComponentsChat modelsChat modelsAI21 LabsAlibaba Cloud PAI EASAnthropic[Deprecated] Experimental Anthropic Tools WrapperAnyscaleAzure OpenAIAzure ML"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 2e83a5c7-27b6-4817-b184-532b908b8709
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 3275, "chroma:document": "AIZillizComponentsChat modelsChat modelsAI21 LabsAlibaba Cloud PAI EASAnthropic[Deprecated] Experimental Anthropic Tools WrapperAnyscaleAzure OpenAIAzure ML EndpointBaichuan ChatBaidu QianfanAWS BedrockCerebrasCloudflare Workers AICohereCoze ChatDappier AIDatabricksDeepInfraDeepSeekEden AIErnie Bot ChatEverlyAIFireworksChatFriendliGigaChatGoodfireGoogle AIGoogle Cloud Vertex AIGPTRouterGroqChatHuggingFaceIBM watsonx.aiJinaChatKineticaKonkoLiteLLMLiteLLM RouterLlama 2 ChatLlama APILlamaEdgeLlama.cppmaritalkMiniMaxMistralAIMLXModelScopeMoonshotNaverNVIDIA AI EndpointsChatOCIModelDeploymentOCIGenAIChatOctoAIOllamaOpenAIOutlinesPerplexityPipeshiftChatPredictionGuardPremAIPromptLayer ChatOpenAIRekaSambaNovaCloudSambaStudioSnowflake CortexsolarSparkLLM ChatNebula (Symbl.ai)Tencent HunyuanTogetherTongyi QwenUpstagevLLM ChatVolc Enging MaasWriterxAIYandexGPTChatYIYuan2.0ZHIPU AIRetrieversRetrieversActiveloop Deep MemoryAmazon KendraArceeArxivAskNewsAzure AI SearchBedrock (Knowledge"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: a8fb3a93-7791-44ed-b341-c02c594f6cd6
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 4083, "chroma:document": "QwenUpstagevLLM ChatVolc Enging MaasWriterxAIYandexGPTChatYIYuan2.0ZHIPU AIRetrieversRetrieversActiveloop Deep MemoryAmazon KendraArceeArxivAskNewsAzure AI SearchBedrock (Knowledge Bases)BM25BoxBREEBS (Open Knowledge)ChaindeskChatGPT pluginCohere rerankerCohere RAGDappierDocArrayDriaElasticSearch BM25ElasticsearchEmbedchainFlashRank rerankerFleet AI ContextGoogle DriveGoogle Vertex AI SearchIBM watsonx.aiJaguarDB Vector DatabaseKay.aiKinetica Vectorstore based RetrieverkNNLinkupSearchRetrieverLLMLingua Document CompressorLOTR (Merger Retriever)MetalMilvus Hybrid SearchNanoPQ (Product Quantization)needleOutlinePinecone Hybrid SearchPubMedQdrant Sparse VectorRAGatouilleRePhraseQueryRememberizerSEC filingSelf-querying retrieversSingleStoreDBSVMTavilySearchAPITF-IDF**NeuralDB**VespaWikipediaYou.comZep CloudZep Open SourceZilliz Cloud PipelineTools/ToolkitsToolsAINetwork ToolkitAlpha VantageAmadeus ToolkitArXivAskNewsAWS LambdaAzure AI Services ToolkitAzure Cognitive Services ToolkitAzure"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: b4b96ece-bd8a-4f28-9965-fc96040c2e55
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 4892, "chroma:document": "CloudZep Open SourceZilliz Cloud PipelineTools/ToolkitsToolsAINetwork ToolkitAlpha VantageAmadeus ToolkitArXivAskNewsAWS LambdaAzure AI Services ToolkitAzure Cognitive Services ToolkitAzure Container Apps dynamic sessionsShell (bash)Bearly Code InterpreterBing SearchBrave SearchCassandra Database ToolkitCDPChatGPT PluginsClickUp ToolkitCogniswitch ToolkitConnery Toolkit and ToolsDall-E Image GeneratorDappierDatabricks Unity Catalog (UC)DataForSEODataheraldDuckDuckGo SearchE2B Data AnalysisEden AIEleven Labs Text2SpeechExa SearchFile SystemFinancialDatasets ToolkitFMP DataGithub ToolkitGitlab ToolkitGmail ToolkitGolden QueryGoogle BooksGoogle Cloud Text-to-SpeechGoogle DriveGoogle FinanceGoogle ImagenGoogle JobsGoogle LensGoogle PlacesGoogle ScholarGoogle SearchGoogle SerperGoogle TrendsGradioGraphQLHuggingFace Hub ToolsHuman as a toolIFTTT WebHooksInfobipIonic Shopping ToolJenkinsJina SearchJira ToolkitJSON ToolkitLemon AgentLinkupSearchToolMemorizeMojeek SearchMultiOn ToolkitNASA"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 37ee1752-e284-461c-824d-f29217572bee
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 5714, "chroma:document": "Hub ToolsHuman as a toolIFTTT WebHooksInfobipIonic Shopping ToolJenkinsJina SearchJira ToolkitJSON ToolkitLemon AgentLinkupSearchToolMemorizeMojeek SearchMultiOn ToolkitNASA ToolkitNuclia UnderstandingNVIDIA Riva: ASR and TTSOffice365 ToolkitOpenAPI ToolkitNatural Language API ToolkitsOpenWeatherMapOracle AI Vector Search: Generate SummaryPandas DataframePassio NutritionAIPaymanAIPlayWright Browser ToolkitPolygon IO Toolkit and ToolsPowerBI ToolkitPubMedPython REPLReddit SearchRequests ToolkitRiza Code InterpreterRobocorp ToolkitSceneXplainScrapeGraphSearchApiSearxNG SearchSemantic Scholar API ToolSerpAPISlack ToolkitSpark SQL ToolkitSQLDatabase ToolkitStackExchangeSteam ToolkitStripeTavily SearchTiloresTwilioUpstageWikidataWikipediaWolfram AlphaYahoo Finance NewsYou.com SearchYouTubeZapier Natural Language ActionsZenGuard AIDocument loadersDocument loadersacreomAirbyteLoaderAirbyte CDK (Deprecated)Airbyte Gong (Deprecated)Airbyte Hubspot (Deprecated)Airbyte JSON (Deprecated)Airbyte"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 6200e5ae-a68c-460a-bf92-2ea00bba2844
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 6516, "chroma:document": "Natural Language ActionsZenGuard AIDocument loadersDocument loadersacreomAirbyteLoaderAirbyte CDK (Deprecated)Airbyte Gong (Deprecated)Airbyte Hubspot (Deprecated)Airbyte JSON (Deprecated)Airbyte Salesforce (Deprecated)Airbyte Shopify (Deprecated)Airbyte Stripe (Deprecated)Airbyte Typeform (Deprecated)Airbyte Zendesk Support (Deprecated)AirtableAlibaba Cloud MaxComputeAmazon TextractApify DatasetArcGISArxivLoaderAssemblyAI Audio TranscriptsAstraDBAsync ChromiumAsyncHtmlAthenaAWS S3 DirectoryAWS S3 FileAZLyricsAzure AI DataAzure Blob Storage ContainerAzure Blob Storage FileAzure AI Document IntelligenceBibTeXBiliBiliBlackboardBlockchainBoxBrave SearchBrowserbaseBrowserlessBSHTMLLoaderCassandraChatGPT DataCollege ConfidentialConcurrent LoaderConfluenceCoNLL-UCopy PasteCouchbaseCSVCube Semantic LayerDatadog LogsDedocDiffbotDiscordDoclingDocugamiDocusaurusDropboxDuckDBEmailEPubEtherscanEverNoteexample_dataFacebook ChatFaunaFigmaFireCrawlGeopandasGitGitBookGitHubGlue CatalogGoogle AlloyDB"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 69b75fb5-f2cd-4a79-add5-9c50a9d3a694
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 7319, "chroma:document": "LayerDatadog LogsDedocDiffbotDiscordDoclingDocugamiDocusaurusDropboxDuckDBEmailEPubEtherscanEverNoteexample_dataFacebook ChatFaunaFigmaFireCrawlGeopandasGitGitBookGitHubGlue CatalogGoogle AlloyDB for PostgreSQLGoogle BigQueryGoogle BigtableGoogle Cloud SQL for SQL serverGoogle Cloud SQL for MySQLGoogle Cloud SQL for PostgreSQLGoogle Cloud Storage DirectoryGoogle Cloud Storage FileGoogle Firestore in Datastore ModeGoogle DriveGoogle El Carro for Oracle WorkloadsGoogle Firestore (Native Mode)Google Memorystore for RedisGoogle SpannerGoogle Speech-to-Text Audio TranscriptsGrobidGutenbergHacker NewsHuawei OBS DirectoryHuawei OBS FileHuggingFace datasetHyperbrowserLoaderiFixitImagesImage captionsIMSDbIuguJoplinJSONLoaderJupyter NotebookKineticalakeFSLangSmithLarkSuite (FeiShu)LLM SherpaMastodonMathPixPDFLoaderMediaWiki DumpMerge Documents LoadermhtmlMicrosoft ExcelMicrosoft OneDriveMicrosoft OneNoteMicrosoft PowerPointMicrosoft SharePointMicrosoft WordNear BlockchainModern"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: ae319978-e29d-4b44-8b8c-d75d52ccc1cb
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 8105, "chroma:document": "SherpaMastodonMathPixPDFLoaderMediaWiki DumpMerge Documents LoadermhtmlMicrosoft ExcelMicrosoft OneDriveMicrosoft OneNoteMicrosoft PowerPointMicrosoft SharePointMicrosoft WordNear BlockchainModern TreasuryMongoDBNeedle Document LoaderNews URLNotion DB 2/2NucliaObsidianOpen Document Format (ODT)Open City DataOracle Autonomous DatabaseOracle AI Vector Search: Document ProcessingOrg-modePandas DataFrameparsersPDFMinerPDFPlumberPebblo Safe DocumentLoaderPolars DataFramePsychicPubMedPullMdLoaderPyMuPDFLoaderPyPDFDirectoryLoaderPyPDFium2LoaderPyPDFLoaderPySparkQuipReadTheDocs DocumentationRecursive URLRedditRoamRocksetrspaceRSS FeedsRSTscrapflyScrapingAntSitemapSlackSnowflakeSource CodeSpiderSpreedlyStripeSubtitleSurrealDBTelegramTencent COS DirectoryTencent COS FileTensorFlow DatasetsTiDB2MarkdownTOMLTrelloTSVTwitterUnstructuredUnstructuredMarkdownLoaderUnstructuredPDFLoaderUpstageURLVsdxWeatherWebBaseLoaderWhatsApp ChatWikipediaUnstructuredXMLLoaderXorbits Pandas DataFrameYouTube"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: abbbf786-575e-41eb-a773-19fd171e0c1f
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 9030, "chroma:document": "ChatWikipediaUnstructuredXMLLoaderXorbits Pandas DataFrameYouTube audioYouTube transcriptsYoutubeLoaderDLYuqueZeroxPDFLoaderVector storesVector storesActiveloop Deep LakeAerospikeAlibaba Cloud OpenSearchAnalyticDBAnnoyApache DorisApertureDBAstra DB Vector StoreAtlasAwaDBAzure Cosmos DB Mongo vCoreAzure Cosmos DB No SQLAzure AI SearchBagelBagelDBBaidu Cloud ElasticSearch VectorSearchBaidu VectorDBApache CassandraChromaClarifaiClickHouseCouchbaseDashVectorDatabricksDingoDBDocArray HnswSearchDocArray InMemorySearchAmazon Document DBDuckDBChina Mobile ECloud ElasticSearch VectorSearchElasticsearchEpsillaFaissFaiss (Async)FalkorDBVectorStoreGoogle AlloyDB for PostgreSQLGoogle BigQuery Vector SearchGoogle Cloud SQL for MySQLGoogle Cloud SQL for PostgreSQLFirestoreGoogle Memorystore for RedisGoogle SpannerGoogle Vertex AI Feature StoreGoogle Vertex AI Vector SearchHippoHologresInfinispanJaguar Vector DatabaseKDB.AIKineticaLanceDBLanternLindormLLMRailsManticoreSearch"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: c455b953-f44e-4c41-8b4c-4ce5bac288ca
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 9805, "chroma:document": "Memorystore for RedisGoogle SpannerGoogle Vertex AI Feature StoreGoogle Vertex AI Vector SearchHippoHologresInfinispanJaguar Vector DatabaseKDB.AIKineticaLanceDBLanternLindormLLMRailsManticoreSearch VectorStoreMarqoMeilisearchAmazon MemoryDBMilvusMomento Vector Index (MVI)MongoDB AtlasMyScaleNeo4j Vector IndexNucliaDBOceanbaseOpenSearchOracle AI Vector Search: Vector StorePathwayPostgres EmbeddingPGVecto.rsPGVectorPineconeQdrantRedisRelytRocksetSAP HANA Cloud Vector EngineScaNNSemaDBSingleStoreDBscikit-learnSQLiteVecSQLite-VSSSQLServerStarRocksSupabase (Postgres)SurrealDBTablestoreTairTencent Cloud VectorDBThirdAI NeuralDBTiDB VectorTigrisTileDBTimescale Vector (Postgres)TypesenseUpstash VectorUSearchValdIntel's Visual Data Management System (VDMS)VearchVectaraVespaviking DBvliteWeaviateXataYellowbrickZepZep CloudZillizEmbedding modelsEmbedding modelsAI21Aleph AlphaAnyscaleascendAwaDBAzureOpenAIBaichuan Text EmbeddingsBaidu QianfanBedrockBGE on Hugging FaceBookend AIClarifaiCloudflare"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: cc41d44a-ad83-444c-8854-88b23712c1bb
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 10625, "chroma:document": "CloudZillizEmbedding modelsEmbedding modelsAI21Aleph AlphaAnyscaleascendAwaDBAzureOpenAIBaichuan Text EmbeddingsBaidu QianfanBedrockBGE on Hugging FaceBookend AIClarifaiCloudflare Workers AIClova EmbeddingsCohereDashScopeDatabricksDeepInfraEDEN AIElasticsearchEmbaasERNIEFake EmbeddingsFastEmbed by QdrantFireworksGigaChatGoogle Generative AI EmbeddingsGoogle Vertex AIGPT4AllGradientHugging FaceIBM watsonx.aiInfinityInstruct Embeddings on Hugging FaceIPEX-LLM: Local BGE Embeddings on Intel CPUIPEX-LLM: Local BGE Embeddings on Intel GPUIntel\u00ae Extension for Transformers Quantized Text EmbeddingsJinaJohn Snow LabsLASER Language-Agnostic SEntence Representations Embeddings by Meta AILindormLlama.cppllamafileLLMRailsLocalAIMiniMaxMistralAImodel2vecModelScopeMosaicMLNaverNLP CloudNomicNVIDIA NIMsOracle Cloud Infrastructure Generative AIOllamaOpenClipOpenAIOpenVINOEmbedding Documents using Optimized and Quantized EmbeddersOracle AI Vector Search: Generate EmbeddingsOVHcloudPinecone"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 944b88fd-e762-4855-bfed-510b173143ff
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 11420, "chroma:document": "NIMsOracle Cloud Infrastructure Generative AIOllamaOpenClipOpenAIOpenVINOEmbedding Documents using Optimized and Quantized EmbeddersOracle AI Vector Search: Generate EmbeddingsOVHcloudPinecone EmbeddingsPredictionGuardEmbeddingsPremAISageMakerSambaStudioSelf HostedSentence Transformers on Hugging FaceSolarSpaCySparkLLM Text EmbeddingsTensorFlow HubText Embeddings InferenceTextEmbed - Embedding Inference ServerTitan TakeoffTogether AIUpstageVolc EngineVoyage AIXorbits inference (Xinference)YandexGPTZhipuAIOtherComponentsVector storesChromaOn this pageChroma This notebook covers how to get started with the Chroma vector store. Chroma is a AI-native open-source vector database focused on developer productivity and happiness. Chroma is licensed under Apache 2.0. View the full docs of Chroma at this page, and find the API reference for the LangChain integration at this page. Setup\u200b To access Chroma vector stores you'll need to install the langchain-chroma integration package. pip install"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 26a5b879-323d-4437-baeb-cd953d937803
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 12218, "chroma:document": "at this page, and find the API reference for the LangChain integration at this page. Setup\u200b To access Chroma vector stores you'll need to install the langchain-chroma integration package. pip install -qU \"langchain-chroma>=0.1.2\" Credentials\u200b You can use the Chroma vector store without any credentials, simply installing the package above is enough! If you want to get best in-class automated tracing of your model calls you can also set your LangSmith API key by uncommenting below: # os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")# os.environ[\"LANGSMITH_TRACING\"] = \"true\" Initialization\u200b Basic Initialization\u200b Below is a basic initialization, including the use of a directory to save the data locally. Select embeddings model:OpenAI\u25beOpenAIAzureGoogleAWSHuggingFaceOllamaCohereMistralAINomicNVIDIAVoyage AIFakepip install -qU langchain-openaiimport getpassimport osif not os.environ.get(\"OPENAI_API_KEY\"): os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"Enter"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: bd41bbd6-6300-4689-890c-9560bacd5493
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 13062, "chroma:document": "AIFakepip install -qU langchain-openaiimport getpassimport osif not os.environ.get(\"OPENAI_API_KEY\"): os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"Enter API key for OpenAI: \")from langchain_openai import OpenAIEmbeddingsembeddings = OpenAIEmbeddings(model=\"text-embedding-3-large\") from langchain_chroma import Chromavector_store = Chroma(  collection_name=\"example_collection\",  embedding_function=embeddings,  persist_directory=\"./chroma_langchain_db\", # Where to save data locally, remove if not necessary) Initialization from client\u200b You can also initialize from a Chroma client, which is particularly useful if you want easier access to the underlying database. import chromadbpersistent_client = chromadb.PersistentClient()collection = persistent_client.get_or_create_collection(\"collection_name\")collection.add(ids=[\"1\", \"2\", \"3\"], documents=[\"a\", \"b\", \"c\"])vector_store_from_client = Chroma(  client=persistent_client,  collection_name=\"collection_name\",  embedding_function=embeddings,)"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 6966d210-dadb-4ff0-a80d-df22fedcf670
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 13893, "chroma:document": "\"2\", \"3\"], documents=[\"a\", \"b\", \"c\"])vector_store_from_client = Chroma(  client=persistent_client,  collection_name=\"collection_name\",  embedding_function=embeddings,) Manage vector store\u200b Once you have created your vector store, we can interact with it by adding and deleting different items. Add items to vector store\u200b We can add items to our vector store by using the add_documents function. from uuid import uuid4from langchain_core.documents import Documentdocument_1 = Document(  page_content=\"I had chocolate chip pancakes and scrambled eggs for breakfast this morning.\",  metadata={\"source\": \"tweet\"},  id=1,)document_2 = Document(  page_content=\"The weather forecast for tomorrow is cloudy and overcast, with a high of 62 degrees.\",  metadata={\"source\": \"news\"},  id=2,)document_3 = Document(  page_content=\"Building an exciting new project with LangChain - come check it out!\",  metadata={\"source\": \"tweet\"},  id=3,)document_4 = Document(  page_content=\"Robbers broke into the city bank"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 5408f738-613c-451e-974e-466de2b09e6c
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 14696, "chroma:document": "page_content=\"Building an exciting new project with LangChain - come check it out!\",  metadata={\"source\": \"tweet\"},  id=3,)document_4 = Document(  page_content=\"Robbers broke into the city bank and stole $1 million in cash.\",  metadata={\"source\": \"news\"},  id=4,)document_5 = Document(  page_content=\"Wow! That was an amazing movie. I can't wait to see it again.\",  metadata={\"source\": \"tweet\"},  id=5,)document_6 = Document(  page_content=\"Is the new iPhone worth the price? Read this review to find out.\",  metadata={\"source\": \"website\"},  id=6,)document_7 = Document(  page_content=\"The top 10 soccer players in the world right now.\",  metadata={\"source\": \"website\"},  id=7,)document_8 = Document(  page_content=\"LangGraph is the best framework for building stateful, agentic applications!\",  metadata={\"source\": \"tweet\"},  id=8,)document_9 = Document(  page_content=\"The stock market is down 500 points today due to fears of a recession.\",  metadata={\"source\": \"news\"},  id=9,)document_10 ="}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: cab4f712-9073-404d-9093-d593e15caffa
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 15492, "chroma:document": "metadata={\"source\": \"tweet\"},  id=8,)document_9 = Document(  page_content=\"The stock market is down 500 points today due to fears of a recession.\",  metadata={\"source\": \"news\"},  id=9,)document_10 = Document(  page_content=\"I have a bad feeling I am going to get deleted :(\",  metadata={\"source\": \"tweet\"},  id=10,)documents = [  document_1,  document_2,  document_3,  document_4,  document_5,  document_6,  document_7,  document_8,  document_9,  document_10,]uuids = [str(uuid4()) for _ in range(len(documents))]vector_store.add_documents(documents=documents, ids=uuids)API Reference:Document ['f22ed484-6db3-4b76-adb1-18a777426cd6', 'e0d5bab4-6453-4511-9a37-023d9d288faa', '877d76b8-3580-4d9e-a13f-eed0fa3d134a', '26eaccab-81ce-4c0a-8e76-bf542647df18', 'bcaa8239-7986-4050-bf40-e14fb7dab997', 'cdc44b38-a83f-4e49-b249-7765b334e09d', 'a7a35354-2687-4bc2-8242-3849a4d18d34', '8780caf1-d946-4f27-a707-67d037e9e1d8', 'dec6af2a-7326-408f-893d-7d7d717dfda9', '3b18e210-bb59-47a0-8e17-c8e51176ea5e']"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 5d34f233-6edc-49d6-a454-56d90d155c43
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 16287, "chroma:document": "'cdc44b38-a83f-4e49-b249-7765b334e09d', 'a7a35354-2687-4bc2-8242-3849a4d18d34', '8780caf1-d946-4f27-a707-67d037e9e1d8', 'dec6af2a-7326-408f-893d-7d7d717dfda9', '3b18e210-bb59-47a0-8e17-c8e51176ea5e'] Update items in vector store\u200b Now that we have added documents to our vector store, we can update existing documents by using the update_documents function. updated_document_1 = Document(  page_content=\"I had chocolate chip pancakes and fried eggs for breakfast this morning.\",  metadata={\"source\": \"tweet\"},  id=1,)updated_document_2 = Document(  page_content=\"The weather forecast for tomorrow is sunny and warm, with a high of 82 degrees.\",  metadata={\"source\": \"news\"},  id=2,)vector_store.update_document(document_id=uuids[0], document=updated_document_1)# You can also update multiple documents at oncevector_store.update_documents(  ids=uuids[:2], documents=[updated_document_1, updated_document_2]) Delete items from vector store\u200b We can also delete items from our vector store as follows:"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 1ecbe53a-f8e6-4100-bbfc-5c9581e7a0fb
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 17088, "chroma:document": "at oncevector_store.update_documents(  ids=uuids[:2], documents=[updated_document_1, updated_document_2]) Delete items from vector store\u200b We can also delete items from our vector store as follows: vector_store.delete(ids=uuids[-1]) Query vector store\u200b Once your vector store has been created and the relevant documents have been added you will most likely wish to query it during the running of your chain or agent. Query directly\u200b Similarity search\u200b Performing a simple similarity search can be done as follows: results = vector_store.similarity_search(  \"LangChain provides abstractions to make working with LLMs easy\",  k=2,  filter={\"source\": \"tweet\"},)for res in results:  print(f\"* {res.page_content} [{res.metadata}]\") * Building an exciting new project with LangChain - come check it out! [{'source': 'tweet'}]* LangGraph is the best framework for building stateful, agentic applications! [{'source': 'tweet'}] Similarity search with score\u200b If you want to execute a similarity search and"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: f42c5a60-9231-42b7-842e-9c144f039750
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 17885, "chroma:document": "[{'source': 'tweet'}]* LangGraph is the best framework for building stateful, agentic applications! [{'source': 'tweet'}] Similarity search with score\u200b If you want to execute a similarity search and receive the corresponding scores you can run: results = vector_store.similarity_search_with_score(  \"Will it be hot tomorrow?\", k=1, filter={\"source\": \"news\"})for res, score in results:  print(f\"* [SIM={score:3f}] {res.page_content} [{res.metadata}]\") * [SIM=1.726390] The stock market is down 500 points today due to fears of a recession. [{'source': 'news'}] Search by vector\u200b You can also search by vector: results = vector_store.similarity_search_by_vector(  embedding=embeddings.embed_query(\"I love green eggs and ham!\"), k=1)for doc in results:  print(f\"* {doc.page_content} [{doc.metadata}]\") * I had chocalate chip pancakes and fried eggs for breakfast this morning. [{'source': 'tweet'}] Other search methods\u200b There are a variety of other search methods that are not covered in this"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: e8cfc059-6125-4994-8e0b-f3a19dbb3330
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 18684, "chroma:document": "* I had chocalate chip pancakes and fried eggs for breakfast this morning. [{'source': 'tweet'}] Other search methods\u200b There are a variety of other search methods that are not covered in this notebook, such as MMR search or searching by vector. For a full list of the search abilities available for AstraDBVectorStore check out the API reference. Query by turning into retriever\u200b You can also transform the vector store into a retriever for easier usage in your chains. For more information on the different search types and kwargs you can pass, please visit the API reference here. retriever = vector_store.as_retriever(  search_type=\"mmr\", search_kwargs={\"k\": 1, \"fetch_k\": 5})retriever.invoke(\"Stealing from the bank is a crime\", filter={\"source\": \"news\"}) [Document(metadata={'source': 'news'}, page_content='Robbers broke into the city bank and stole $1 million in cash.')] Usage for retrieval-augmented generation\u200b For guides on how to use this vector store for retrieval-augmented generation"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: fc37e4c9-7754-4f73-a03e-40d968442727
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 19483, "chroma:document": "page_content='Robbers broke into the city bank and stole $1 million in cash.')] Usage for retrieval-augmented generation\u200b For guides on how to use this vector store for retrieval-augmented generation (RAG), see the following sections: Tutorials How-to: Question and answer with RAG Retrieval conceptual docs API reference\u200b For detailed documentation of all Chroma vector store features and configurations head to the API reference: https://python.langchain.com/api_reference/chroma/vectorstores/langchain_chroma.vectorstores.Chroma.html Related\u200b Vector store conceptual guide Vector store how-to guides Edit this pageWas this page helpful?PreviousApache CassandraNextClarifaiSetupCredentialsInitializationBasic InitializationInitialization from clientManage vector storeAdd items to vector storeUpdate items in vector storeDelete items from vector storeQuery vector storeQuery directlyQuery by turning into retrieverUsage for retrieval-augmented generationAPI"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 58323b85-3862-4adb-81a1-df07e1121fb1
encoding: FLOAT32
metadata:
{"source": "https://python.langchain.com/docs/integrations/vectorstores/chroma/", "doc_id": "474cc0af-bd33-9f5e-b1f2-aaca8377e551", "source_type": "web", "start_index": 20248, "chroma:document": "storeAdd items to vector storeUpdate items in vector storeDelete items from vector storeQuery vector storeQuery directlyQuery by turning into retrieverUsage for retrieval-augmented generationAPI referenceRelatedCommunityTwitterGitHubOrganizationPythonJS/TSMoreHomepageBlogYouTubeCopyright \u00a9 2025 LangChain, Inc."}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 008a7981-133f-4d71-bf19-ad5072568bd2
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 0, "chroma:document": "LLM Powered Autonomous Agents | Lil'Log        Lil'Log   |  Posts Archive Search Tags FAQ emojisearch.app    LLM Powered Autonomous Agents Date: June 23, 2023 | Estimated Reading Time: 31 min | Author: Lilian Weng  Table of Contents  Agent System Overview Component One: Planning Task Decomposition Self-Reflection  Component Two: Memory Types of Memory Maximum Inner Product Search (MIPS)  Component Three: Tool Use Case Studies Scientific Discovery Agent Generative Agents Simulation Proof-of-Concept Examples  Challenges Citation References Building agents with LLM (large language model) as its core controller is a cool concept. Several proof-of-concepts demos, such as AutoGPT, GPT-Engineer and BabyAGI, serve as inspiring examples. The potentiality of LLM extends beyond generating well-written copies, stories, essays and programs; it can be framed as a powerful general problem solver. Agent System Overview# In a LLM-powered autonomous agent system, LLM functions as the agent\u2019s brain,"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: d428887c-0d74-4c1d-8b72-c4d6ef7eff20
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 802, "chroma:document": "copies, stories, essays and programs; it can be framed as a powerful general problem solver. Agent System Overview# In a LLM-powered autonomous agent system, LLM functions as the agent\u2019s brain, complemented by several key components: Planning Subgoal and decomposition: The agent breaks down large tasks into smaller, manageable subgoals, enabling efficient handling of complex tasks. Reflection and refinement: The agent can do self-criticism and self-reflection over past actions, learn from mistakes and refine them for future steps, thereby improving the quality of final results.  Memory Short-term memory: I would consider all the in-context learning (See Prompt Engineering) as utilizing short-term memory of the model to learn. Long-term memory: This provides the agent with the capability to retain and recall (infinite) information over extended periods, often by leveraging an external vector store and fast retrieval.  Tool use The agent learns to call external APIs for extra"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 1af82615-7cc1-48d7-9303-c54531ca3d50
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 1600, "chroma:document": "to retain and recall (infinite) information over extended periods, often by leveraging an external vector store and fast retrieval.  Tool use The agent learns to call external APIs for extra information that is missing from the model weights (often hard to change after pre-training), including current information, code execution capability, access to proprietary information sources and more. Fig. 1. Overview of a LLM-powered autonomous agent system. Component One: Planning# A complicated task usually involves many steps. An agent needs to know what they are and plan ahead. Task Decomposition# Chain of thought (CoT; Wei et al. 2022) has become a standard prompting technique for enhancing model performance on complex tasks. The model is instructed to \u201cthink step by step\u201d to utilize more test-time computation to decompose hard tasks into smaller and simpler steps. CoT transforms big tasks into multiple manageable tasks and shed lights into an interpretation of the model\u2019s thinking"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: c97b31e6-a625-4a30-a560-f96908575589
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 2396, "chroma:document": "test-time computation to decompose hard tasks into smaller and simpler steps. CoT transforms big tasks into multiple manageable tasks and shed lights into an interpretation of the model\u2019s thinking process. Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning possibilities at each step. It first decomposes the problem into multiple thought steps and generates multiple thoughts per step, creating a tree structure. The search process can be BFS (breadth-first search) or DFS (depth-first search) with each state evaluated by a classifier (via a prompt) or majority vote. Task decomposition can be done (1) by LLM with simple prompting like \"Steps for XYZ.\\n1.\", \"What are the subgoals for achieving XYZ?\", (2) by using task-specific instructions; e.g. \"Write a story outline.\" for writing a novel, or (3) with human inputs. Another quite distinct approach, LLM+P (Liu et al. 2023), involves relying on an external classical planner to do long-horizon planning. This"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 0f245632-c9ea-41c8-9688-326d3e9e2777
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 3190, "chroma:document": "outline.\" for writing a novel, or (3) with human inputs. Another quite distinct approach, LLM+P (Liu et al. 2023), involves relying on an external classical planner to do long-horizon planning. This approach utilizes the Planning Domain Definition Language (PDDL) as an intermediate interface to describe the planning problem. In this process, LLM (1) translates the problem into \u201cProblem PDDL\u201d, then (2) requests a classical planner to generate a PDDL plan based on an existing \u201cDomain PDDL\u201d, and finally (3) translates the PDDL plan back into natural language. Essentially, the planning step is outsourced to an external tool, assuming the availability of domain-specific PDDL and a suitable planner which is common in certain robotic setups but not in many other domains. Self-Reflection# Self-reflection is a vital aspect that allows autonomous agents to improve iteratively by refining past action decisions and correcting previous mistakes. It plays a crucial role in real-world tasks where"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 4c637f9d-37dc-4ee4-8bd5-cff45a91f578
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 3998, "chroma:document": "is a vital aspect that allows autonomous agents to improve iteratively by refining past action decisions and correcting previous mistakes. It plays a crucial role in real-world tasks where trial and error are inevitable. ReAct (Yao et al. 2023) integrates reasoning and acting within LLM by extending the action space to be a combination of task-specific discrete actions and the language space. The former enables LLM to interact with the environment (e.g. use Wikipedia search API), while the latter prompting LLM to generate reasoning traces in natural language. The ReAct prompt template incorporates explicit steps for LLM to think, roughly formatted as: Thought: ... Action: ... Observation: ... ... (Repeated many times) Fig. 2. Examples of reasoning trajectories for knowledge-intensive tasks (e.g. HotpotQA, FEVER) and decision-making tasks (e.g. AlfWorld Env, WebShop). (Image source: Yao et al. 2023). In both experiments on knowledge-intensive tasks and decision-making tasks, ReAct"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 849b0f15-3b8f-4e53-bf2e-53ec5a8cd260
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 4793, "chroma:document": "tasks (e.g. HotpotQA, FEVER) and decision-making tasks (e.g. AlfWorld Env, WebShop). (Image source: Yao et al. 2023). In both experiments on knowledge-intensive tasks and decision-making tasks, ReAct works better than the Act-only baseline where Thought: \u2026 step is removed. Reflexion (Shinn & Labash 2023) is a framework to equip agents with dynamic memory and self-reflection capabilities to improve reasoning skills. Reflexion has a standard RL setup, in which the reward model provides a simple binary reward and the action space follows the setup in ReAct where the task-specific action space is augmented with language to enable complex reasoning steps. After each action $a_t$, the agent computes a heuristic $h_t$ and optionally may decide to reset the environment to start a new trial depending on the self-reflection results. Fig. 3. Illustration of the Reflexion framework. (Image source: Shinn & Labash, 2023) The heuristic function determines when the trajectory is inefficient or"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: a6d3abe9-9644-494b-be98-c72d911ed580
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 5586, "chroma:document": "depending on the self-reflection results. Fig. 3. Illustration of the Reflexion framework. (Image source: Shinn & Labash, 2023) The heuristic function determines when the trajectory is inefficient or contains hallucination and should be stopped. Inefficient planning refers to trajectories that take too long without success. Hallucination is defined as encountering a sequence of consecutive identical actions that lead to the same observation in the environment. Self-reflection is created by showing two-shot examples to LLM and each example is a pair of (failed trajectory, ideal reflection for guiding future changes in the plan). Then reflections are added into the agent\u2019s working memory, up to three, to be used as context for querying LLM. Fig. 4. Experiments on AlfWorld Env and HotpotQA. Hallucination is a more common failure than inefficient planning in AlfWorld. (Image source: Shinn & Labash, 2023) Chain of Hindsight (CoH; Liu et al. 2023) encourages the model to improve on its own"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 2c22c2a8-aa6e-401d-af37-fc678a2ec38d
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 6385, "chroma:document": "Hallucination is a more common failure than inefficient planning in AlfWorld. (Image source: Shinn & Labash, 2023) Chain of Hindsight (CoH; Liu et al. 2023) encourages the model to improve on its own outputs by explicitly presenting it with a sequence of past outputs, each annotated with feedback. Human feedback data is a collection of $D_h = \\{(x, y_i , r_i , z_i)\\}_{i=1}^n$, where $x$ is the prompt, each $y_i$ is a model completion, $r_i$ is the human rating of $y_i$, and $z_i$ is the corresponding human-provided hindsight feedback. Assume the feedback tuples are ranked by reward, $r_n \\geq r_{n-1} \\geq \\dots \\geq r_1$ The process is supervised fine-tuning where the data is a sequence in the form of $\\tau_h = (x, z_i, y_i, z_j, y_j, \\dots, z_n, y_n)$, where $\\leq i \\leq j \\leq n$. The model is finetuned to only predict $y_n$ where conditioned on the sequence prefix, such that the model can self-reflect to produce better output based on the feedback sequence. The model can optionally"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 61dcba75-6266-427f-b007-611b23ffaa3b
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 7189, "chroma:document": "is finetuned to only predict $y_n$ where conditioned on the sequence prefix, such that the model can self-reflect to produce better output based on the feedback sequence. The model can optionally receive multiple rounds of instructions with human annotators at test time. To avoid overfitting, CoH adds a regularization term to maximize the log-likelihood of the pre-training dataset. To avoid shortcutting and copying (because there are many common words in feedback sequences), they randomly mask 0% - 5% of past tokens during training. The training dataset in their experiments is a combination of WebGPT comparisons, summarization from human feedback and human preference dataset. Fig. 5. After fine-tuning with CoH, the model can follow instructions to produce outputs with incremental improvement in a sequence. (Image source: Liu et al. 2023) The idea of CoH is to present a history of sequentially improved outputs in context and train the model to take on the trend to produce better"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: b42c0345-c1fb-4241-a1ae-43c00d56243b
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 7992, "chroma:document": "in a sequence. (Image source: Liu et al. 2023) The idea of CoH is to present a history of sequentially improved outputs in context and train the model to take on the trend to produce better outputs. Algorithm Distillation (AD; Laskin et al. 2023) applies the same idea to cross-episode trajectories in reinforcement learning tasks, where an algorithm is encapsulated in a long history-conditioned policy. Considering that an agent interacts with the environment many times and in each episode the agent gets a little better, AD concatenates this learning history and feeds that into the model. Hence we should expect the next predicted action to lead to better performance than previous trials. The goal is to learn the process of RL instead of training a task-specific policy itself. Fig. 6. Illustration of how Algorithm Distillation (AD) works. (Image source: Laskin et al. 2023). The paper hypothesizes that any algorithm that generates a set of learning histories can be distilled into a neural"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 67d82733-d6e3-489e-9276-9f784f006e5b
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 8798, "chroma:document": "of how Algorithm Distillation (AD) works. (Image source: Laskin et al. 2023). The paper hypothesizes that any algorithm that generates a set of learning histories can be distilled into a neural network by performing behavioral cloning over actions. The history data is generated by a set of source policies, each trained for a specific task. At the training stage, during each RL run, a random task is sampled and a subsequence of multi-episode history is used for training, such that the learned policy is task-agnostic. In reality, the model has limited context window length, so episodes should be short enough to construct multi-episode history. Multi-episodic contexts of 2-4 episodes are necessary to learn a near-optimal in-context RL algorithm. The emergence of in-context RL requires long enough context. In comparison with three baselines, including ED (expert distillation, behavior cloning with expert trajectories instead of learning history), source policy (used for generating"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 40ce8284-230c-4e48-8bad-cc342cdef50b
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 9591, "chroma:document": "long enough context. In comparison with three baselines, including ED (expert distillation, behavior cloning with expert trajectories instead of learning history), source policy (used for generating trajectories for distillation by UCB), RL^2 (Duan et al. 2017; used as upper bound since it needs online RL), AD demonstrates in-context RL with performance getting close to RL^2 despite only using offline RL and learns much faster than other baselines. When conditioned on partial training history of the source policy, AD also improves much faster than ED baseline. Fig. 7. Comparison of AD, ED, source policy and RL^2 on environments that require memory and exploration. Only binary reward is assigned. The source policies are trained with A3C for \"dark\" environments and DQN for watermaze.(Image source: Laskin et al. 2023) Component Two: Memory# (Big thank you to ChatGPT for helping me draft this section. I\u2019ve learned a lot about the human brain and data structure for fast MIPS in my"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 0e091aa8-2e0d-44dc-bedd-7c1461d3e312
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 10390, "chroma:document": "source: Laskin et al. 2023) Component Two: Memory# (Big thank you to ChatGPT for helping me draft this section. I\u2019ve learned a lot about the human brain and data structure for fast MIPS in my conversations with ChatGPT.) Types of Memory# Memory can be defined as the processes used to acquire, store, retain, and later retrieve information. There are several types of memory in human brains.  Sensory Memory: This is the earliest stage of memory, providing the ability to retain impressions of sensory information (visual, auditory, etc) after the original stimuli have ended. Sensory memory typically only lasts for up to a few seconds. Subcategories include iconic memory (visual), echoic memory (auditory), and haptic memory (touch).  Short-Term Memory (STM) or Working Memory: It stores information that we are currently aware of and needed to carry out complex cognitive tasks such as learning and reasoning. Short-term memory is believed to have the capacity of about 7 items (Miller 1956) and"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 36ba3c5b-24ca-4df6-b0b5-9a0ae110411d
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 11193, "chroma:document": "that we are currently aware of and needed to carry out complex cognitive tasks such as learning and reasoning. Short-term memory is believed to have the capacity of about 7 items (Miller 1956) and lasts for 20-30 seconds.  Long-Term Memory (LTM): Long-term memory can store information for a remarkably long time, ranging from a few days to decades, with an essentially unlimited storage capacity. There are two subtypes of LTM: Explicit / declarative memory: This is memory of facts and events, and refers to those memories that can be consciously recalled, including episodic memory (events and experiences) and semantic memory (facts and concepts). Implicit / procedural memory: This type of memory is unconscious and involves skills and routines that are performed automatically, like riding a bike or typing on a keyboard. Fig. 8. Categorization of human memory. We can roughly consider the following mappings: Sensory memory as learning embedding representations for raw inputs, including"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 2b9a6d23-7c19-4c9c-bc0c-3d140d45b7da
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 11989, "chroma:document": "a bike or typing on a keyboard. Fig. 8. Categorization of human memory. We can roughly consider the following mappings: Sensory memory as learning embedding representations for raw inputs, including text, image or other modalities; Short-term memory as in-context learning. It is short and finite, as it is restricted by the finite context window length of Transformer. Long-term memory as the external vector store that the agent can attend to at query time, accessible via fast retrieval. Maximum Inner Product Search (MIPS)# The external memory can alleviate the restriction of finite attention span. A standard practice is to save the embedding representation of information into a vector store database that can support fast maximum inner-product search (MIPS). To optimize the retrieval speed, the common choice is the approximate nearest neighbors (ANN)\u200b algorithm to return approximately top k nearest neighbors to trade off a little accuracy lost for a huge speedup. A couple common choices"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 50f294a3-d210-4960-a4ce-7a1ea8bfe221
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 12789, "chroma:document": "the common choice is the approximate nearest neighbors (ANN)\u200b algorithm to return approximately top k nearest neighbors to trade off a little accuracy lost for a huge speedup. A couple common choices of ANN algorithms for fast MIPS: LSH (Locality-Sensitive Hashing): It introduces a hashing function such that similar input items are mapped to the same buckets with high probability, where the number of buckets is much smaller than the number of inputs. ANNOY (Approximate Nearest Neighbors Oh Yeah): The core data structure are random projection trees, a set of binary trees where each non-leaf node represents a hyperplane splitting the input space into half and each leaf stores one data point. Trees are built independently and at random, so to some extent, it mimics a hashing function. ANNOY search happens in all the trees to iteratively search through the half that is closest to the query and then aggregates the results. The idea is quite related to KD tree but a lot more scalable. HNSW"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: ceb627ad-aa9c-48e7-b368-1d5238f008ee
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 13588, "chroma:document": "search happens in all the trees to iteratively search through the half that is closest to the query and then aggregates the results. The idea is quite related to KD tree but a lot more scalable. HNSW (Hierarchical Navigable Small World): It is inspired by the idea of small world networks where most nodes can be reached by any other nodes within a small number of steps; e.g. \u201csix degrees of separation\u201d feature of social networks. HNSW builds hierarchical layers of these small-world graphs, where the bottom layers contain the actual data points. The layers in the middle create shortcuts to speed up search. When performing a search, HNSW starts from a random node in the top layer and navigates towards the target. When it can\u2019t get any closer, it moves down to the next layer, until it reaches the bottom layer. Each move in the upper layers can potentially cover a large distance in the data space, and each move in the lower layers refines the search quality. FAISS (Facebook AI Similarity"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 4ceb51d3-6919-4b3f-a166-d88f8fdaa31f
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 14388, "chroma:document": "the bottom layer. Each move in the upper layers can potentially cover a large distance in the data space, and each move in the lower layers refines the search quality. FAISS (Facebook AI Similarity Search): It operates on the assumption that in high dimensional space, distances between nodes follow a Gaussian distribution and thus there should exist clustering of data points. FAISS applies vector quantization by partitioning the vector space into clusters and then refining the quantization within clusters. Search first looks for cluster candidates with coarse quantization and then further looks into each cluster with finer quantization. ScaNN (Scalable Nearest Neighbors): The main innovation in ScaNN is anisotropic vector quantization. It quantizes a data point $x_i$ to $\\tilde{x}_i$ such that the inner product $\\langle q, x_i \\rangle$ is as similar to the original distance of $\\angle q, \\tilde{x}_i$ as possible, instead of picking the closet quantization centroid points.  Fig. 9."}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 3f488b17-5a0c-4dcb-913c-41f2ba9fa749
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 15188, "chroma:document": "that the inner product $\\langle q, x_i \\rangle$ is as similar to the original distance of $\\angle q, \\tilde{x}_i$ as possible, instead of picking the closet quantization centroid points.  Fig. 9. Comparison of MIPS algorithms, measured in recall@10. (Image source: Google Blog, 2020) Check more MIPS algorithms and performance comparison in ann-benchmarks.com. Component Three: Tool Use# Tool use is a remarkable and distinguishing characteristic of human beings. We create, modify and utilize external objects to do things that go beyond our physical and cognitive limits. Equipping LLMs with external tools can significantly extend the model capabilities. Fig. 10. A picture of a sea otter using rock to crack open a seashell, while floating in the water. While some other animals can use tools, the complexity is not comparable with humans. (Image source: Animals using tools) MRKL (Karpas et al. 2022), short for \u201cModular Reasoning, Knowledge and Language\u201d, is a neuro-symbolic architecture for"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 973d8054-8eca-4224-a78a-df689cbaaa81
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 15990, "chroma:document": "complexity is not comparable with humans. (Image source: Animals using tools) MRKL (Karpas et al. 2022), short for \u201cModular Reasoning, Knowledge and Language\u201d, is a neuro-symbolic architecture for autonomous agents. A MRKL system is proposed to contain a collection of \u201cexpert\u201d modules and the general-purpose LLM works as a router to route inquiries to the best suitable expert module. These modules can be neural (e.g. deep learning models) or symbolic (e.g. math calculator, currency converter, weather API). They did an experiment on fine-tuning LLM to call a calculator, using arithmetic as a test case. Their experiments showed that it was harder to solve verbal math problems than explicitly stated math problems because LLMs (7B Jurassic1-large model) failed to extract the right arguments for the basic arithmetic reliably. The results highlight when the external symbolic tools can work reliably, knowing when to and how to use the tools are crucial, determined by the LLM capability. Both"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 1fe2996d-8e81-4b52-989f-ad38de2e7ddf
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 16792, "chroma:document": "the basic arithmetic reliably. The results highlight when the external symbolic tools can work reliably, knowing when to and how to use the tools are crucial, determined by the LLM capability. Both TALM (Tool Augmented Language Models; Parisi et al. 2022) and Toolformer (Schick et al. 2023) fine-tune a LM to learn to use external tool APIs. The dataset is expanded based on whether a newly added API call annotation can improve the quality of model outputs. See more details in the \u201cExternal APIs\u201d section of Prompt Engineering. ChatGPT Plugins and OpenAI API function calling are good examples of LLMs augmented with tool use capability working in practice. The collection of tool APIs can be provided by other developers (as in Plugins) or self-defined (as in function calls). HuggingGPT (Shen et al. 2023) is a framework to use ChatGPT as the task planner to select models available in HuggingFace platform according to the model descriptions and summarize the response based on the execution"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 9e1cbebd-20dc-4d03-991b-7f190c3ca68c
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 17590, "chroma:document": "et al. 2023) is a framework to use ChatGPT as the task planner to select models available in HuggingFace platform according to the model descriptions and summarize the response based on the execution results. Fig. 11. Illustration of how HuggingGPT works. (Image source: Shen et al. 2023) The system comprises of 4 stages: (1) Task planning: LLM works as the brain and parses the user requests into multiple tasks. There are four attributes associated with each task: task type, ID, dependencies, and arguments. They use few-shot examples to guide LLM to do task parsing and planning. Instruction: The AI assistant can parse user input to several tasks: [{\"task\": task, \"id\", task_id, \"dep\": dependency_task_ids, \"args\": {\"text\": text, \"image\": URL, \"audio\": URL, \"video\": URL}}]. The \"dep\" field denotes the id of the previous task which generates a new resource that the current task relies on. A special tag \"-task_id\" refers to the generated text image, audio and video in the dependency task"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 553d470d-ad2b-46d9-ad87-41ab5eedb62d
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 18387, "chroma:document": "denotes the id of the previous task which generates a new resource that the current task relies on. A special tag \"-task_id\" refers to the generated text image, audio and video in the dependency task with id as task_id. The task MUST be selected from the following options: {{ Available Task List }}. There is a logical relationship between tasks, please note their order. If the user input can't be parsed, you need to reply empty JSON. Here are several cases for your reference: {{ Demonstrations }}. The chat history is recorded as {{ Chat History }}. From this chat history, you can find the path of the user-mentioned resources for your task planning. (2) Model selection: LLM distributes the tasks to expert models, where the request is framed as a multiple-choice question. LLM is presented with a list of models to choose from. Due to the limited context length, task type based filtration is needed. Instruction: Given the user request and the call command, the AI assistant helps the user"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: ea5e5d36-61e4-4aea-a03e-7424383306a8
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 19190, "chroma:document": "a list of models to choose from. Due to the limited context length, task type based filtration is needed. Instruction: Given the user request and the call command, the AI assistant helps the user to select a suitable model from a list of models to process the user request. The AI assistant merely outputs the model id of the most appropriate model. The output must be in a strict JSON format: \"id\": \"id\", \"reason\": \"your detail reason for the choice\". We have a list of models for you to choose from {{ Candidate Models }}. Please select one model from the list. (3) Task execution: Expert models execute on the specific tasks and log results. Instruction: With the input and the inference results, the AI assistant needs to describe the process and results. The previous stages can be formed as - User Input: {{ User Input }}, Task Planning: {{ Tasks }}, Model Selection: {{ Model Assignment }}, Task Execution: {{ Predictions }}. You must first answer the user's request in a straightforward"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: c538b531-6429-4e89-91ff-2220a2fec1c6
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 19987, "chroma:document": "- User Input: {{ User Input }}, Task Planning: {{ Tasks }}, Model Selection: {{ Model Assignment }}, Task Execution: {{ Predictions }}. You must first answer the user's request in a straightforward manner. Then describe the task process and show your analysis and model inference results to the user in the first person. If inference results contain a file path, must tell the user the complete file path. (4) Response generation: LLM receives the execution results and provides summarized results to users. To put HuggingGPT into real world usage, a couple challenges need to solve: (1) Efficiency improvement is needed as both LLM inference rounds and interactions with other models slow down the process; (2) It relies on a long context window to communicate over complicated task content; (3) Stability improvement of LLM outputs and external model services. API-Bank (Li et al. 2023) is a benchmark for evaluating the performance of tool-augmented LLMs. It contains 53 commonly used API tools,"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 8f5c73b1-5a76-4371-90da-c3ca780efe5e
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 20794, "chroma:document": "improvement of LLM outputs and external model services. API-Bank (Li et al. 2023) is a benchmark for evaluating the performance of tool-augmented LLMs. It contains 53 commonly used API tools, a complete tool-augmented LLM workflow, and 264 annotated dialogues that involve 568 API calls. The selection of APIs is quite diverse, including search engines, calculator, calendar queries, smart home control, schedule management, health data management, account authentication workflow and more. Because there are a large number of APIs, LLM first has access to API search engine to find the right API to call and then uses the corresponding documentation to make a call. Fig. 12. Pseudo code of how LLM makes an API call in API-Bank. (Image source: Li et al. 2023) In the API-Bank workflow, LLMs need to make a couple of decisions and at each step we can evaluate how accurate that decision is. Decisions include: Whether an API call is needed. Identify the right API to call: if not good enough, LLMs"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: bc64b2dd-28b2-4c03-9680-2463f3c71f66
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 21594, "chroma:document": "make a couple of decisions and at each step we can evaluate how accurate that decision is. Decisions include: Whether an API call is needed. Identify the right API to call: if not good enough, LLMs need to iteratively modify the API inputs (e.g. deciding search keywords for Search Engine API). Response based on the API results: the model can choose to refine and call again if results are not satisfied. This benchmark evaluates the agent\u2019s tool use capabilities at three levels: Level-1 evaluates the ability to call the API. Given an API\u2019s description, the model needs to determine whether to call a given API, call it correctly, and respond properly to API returns. Level-2 examines the ability to retrieve the API. The model needs to search for possible APIs that may solve the user\u2019s requirement and learn how to use them by reading documentation. Level-3 assesses the ability to plan API beyond retrieve and call. Given unclear user requests (e.g. schedule group meetings, book"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: fb784c7f-041a-4dc4-aac9-9036b4049e53
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 22385, "chroma:document": "requirement and learn how to use them by reading documentation. Level-3 assesses the ability to plan API beyond retrieve and call. Given unclear user requests (e.g. schedule group meetings, book flight/hotel/restaurant for a trip), the model may have to conduct multiple API calls to solve it. Case Studies# Scientific Discovery Agent# ChemCrow (Bran et al. 2023) is a domain-specific example in which LLM is augmented with 13 expert-designed tools to accomplish tasks across organic synthesis, drug discovery, and materials design. The workflow, implemented in LangChain, reflects what was previously described in the ReAct and MRKLs and combines CoT reasoning with tools relevant to the tasks: The LLM is provided with a list of tool names, descriptions of their utility, and details about the expected input/output. It is then instructed to answer a user-given prompt using the tools provided when necessary. The instruction suggests the model to follow the ReAct format - Thought, Action, Action"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 61a86a07-d36e-4c8c-95d2-103e6f573b8a
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 23190, "chroma:document": "input/output. It is then instructed to answer a user-given prompt using the tools provided when necessary. The instruction suggests the model to follow the ReAct format - Thought, Action, Action Input, Observation. One interesting observation is that while the LLM-based evaluation concluded that GPT-4 and ChemCrow perform nearly equivalently, human evaluations with experts oriented towards the completion and chemical correctness of the solutions showed that ChemCrow outperforms GPT-4 by a large margin. This indicates a potential problem with using LLM to evaluate its own performance on domains that requires deep expertise. The lack of expertise may cause LLMs not knowing its flaws and thus cannot well judge the correctness of task results. Boiko et al. (2023) also looked into LLM-empowered agents for scientific discovery, to handle autonomous design, planning, and performance of complex scientific experiments. This agent can use tools to browse the Internet, read documentation,"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 421c16a7-b81b-4b2a-918d-b7f259baa8d4
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 23991, "chroma:document": "agents for scientific discovery, to handle autonomous design, planning, and performance of complex scientific experiments. This agent can use tools to browse the Internet, read documentation, execute code, call robotics experimentation APIs and leverage other LLMs. For example, when requested to \"develop a novel anticancer drug\", the model came up with the following reasoning steps: inquired about current trends in anticancer drug discovery; selected a target; requested a scaffold targeting these compounds; Once the compound was identified, the model attempted its synthesis. They also discussed the risks, especially with illicit drugs and bioweapons. They developed a test set containing a list of known chemical weapon agents and asked the agent to synthesize them. 4 out of 11 requests (36%) were accepted to obtain a synthesis solution and the agent attempted to consult documentation to execute the procedure. 7 out of 11 were rejected and among these 7 rejected cases, 5 happened after"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 8c8923a7-8df9-45f5-8462-b1236cd3ced4
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 24793, "chroma:document": "were accepted to obtain a synthesis solution and the agent attempted to consult documentation to execute the procedure. 7 out of 11 were rejected and among these 7 rejected cases, 5 happened after a Web search while 2 were rejected based on prompt only. Generative Agents Simulation# Generative Agents (Park, et al. 2023) is super fun experiment where 25 virtual characters, each controlled by a LLM-powered agent, are living and interacting in a sandbox environment, inspired by The Sims. Generative agents create believable simulacra of human behavior for interactive applications. The design of generative agents combines LLM with memory, planning and reflection mechanisms to enable agents to behave conditioned on past experience, as well as to interact with other agents. Memory stream: is a long-term memory module (external database) that records a comprehensive list of agents\u2019 experience in natural language. Each element is an observation, an event directly provided by the agent. -"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: ee5f1904-c6a1-43f7-9992-357c437ca025
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 25589, "chroma:document": "a long-term memory module (external database) that records a comprehensive list of agents\u2019 experience in natural language. Each element is an observation, an event directly provided by the agent. - Inter-agent communication can trigger new natural language statements.  Retrieval model: surfaces the context to inform the agent\u2019s behavior, according to relevance, recency and importance. Recency: recent events have higher scores Importance: distinguish mundane from core memories. Ask LM directly. Relevance: based on how related it is to the current situation / query.  Reflection mechanism: synthesizes memories into higher level inferences over time and guides the agent\u2019s future behavior. They are higher-level summaries of past events (<- note that this is a bit different from self-reflection above) Prompt LM with 100 most recent observations and to generate 3 most salient high-level questions given a set of observations/statements. Then ask LM to answer those questions.  Planning &"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 9961d104-5a8e-4e27-8b16-37f540f87af7
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 26389, "chroma:document": "above) Prompt LM with 100 most recent observations and to generate 3 most salient high-level questions given a set of observations/statements. Then ask LM to answer those questions.  Planning & Reacting: translate the reflections and the environment information into actions Planning is essentially in order to optimize believability at the moment vs in time. Prompt template: {Intro of an agent X}. Here is X's plan today in broad strokes: 1) Relationships between agents and observations of one agent by another are all taken into consideration for planning and reacting. Environment information is present in a tree structure. Fig. 13. The generative agent architecture. (Image source: Park et al. 2023) This fun simulation results in emergent social behavior, such as information diffusion, relationship memory (e.g. two agents continuing the conversation topic) and coordination of social events (e.g. host a party and invite many others). Proof-of-Concept Examples# AutoGPT has drawn a lot of"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: efba9042-5982-4198-a540-70fca6963aa3
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 27197, "chroma:document": "memory (e.g. two agents continuing the conversation topic) and coordination of social events (e.g. host a party and invite many others). Proof-of-Concept Examples# AutoGPT has drawn a lot of attention into the possibility of setting up autonomous agents with LLM as the main controller. It has quite a lot of reliability issues given the natural language interface, but nevertheless a cool proof-of-concept demo. A lot of code in AutoGPT is about format parsing. Here is the system message used by AutoGPT, where {{...}} are user inputs: You are {{ai-name}}, {{user-provided AI bot description}}. Your decisions must always be made independently without seeking user assistance. Play to your strengths as an LLM and pursue simple strategies with no legal complications. GOALS: 1. {{user-provided goal 1}} 2. {{user-provided goal 2}} 3. ... 4. ... 5. ... Constraints: 1. ~4000 word limit for short term memory. Your short term memory is short, so immediately save important information to files. 2."}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: d955e43f-d824-4669-9bbe-6da4d12720b8
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 27998, "chroma:document": "1}} 2. {{user-provided goal 2}} 3. ... 4. ... 5. ... Constraints: 1. ~4000 word limit for short term memory. Your short term memory is short, so immediately save important information to files. 2. If you are unsure how you previously did something or want to recall past events, thinking about similar events will help you remember. 3. No user assistance 4. Exclusively use the commands listed in double quotes e.g. \"command name\" 5. Use subprocesses for commands that will not terminate within a few minutes Commands: 1. Google Search: \"google\", args: \"input\": \"<search>\" 2. Browse Website: \"browse_website\", args: \"url\": \"<url>\", \"question\": \"<what_you_want_to_find_on_website>\" 3. Start GPT Agent: \"start_agent\", args: \"name\": \"<name>\", \"task\": \"<short_task_desc>\", \"prompt\": \"<prompt>\" 4. Message GPT Agent: \"message_agent\", args: \"key\": \"<key>\", \"message\": \"<message>\" 5. List GPT Agents: \"list_agents\", args: 6. Delete GPT Agent: \"delete_agent\", args: \"key\": \"<key>\" 7. Clone Repository:"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: bc061dc4-6535-4d94-afa6-f3764d43629f
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 28799, "chroma:document": "GPT Agent: \"message_agent\", args: \"key\": \"<key>\", \"message\": \"<message>\" 5. List GPT Agents: \"list_agents\", args: 6. Delete GPT Agent: \"delete_agent\", args: \"key\": \"<key>\" 7. Clone Repository: \"clone_repository\", args: \"repository_url\": \"<url>\", \"clone_path\": \"<directory>\" 8. Write to file: \"write_to_file\", args: \"file\": \"<file>\", \"text\": \"<text>\" 9. Read file: \"read_file\", args: \"file\": \"<file>\" 10. Append to file: \"append_to_file\", args: \"file\": \"<file>\", \"text\": \"<text>\" 11. Delete file: \"delete_file\", args: \"file\": \"<file>\" 12. Search Files: \"search_files\", args: \"directory\": \"<directory>\" 13. Analyze Code: \"analyze_code\", args: \"code\": \"<full_code_string>\" 14. Get Improved Code: \"improve_code\", args: \"suggestions\": \"<list_of_suggestions>\", \"code\": \"<full_code_string>\" 15. Write Tests: \"write_tests\", args: \"code\": \"<full_code_string>\", \"focus\": \"<list_of_focus_areas>\" 16. Execute Python File: \"execute_python_file\", args: \"file\": \"<file>\" 17. Generate Image: \"generate_image\", args:"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: d0a177b0-2eec-47f0-a0dc-b32933c7834c
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 29600, "chroma:document": "\"write_tests\", args: \"code\": \"<full_code_string>\", \"focus\": \"<list_of_focus_areas>\" 16. Execute Python File: \"execute_python_file\", args: \"file\": \"<file>\" 17. Generate Image: \"generate_image\", args: \"prompt\": \"<prompt>\" 18. Send Tweet: \"send_tweet\", args: \"text\": \"<text>\" 19. Do Nothing: \"do_nothing\", args: 20. Task Complete (Shutdown): \"task_complete\", args: \"reason\": \"<reason>\" Resources: 1. Internet access for searches and information gathering. 2. Long Term memory management. 3. GPT-3.5 powered Agents for delegation of simple tasks. 4. File output. Performance Evaluation: 1. Continuously review and analyze your actions to ensure you are performing to the best of your abilities. 2. Constructively self-criticize your big-picture behavior constantly. 3. Reflect on past decisions and strategies to refine your approach. 4. Every command has a cost, so be smart and efficient. Aim to complete tasks in the least number of steps. You should only respond in JSON format as described below"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 962af639-bbb8-4734-9439-6183b1e000a0
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 30406, "chroma:document": "to refine your approach. 4. Every command has a cost, so be smart and efficient. Aim to complete tasks in the least number of steps. You should only respond in JSON format as described below Response Format: { \"thoughts\": {   \"text\": \"thought\",   \"reasoning\": \"reasoning\",   \"plan\": \"- short bulleted\\n- list that conveys\\n- long-term plan\",   \"criticism\": \"constructive self-criticism\",   \"speak\": \"thoughts summary to say to user\" }, \"command\": {   \"name\": \"command name\",   \"args\": {   \"arg name\": \"value\"   } } } Ensure the response can be parsed by Python json.loads GPT-Engineer is another project to create a whole repository of code given a task specified in natural language. The GPT-Engineer is instructed to think over a list of smaller components to build and ask for user input to clarify questions as needed. Here are a sample conversation for task clarification sent to OpenAI ChatCompletion endpoint used by GPT-Engineer. The user inputs are wrapped in {{user input text}}. [  {"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: c734f25e-5119-462b-bc98-0f1dc36f6787
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 31208, "chroma:document": "questions as needed. Here are a sample conversation for task clarification sent to OpenAI ChatCompletion endpoint used by GPT-Engineer. The user inputs are wrapped in {{user input text}}. [  { \"role\": \"system\", \"content\": \"You will read instructions and not carry them out, only seek to clarify them.\\nSpecifically you will first summarise a list of super short bullets of areas that need clarification.\\nThen you will pick one clarifying question, and wait for an answer from the user.\\n\"  },  { \"role\": \"user\", \"content\": \"We are writing {{a Super Mario game in python. MVC components split in separate files. Keyboard control.}}\\n\"  },  { \"role\": \"assistant\", \"content\": \"Summary of areas that need clarification:\\n1. Specifics of the Super Mario game (e.g. level design, characters, gameplay mechanics)\\n2. Details about the MVC components (e.g. which components are in each file)\\n3. Keyboard control implementation (e.g. which keys to use, how to handle input)\\n\\nClarifying question:\\nCan you"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 31da96af-f35b-4017-a7a3-99874798027b
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 32019, "chroma:document": "Details about the MVC components (e.g. which components are in each file)\\n3. Keyboard control implementation (e.g. which keys to use, how to handle input)\\n\\nClarifying question:\\nCan you provide more details about the Super Mario game, such as level design, characters, and gameplay mechanics?\"  },  { \"role\": \"user\", \"content\": \"{{There are 10 levels in total. The main character is a plumber named Mario, who can walk and jump. It is a classical platform game just like Super Mario. The main character moves from left to right, trying to get to the destination, where there are many obstacles and attacks from enemies in the process.}}\\n\\nIs anything else unclear? If yes, only answer in the form:\\n{remaining unclear areas} remaining questions.\\n{Next question}\\nIf everything is sufficiently clear, only answer \\\"Nothing more to clarify.\\\".\"  },  { \"role\": \"assistant\", \"content\": \"Remaining unclear areas: 2 remaining questions.\\nCan you provide more information about how the MVC components"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: b7fe6159-61f6-45c9-a435-a5f694b50409
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 32824, "chroma:document": "only answer \\\"Nothing more to clarify.\\\".\"  },  { \"role\": \"assistant\", \"content\": \"Remaining unclear areas: 2 remaining questions.\\nCan you provide more information about how the MVC components are split into separate files?\"  },  { \"role\": \"user\", \"content\": \"{{Make your own assumptions and state them explicitly before starting}}\"  } ] Then after these clarification, the agent moved into the code writing mode with a different system message. System message: You will get instructions for code to write. You will write a very long answer. Make sure that every detail of the architecture is, in the end, implemented as code. Make sure that every detail of the architecture is, in the end, implemented as code. Think step by step and reason yourself to the right decisions to make sure we get it right. You will first lay out the names of the core classes, functions, methods that will be necessary, as well as a quick comment on their purpose. Then you will output the content of each file"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: ee764f0b-d7f3-4fb0-8f6c-edcfcea58c36
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 33619, "chroma:document": "it right. You will first lay out the names of the core classes, functions, methods that will be necessary, as well as a quick comment on their purpose. Then you will output the content of each file including ALL code. Each file must strictly follow a markdown code block format, where the following tokens must be replaced such that FILENAME is the lowercase file name including the file extension, LANG is the markup code block language for the code\u2019s language, and CODE is the code: FILENAME CODE You will start with the \u201centrypoint\u201d file, then go to the ones that are imported by that file, and so on. Please note that the code should be fully functional. No placeholders. Follow a language and framework appropriate best practice file naming convention. Make sure that files contain all imports, types etc. Make sure that code in different files are compatible with each other. Ensure to implement all code, if you are unsure, write a plausible implementation. Include module dependency or"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 9b34b783-fd3c-429d-acbb-4fd5ea6e382e
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 34419, "chroma:document": "types etc. Make sure that code in different files are compatible with each other. Ensure to implement all code, if you are unsure, write a plausible implementation. Include module dependency or package manager dependency definition file. Before you finish, double check that all parts of the architecture is present in the files. Useful to know: You almost always put different classes in different files. For Python, you always create an appropriate requirements.txt file. For NodeJS, you always create an appropriate package.json file. You always add a comment briefly describing the purpose of the function definition. You try to add comments explaining very complex bits of logic. You always follow the best practices for the requested languages in terms of describing the code written as a defined package/project. Python toolbelt preferences: pytest dataclasses  Conversatin samples: [  { \"role\": \"system\", \"content\": \"You will get instructions for code to write.\\nYou will write a very long"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 4e5b10b1-c5e3-4089-9f29-783af46ee544
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 35222, "chroma:document": "package/project. Python toolbelt preferences: pytest dataclasses  Conversatin samples: [  { \"role\": \"system\", \"content\": \"You will get instructions for code to write.\\nYou will write a very long answer. Make sure that every detail of the architecture is, in the end, implemented as code.\\nMake sure that every detail of the architecture is, in the end, implemented as code.\\n\\nThink step by step and reason yourself to the right decisions to make sure we get it right.\\nYou will first lay out the names of the core classes, functions, methods that will be necessary, as well as a quick comment on their purpose.\\n\\nThen you will output the content of each file including ALL code.\\nEach file must strictly follow a markdown code block format, where the following tokens must be replaced such that\\nFILENAME is the lowercase file name including the file extension,\\nLANG is the markup code block language for the code's language, and CODE is the code:\\n\\nFILENAME\\n```LANG\\nCODE\\n```\\n\\nYou will"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 1a851c5e-ddd6-414f-b6f5-aea4fed18621
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 36029, "chroma:document": "is the lowercase file name including the file extension,\\nLANG is the markup code block language for the code's language, and CODE is the code:\\n\\nFILENAME\\n```LANG\\nCODE\\n```\\n\\nYou will start with the \\\"entrypoint\\\" file, then go to the ones that are imported by that file, and so on.\\nPlease note that the code should be fully functional. No placeholders.\\n\\nFollow a language and framework appropriate best practice file naming convention.\\nMake sure that files contain all imports, types etc. Make sure that code in different files are compatible with each other.\\nEnsure to implement all code, if you are unsure, write a plausible implementation.\\nInclude module dependency or package manager dependency definition file.\\nBefore you finish, double check that all parts of the architecture is present in the files.\\n\\nUseful to know:\\nYou almost always put different classes in different files.\\nFor Python, you always create an appropriate requirements.txt file.\\nFor NodeJS, you always create"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: b27d4832-e84e-4e70-a414-d9d3e57ef074
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 36835, "chroma:document": "in the files.\\n\\nUseful to know:\\nYou almost always put different classes in different files.\\nFor Python, you always create an appropriate requirements.txt file.\\nFor NodeJS, you always create an appropriate package.json file.\\nYou always add a comment briefly describing the purpose of the function definition.\\nYou try to add comments explaining very complex bits of logic.\\nYou always follow the best practices for the requested languages in terms of describing the code written as a defined\\npackage/project.\\n\\n\\nPython toolbelt preferences:\\n- pytest\\n- dataclasses\\n\"  }, # \u2026 same conversation as earlier, ended with \"Make your own assumptions and state them explicitly before starting\".  { \"role\": \"assistant\", \"content\": \"Assumptions:\\n1. Model: The model will contain the game's data, such as level information, character states, and enemy positions.\\n2. View: The view will handle the game's visuals, including rendering the game objects, backgrounds, and updating the display.\\n3."}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 1f33a1ef-33ce-4ad9-9a3a-e79afd21d3fe
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 37631, "chroma:document": "such as level information, character states, and enemy positions.\\n2. View: The view will handle the game's visuals, including rendering the game objects, backgrounds, and updating the display.\\n3. Controller: The controller will manage user input, such as keyboard controls, and update the model accordingly.\\n\\nWith these assumptions in mind, is there any specific aspect of the keyboard control implementation you would like me to clarify?\"  },  { \"role\": \"user\", \"content\": \"Please now remember the steps:\\n\\nThink step by step and reason yourself to the right decisions to make sure we get it right.\\nFirst lay out the names of the core classes, functions, methods that will be necessary, As well as a quick comment on their purpose.\\n\\nThen you will output the content of each file including ALL code.\\nEach file must strictly follow a markdown code block format, where the following tokens must be replaced such that\\nFILENAME is the lowercase file name including the file extension,\\nLANG is"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: ddf03da7-1bcc-4923-b4c4-fabd6d1610b2
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 38433, "chroma:document": "code.\\nEach file must strictly follow a markdown code block format, where the following tokens must be replaced such that\\nFILENAME is the lowercase file name including the file extension,\\nLANG is the markup code block language for the code's language, and CODE is the code:\\n\\nFILENAME\\n```LANG\\nCODE\\n```\\n\\nPlease note that the code should be fully functional. No placeholders.\\n\\nYou will start with the \\\"entrypoint\\\" file, then go to the ones that are imported by that file, and so on.\\nFollow a language and framework appropriate best practice file naming convention.\\nMake sure that files contain all imports, types etc. The code should be fully functional. Make sure that code in different files are compatible with each other.\\nBefore you finish, double check that all parts of the architecture is present in the files.\\n\"  } ] Challenges# After going through key ideas and demos of building LLM-centered agents, I start to see a couple common limitations:  Finite context length: The"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: 0d21e9c2-5958-4994-add3-3407a430d13f
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 39239, "chroma:document": "is present in the files.\\n\"  } ] Challenges# After going through key ideas and demos of building LLM-centered agents, I start to see a couple common limitations:  Finite context length: The restricted context capacity limits the inclusion of historical information, detailed instructions, API call context, and responses. The design of the system has to work with this limited communication bandwidth, while mechanisms like self-reflection to learn from past mistakes would benefit a lot from long or infinite context windows. Although vector stores and retrieval can provide access to a larger knowledge pool, their representation power is not as powerful as full attention.  Challenges in long-term planning and task decomposition: Planning over a lengthy history and effectively exploring the solution space remain challenging. LLMs struggle to adjust plans when faced with unexpected errors, making them less robust compared to humans who learn from trial and error.  Reliability of natural"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: ea3b8f3b-7dc5-47de-909f-29b2f3302fdd
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 40035, "chroma:document": "solution space remain challenging. LLMs struggle to adjust plans when faced with unexpected errors, making them less robust compared to humans who learn from trial and error.  Reliability of natural language interface: Current agent system relies on natural language as an interface between LLMs and external components such as memory and tools. However, the reliability of model outputs is questionable, as LLMs may make formatting errors and occasionally exhibit rebellious behavior (e.g. refuse to follow an instruction). Consequently, much of the agent demo code focuses on parsing model output.  Citation# Cited as: Weng, Lilian. (Jun 2023). \u201cLLM-powered Autonomous Agents\u201d. Lil\u2019Log. https://lilianweng.github.io/posts/2023-06-23-agent/. Or @article{weng2023agent,  title  = \"LLM-powered Autonomous Agents\",  author = \"Weng, Lilian\",  journal = \"lilianweng.github.io\",  year  = \"2023\",  month  = \"Jun\",  url = \"https://lilianweng.github.io/posts/2023-06-23-agent/\" } References# [1] Wei et al."}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: efeee4b2-2217-42c5-83cc-aa84291ace52
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 40839, "chroma:document": "Agents\",  author = \"Weng, Lilian\",  journal = \"lilianweng.github.io\",  year  = \"2023\",  month  = \"Jun\",  url = \"https://lilianweng.github.io/posts/2023-06-23-agent/\" } References# [1] Wei et al. \u201cChain of thought prompting elicits reasoning in large language models.\u201d NeurIPS 2022 [2] Yao et al. \u201cTree of Thoughts: Dliberate Problem Solving with Large Language Models.\u201d arXiv preprint arXiv:2305.10601 (2023). [3] Liu et al. \u201cChain of Hindsight Aligns Language Models with Feedback \u201c arXiv preprint arXiv:2302.02676 (2023). [4] Liu et al. \u201cLLM+P: Empowering Large Language Models with Optimal Planning Proficiency\u201d arXiv preprint arXiv:2304.11477 (2023). [5] Yao et al. \u201cReAct: Synergizing reasoning and acting in language models.\u201d ICLR 2023. [6] Google Blog. \u201cAnnouncing ScaNN: Efficient Vector Similarity Search\u201d July 28, 2020. [7] https://chat.openai.com/share/46ff149e-a4c7-4dd7-a800-fc4a642ea389 [8] Shinn & Labash. \u201cReflexion: an autonomous agent with dynamic memory and self-reflection\u201d arXiv"}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: e1019854-4bb5-4a50-b242-279c98aa17a9
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 41646, "chroma:document": "Search\u201d July 28, 2020. [7] https://chat.openai.com/share/46ff149e-a4c7-4dd7-a800-fc4a642ea389 [8] Shinn & Labash. \u201cReflexion: an autonomous agent with dynamic memory and self-reflection\u201d arXiv preprint arXiv:2303.11366 (2023). [9] Laskin et al. \u201cIn-context Reinforcement Learning with Algorithm Distillation\u201d ICLR 2023. [10] Karpas et al. \u201cMRKL Systems A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning.\u201d arXiv preprint arXiv:2205.00445 (2022). [11] Nakano et al. \u201cWebgpt: Browser-assisted question-answering with human feedback.\u201d arXiv preprint arXiv:2112.09332 (2021). [12] Parisi et al. \u201cTALM: Tool Augmented Language Models\u201d [13] Schick et al. \u201cToolformer: Language Models Can Teach Themselves to Use Tools.\u201d arXiv preprint arXiv:2302.04761 (2023). [14] Weaviate Blog. Why is Vector Search so fast? Sep 13, 2022. [15] Li et al. \u201cAPI-Bank: A Benchmark for Tool-Augmented LLMs\u201d arXiv preprint arXiv:2304.08244 (2023)."}


created_at: 2025-02-04 15:04:55
topic: persistent://default/default/d96bc087-722f-46b1-86af-cbd39f5d55a2
id: b7f53e0c-9b8c-49bd-a813-919a93457cf5
encoding: FLOAT32
metadata:
{"source": "https://lilianweng.github.io/posts/2023-06-23-agent/", "doc_id": "df7ab93c-45a5-e87e-221f-90f6455b7c04", "source_type": "web", "start_index": 42450, "chroma:document": "arXiv:2302.04761 (2023). [14] Weaviate Blog. Why is Vector Search so fast? Sep 13, 2022. [15] Li et al. \u201cAPI-Bank: A Benchmark for Tool-Augmented LLMs\u201d arXiv preprint arXiv:2304.08244 (2023). [16] Shen et al. \u201cHuggingGPT: Solving AI Tasks with ChatGPT and its Friends in HuggingFace\u201d arXiv preprint arXiv:2303.17580 (2023). [17] Bran et al. \u201cChemCrow: Augmenting large-language models with chemistry tools.\u201d arXiv preprint arXiv:2304.05376 (2023). [18] Boiko et al. \u201cEmergent autonomous scientific research capabilities of large language models.\u201d arXiv preprint arXiv:2304.05332 (2023). [19] Joon Sung Park, et al. \u201cGenerative Agents: Interactive Simulacra of Human Behavior.\u201d arXiv preprint arXiv:2304.03442 (2023). [20] AutoGPT. https://github.com/Significant-Gravitas/Auto-GPT [21] GPT-Engineer. https://github.com/AntonOsika/gpt-engineer  Nlp Language-Model Agent Steerability Prompting  \u00ab  Adversarial Attacks on LLMs  \u00bb Prompt Engineering        \u00a9 2024 Lil'Log   Powered by   Hugo &   PaperMod"}


# collections

id: d96bc087-722f-46b1-86af-cbd39f5d55a2
name: general_vstore
database_id: 00000000-0000-0000-0000-000000000000
config_json_str:
{"hnsw_configuration": {"space": "l2", "ef_construction": 100, "ef_search": 100, "num_threads": 10, "M": 16, "resize_factor": 1.2, "batch_size": 100, "sync_threshold": 1000, "_type": "HNSWConfigurationInternal"}, "_type": "CollectionConfigurationInternal"}


# embedding_metadata

key: chroma:document
string_value:
A Review of Deep Learning Techniques

Abstract: In today‚Äôs world, Deep learning techniques are playing a vital role in all areas. It has already
made a massive impact in almost every field, such as self-driving cars, cancer diagnosis, predictive
forecasting, precision medicine and speech recognition. The limitations of traditional learning techniques
are overcome by deep learning techniques. In this paper deep learning and its techniques are discussed.
Keywords‚ÄîDeep Neural Network, Recurrent Neural Network, Convolutional Neural Network, Restricted
Boltzmann Machine.


key: chroma:document
string_value:
1. Introduction: With a massive influx of multimodality data, the role of machine learning has grown
rapidly in the last few years. In last decade various machine learning techniques like decision tree, naive
Bayesian, support vector machine and K-Nearest Neighbour have been utilized for developing models to
solve real life problems such as predictive forecasting, speech recognition, diseases diagnosis and NLP.
These techniques have provided successful results but fails when amount of data increases abruptly. And
one of the other reasons for traditional machine learning failure is its features extraction capability. These
classification techniques do not give good result if we forget to give some parameter. These problems are
overcome as the concept of neural network was introduced in AI. After that machine learning is
revolutionized completely. Deep learning techniques are based on the concept of neural network; provide


key: chroma:document
string_value:
overcome as the concept of neural network was introduced in AI. After that machine learning is
revolutionized completely. Deep learning techniques are based on the concept of neural network; provide
amazing results in all fields. In this paper we discussed deep learning, various techniques of deep learning.
In the first section of paper introduction is given. In second section of the paper deep learning and its
working is described. In next section various deep learning techniques such as CNN, RNN, LSTM and
Autoencoder and Restricted Boltzman Machine are described. In fourth section of the paper conclusion is
given and last section contains references.


key: chroma:document
string_value:
2. Deep Learning: Deep learning is a subarea of machine learning that is emerged from the concept of
neural network. Neural network is inspired by and resembles the human nervous system and the structure of
the brain [1][2]. It is an application of Artificial Neural Network in which number of hidden layer is one but
as number of hidden number of hidden layer increases network goes deeper and it refers to deep neural
network. Deep learning can be applied in almost every machine learning problem. The layers in DNN are
divided into three categories broadly input layer, multiple hidden layer and output layers. Input layer is used
to give input to the network, hidden layer process the input provided and output layer generates output to the


key: chroma:document
string_value:
system. Input in hidden layers processed a distributed representation and the main driving variables of the
input data. Deep learning working model is divided into two phases, training and testing. In training phase
of model, model parameters with random numbers are introduced and the pre-train some models are done.
After first iteration completes then next step is to read and processes the training data and training errors are
calculated by comparing the obtained output with expected output. Then parameters are upgraded according
to training error through error backpropagation. Then testing phase is performed to find whether conditions
of the iterative training are met for termination or continue the iterative process of training [3]. For
designing a complex deep neural network requires complex and high level resources for computation along
with large amount of training data. Deep learning models have automatic feature extraction capability that


key: chroma:document
string_value:
with large amount of training data. Deep learning models have automatic feature extraction capability that
make it different from traditional machine learning algorithms, means user need not to specify the feature to
the model, neural network architecture itself find out important feature required for desired results. Deep
learning network provides high level of abstraction; even developer does not know how neurons in the
network are connected and how data is processed within the network. For solving problems deep learning
uses end to end process. Suppose we have a task of multiple object detection, when we solve this problem
with earlier machine learning model then whole task is divided into two phases in first phase we apply
bounding box detection algorithm to find all the objects on a image and then we apply a recognizer
algorithm like support vector machine to detect a particular object on the image. Whereas deep learning


key: chroma:document
string_value:
algorithm like support vector machine to detect a particular object on the image. Whereas deep learning
performs whole task in one go. We apply euro net deep learning algorithm in which we pass image to the
model it gives the location along with the name of the object. As deep neural network is very complex, it
requires Graphics processing units for processing large matrix multiplication and other complex operations.
Deep learning requires large amount of data to train the machine so that it can generate accurate results so
training time of DNN is more than traditional machine learning models. Where is best to apply deep
learning:
Deep learning is ideal to use where we have to predict results from large amount of data.
Deep learning is applied to solve those complex problems that are very expensive to solve with human
decision making.


key: chroma:document
string_value:
3. Techniques of deep learning [1]:
CNN: is a feedforward neural network that is generally used to analyse the static or image data and is also
known as ConvNet. In CNN, whole architecture is divided into three layered structure. In image


key: chroma:document
string_value:
identification, CNN take the input image, process it, and classify it in a certain category e.g. cat, dog, tiger.
In computer image is stored as an array of pixels. Like the traditional machine learning architecture, CNN
also have to train for data to solve a particular problem. For this first architecture of Neural network is
decided like how many layers are used in network, how we arrange the layers, which layer to use, and how
many neuron to be used in a layer. Various CNN architectures are AlexNet, GoogleNet, Inception ResNet,
and VGG. Once network architecture is decided after that various biases and parameters for the network are
selected bases on the problem. At first these are selected randomly but further they are changed through
back propagation. Objective of this phase is to find the best possible values of network parameters and data
features so that further identification of data can be accurately done. For e.g. when we try to build a


key: chroma:document
string_value:
features so that further identification of data can be accurately done. For e.g. when we try to build a
classifier for cat and dog then we are looking to find the parameters that gives the probability of dog 1 or
higher than cat and for all the images of dog or 0 or less than dog for cat image. Whole CNN process is
divided into two parts feature learning and classification. In feature learning there are three steps performed
many times for different feature detection. These are convolutional operation, ReLu and Pooling. In
classification image is classification is performed. In this phase three operation is performed flattened, fully
connected and softmax operations [4][5].


key: chroma:document
string_value:
Convolutional layer: in this layer filter works on every part of the image. And search same feature
everywhere in the image. This layer involves shift, multiply and sum operations. The purpose of this layer is
to identify the basic pattern of which the object in the image is made up of. Output of this layer is a new
modified filtered image. In this layer features are identified. In training phase this layer identified the most
accurate feature for image classification.

ReLU layer: is rectified linear unit. Once features are extracted then next step is to move them to ReLU
layer. This layer mainly performs element wise operations, sets all the negative values to zero and
introduces non linearity to the network. At this layer sigmoid function are applied. This function removes all
the black elements from the images.

Pooling layer: provides the down sampling to the output that reduces the dimensionality of the feature map.


key: chroma:document
string_value:
Pooling layer reduces size of each feature map by 2. Two types of pooling can be applied in CNN, Average
pooling and max pooling. In average pooling feature map is patched with average value and in max pooling
feature map is patched by maximum value of the matrix.

These three steps are applied multiple times to find best features of the data.
Flattening layer: this layer transforms the matrix into a vector form so that it can be fed into a fully
connected NN classifier.

Fully connected layer: at this layer data is in one dimensional structure. This layer helps in classifying the
input pattern with high-level features extracted by previous layer. This layer gives a probability that a
certain feature belongs to a label. For example, if the image is of a cat, features representing things like
whiskers or fur provide high probabilities for the label ‚Äúcat‚Äù. Surtax activation function is used to provide
probability to each label.


key: chroma:document
string_value:
RNN: is Recurrent Neural Network that is applied to problems where data changes according to time. A
single time step of input is provided to the network in RNN. In first step current state is calculated by using
current input and output from the previous state. In next step current state become the previous state. Once
all time steps are completed then output state is calculated from the current state. Output obtained is then
compared with expected output and error is calculated. This error is back propagated in the network to
upgrade the parameters and weights. In this way RNN network is trained. The process of calculating error
rate and upgrading weights and parameters is called vanishing gradient. In RNN vanishing gradient can only
remember one step output but sometime problem requires outputs from long distant states for this LSTM is
used. LSTM have a chain like structure. In LSTM first architecture identify unnecessary information that is


key: chroma:document
string_value:
used. LSTM have a chain like structure. In LSTM first architecture identify unnecessary information that is
throwing away from the cell state. This is done by sigmoid layer called forget layer. Then model identify the
information that is necessary for further processing. This is done through tanh activation function. After this
output from previous stages is combined to update new cell state [7][8].


key: chroma:document
string_value:
Autoencoders: is an unsupervised algorithm which uses back propagation algorithm for setting the desired
output equal to the input. Autoencoders are based on the concept of principal component analysis (PCA). It


key: chroma:document
string_value:
uses layer by layer fin-tuning with backpropagation and unsupervised pretraining [9] It is lossy
compression. Autoencoder can easily works on non linear data. Autoencoder have multiple representations
of data. Autoencoder uses convolutional layer for learning feature from data so it does not require learning
from dense layer. It is widely used in image reconstruction, image colorization, and dimensionality
reduction. . Hinton et al. achieved a perfect reconstruction of 784- pixel images using autoencoders which
was better than principal component analysis technique [10]. Autoencoder are used only for which they are
trained only we cannot apply autoencoder for another applications.


key: chroma:document
string_value:
Restricted Boltzmann Machine (RBM): Restricted Boltzmann Machine is a deep learning technique
applied on unlabeled data to build non-linear generative models [11]. RBM contains two layers called
visible layer and hidden layer. Each node of visible layer is connected to all nodes in the hidden layer and
no nodes are connected to other nodes in the same layer.RBM increases at the probability of vectors in the
visible layers so that it can probabilistically reconstruct the unlabeled data. The energy (E) function of the
configuration is used for this [12]


key: chroma:document
string_value:
4. Conclusion: The unmatched learning capability of deep learning made it an attractive and
indispensable technique for analyzing data and images. It is successfully used in bioinformatics, medical
diagnosing, precision medicine and speech recognition. Deep learning techniques are contributing to the
high level of services in every field. In this paper we discussed various deep learning techniques CNN,
RNN, Autoencoders and RBM. CNN technique is successfully used on image data whereas RNN and
LSTM provide amazing results in textual data. RBM is usually used to build non-linear generative models.
Feature extraction capabilities of deep learning techniques make it indispensably useful in every field


key: chroma:document
string_value:
Official (Closed) and Non-Sensitive
Develop computer vision solutions with Azure AI Vision
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Agenda
· Analyze and manipulate images
· Create a custom vision model
· Detect and recognize faces
· Analyze video
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive
Analyze and manipulate images
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Learning Objectives
After completing this module, you will be able to:
1 Understand features and functionality of Image Analysis
2 Perform Optical Character Recognition (OCR)
3
Connect an app to Image Analysis APIs
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Image analysis:
· Caption and tag generation
· Object detection
· People detection
· Optical character recognition
· Smart crop thumbnails
· Background removal
· Multi-modal embeddings
· Product recognition


key: chroma:document
string_value:
Can be used as:
· Standalone Azure AI Vision resource
· Multi-service Azure AI Services resource
* Some new features are limited to specific regions
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Image Analysis APIs
· Single Analyze call to retrieve specified features in VisualFeatures enum
· VisualFeatures.Caption
· VisualFeatures.DenseCaptions
· VisualFeatures.Tags
· VisualFeatures.Objects
· VisualFeatures.SmartCrops
· VisualFeatures.People
· VisualFeatures.Read
· SDKs define the client , then call the analyze() function from it
· Client defines endpoint and key of resource
· Analyze() needs
· Image data from file or URL
· Visual features to analyze for
· (Optional) Analysis options: Which features, language, and other options for the analysis


key: chroma:document
string_value:
REST
https://<endpoint>/computervision/imageanalysis:analyze? features=caption,people&model-version=latest& language=en&api-version={version}


key: chroma:document
string_value:
C#
ImageAnalysisResult result = client.Analyze( new Uri('<uri-to-image>'), VisualFeatures.CAPTION | VisualFeatures.PEOPLE, analysisOptions // Optional );


key: chroma:document
string_value:
Python
result = client.analyze( image_url='<uri-to-image>', visual_features=[ VisualFeatures.CAPTION, VisualFeatures.PEOPLE, ], <analysis_options> # Optional )
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Image Analysis Options
· Analysis options
· Cropping aspect ratios
· Gender neutral caption
· Language
· Model version
C# ImageAnalysisClient client = new ImageAnalysisClient( new Uri(endpoint), new AzureKeyCredential(key)); ImageAnalysisOptions options = new ImageAnalysisOptions { GenderNeutralCaption = true, Language = "en" }; ImageAnalysisResult result = client.Analyze( imageURL, visualFeatures, options ); Python client = ImageAnalysisClient( endpoint=endpoint, credential=AzureKeyCredential(key)) result = client.analyze( image_url=image_url, visual_features=visual_features, gender_neutral_caption=True, language="en" )
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Image Analysis Result
· Successful image analysis returns JSON (REST) or an object (SDKs)
· Results may have one or several layers of depth
· Tags > values[ ] > name
· Text > lines > words
{ "captionResult": { "text": "a man pointing at a screen", "confidence": 0.4891590476036072 }, "objectsResult": { "values": [ { "name": "string", "confidence": 0.0 }] }, "smartCropsResult": { "values": [ { "aspectRatio": 0.0, "boundingBox": { "x": 0, "y": 0, "w": 0, "h": 0 }}]}, ... }
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
[Optional] Exercise - Explore features in Vision Studio
Search photos with image retrieval
Create smart-cropped images
Open https://portal.vision.cognitive.azure.com/ and select 'Try it out' under these features and use the provided images
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Lab - Analyze Images with Azure AI Vision
Generate captions
Create tags
Detect objects and people
Remove background
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Azure AI Vision - OCR
Use Image analysis with READ feature
Vision OCR vs Document Intelligence:
· OCR: General, non-document images with smaller amounts of text. Synchronous API.
· Document Intelligence: Ideal for larger text heavy documents. Asynchronous API.
Results in JSON (REST) or object (SDK) of similar structure
© Copyright Microsoft Corporation. All rights reserved.
[{ "lines": [{ "text": "You must be the change you", "boundingPolygon": [ { "x": 251, "y": 265 }, { "x": 673, "y": 260 }, { "x": 674, "y": 308 }, { "x": 252, "y": 318 } ], "words": [ { "text": "You", "boundingPolygon": ... "confidence": 0.996 }, { "text": "must", "boundingPolygon": ... ...
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Use the Azure AI Vision for OCR
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Detecting Faces with the Azure AI Vision
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Learning Objectives
After completing this module, you will be able to:
2
Understand features, use cases, and responsibility of the Azure AI Vision Face API
Use the Face API in an app
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Face Service
· Face detection
· Comprehensive facial feature analysis
· Face comparison and identification*
· Facial recognition*
* Require Limited Access approval
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Considerations for Face Detection and Facial Recognition
Principles of responsible AI apply to all kinds of application, but systems that rely on facial data can be particularly problematic. As a safeguard for responsible AI usage, facial recognition, identification, verification, and comparison is behind a Limited Access policy, requiring users to be approved my Microsoft before enabling these features.


key: chroma:document
string_value:
Data privacy and security
Systems based on facial data should protect individual privacy, ensuring that personally identifiable data is not accessed inappropriately


key: chroma:document
string_value:
Transparency
Users should be informed about how their image will be used, and who will have access to it.


key: chroma:document
string_value:
Fairness and Inclusiveness
Facial recognition should not be used in a manner that is prejudicial to individuals based on their appearance, or to unfairly target individuals
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Face detection
Face attribute analysis
Facial landmark location
· Nose, eyes, mouth, …
Face comparison*
Facial recognition and identification*
Facial liveness*
Can be used as:
· Standalone Face resource
· Multi-service Azure AI Services resource
* Require Limited Access approval
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Every detected face is assigned an anonymous ID
· Retained in your service resource for 24 hours
· Can be used to compare faces in multiple images
-Verify faces to determine if they're the same individual
-Find similar faces to identify faces with similar features
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Train a facial recognition model using face images
1. Create a Person Group for the people you want to identify
2. Add a Person for each individual
3. Add multiple detected Faces to each person
- These become persisted faces
4. Train the model


key: chroma:document
string_value:
Use the model for facial recognition
· Identify an individual person
· Verify the face of an individual person
· Find similar faces to a persisted face
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Face Detection with Azure AI Vision
Use the Face endpoint, specifying Faces as a visual feature Note: Most features like recognition and identification are not enabled for new users


key: chroma:document
string_value:
Optional request parameters
returnFaceId
recognitionModel
returnFaceLandmarks
returnRecognitionModel
returnFaceAttributes
detectionModel
Request: https://{endpoint}/face/v1.0/detect[?options] Body: {"url": "http://path-to-image}
Response:
[
{
"faceId": "c5c24a82-6845-4031-9d5d-978df9175426", "recognitionModel": "recognition_03",
"faceRectangle": {
"width": 78,
"height": 78,
"left": 394,
"top": 54
},
"faceLandmarks": {
"pupilLeft": {
"x": 412.7,
"y": 78.4
},
"pupilRight": {
"x": 446.8,
"y": 74.2
},
...
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
[Optional Exercise]- Detect and Analyze Faces
Detect faces with the Azure AI Vision service
Detect and analyze faces with the Face service
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Custom vision models with Azure AI Vision
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Learning Objectives
After completing this module, you will be able to:
1 Understand use cases of custom Vision models
2 Label data in Azure ML for both image classification and object detection
3
Use a custom vision model in an app
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Azure AI Custom Vision (previous service)
· Portal: customvision.ai
· Base model:
· Convolutional neural network (CNN)
· Tasks:
· Image classification
· Object detection
· Labeling:
· Customvision.ai
· Minimum training data needed:
· 15 images per category
· Training data storage
· Uploaded to Custom Vision service
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Custom AI Vision models (new Florence model)
· Portal: Vision Studio
· Base model:
· Transformer (multi modal)
· Tasks:
· Image classification
· Object detection
· Product recognition
· Labeling:
· AML Studio or COCO file
· Minimum training data needed:
· 2-5 images per category
· Training data storage
· In user's blob storage account
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Train custom models with your own images
· Upload your images
· Label your images
· Train your model
· Query your model with new images to predict labels
Labeling data lives in COCO file
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
About COCO files
JSON file with specific fields
· images
· annotations
· categories
Defines labeling data from Azure ML project
© Copyright Microsoft Corporation. All rights reserved.
{ "images": [ { "id": 1, "width": 500, "height": 828, "file_name": 'file.jpg", "absolute_url": '{url}" }, ... ], "annotations": [ { "id": 1, "category_id": 7, "image_id": 1, "area": 0.407, "bbox": [ 0.02663142641129032, 0.40691584277841153, 0.9524163571731749, 0.42766634515266866 ] }, ... ], "categories": [ { "id": 1, "name": "apple" }, ...
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Train a model to predict the class label for the image
In other words, what is this a picture of?
Apple
Banana
Orange
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Train a model to detect and locate specific classes of object in images
In other words, what objects are in this image, and where?
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Use the Azure Vision Studio
1. Create a custom model project, or retrieve an existing one
2. Select your resource, if necessary
3. Add your dataset(s) and specify model type
1. Image classification: Assign label to each image
2. Object detection: Define bounding boxes for objects in each image
4. Create your Azure ML project and label your images (if no COCO file exists)
5. Add COCO file (from Azure ML project or previously built)
6. Train a new model
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Lab - Classify Images with Azure AI Vision custom model
Create a custom model
Add COCO file
Integrate custom model into app
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Analyzing Videos
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Video analysis:
· Facial recognition (limited access)
· Optical character recognition
· Speech transcription
· Topics
· Sentiment
· Labels
· Content moderation
· Scene segmentation
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Custom Insights
Pre-defined models for recognizing language, well-known celebrities, brands, …
Create your own models for:
* Facial recognition and celebrity identification requires Limited Access approval, in accordance with our Responsible AI Standard
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Video Indexer Widgets and API
Share insights and analysis in web pages
Automate video analysis with the REST API
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
[Optional] Exercise- Analyze Video
Analyze a video
Use Video Indexer widgets
Use the Video Indexer API
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Knowledge check
You want to use the Azure AI Vision Analyze Image function to generate an appropriate caption for an image. Which visual feature should you specify?
⃣
Tags
⃣
Caption
⃣
Text
2 What is the effect of the Smart Cropping option when using Azure AI Vision to generate a thumbnail?
⃣
The aspect ratio of the original image is maintained.
⃣
The thumbnail is skewed to fit the specified proportions.
⃣
The region of interest is centered in the thumbnail.
3 You want Video Analyzer to recognize colleagues in videos recorded from conference calls. What should you do?
⃣
Create a custom model containing a Person for each colleague, with example images of their faces.
⃣
Edit the conference call videos to include a caption of each person's name on their first appearance.
⃣
Embed the Video Analyzer widgets in a custom web site that employees access using their own user credentials.
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive
5


key: chroma:document
string_value:
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive
5
Which of the following facial attributes can the Azure AI Vision service predict?
⃣
Type of eye-classes
⃣
Location
⃣
Occlusion
You need to create a facial recognition solution that can identify named employees. Which service should you use?
⃣
Vision
⃣
Personalizer
⃣
Face
6 You need to detect if a specific dangerous item is in your company's inspection photos. What should you do?
⃣
Create an image classification custom model.
⃣
Use the Object feature in Image Analysis.
⃣
Train a custom model to detect that item.
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
In this learning path, we:
Used Image Analysis to analyze images, extract insights, remove background, and perform OCR.
Detected faces and facial recognition.
Created custom vision models trained on your own images.
Extracted insights from videos with Azure Video Indexer.
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Fundamentals of Deep Learning
Part 1: An Introduction to Deep Learning
•, 1 = Part 1:  An Introduction to Deep Learning. •, 1 = Part 2: How a Neural Network Trains. •, 1 = Part 3: Convolutional Neural Networks. •, 1 = Part 4: Data Augmentation and Deployment. •, 1 = Part 5: Pre-Trained Models. •, 1 = Part 6: Advanced Architectures
Agenda
To see lecture notes, make full screen and click the “notes” button
Welcome!


key: chroma:document
string_value:
Human Vs Machine Learning
Relaxed Alertness
Human, 1 = Machine. Rest and Digest, 1 = Training. Fight-or-flight, 1 = Prediction


key: chroma:document
string_value:
Early Neural Networks
Inspired by biology
Created in the 1950’s
Outclassed by Von Neumann Architecture


key: chroma:document
string_value:
Expert Systems - Limitations
What are these three images?
Expose them to lots of data
Give them the “correct answer”
They will pick up the important patterns on their own


key: chroma:document
string_value:
Data
Networks need a lot of information to learn from
The digital era and the internet has supplied that data


key: chroma:document
string_value:
Computing Power
Need a way for our artificial “brain” to observe lots of data within a practical amount of time.


key: chroma:document
string_value:
The Importance of the GPU
A Neural Network
A Rendered Image
What is Deep Learning?
Deep learning flips traditional programming on its head


key: chroma:document
string_value:
Machine Learning
Building a Classifier
This is a Fundamental Shift
Classic Programming
Deep Learning
When to Choose Deep Learning
Deep Learning Compared to Other AI
How Deep Learning is Transforming the World
Computer Vision
Natural Language Processing
Recommender Systems
Reinforcement Learning
Overview of the Course


key: chroma:document
string_value:
Hands on Exercises
Get comfortable with the process of deep learning
Exposure to different models and datatypes
Get a jump-start to tackle your own projects
Structure of the Course
Platform of the Course
Major deep learning platforms: 
TensorFlow + Keras (Google)
PyTorch (Meta)
MXNet (Apache)
We’ll be using PyTorch
Good idea to gain exposure to others moving forward
Software of This Course
First Exercise:Classify Handwritten Digits


key: chroma:document
string_value:
Agenda
· Implementing an Intelligent Search Solution
· Developing Custom Skills for an Enrichment Pipeline
· Creating a Knowledge Store
© Copyright Microsoft Corporation. All rights reserved.
Implementing an Intelligent Search Solution
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Learning Objectives
After completing this module, you will be able to:
1 Create an Azure AI Search Solution
2 Implement a custom skill for Azure AI Search and integrate it into a skillset
3 Create a knowledge store with object, file, and table projections
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
AI-Powered Knowledge Mining
· Index documents and data from a range of sources
· Use skills to enrich index data
· Store extracted insights in a knowledge store for analysis and integration


key: chroma:document
string_value:
Azure Resources:
· Azure AI Search for core indexing and querying
· Azure AI Services for index enrichment
· Storage account for knowledge store persistence
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Data Source
The data store to be searched:
· Blob storage container
· SQL Database
· Cosmos DB
You can also push JSON documents directly into an index


key: chroma:document
string_value:
Skillset
Defines an enrichment pipeline of AI skills to enhance data during indexing:
· Built-in AI skills
· Custom skills
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Indexer
Maps data source fields and skillset outputs to index fields
· Running the indexer builds the index


key: chroma:document
string_value:
[Optional] Create an Azure Cognitive Search Solution
Create an indexing solution
Modify an indexing solution
Query an index from a client application
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
When builtin skills don't provide what you need…
Create a custom skill, for example:
· Integrate Document Intelligence
· Consume an Azure Machine Learning model
· Any other custom logic
Custom skills are implemented as Web APIs
· Commonly Azure Functions
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Input Schema
{ "values": [ { "recordId": "<unique_identifier>", "data": { "<input1_name>":  "<input1_value>", "<input2_name>": "<input2_value>", ... } }, { "recordId": "<unique_identifier>", "data": { "<input1_name>":  "<input1_value>", "<input2_name>": "<input2_value>", ... } }, ... ] }


key: chroma:document
string_value:
Output Schema
{ "values": [ { "recordId": "<unique_identifier_from_input>", "data": { "<output1_name>":  "<output1_value>", ... }, "errors": [...], "warnings": [...] }, { "recordId": "< unique_identifier_from_input>", "data": { "<output1_name>":  "<output1_value>", ... }, "errors": [...], "warnings": [...] }, ... ] } This is a property bag of values -it can be a single value or a complex JSON structure
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Specify the URI to your web API endpoint
· Optionally add parameters and headers
Set the context to specify at which point in the document hierarchy the skill should be called


key: chroma:document
string_value:
Store output in a new field
· Optionally, specify a target field name (otherwise the output name is used)
{ "skills": [ ..., { "@odata.type": "#Microsoft.Skills.Custom.WebApiSkill", "description": "<custom skill description>", "uri": "https://<web_api_endpoint>?<params>", "httpHeaders": { "<header_name>": "<header_value>" }, "context": "/document/<where_to_apply_skill>", "inputs": [ { "name": "<input1_name>", "source": "/document/<path_to_input_field>" } ], "outputs": [ { "name": "<output1_name>", "targetName": "<optional_field_name>" } ] } ] }


key: chroma:document
string_value:
Exercise -Create a Custom Skill for Azure AI Search
Use an Azure Function to implement a custom skill Integrate a custom skill into a skillset
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Stored as projections in Azure Storage
· Tables: Relational tables with keys for joining
· Objects: JSON structures of document fields
· Files: Extracted images saved in JPG format
Used for analysis or integration into data processing workflows
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Restructure fields to simplify projections
· Create a JSON object with the fields you want to persist
· Use sourceContext and inputs to map primitives to well-formed JSON objects
{ "@odata.type": "#Microsoft.Skills.Util.ShaperSkill", "name": "define-projection", "description": "Prepare projection fields", "context": "/document", "inputs": [ { "name": "url", "source": "/document/url" }, { "name": "sentiment", "source": "/document/sentiment" }, { "name": "key_phrases", "source": null, "sourceContext": "/document/merged_content/keyphrases/*", "inputs": [ { "name": "phrase", "source": "/document/merged_content/keyphrases/*" } ] } ], "outputs": [ { "name": "output", "targetName": "projection" } ] }


key: chroma:document
string_value:
Extended interactive exercises -Create a Knowledge Store with Azure AI Search
https://aka.ms/km-ai-lp
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Knowledge check
1
You want to enrich an index by extracting any geographical locations mentioned in the source data. Which built-in skill should you use?
⃣
Entity Recognition
⃣
Key Phrase Extraction
⃣
Language Detection
2 You have implemented a custom skill as an Azure function. How can you include the custom skill in your indexing process?
⃣
Add a Merge skill to the skillset to combine output from built-in skills with your custom skill.
⃣
Add a WebApiSkill to a skillset, referencing the Azure function's URI
⃣
Add a Shaper skill to the skillset to create a collection of records with unique IDs generated by your custom
3 You want to create a knowledge store that contains JSON representations of the extracted data. What kind of projection should you define?
⃣
File
⃣
Object
⃣
Table
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Learning Path Recap
In this learning path, we learned how to:
Create an Azure AI Search Solution
Implement a custom skill for Azure AI Search and integrate it into a skillset
Create a knowledge store with object, file, and table projections
© Copyright Microsoft Corporation. All rights reserved.
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Fundamentals of Deep Learning
Part 6: Advanced Architectures
•, 1 = Part 1:  An Introduction to Deep Learning. •, 1 = Part 2: How a Neural Network Trains. •, 1 = Part 3: Convolutional Neural Networks. •, 1 = Part 4: Data Augmentation and Deployment. •, 1 = Part 5: Pre-Trained Models. •, 1 = Part 6: Advanced Architectures
Agenda
Moving Forward


key: chroma:document
string_value:
Fields of AI
Natural Language Processing
Dictionary
A
An
And
At
Ate
Bark
Barked
Cat
Cats
Dog
Dogs
Eat
“A dog barked at a cat.”
[1, 10, 7, 4, 1, 8]


key: chroma:document
string_value:
From Words to Numbers
Barked
Bark
Ate
And
At
An
A
Cat
Cats
Dog
Dogs
Eat
Barked
Bark
Ate
And
At
An
A
Cat
Cats
Dog
Dogs
Eat
Inputs
Outputs
Dictionary
A
An
And
At
Ate
Bark
Barked
Cat
Cats
Dog
Dogs
Eat
50%
0%
35%
10%
5%
0%
0%
0%
0%
0%
0%
0%
0
0
0
0
0
0
0
0
0
1
0
0
Inputs
Outputs
Dictionary
A
An
And
At
Ate
Bark
Barked
Cat
Cats
Dog
Dogs
Eat
Bigger Dictionary
A
An
And
At
Ate
Bark
Barked
Cat
Cats
Dog
Dogs
Eat
Eaten
A
An
And
At
Ate
Bark
Barked
Cat
Cats
Dog
Dogs
Eat
Eaten
A
An
And
At
Ate
Bark
Barked
Cat
Cats
Dog
Dogs
Eat
Eaten
A
An
And
At
Ate
Bark
Barked
Cat
Cats
Dog
Dogs
Eat
Eaten
A
An
And
At
Ate
Bark
Barked
Cat
Cats
Dog
Dogs
Eat
Eaten
A
An
And
At
Ate
Bark
Barked
Cat
Cats
Dog
Dogs
Eat
Eaten
…
…
…
…
Domestic
Wild
Big
Small
Kitty
(-.75, -.8)
Llama
(-.9, .1)
Falcon
(.15, -.4)
Puffin
(.85, -.65)
Giraffe
(.9, .9)
Barked
Bark
Ate
And
At
An
A
Cat
Cats
Dog
Dogs
Eat
Barked
Bark
Ate
And
At
An
A
Cat
Cats
Dog
Dogs
Eat
Technically an Embedding
Inputs
Outputs
Bigger Dictionary
A
An
And
At
Ate
Bark
Barked
Cat


key: chroma:document
string_value:
(.9, .9)
Barked
Bark
Ate
And
At
An
A
Cat
Cats
Dog
Dogs
Eat
Barked
Bark
Ate
And
At
An
A
Cat
Cats
Dog
Dogs
Eat
Technically an Embedding
Inputs
Outputs
Bigger Dictionary
A
An
And
At
Ate
Bark
Barked
Cat
Cats
Dog
Dogs
Eat
Eaten
A
An
And
At
Ate
Bark
Barked
Cat
Cats
Dog
Dogs
Eat
Eaten
A
An
And
At
Ate
Bark
Barked
Cat
Cats
Dog
Dogs
Eat
Eaten
A
An
And
At
Ate
Bark
Barked
Cat
Cats
Dog
Dogs
Eat
Eaten
A
An
And
At
Ate
Bark
Barked
Cat
Cats
Dog
Dogs
Eat
Eaten
A
An
And
At
Ate
Bark
Barked
Cat
Cats
Dog
Dogs
Eat
Eaten
…
…
…
…
Attention


key: chroma:document
string_value:
Sentence Prediction
I am the very model of a modern Major-Gineral,
I've information vegetable, animal, and mineral,
...
I'm very good at integral and differential calculus;
I know the scientific names of beings animalculous:
In short, in matters vegetable, animal, and mineral,
I am the very model of a modern Major-Gineral.
~ Major-General Stanley
I am the very model of a modern Major-Gineral,
I've information vegetable, animal, and mineral,
...
I'm very good at integral and differential calculus;
I know the scientific names of beings animalculous:
In short, in matters vegetable, animal, and mineral,
I am the very model of a modern Major-Gineral.
~ Major-General Stanley


key: chroma:document
string_value:
Attention
5 x 3
5 x 3
Q
K
Query
Key
5 x 3
5 x 3
Q
K
Query
Key
3 x 5
5 x 3
Attention
3 x 5
5 x 3
5 x 5
, 1 = I. , 2 = Understand. , 3 = Equations. , 4 = Both. , 5 = Simple. , 6 = and. , 7 = Quadraical. I, 1 = . I, 2 = . I, 3 = . I, 4 = . I, 5 = . I, 6 = . I, 7 = . Understand, 1 = . Understand, 2 = . Understand, 3 = . Understand, 4 = . Understand, 5 = . Understand, 6 = . Understand, 7 = . Equations, 1 = . Equations, 2 = . Equations, 3 = . Equations, 4 = . Equations, 5 = . Equations, 6 = . Equations, 7 = . Both, 1 = . Both, 2 = . Both, 3 = . Both, 4 = . Both, 5 = . Both, 6 = . Both, 7 = . Simple, 1 = . Simple, 2 = . Simple, 3 = . Simple, 4 = . Simple, 5 = . Simple, 6 = . Simple, 7 = . And, 1 = . And, 2 = . And, 3 = . And, 4 = . And, 5 = . And, 6 = . And, 7 = . Quadratical, 1 = . Quadratical, 2 = . Quadratical, 3 = . Quadratical, 4 = . Quadratical, 5 = . Quadratical, 6 = . Quadratical, 7 = 
5 x 3
5 x 3
5 x 3
Q
K
V
Query
Key
Value
5 x 3
5 x 3
5 x 3
5 x 3
Q
K
V
Z
Query
Key
Value


key: chroma:document
string_value:
5 x 3
5 x 3
5 x 3
Q
K
V
Query
Key
Value
5 x 3
5 x 3
5 x 3
5 x 3
Q
K
V
Z
Query
Key
Value
Attention Score


key: chroma:document
string_value:
ENABLING PORTABILITY WITH NGC CONTAINERS
Extensive
Diverse range of workloads and industry specific use cases
DL containers updated monthly
Packed with latest features and superior performance
Scanned for vulnerabilities and crypto
Tested on workstations, servers, & cloud instances
Supports multi-GPU & multi-node systems
Supports Docker, Singularity & other runtimes
Bare metal, VMs, Kubernetes
x86, ARM, POWER
Multi-cloud, on-prem, hybrid, edge
Optimized
Secure & Reliable
Scalable
Designed for Enterprise & HPC
Run Anywhere
NGC Deep Learning Containers
Learn more about NGC Containers
Riva


key: chroma:document
string_value:
Introduction to AI and AI on Azure
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Agenda
· Introduction to AI
· AI on Azure
· Get started with Azure AI services
· Using Azure AI Services for enterprise applications
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Introduction to AI and Azure AI services
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Learning Objectives
After completing this module, you will be able to:
Describe artificial intelligence and how it compares to machine learning and data science.
Describe Azure AI services.
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
What is Artificial Intelligence?
Software that exhibits human-like capabilities, such as:
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Data Science
Application of mathematical and statistical techniques to analyze data
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Software Development Skills
· Coding (C#, Python, Node.js, …)
· Consuming APIs (REST or SDKs)
· DevOps (source control, CI/CD)
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Conceptual AI Understanding
· Model training and inferencing
· Probability and confidence scores
· Responsible AI and ethics
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Considerations for Responsible AI
https://www.microsoft.com/ai/responsible-ai
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Azure Machine Learning
Cloud platform for creating and operating machine learning solutions
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Capabilities include:
· Text analysis, Speech = · Speech recognition. · Text analysis, Vision = · Image and video analysis. · Text analysis, Generative = · Generate text completions. · Question answering, Speech = · Speech synthesis. · Question answering, Vision = · Image classification. · Question answering, Generative = · Image generation. · Language understanding, Speech = · Speech Translation. · Language understanding, Vision = · Object detection. · Language understanding, Generative = · Image generation. · Translation, Speech = · Speaker Recognition. · Translation, Vision = · Optical character recognition. · Translation, Generative = · Image generation
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Azure AI Services
· Azure AI Document Intelligence
· Azure AI Language
· Azure AI Vision
· Azure OpenAI
· Azure AI Search
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
AI-enriched indexing for search and knowledge mining
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Knowledge check
Which of the following best describes the predictions made by a machine learning model?
⃣
Absolutely correct values based on conditional logic.
⃣
Export the model as an Azure AI service.
⃣
You must build your application using the Azure Machine Learning designer.
3 You want to index a collection of text documents, and search them from a mobile application. Which service should you use to create the index.
⃣
Azure AI Language
⃣
Azure AI Search
⃣
Azure AI Speech
© Copyright Microsoft Corporation. All rights reserved.
⃣
Randomly selected values with an equal chance of selection.
⃣
Probabilistic values based on correlations found in training data.
2 A data scientist has used Azure Machine Learning to train a machine learning model.
How can you use the model in your application?
⃣
Use Azure Machine Learning to publish the model as a web service.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Get Started with Azure AI services
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Learning Objectives
After completing this module, you will be able to:
Understand Azure AI APIs.
Create and consume Azure AI services resources.
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Create a resource in your Azure subscription
· You will create either a single-service resource or a multiservice resource:
· Multi-service resource (Azure AI Services):
- Access multiple Azure AI Services with a single key and endpoint.
- Consolidates billing from the services you use.
· Single-service resource (for example, Language):
- Access a single Azure AI service with a unique key and endpoint for each service created.
- Use the free tier to try out the service.
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Location:
· Azure data center in which resource is provisioned
· Required by some SDK clients
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Clients submit HTTP requests to the resource endpoint
· Key specified in request header
· Input data in JSON format
· Specific schema varies by service and method
Service returns JSON response
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Azure AI Services SDKs
Runtime library abstracts REST interface
Multiple SDKs for each service:
· .NET
· Python
· Node.js
· Java
· Others…
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Exercise - Get Started with Azure AI Services
Provision an Azure AI Services resource
Use a REST interface
Use an SDK
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive
Using Azure AI Services for enterprise applications
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Learning Objectives
After completing this module, you will be able to:
Consider and manage authentication and network security for Azure AI services.
2 Manage costs, view metrics, and manage alerts and diagnostic logging.
3 Deploy to secure containers and consume Azure AI services from containers.
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Regenerate keys regularly to protect access
· To avoid service interruption, switch apps to use key 2 before regenerating key 1; and vice-versa


key: chroma:document
string_value:
Consider protecting keys by storing them in Azure Key Vault
· Apps can use a Service Principal as a managed identity to retrieve keys from Key Vault
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
[Optional] Exercise - Manage Azure AI Services security
Manage Authorization Keys
Secure Key Access with Azure KeyVault
References:
1. Lab 2: mslearn-ai-services/Instructions/Exercises at main · MicrosoftLearning/mslearn-ai-services (github.com)
2. Exercise - Manage Azure AI Services Security Training | Microsoft Learn
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Alerts
· Alerts will ensure that the correct team knows when a problem arises.
· Every alert or notification available in Azure Monitor is the product of a rule


key: chroma:document
string_value:
Metrics
· Metrics are numerical values
· The metrics are collected at regular intervals and are useful for alerting.
· Metrics are stored in a time-series database.
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Diagnostic settings
· Configure diagnostic settings is to provide detailed information for diagnostics and auditing.
· Diagnostic Destinations:
-Log Analytics Workspace
-Event Hubs
-Azure Storage


key: chroma:document
string_value:
Logs
· Logs contain timestamped information about changes made to resources.
· The log data is organized into record
· The logs can include numeric values, but most include text data
· The most common type of log entry records an event
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
[Optional] Exercise - Monitor Azure AI Services
Configure an alert
Visualize a metric
You may refer to lab 3 mslearn-aiservices/Instructions/Exercises at main · MicrosoftLearning/mslearn-ai-services (github.com)
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Container images are available for commonly used Azure AI services APIs
· Deploy containers to:
· Local Docker hosts
· Azure Container Instances
· Azure Kubernetes Services clusters
· others…


key: chroma:document
string_value:
Enables more control over data sent to public Azure AI service endpoint
· An Azure AI services resource is still required, and the container must communicate with it to send billing data
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
Extended interactive exercises - Use an Azure AI Services container
https://aka.ms/ai-services-lp
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
How are client applications typically granted access to an Azure AI services endpoint?
⃣
The application must specify a valid subscription key for the Azure resource.
⃣
All data passed from the client application to the container is forwarded to the Azure resource endpoint.
⃣
The container must be able to connect to the Azure resource endpoint to send usage data for billing.
© Copyright Microsoft Corporation. All rights reserved.
⃣
The user of the application must enter a user name and password associated with the Azure subscription.
⃣
Access to Azure AI services is granted to anonymous users by default.
2 You want to keep track of how often the subscription keys for your Azure AI services resource are retrieved. What should you do?
⃣
Regenerate the keys for your Azure AI services resource.
⃣
Create an alert for your Azure AI services resource.
⃣
Store the keys in Azure Key Vault.
3 You plan to use an Azure AI services container in a local Docker host. Which of the following is true?
⃣


key: chroma:document
string_value:
⃣
Create an alert for your Azure AI services resource.
⃣
Store the keys in Azure Key Vault.
3 You plan to use an Azure AI services container in a local Docker host. Which of the following is true?
⃣
Client applications must pass a subscription key to the Azure resource endpoint before using the container.
Official (Closed) and Non-Sensitive


key: chroma:document
string_value:
In this learning path, we:
Described artificial intelligence and how it compares to machine learning and data science.
Described Azure AI services.
Understood how to get started with Azure AI services
Understood how to use Azure AI Services for enterprise applications
© Copyright Microsoft Corporation. All rights reserved.
Official (Closed) and Non-Sensitive
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Develop Generative AI Solutions with Azure OpenAI Service
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Agenda
· Get started with Azure OpenAI Service
· Develop apps with Azure OpenAI Service
· Apply prompt engineering with Azure OpenAI Service
· Use your own data with Azure OpenAI Service
© Copyright Microsoft Corporation. All rights reserved.
Get started with Azure OpenAI Service
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Learning Objectives
After completing this module, you will be able to:
1 Describe what generative AI is
2 Provision a resource and deploy a model
3 Use Azure AI Studio
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Artificial Intelligence
the field of computer science that seeks to create intelligent machines that can replicate or exceed human intelligence


key: chroma:document
string_value:
Machine Learning
subset of AI that enables machines to learn from existing data and improve upon that data to make decisions or predictions


key: chroma:document
string_value:
Deep Learning
a machine learning technique in which layers of neural networks are used to process data and make decisions


key: chroma:document
string_value:
Generative AI
Create new written, visual, and auditory content given prompts or existing data.
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Provision an Azure OpenAI resource in Azure
Home Azure Al services Azure OpenAl
Create an Azure OpenAI resource in the Azure portal
Deploy a resource in Azure, then deploy and use a model to in Azure AI Studio


key: chroma:document
string_value:
Alternatively, use the Azure CLI
az cognitiveservices account create \
-n MyOpenAIResource \
-g MyResourceGroup \
-l eastus \
--kind OpenAI \
--sku s0 \
--subscription subscriptionID
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Create Azure OpenAl
Basics
Network
Tags
Review + submit
Enable new business solutions with OpenAl's language generation capabilities powered by GPT-3 models These models have been pretrained with trillions of words and can easily adapt to your scenario with a few short examples provided at inference. Apply them to numerous scenarios, from summarization to content and code generation


key: chroma:document
string_value:
Project Details
Subscription
Resource group
Create new
Instance Details
Region
East US
Name
Pricing tier


key: chroma:document
string_value:
Azure AI Studio
· Web portal for working with AI models: https://ai.azure.com/
· View and deploy base models
· Connect to other AI Services or your own data source
· Manage fine tuning and data files for custom models
· Test models in visual playgrounds:
· Chat (GPT-3.5-Turbo and later models)
· DALL-E (Image generations)
· Assistants (Custom and Copilot-like experiences)
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Types of generative AI model
GPT-4, Description = Newest, most capable chat-based  models for language and code  generation. GPT-3, Description = Natural language and code- generation models. Embeddings, Description = Models that use embeddings for  specific tasks (similarity, text search,  and code search). DALL-E, Description = Image-generation model ( preview,  restricted regions )
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Deploy a model in Azure AI Studio to use it
· You can deploy one or more instances of each available model
· The number of deployments depends on your quota, which you can see in the portal
· Alternatively, use the Azure CLI
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Using prompts to get completions from models
Classifying content, Prompt = Tweet: I enjoyed the training course. Sentiment:. Classifying content, Completion = Positive. Generating new content, Prompt = Write a poem about databases. Generating new content, Completion = Databases, oh databases, You keep our information safe, From the small to the large, You store our data in a place.. Transformation/Translation, Prompt = English: Hello French:. Transformation/Translation, Completion = Bonjour. Summarization, Prompt = Scotland is [ long description of  Scotland… ]. Summarization, Completion = Scotland is [ summarized description… ]. Continuation, Prompt = One way to grow tomatoes is to. Continuation, Completion = start with seeds…. Question answering, Prompt = How many moons does Earth have?. Question answering, Completion = Earth has one moon.. Chat, Prompt = Setup, followed by messages…. Chat, Completion = A sequence of relevant responses


key: chroma:document
string_value:
Testing models in Azure AI Studio playground
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
[Optional Exercise]-Get started with Azure OpenAI Service
Deploy Azure OpenAI resource and model Test deployed model in playground
Azure OpenAI Service models - Azure OpenAI | Microsoft Learn
© Copyright Microsoft Corporation. All rights reserved.
Develop apps with Azure OpenAI Service


key: chroma:document
string_value:
Learning Objectives
After completing this module, you will be able to:
Integrate Azure OpenAI into your app
Use the REST API 2
Use language specific SDKs 3
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Integrating Azure OpenAI into your app
Applications submit prompts to deployed models. Responses are completions.
Three REST API endpoints:
· Completion - model takes an input prompt, and generates one or more predicted completions
· Embeddings - model takes input and returns a vector representation of that input
· ChatCompletion - model takes input in the form of a chat conversation (where roles are specified with the message they send), and the next chat completion is generated
ChatCompletion will be the endpoint we focus on for this course
Use Completion and Embeddings with GPT-3 based models Use ChatCompletion with GPT-35-Turbo and later models
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Using the Azure OpenAI REST API
Completion Endpoint
https:// endpoint .openai.azure.com/openai/deployments/ deployment / completions
{ "prompt": "Your favorite Shakespeare play is", "max_tokens": 5 } { "id": '1234….", "object": "text_completion", "created": 1679001781, "model": 'gpt-35-turbo", "choices": [ { "text": "Macbeth", "index": 0, "logprobs": null, "finish_reason": "stop" } ] }
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Embedding Endpoint
https:// endpoint .openai.azure.com/openai/deployments/ deployment / embeddings
{ "input": "The food was delicious and the waiter was very friendly..." } { "object": "list", "data": [ { "object": "embedding", "embedding": [ 0.0172990688066482523, .... 0.0134544348834753042, ], "index": 0 } ], "model": "text-embedding-ada:002" }
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Using the Azure OpenAI REST API
ChatCompletion Endpoint
https:// endpoint .openai.azure.com/openai/deployments/ deployment / chat / completions
{ "messages":[ {"role": "system", "content": "You are an assistant that teaches people about AI."}, {"role": "user", "content": "Does Azure OpenAI support multiple languages?"}, {"role": "assistant", "content": "Yes, Azure OpenAI supports several languages."}, {"role": "user", "content": "Do other Cognitive Services support translation?"} ] }
{ "id": 'unique_id", "object": "chat.completion", "created": 1679001781, "model": "gpt-35-turbo", "usage": { "prompt_tokens": 95, "completion_tokens": 84, "total_tokens": 179}, "choices": [ {"message": {  "role": "assistant", "content": "Yes, other Azure Cognitive Services also support translation…"}, "finish_reason": "stop", "index": 0} ] }
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Using the Azure OpenAI SDKs
Language specific SDKs are available for use in your applications, in both C# and Python.
Code structure follows a similar pattern for both languages.
Parameters such as Max Response and Temperature are defined in the chat options.
Both synchronous and asynchronous API versions are available.
Pseudo code structure:
<include library>
<create client>
<define chat messages and options>
<send request>
<extract response content>
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Exercise: Integrate Azure OpenAI into your app
Configure your application to connect to Azure OpenAI
Test your app by sending a request to your deployed model
Use the hosted lab environment if provided, or view the lab instructions at the link below:
https://aka.ms/mslearn-azure-openai-api
© Copyright Microsoft Corporation. All rights reserved.
Apply prompt engineering with Azure OpenAI Service


key: chroma:document
string_value:
Learning Objectives
After completing this module, you will be able to:
Understand what prompt engineering is
Understand considerations for different endpoints 2
Explore different techniques of prompt engineering 3
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Constructing prompts to:
Maximize relevancy and accuracy of completions
Specify formatting and style of completions
Provide conversational context
Mitigate bias and improve fairness
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Providing clear instructions
Write a product description for a new water bottle
Introducing the latest addition to our product line - the innovative and eco-friendly water bottle. Made from highquality, BPA-free materials, this bottle is designed to keep your beverages at the perfect temperature for hours. Its sleek and stylish design makes it the perfect accessory for any outdoor activity or workout session. With its convenient carrying handle and leak-proof lid, you  can take it with you wherever you go, without worrying about spills or leaks. Plus, its easy-to-clean design ensures that you can use it again and again, making it the perfect choice for anyone looking for a reliable and sustainable water bottle.


key: chroma:document
string_value:
Introducing our newest water bottle, the eco-friendly choice for the conscious consumer. Made from 100% recycled materials and available in natural colors with no dyes, this bottle is not only stylish but also sustainable. Each purchase of this bottle removes 10 pounds of plastic from our oceans, making it a responsible choice for those who care about the environment. Its durable and leak-proof design makes it perfect for outdoor activities, while its easy-to-carry handle makes it convenient to take on the go. Join us in our mission to reduce plastic waste and choose this eco-friendly water bottle for your daily hydration needs.
© Copyright Microsoft Corporation. All rights reserved.
Write a product description for a new water bottle that is 100% recycled. Be sure to include that it comes in natural colors with no dyes, and each purchase removes 10 pounds of plastic from our oceans


key: chroma:document
string_value:
Cues
Cues provide a starting point on which the completion builds
Use cues to set the scope and format for the output
Cues are particularly useful when generating code. For example:
Provide the first keyword in a SQL SELECT statement
Define a name and parameters for a Python function
---
Review: The writing was good but the movie was too long.
Review: The special effects were terrible.
<more>…
---
Summarize the reviews above.
Most common complaints:
-
· The movie was too long.
· The special effects were terrible.
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Using a system message
When using the ChatCompletions API, a system message sets context that can affect completions
{"role": "system", "content": "You are a casual, helpful assistant. You will talk like an American old western film character."}, {"role": "user", "content": "Can you direct me to the library?"} {"role": "assistant", "content": "Well howdy there, stranger! The library, huh? Y'all just head down the main road till you hit the town square. Once you're there, take a left and follow the street for a spell. You'll see the library on your right, can't miss it. Happy trails!"}
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Conversation history and few shot learning
· Including previous messages helps retain the context and style of a conversation
· Starting a conversation with user-defined messages and responses ( few shot learning) sets examples for the expected responses
{"role": "system", "content": "You are an assistant that evaluates the sentiment of customer feedback."}, {"role": "user", "content": "That was an awesome experience"}, {"role": "assistant", "content": "positive"}, {"role": "user", "content": "I won't do that again"}, {"role": "assistant", "content": "negative"}, {"role": "user", "content": "That was not worth my time"}, {"role": "assistant", "content": "negative"}, {"role": "user", "content": "You can't miss this"}
{"role": "assistant", "content": "positive"},
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Chain of thought
Ask a model to break down its response and explain its reasoning
What sport is easiest to learn but hardest to master? Give a step by step approach of your thoughts, ending in your answer
Step 1: Identify the criteria for "easy to learn" and "hard to master" sports.
For a sport to be considered easy to learn, it should have simple rules and require minimal equipment For a sport to be considered hard to master, it should require years of practice to perfect and have a large variety of techniques and strategies…
Step 2: Consider different sports that fit these criteria.
Some sports that could be considered easy to learn but hard to master include:
-Tennis: Simple rules, easy to pick up, but requires years of practice to master all the different shots and strategies.
-Golf: Basic swing mechanics are easy to learn but perfecting them takes a lifetime of practice.
-
…
Step 3: Evaluate each sport based on the criteria.


key: chroma:document
string_value:
-Golf: Basic swing mechanics are easy to learn but perfecting them takes a lifetime of practice.
-
…
Step 3: Evaluate each sport based on the criteria.
Tennis and golf both require expensive equipment, which could make them less accessible to beginners. In terms of skill ceiling and time required to master, tennis and golf both have a high skill ceiling and require years of practice to perfect….
Step 4: Make a decision based on the evaluation.
Based on the above criteria and evaluation, I would say that tennis is the sport that is easiest to learn but hardest to master. While it may require expensive equipment, it is still accessible to many people. The basic rules and techniques are easy to learn, but mastering all the different shots and strategies takes years of practice…
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Exercise: Utilize prompt engineering in your app
Explore prompt engineering techniques
Connect your app to Azure OpenAI and test prompts with increasing prompt engineering
Use the hosted lab environment if provided, or view the lab instructions at the link below:
https://aka.ms/openai-prompt-engineering-lab
© Copyright Microsoft Corporation. All rights reserved.
Implement Retrieval Augmented Generation (RAG) with Azure OpenAI Service


key: chroma:document
string_value:
Learning Objectives
After completing this module, you will be able to:
Understand how RAG using your own data works 1
Use the REST API 2
Use language specific SDKs 3
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Set up your data source
· Use an existing data source, such as an Azure search resource
· Use the Azure AI Studio to create that data source, if you don't already have one
· When creating the data source, you can use data already in your account such as blob storage
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Configure the studio or your app to connect to that data source
· In the studio, set up the connection by pointing it to the data source
· In your app, specify the data source in the prompt parameters
· Both configurations allow the search resource to augment the prompt


key: chroma:document
string_value:
Use the Azure OpenAI model, which now uses your data for grounding
· Chat with the AI models like normal
· If the data source has relevant information about the prompt, it will use that data
· You can specify if the AI model is limited to just your data source


key: chroma:document
string_value:
Connect to your data source
· Add your data source in the Chat playground, under Add your data
· Use an existing data source, or use that wizard to create a new one
· Once connected, a new chat session will start. Chat like normal, and see how the AI model references that data
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Using your own data
https:// endpoint .openai.azure.com/openai/deployments/ deployment /chat/completions ?api-version= version
· With each call, you need to specify the data source values, along with the messages array and any other parameters
· Authentication in the data source definition is for your search resource, not your Azure OpenAI resource
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Using the Azure OpenAI SDKs
Language specific SDKs are available for use in your applications, in both C# and Python.
Code structure follows a similar pattern for both languages.
Current supported data sources are:
· Azure AI Search
· Azure Cosmos DB for MangoDB vCore
· Plus others in preview, soon to be released GA
Pseudo code structure:
<include library>
<create client>
<define chat messages and options>
<define data source object to include with request>
<send request>
<extract response content>
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Exercise: Implement Retrieval Augmented Generation (RAG) with Azure OpenAI Service
Set up and connect your data in the chat playground
Configure your app to use your own data for augmenting the prompt
Use the hosted lab environment if provided, or view the lab instructions at the link below:
https://aka.ms/mslearn-openai-own-data
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Extended interactive exercises
Generate code
Generate images
https://aka.ms/develop-azure-openai
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
What is the purpose of providing conversation history to an AI model?
⃣
Providing conversation history to an AI model is irrelevant and has no effect on the AI's performance.
⃣
Stop sequence
3 You plan to implement a multi-turn conversation with Azure OpenAI. Which endpoint should you use?
⃣
Completions
⃣
Chat
⃣
Embeddings
© Copyright Microsoft Corporation. All rights reserved.
⃣
To limit the number of input tokens used by the model
To enable the model to continue responding in a similar way and allow the user to reference previous content in subsequent queries
⃣
Which parameter could you adjust to change the randomness or creativeness of completions?
⃣
Frequency penalty
⃣
Temperature


key: chroma:document
string_value:
Learning Recap
In this learning path, we:
Created and deployed Azure OpenAI resources
Integrated Azure OpenAI into your application through REST APIs and SDKs
Explored prompt engineering techniques to improve model responses
Connected your own data for grounding an Azure OpenAI model
© Copyright Microsoft Corporation. All rights reserved.
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Fundamentals of Deep Learning
Part 4: Data Augmentation and Deployment
•, 1 = Part 1:  An Introduction to Deep Learning. •, 1 = Part 2: How a Neural Network Trains. •, 1 = Part 3: Convolutional Neural Networks. •, 1 = Part 4: Data Augmentation and Deployment. •, 1 = Part 5: Pre-Trained Models. •, 1 = Part 6: Advanced Architectures
Agenda


key: chroma:document
string_value:
Recap of the Exercise
Analysis
CNN increased validation accuracy
Still seeing training accuracy higher than validation
CNN increased validation accuracy
Still seeing training accuracy higher than validation
Clean data provides better examples
Dataset variety helps the model generalize
Analysis
Solution
Recap of the Exercise
Data Augmentation


key: chroma:document
string_value:
Model Deployment
(1, 28, 28)
Image Input
(2, 1, 3, 3)
Kernels
(2, 28, 28)
Stacked Images
(2, 2, 3, 3)
Kernels
(2, 28, 28)
Stacked Images
(1568)
Flattened Image
Vector
…                 …
…                 …
(10)
Output Prediction
(512)
Linear
(512)
Linear
Training Batch Input
Convolution
…
Max Pooling
Resize
Greyscale
“Batch”
(1, 220, 155)
(3, 287, 433)
(3, 220, 155)
(1, 1, 220, 155)
Let’s Try It Out!


key: chroma:document
string_value:
Develop natural language processing solutions
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Agenda
· Analyzing and translating text
· Build a question answering solution
· Build a conversational language understanding app
· Custom classification and named entity extraction
· Speech recognition, synthesis, and translation
© Copyright Microsoft Corporation. All rights reserved.
Analyzing text
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Learning Objectives
After completing this module, you will be able to:
Detect language and extract key phrases
2 Analyze sentiment and detect PII
3 Summarize text
4 Extract entities and linked entities
5 Translate text
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
The Azure AI Language Service
Preconfigured features:
· Language detection
· Key phrase extraction
· Sentiment analysis
· Named entity recognition
· Entity linking
· Summarization
· Personal Identifying Information (PII) detection
Customizable features are covered in another section
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Language detection
· Determine the language in which text is written
· Often useful as a pre-cursor to further analysis that requires a known language
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Key phrase extraction
· Identify the main 'talking points' of the text
· Works best with larger documents (up to 5,120 characters)
{ "documents": [ { "id": "1", "language": "en", "text": "You must be the change you wish to see in the world." }, { "id": "2", "language": "en", "text": "The journey of a thousand miles begins with a single step." } ] } Language (defaults to English if not present)
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Sentiment analysis
· Scores overall document sentiment and individual sentence sentiment
· Sentence sentiment is based on confidence scores for positive , negative , and neutral
· Overall document sentiment is based on sentences:
-All sentences are neutral = neutral
-Sentences include positive and neutral = positive
-Sentences include negative and neutral = negative
-Sentences include positive and negative = mixed
{
"documents": [
{
"language": "en",
"id": "1",
"text": "Smile! Life is good!"
}
]
}
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Entity Linking
· Used to disambiguate entities of the same name
-For example, is 'Venus' a planet or a goddess?
· Wikipedia provides the knowledge base
· Specific article links are determined based on entity context within the text
'I saw Venus shining in the sky':
https://en.wikipedia.org/wiki/Venus
'Venus, the goddess of beauty':
https://en.wikipedia.org/wiki/Venus_(mythology)
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Summarization
· Can provide two different types of summarization
· Extractive summarization: Produces summary by using most important sentences
· Abstractive summarization: Produces a summary capturing the main idea, but not necessarily using the same words as the source document
· Can be customized by training on your own data
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Personally Identifiable Information detection
· Used to detect and remove sensitive information
· Entity categories include Person, PhoneNumber, Email, Address, Credit card, and financial account identification
· Can be used in situations like applying sensitivity labels, removing information to reduce bias, and clean data for data science
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Exercise -Analyze Text
Detect Language
Evaluate Sentiment
Identify Key Phrases
Extract Entities
Extract Linked Entities
© Copyright Microsoft Corporation. All rights reserved.
Translating Text
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Multilingual text translation REST API
· Language detection
· One-to-many translation
· Script transliteration
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Create a custom translation model
1. Use the Custom Translator portal
2. Link a workspace to your Azure AI Translator resource
3. Create a project
4. Upload training data files
5. Train a model


key: chroma:document
string_value:
Call your model through the Translator API
· Specify a category parameter with the project category Id
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
[Optional Exercise] -Translate Text
Detect language
Translate text
mslearn-ai-language/Instructions/Exercises at main · MicrosoftLearning/mslearn-ailanguage (github.com)
© Copyright Microsoft Corporation. All rights reserved.
Build a question answering solution
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Learning Objectives
After completing this module, you will be able to:
1 Describe the question answering capabilities
of Azure AI Language.
2 Describe the differences between question answering and conversational language understanding.
Create a knowledge base.
Implement multi-turn conversation.
© Copyright Microsoft Corporation. All rights reserved.
Test and publish a knowledge base.
Consume a published knowledge base.
Implement active learning.


key: chroma:document
string_value:
Introduction to Question Answering
· Knowledge base of question and answer pairs with natural language understanding
· Published as a REST endpoint for applications to consume
· Available through language specific SDKs
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Question answering
· User submits a question, expecting an answer
· Service uses natural language understanding to match the question to an answer in the knowledge base
· Response is a static answer to a known question
· Client application presents the answer to the user
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Language understanding
· User submits an utterance, expecting an appropriate response or action
· Service uses natural language understanding to interpret the utterance, match it to an intent, and identify entities
· Response indicates the most likely intent and referenced entities
· Client application is responsible for performing appropriate action based on the detected intent


key: chroma:document
string_value:
Use the Language Studio portal
1. Create an Azure AI Language service resource in your Azure subscription
2. In Language Studio, select your Azure AI Language resource and create a Custom question answering project.
3. Populate the knowledge base:
· Import from existing FAQ web page
· Upload document files
· Add predefined 'chit -chat' pairs
5. Create the knowledge base and edit question and answer pairs
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Add follow-up prompts to define multi-turn exchanges
· Can reference existing question and answer pairs
· Can be restricted to follow-up responses only
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Test interactively in Language Studio
· Inspect results to see confidence scores
· Add alternative phrases to improve scores as necessary


key: chroma:document
string_value:
Publish the trained knowledge base
· Creates an HTTP REST-based endpoint for client apps to consume
· Published knowledge base can be used with SDKs within your app
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Creating client apps
· REST Interface or SDKs
· Submit questions to the endpoint
{ "question": "What do I need to do to cancel a reservation?", "top": 2, "scoreThreshold": 20, "strictFilters": [ { "name": "category", "value": "api" } ] }
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Improving Question Answering Performance
Enable Active Learning to suggest alternatives when multiple questions have similar scores for user input
· Implicit : The service identifies potential alternative phrases for questions; and presents suggestions in the Language Studio. Periodically review and accept/reject the suggestions.
· Explicit : The service returns multiple possible question matches to the user, and the user identifies the correct one. The client app then uses the API to submit feedback items, identifying the correct answer.


key: chroma:document
string_value:
Create Synonyms for terms with the same meaning
· Add synonyms to the knowledge base through the API or Language Studio interface.
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Exercise -Create a Question Answering solution
Create and edit a knowledge base
Train, test, and deploy the knowledge base
© Copyright Microsoft Corporation. All rights reserved.
Build a conversational language understanding app
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Learning objectives
After completing this module, you will be able to:
1 Provision an Azure AI Language resource
2 Define intents, entities, and utterances
3 Use patterns to differentiate similar utterances and use pre-built entity components
4 Train, test, publish, and review a model
Describe Azure AI Language Understanding
5 features
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Introduction to language understanding
An app accepts natural language input from a user
A language model is used to determine semantic meaning (the user's intent )
The app performs an appropriate action


key: chroma:document
string_value:
Natural Language Processing (NLP) requires a language model to interpret user input
Often this activity is referred to as natural language understanding (NLU)
Conversational language understanding (CLU) is an Azure service to enable you to build natural language understanding component to be used in an end-to-end conversational application.
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
To train a language understanding model:
· Specify utterances that represent expected natural language input
· Map utterances to intents that assign semantic meaning
What time is it?, Intent = GetTime. Tell me the time., Intent = GetTime. What is the weather forecast?, Intent = GetWeather. Do I need an umbrella?, Intent = GetWeather. Turn the light on., Intent = GetWeather. Switch on the fan., Intent = GetWeather. Hello, Intent = None
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Define entities to add specific context to intents
What time is it in London?, Intent.GetTime = GetTime. What time is it in London?, Entities. = Location (London). What's the weather forecast for Paris?, Intent.GetTime = GetWeather. What's the weather forecast for Paris?, Entities. = Location (Paris). Will I need an umbrella tonight?, Intent.GetTime = GetWeather. Will I need an umbrella tonight?, Entities. = Time (tonight). What's the forecast for Seattle tomorrow?, Intent.GetTime = GetWeather. What's the forecast for Seattle tomorrow?, Entities. = Location (Seattle), Time (tomorrow). Turn the light on., Intent.GetTime = TurnOnDevice. Turn the light on., Entities. = Device (light). Switch on the fan., Intent.GetTime = TurnOnDevice. Switch on the fan., Entities. = Device (fan)


key: chroma:document
string_value:
Entity types:
Learned, 1 = List. Learned, 2 = Prebuilt. , 1 = Term in a defined list. , 2 = Machine learned through training Common types like numbers and date/times
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Azure AI Language service capabilities
Features fall into two categories:
Preconfigured features -Can be used without labeling or training:
· Summarization
· Named entity recognition
· PII detection
· Key phrase detection
· Sentiment analysis
· Language detection
© Copyright Microsoft Corporation. All rights reserved.
Learned features -Require labeling, training, and deploying to utilize
· Conversational language understanding
· Custom named entity recognition
· Custom text classification
· Question answering


key: chroma:document
string_value:
Submit a request to a published slot, specifying:
· Kind -Indicates which language feature you're requesting. For example, kind is defined as Conversation for conversational language understanding, or EntityRecognition to detect entities
· Parameters -Indicates the values for various input parameters. These parameters vary depending on the feature.
· Analysis input -Specifies the input documents or text strings to be analyzed by the Azure AI Language service.
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Training, testing, publishing, and reviewing
Train a model to learn intents and entities from sample utterances
Test the model interactively or using a testing dataset with known labels
Deploy a trained model to a public endpoint so client apps can use it
Review predictions and iterate on utterances to train your model
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Exercise -Create a conversational language understanding app
Create intents
Create entities
Test and publish a language model
Query your model from a client app
© Copyright Microsoft Corporation. All rights reserved.
Custom classification and named entity extraction
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Learning Objectives
After completing this module, you will be able to:
Label documents, train and deploy models for custom classification
Understand model performance and see where to improve your model 2
3
Use your custom model in an app
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Assign custom labels to documents
1. Connect to documents in Azure
2. Define class labels to assign to your documents
3. Label documents
4. Train your model


key: chroma:document
string_value:
Call your model through the Language API
· Specify project and deployment name
Can be single label or multi label projects
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Assign custom labels to entities in your documents
1. Connect to documents in Azure
2. Define entity labels to assign to your documents
3. Label documents completely and consistently
4. Train your model


key: chroma:document
string_value:
Call your model through the Language API
· Specify project and deployment name
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Review and improve a model
Train a model to teach labels or entities
Review model performance to determine how to improve performance, including Confusion matrix
Determine what cases need to be added to your training data
Retrain your model with new data included, and repeat as necessary
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
[Optional Exercise] -Extract custom entities
Create a custom named entity recognition project
Create and label entities
Test and publish a custom model
Query your model from a client app
mslearn-ai-language/Instructions/Exercises at main · MicrosoftLearning/mslearn-ai-language (github.com)
© Copyright Microsoft Corporation. All rights reserved.
Speech recognition, translation and synthesis
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Learning Objectives
After completing this module, you will be able to:
2
Provision an Azure resource for the Azure AI Speech service
Use the Speech to text API to implement speech recognition
Use the Text to speech API to implement speech synthesis 3
Configure audio format and voices 4
5
Use Speech Synthesis Markup Language (SSML)
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Speech APIs
· Speech-to-Text API (speech recognition)
· Text-to-Speech API (speech synthesis)
· Speech Translation API
· Speaker Recognition API
· Intent Recognition (uses conversational language understanding)
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Two REST APIs:
· Speech-to-text API -Used by Azure AI Speech SDK -preferred for most scenarios
· Speech-to-text Short Audio API -Useful for short (up to 60s) of audio
© Copyright Microsoft Corporation. All rights reserved.
Azure AI Speech SDK (.NET, Python, JavaScript, etc.)
· Text-to-speech API -Suitable for most scenarios
· Batch synthesis API -Convert large volumes of text to audio files
© Copyright Microsoft Corporation. All rights reserved.
Azure AI Speech SDK (.NET, Python, JavaScript, etc.)


key: chroma:document
string_value:
XML-based language with customization options:
· Speaking styles (Neural voices only)
· Pauses and silence
· Phonemes (phonetic pronunciations)
· Prosody (speaking pitch, range, rate, etc.)
· "say-as" (number, date, time, address, etc.)
· Insert recorded speech or background audio
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Exercise -Recognize and Synthesize Speech
Recognize Speech
Synthesize Speech
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Translation builds on speech recognition:
1. Recognize and transcribe spoken input in speech recognition language
2. Return translations for one or more target languages
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Event-based synthesis
· Only supported for 1:1 translation (single target language)
· Specify desired voice in the TranslationConfig
· Use the Synthesizing event to retrieve audio stream
· Create an event handler
· Use Result.GetAudio() to retrieve byte stream


key: chroma:document
string_value:
Manual synthesis
· Use for multiple target languages
· Translate to text then use Text-to-Speech API to synthesize each translation in the results
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Extended interactive exercises
Custom text classification
Translate speech
https://aka.ms/azure-ai-language-lp
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Knowledge check
Which object should you use to specify that the speech input to be transcribed to text is in an audio file?
⃣
SpeechConfig
⃣
Use the Azure AI Language service to analyze sentiment.
⃣
Use the Azure AI Language service to extract linked entities.
When translating speech, in which cases can you can use the Synthesizing event to synthesize the translations and speech? 3
⃣
Only when translating to a single target language
⃣
AudioConfig
⃣
SpeechRecognizer
You have analyzed text that contains the word 'Paris'. How might you determine if this word refers to the French city or the character in Homer's The Iliad? 2
⃣
Use the Azure AI Language service to extract key phrases.
⃣
Only when translating to multiple target languages
⃣
When translating to one or more target languages
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
⃣
Only when translating to multiple target languages
⃣
When translating to one or more target languages
© Copyright Microsoft Corporation. All rights reserved.
Your app must interpret a command to book a flight to a specified city, such as 'Book a flight to Paris.' How should you model the city element of the command?
⃣
As an intent.
⃣
As an utterance.
⃣
As an entity.
5 Your language model needs to detect an email when present in an utterance. What is the simplest way to extract that email?
⃣
Use Regular Expression entities.
⃣
Use Prebuilt entity components
⃣
Use Learned entity components.
6 How should you create an application that monitors the comments on your company's web site and flags any indication that customers are unhappy?
⃣
Use the Azure AI Translator service to detect profanities in comments.
⃣
Use the Azure AI Language service to perform sentiment analysis of the comments.
⃣
Use the Azure AI Language service to extract named entities from the comments


key: chroma:document
string_value:
⃣
Use the Azure AI Language service to perform sentiment analysis of the comments.
⃣
Use the Azure AI Language service to extract named entities from the comments
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
In this learning path, we learned to:
· Analyze and translate text
· Build a conversational language understanding model
· Build a question answering solution
· Speech recognition, synthesis, and translation
· Connect an app to Azure AI Language resources
© Copyright Microsoft Corporation. All rights reserved.
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Fundamentals of Deep Learning
Part 5: Pre-trained Models
•, 1 = Part 1:  An Introduction to Deep Learning. •, 1 = Part 2: How a Neural Network Trains. •, 1 = Part 3: Convolutional Neural Networks. •, 1 = Part 4: Data Augmentation and Deployment. •, 1 = Part 5: Pre-Trained Models. •, 1 = Part 6: Advanced Architectures
Agenda
Review So Far


key: chroma:document
string_value:
Review So Far
Learning Rate
Number of Layers
Neurons per Layer
Activation Functions
Dropout
Data
Pre-Trained Models


key: chroma:document
string_value:
Transfer Learning
(1568)
Flattened Image
Vector
…                 …
…                 …
(10)
Output Prediction
(512)
Linear
(512)
Linear
(1, 28, 28)
Image Input
(2, 1, 3, 3)
Kernels
(2, 28, 28)
Stacked Images
(2, 2, 3, 3)
Kernels
(2, 28, 28)
Stacked Images
Convolution
Input
Convolution
Convolution
Linear
Output
Linear
Max Pooling
Dropout
Max Pooling
Max Pooling
More Generalized
More Specialized
Freezing the Model?
Let’s Get Started!


key: chroma:document
string_value:
Develop solutions with Azure AI Document Intelligence
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Agenda
· Use prebuilt Document Intelligence models
· Train a custom Document Intelligence model
© Copyright Microsoft Corporation. All rights reserved.
Develop a Document Intelligence solution
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Learning Objectives
After completing this module, you will be able to:
1 Understand models in Azure AI Document Intelligence
2 Train a custom Document Intelligence model
3 Connect an app to Document Intelligence APIs
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
The Document Intelligence Service
Data extraction from forms and documents:
· Document analysis from general documents
· Read: OCR for printed and written text
· Layout: Extract text and structure
· General document: Extract text, structure, and key-value pairs
· Prebuilt models for common form types
· Train custom models for your own forms
· Custom template: Extract data from static layouts
· Custom neural: Extract data from mixed-type documents
· Custom composed: Collection of multiple models assigned to a single model
Provision as single-service Document Intelligence resource or multi-service Azure AI Services resource
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Calling the API
· Each request is configured with your resource endpoint and needs your resource key
· Send the request, which when successful returns a poller to get the results
· REST returns it in Operation-Location header
· SDKs return an object from the request
· Query the poller received for the extracted data


key: chroma:document
string_value:
REST
Request POST:
{endpoint}/documentintelligence/documentModels/prebuiltlayout:analyze?api-version={version}
Operation-Location:
{endpoint}/documentintelligence/documentModels/prebuiltlayout/analyzeResults/ab12345c-12ab-23cd-b19c2322a7f11034?api-version={version}


key: chroma:document
string_value:
C#
AnalyzeDocumentOperation operation = await client.AnalyzeDocumentFromUriAsync(WaitUntil.Completed, "prebuilt-layout", fileUri);
AnalyzeResult result = operation.Value;


key: chroma:document
string_value:
Python
poller=document_analysis_client.begin_analyze_document_ from_url("prebuilt-document", docUrl)
result = poller.result()
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
API response
· Response it broken down by page, lines, and words
· Subset of REST response included here
· SDK response objects have similar structure, broken down similarly
· Additional data about detected text or selection marks, such as bounding box and handwritten style
{
"analyzeResult": { "apiVersion ": '{version}", "modelId": "prebuilt-invoice", ... "pages": [{ "pageNumber": 1, "angle": 0, "width": 8.5, "height": 11, "unit": "inch", "words": [{ "content": 'Margie's", "boundingBox": [ 0.5911, 0.6857, 1.7451, 0.6857, 1.7451, ... ], "confidence": 1, "span": {...} }], }] }
}
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
[Optional] -Use prebuilt Document Intelligence models
Use the Read model
Use an app to use document analysis
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Custom classification
· Apply a label to the entire document
· Ideal for sorting large numbers of incoming documents into types
· Requires two different classes, and a minimum of five labeled documents per class
· One type of training model
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Custom extraction
· Apply label to specific text
· Ideal for extracting custom labels from documents
· Requires five examples of the same document type
· Two training methods:
· Custom template (custom form)
· Training time: 1-5 minutes
· Document structure: forms, templates, other structured documents
· Custom neural (custom document)
· Training time: 20-60 minutes
· Document structure: structured and unstructured documents


key: chroma:document
string_value:
Training Custom Models
Create project and upload training files to your project, or connect to blob storage containing files 1
Add data type (such as field or signature) to start labeling your dataset 2
Select a word in the document, and assign one of the fields to label it 3
Repeat for all fields and files in your dataset 4
Layout and auto label (using a prebuilt model) can assist in this process 5
Train the model, providing a Model ID used in API requests 6
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Accuracy and confidence scores
· After training, a custom model has an estimated accuracy score
· Score is calculated by running combinations of training data predictions against the labeled values
· Confidence score is the same as using prebuilt models, indicated how accurate the model thinks that specific prediction is
· Confidence scores are provided in the response from the model for each predicted label
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Analyze document using custom model
· Requires endpoint and key from deployed resource, similar to prebuilt models
· Needs to also include the ID of your deployed custom model
· Query the poller received for the extracted data


key: chroma:document
string_value:
C#
AnalyzeDocumentOperation operation = await client.AnalyzeDocumentFromUriAsync(WaitUntil.Completed, {modelId} , {fileUri});
AnalyzeResult result = operation.Value;


key: chroma:document
string_value:
Python
poller = document_analysis_client.begin_analyze_document( model_id={modelId} , document={fileUri}
result = poller.result()
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Lab -Extract Data from Forms
Train a custom model in Document Intelligence Studio
Test your custom Document Intelligence model
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Knowledge check
You have scanned a letter into PDF format and need to extract the text it contains. What should you do?
⃣
Use the Image Analysis feature in Azure AI Vision.
⃣
Use the Read model in the Document Intelligence service.
⃣
Use a custom model in the Document Intelligence service.
2 You need to build an application that submits expense claims, extracting the merchant, date, and amount from scanned receipts. What's the best way to do this?
⃣
Use the general document model.
⃣
Use the prebuilt Contract model.
⃣
Use the prebuilt Receipt model.
3 You need to extract only data from specific fields in cargo manifest forms using Document Intelligence. What should you do?
⃣
Use a prebuilt model.
⃣
Build a custom composed model from several custom models.
⃣
Train custom template model with labeled documents.
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
In this learning path, we:
Explored available prebuilt models, and how to use them in Document Intelligence Studio
Trained and deployed a custom model
Connected an app to use Document Intelligence APIs
© Copyright Microsoft Corporation. All rights reserved.
© Copyright Microsoft Corporation. All rights reserved.


key: chroma:document
string_value:
Fundamentals of Deep Learning
Part 2: How a Neural Network Trains
•, 1 = Part 1:  An Introduction to Deep Learning. •, 1 = Part 2: How a Neural Network Trains. •, 1 = Part 3: Convolutional Neural Networks. •, 1 = Part 4: Data Augmentation and Deployment. •, 1 = Part 5: Pre-Trained Models. •, 1 = Part 6: Advanced Architectures
Agenda


key: chroma:document
string_value:
Data Preparation
Input as an Array
28
28
[0,0,0,24,75,184,185,78,32,55,0,0,0…]
Targets as Categories
0
[1,0,0,0,0,0,0,0,0,0] 
1
[0,1,0,0,0,0,0,0,0,0] 
2
[0,0,1,0,0,0,0,0,0,0] 
3
[0,0,0,1,0,0,0,0,0,0] 
.
.
.


key: chroma:document
string_value:
An Untrained Model
[ 0, 0, …, 0]
…                 …                 …
…                 …                 …
(784,)
(512,)
(512,)
(10,)
Layer
Size
A Simpler Model


key: chroma:document
string_value:
Building a Network
w2
w1
Scales to more inputs
w6
w5
w1
w2
x1
x2
w3
w4
Scales to more inputs
Can chain neurons
Scales to more inputs
Can chain neurons
If all regressions are linear, then output will also be a linear regression
w6
w5
w1
w2
x1
x2
w3
w4
Activation Functions


key: chroma:document
string_value:
Overfitting
Why not have a super large neural network?
Which Trendline is Better?
MSE = .0000
MSE = .0113
Which Trendline is Better?
MSE = .0308
MSE = .0062


key: chroma:document
string_value:
An MNIST Model
(784,)
(512,)
(512,)
(10,)
Layer
Size
[ 0, 0, …, 0]
…                 …                 …
…                 …                 …
(784,)
(512,)
(512,)
(10,)
Layer
Size
ReLU
Linear
ReLU
[ 0, 0, …, 0]
…                 …                 …
…                 …                 …
During Prediction
(784,)
(512,)
(512,)
(10,)
ReLU
Softmax
Layer
Size
ReLU
[ 0, 0, …, 0]
…                 …                 …
…                 …                 …
During Training


key: chroma:document
string_value:
Fundamentals of Deep Learning
Part 3: Convolutional Neural Networks
•, 1 = Part 1:  An Introduction to Deep Learning. •, 1 = Part 2: How a Neural Network Trains. •, 1 = Part 3: Convolutional Neural Networks. •, 1 = Part 4: Data Augmentation and Deployment. •, 1 = Part 5: Pre-Trained Models. •, 1 = Part 6: Advanced Architectures
Agenda


key: chroma:document
string_value:
Kernels and Convolution
Darken
Sharpen
Brighten
Blur
Original Image
Darken
Sharpen
Brighten
Blur
Original Image
0, 1 = 0. 0, 2 = 0. 0, 1 = 1.5. 0, 2 = 0. 0, 1 = 0. 0, 2 = 0
0, 1 = 0. 0, 2 = 0. 0, 1 = 0.5. 0, 2 = 0. 0, 1 = 0. 0, 2 = 0
.06, 1 = .13. .06, 2 = .06. .13, 1 = .25. .13, 2 = .13. .06, 1 = .13. .06, 2 = .06
0, 1 = -1. 0, 2 = 0. -1, 1 = 5. -1, 2 = -1. 0, 1 = -1. 0, 2 = 0
1, 1 = 0. 1, 2 = 1. 1, 3 = 1. 1, 4 = 0. 1, 5 = 1. 0, 1 = 1. 0, 2 = 0. 0, 3 = 0. 0, 4 = 1. 0, 5 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0. 1, 1 = 0. 1, 2 = 1. 1, 3 = 1. 1, 4 = 0. 1, 5 = 1. 1, 1 = 1. 1, 2 = 0. 1, 3 = 0. 1, 4 = 1. 1, 5 = 1
.06, 1 = .13. .06, 2 = .06. .13, 1 = .25. .13, 2 = .13. .06, 1 = .13. .06, 2 = .06
Original Image
Blur Kernel
Convolved Image


key: chroma:document
string_value:
Kernels and Convolution
1, 1 = 0. 1, 2 = 1. 1, 3 = 1. 1, 4 = 0. 1, 5 = 1. 0, 1 = 1. 0, 2 = 0. 0, 3 = 0. 0, 4 = 1. 0, 5 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0. 1, 1 = 0. 1, 2 = 1. 1, 3 = 1. 1, 4 = 0. 1, 5 = 1. 1, 1 = 1. 1, 2 = 0. 1, 3 = 0. 1, 4 = 1. 1, 5 = 1
.06, 1 = .13. .06, 2 = .06. .13, 1 = .25. .13, 2 = .13. .06, 1 = .13. .06, 2 = .06
Original Image
Blur Kernel
Convolved Image
1, 1 = 0. 1, 2 = 1. 1, 3 = 1. 1, 4 = 0. 1, 5 = 1. 0, 1 = 1. 0, 2 = 0. 0, 3 = 0. 0, 4 = 1. 0, 5 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0. 1, 1 = 0. 1, 2 = 1. 1, 3 = 1. 1, 4 = 0. 1, 5 = 1. 1, 1 = 1. 1, 2 = 0. 1, 3 = 0. 1, 4 = 1. 1, 5 = 1
.06, 1 = .13. .06, 2 = .06. .13, 1 = .25. .13, 2 = .13. .06, 1 = .13. .06, 2 = .06
Original Image
Blur Kernel
Multiply
Convolved Image


key: chroma:document
string_value:
Kernels and Convolution
1, 1 = 0. 1, 2 = 1. 1, 3 = 1. 1, 4 = 0. 1, 5 = 1. 0, 1 = 1. 0, 2 = 0. 0, 3 = 0. 0, 4 = 1. 0, 5 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0. 1, 1 = 0. 1, 2 = 1. 1, 3 = 1. 1, 4 = 0. 1, 5 = 1. 1, 1 = 1. 1, 2 = 0. 1, 3 = 0. 1, 4 = 1. 1, 5 = 1
.06, 1 = .13. .06, 2 = .06. .13, 1 = .25. .13, 2 = .13. .06, 1 = .13. .06, 2 = .06
Original Image
Blur Kernel
.56, 1 = . .56, 2 = . .56, 3 = . , 1 = . , 2 = . , 3 = . , 1 = . , 2 = . , 3 = . , 1 = . , 2 = . , 3 = 
Convolved Image
.06, 1 = 0. .06, 2 = .06. 0, 1 = .25. 0, 2 = 0. 0, 1 = .13. 0, 2 = .06
Total


key: chroma:document
string_value:
Kernels and Convolution
1, 1 = 0. 1, 2 = 1. 1, 3 = 1. 1, 4 = 0. 1, 5 = 1. 0, 1 = 1. 0, 2 = 0. 0, 3 = 0. 0, 4 = 1. 0, 5 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0. 1, 1 = 0. 1, 2 = 1. 1, 3 = 1. 1, 4 = 0. 1, 5 = 1. 1, 1 = 1. 1, 2 = 0. 1, 3 = 0. 1, 4 = 1. 1, 5 = 1
Original Image
Convolved Image
.56, 1 = .57. .56, 2 = . .56, 3 = . , 1 = . , 2 = . , 3 = . , 1 = . , 2 = . , 3 = . , 1 = . , 2 = . , 3 = 
0, 1 = .13. 0, 2 = .06. .13, 1 = 0. .13, 2 = 0. .06, 1 = .13. .06, 2 = .06
.06, 1 = .13. .06, 2 = .06. .13, 1 = .25. .13, 2 = .13. .06, 1 = .13. .06, 2 = .06
Blur Kernel


key: chroma:document
string_value:
Kernels and Convolution
1, 1 = 0. 1, 2 = 1. 1, 3 = 1. 1, 4 = 0. 1, 5 = 1. 0, 1 = 1. 0, 2 = 0. 0, 3 = 0. 0, 4 = 1. 0, 5 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0. 1, 1 = 0. 1, 2 = 1. 1, 3 = 1. 1, 4 = 0. 1, 5 = 1. 1, 1 = 1. 1, 2 = 0. 1, 3 = 0. 1, 4 = 1. 1, 5 = 1
.56, 1 = .57. .56, 2 = .57. .56, 3 = .56. .7, 1 = .82. .7, 2 = .82. .7, 3 = .7. .69, 1 = .95. .69, 2 = .95. .69, 3 = .69. .64, 1 = .69. .64, 2 = .69. .64, 3 = .64
.06, 1 = .13. .06, 2 = .06. .13, 1 = .25. .13, 2 = .13. .06, 1 = .13. .06, 2 = .06
Blur Kernel
Original Image
Convolved Image
1, 1 = 0. 1, 2 = 1. 1, 3 = 1. 1, 4 = 0. 1, 5 = 1. 0, 1 = 1. 0, 2 = 0. 0, 3 = 0. 0, 4 = 1. 0, 5 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0


key: chroma:document
string_value:
Kernels and Convolution
1, 1 = 0. 1, 2 = 1. 1, 3 = 1. 1, 4 = 0. 1, 5 = 1. 0, 1 = 1. 0, 2 = 0. 0, 3 = 0. 0, 4 = 1. 0, 5 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0


key: chroma:document
string_value:
Stride
Stride 1
.56, 1 = .57
Stride 2
Stride 3
.56, 1 = .57. .56, 2 = .57. .56, 3 = .56
.56, 1 = .56
1, 1 = 0. 1, 2 = 1. 1, 3 = 1. 1, 4 = 0. 1, 5 = 1. 0, 1 = 1. 0, 2 = 0. 0, 3 = 0. 0, 4 = 1. 0, 5 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0


key: chroma:document
string_value:
Padding
Original Image
0, 1 = 0. 0, 2 = 0. 0, 3 = 0. 0, 4 = 0. 0, 5 = 0. 0, 6 = 0. 0, 7 = 0. 0, 1 = 1. 0, 2 = 0. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0. 0, 6 = 1. 0, 7 = 0. 0, 1 = 0. 0, 2 = 1. 0, 3 = 0. 0, 4 = 0. 0, 5 = 1. 0, 6 = 0. 0, 7 = 0. 0, 1 = 0. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 1. 0, 6 = 0. 0, 7 = 0. 0, 1 = 0. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 1. 0, 6 = 0. 0, 7 = 0. 0, 1 = 1. 0, 2 = 0. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0. 0, 6 = 1. 0, 7 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 0. 0, 4 = 0. 0, 5 = 1. 0, 6 = 1. 0, 7 = 0. 0, 1 = 0. 0, 2 = 0. 0, 3 = 0. 0, 4 = 0. 0, 5 = 0. 0, 6 = 0. 0, 7 = 0
Zero Padding


key: chroma:document
string_value:
Padding
1, 1 = 0. 1, 2 = 1. 1, 3 = 1. 1, 4 = 0. 1, 5 = 1. 0, 1 = 1. 0, 2 = 0. 0, 3 = 0. 0, 4 = 1. 0, 5 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0. 1, 1 = 0. 1, 2 = 1. 1, 3 = 1. 1, 4 = 0. 1, 5 = 1. 1, 1 = 1. 1, 2 = 0. 1, 3 = 0. 1, 4 = 1. 1, 5 = 1
Original Image


key: chroma:document
string_value:
Padding
1, 1 = 1. 1, 2 = 0. 1, 3 = 1. 1, 4 = 1. 1, 5 = 0. 1, 6 = 1. 1, 7 = 1. 1, 1 = 1. 1, 2 = 0. 1, 3 = 1. 1, 4 = 1. 1, 5 = 0. 1, 6 = 1. 1, 7 = 1. 0, 1 = 0. 0, 2 = 1. 0, 3 = 0. 0, 4 = 0. 0, 5 = 1. 0, 6 = 0. 0, 7 = 0. 0, 1 = 0. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 1. 0, 6 = 0. 0, 7 = 0. 0, 1 = 0. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 1. 0, 6 = 0. 0, 7 = 0. 1, 1 = 1. 1, 2 = 0. 1, 3 = 1. 1, 4 = 1. 1, 5 = 0. 1, 6 = 1. 1, 7 = 1. 1, 1 = 1. 1, 2 = 1. 1, 3 = 0. 1, 4 = 0. 1, 5 = 1. 1, 6 = 1. 1, 7 = 1. 1, 1 = 1. 1, 2 = 1. 1, 3 = 0. 1, 4 = 0. 1, 5 = 1. 1, 6 = 1. 1, 7 = 1
Mirror Padding


key: chroma:document
string_value:
Padding
1, 1 = 0. 1, 2 = 1. 1, 3 = 1. 1, 4 = 0. 1, 5 = 1. 0, 1 = 1. 0, 2 = 0. 0, 3 = 0. 0, 4 = 1. 0, 5 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0. 0, 1 = 1. 0, 2 = 1. 0, 3 = 1. 0, 4 = 1. 0, 5 = 0. 1, 1 = 0. 1, 2 = 1. 1, 3 = 1. 1, 4 = 0. 1, 5 = 1. 1, 1 = 1. 1, 2 = 0. 1, 3 = 0. 1, 4 = 1. 1, 5 = 1
Kernels and Neural Networks


key: chroma:document
string_value:
Kernels and Neural Networks
w1, 1 = w2. w1, 2 = w3. w4, 1 = w5. w4, 2 = w6. w7, 1 = w8. w7, 2 = w9
Kernel
w1, 1 = w2. w1, 2 = w3. w4, 1 = w5. w4, 2 = w6. w7, 1 = w8. w7, 2 = w9
Kernel
w6
w5
w1
w2
x1
x2
w3
w4
Neuron
(1568)
Flattened Image
Vector
…                 …
…                 …
(10)
Output Prediction
(512)
Linear
(512)
Linear
(1, 28, 28)
Image Input
(2, 1, 3, 3)
Kernels
(2, 28, 28)
Stacked Images
(2, 2, 3, 3)
Kernels
(2, 28, 28)
Stacked Images


key: chroma:document
string_value:
Finding Edges
Original Image
Vertical Edges
Horizontal Edges
1, 1 = 0. 1, 2 = -1. 2, 1 = 0. 2, 2 = -2. 1, 1 = 0. 1, 2 = -1
0, 1 = 0. 0, 2 = 0. 0, 1 = 1. 0, 2 = 0. 0, 1 = 0. 0, 2 = 0
1, 1 = 2. 1, 2 = 1. 0, 1 = 0. 0, 2 = 0. -1, 1 = -2. -1, 2 = -1


key: chroma:document
string_value:
Neural Network Perception
Convolution
Input
Convolution
Convolution
Convolution
Linear
Output
Linear
Textures
Edges
Objects
Other Layers in the Model


key: chroma:document
string_value:
Max Pooling
110, 1 = 256. 110, 2 = 153. 110, 3 = 67. 12, 1 = 89. 12, 2 = 88. 12, 3 = 43. 10, 1 = 15. 10, 2 = 50. 10, 3 = 55. 23, 1 = 9. 23, 2 = 49. 23, 3 = 23
256, 1 = 153. 23, 1 = 55


key: chroma:document
string_value:
Whole Architecture
Convolution
Input
Convolution
Convolution
Linear
Output
Linear
Max Pooling
Dropout
Max Pooling
Max Pooling
Let’s go!


key: chroma:document
string_value:
Generative artificial intelligence - Wikipedia       Jump to content  Main menu Main menu move to sidebar hide Navigation Main pageContentsCurrent eventsRandom articleAbout WikipediaContact us  Contribute HelpLearn to editCommunity portalRecent changesUpload file    Search  Search    Appearance   Donate Create account Log in   Personal tools Donate Create account Log in  Pages for logged out editors learn more  ContributionsTalk     Contents move to sidebar hide (Top) 1 History Toggle History subsection 1.1 Early history   1.2 Academic artificial intelligence   1.3 Generative neural nets (2014-2019)   1.4 Generative AI boom (2020-)  2 Modalities Toggle Modalities subsection 2.1 Text   2.2 Code   2.3 Images   2.4 Audio  2.4.1 Music  2.5 Video   2.6 Actions   2.7 3D modeling  3 Software and hardware   4 Law and regulation Toggle Law and regulation subsection 4.1 Copyright  4.1.1 Training with copyrighted content   4.1.2 Copyright of AI-generated content   5 Concerns Toggle Concerns


key: chroma:document
string_value:
and hardware   4 Law and regulation Toggle Law and regulation subsection 4.1 Copyright  4.1.1 Training with copyrighted content   4.1.2 Copyright of AI-generated content   5 Concerns Toggle Concerns subsection 5.1 Job losses   5.2 Racial and gender bias   5.3 Deepfakes  5.3.1 Audio deepfakes  5.4 Cybercrime   5.5 Reliance on industry giants   5.6 Energy and environment   5.7 Content quality   5.8 Misuse in journalism  6 See also   7 References    Toggle the table of contents  Generative artificial intelligence  45 languages العربيةAzərbaycancaবাংলাБеларускаяBosanskiCatalàČeštinaDanskDeutschΕλληνικάEspañolEsperantoEuskaraفارسیFrançais한국어ՀայերենBahasa IndonesiaIsiZuluItalianoעבריתKiswahiliMagyarमराठीМонголNederlands日本語Norsk bokmålOʻzbekcha / ўзбекчаPolskiPortuguêsQaraqalpaqshaRomânăRuna SimiРусскийSimple EnglishکوردیСрпски / srpskiSvenskaไทยTürkçeУкраїнськаTiếng Việt粵語中文 Edit links  ArticleTalk English   ReadEditView history  Tools Tools move to sidebar hide Actions ReadEditView history


key: chroma:document
string_value:
EnglishکوردیСрпски / srpskiSvenskaไทยTürkçeУкраїнськаTiếng Việt粵語中文 Edit links  ArticleTalk English   ReadEditView history  Tools Tools move to sidebar hide Actions ReadEditView history  General What links hereRelated changesUpload fileSpecial pagesPermanent linkPage informationCite this pageGet shortened URLDownload QR code  Print/export Download as PDFPrintable version  In other projects Wikimedia CommonsWikidata item     Appearance move to sidebar hide  From Wikipedia, the free encyclopedia  AI system capable of generating content in response to prompts  Not to be confused with Artificial general intelligence. This page focuses on statistical machine learning AI. For other topics, see Algorithmic composition, Algorithm art, Generative art, Procedural generation. Théâtre D'opéra Spatial, an image made using generative artificial intelligence Part of a series onArtificial intelligence (AI) Major goals Artificial general intelligence Intelligent agent Recursive self-improvement


key: chroma:document
string_value:
an image made using generative artificial intelligence Part of a series onArtificial intelligence (AI) Major goals Artificial general intelligence Intelligent agent Recursive self-improvement Planning Computer vision General game playing Knowledge reasoning Natural language processing Robotics AI safety Approaches Machine learning Symbolic Deep learning Bayesian networks Evolutionary algorithms Hybrid intelligent systems Systems integration Applications Bioinformatics Deepfake Earth sciences Finance Generative AI Art Audio Music Government Healthcare Mental health Industry Translation Military Physics Projects Philosophy Artificial consciousness Chinese room Friendly AI Control problem/Takeover Ethics Existential risk Turing test Uncanny valley History Timeline Progress AI winter AI boom Glossary Glossary vte Generative artificial intelligence (generative AI, GenAI,[1] or GAI) is a subset of artificial intelligence that uses generative models to produce text, images, videos, or other


key: chroma:document
string_value:
Glossary Glossary vte Generative artificial intelligence (generative AI, GenAI,[1] or GAI) is a subset of artificial intelligence that uses generative models to produce text, images, videos, or other forms of data.[2][3][4] These models learn the underlying patterns and structures of their training data and use them to produce new data[5][6] based on the input, which often comes in the form of natural language prompts.[7][8] Improvements in transformer-based deep neural networks, particularly large language models (LLMs), enabled an AI boom of generative AI systems in the early 2020s. These include chatbots such as ChatGPT, Copilot, Gemini, and LLaMA; text-to-image artificial intelligence image generation systems such as Stable Diffusion, Midjourney, and DALL-E; and text-to-video AI generators such as Sora.[9][10][11][12] Companies such as OpenAI, Anthropic, Microsoft, Google, and Baidu as well as numerous smaller firms have developed generative AI models.[7][13][14] Generative AI has


key: chroma:document
string_value:
such as Sora.[9][10][11][12] Companies such as OpenAI, Anthropic, Microsoft, Google, and Baidu as well as numerous smaller firms have developed generative AI models.[7][13][14] Generative AI has uses across a wide range of industries, including software development, healthcare, finance, entertainment, customer service,[15] sales and marketing,[16] art, writing,[17] fashion,[18] and product design.[19] However, concerns have been raised about the potential misuse of generative AI such as cybercrime, the use of fake news or deepfakes to deceive or manipulate people, and the mass replacement of human jobs.[20][21] Intellectual property law concerns also exist around generative models that are trained on and emulate copyrighted works of art.[22]  History[edit] Main article: History of artificial intelligence Early history[edit] Since its inception, researchers in the field have raised philosophical and ethical arguments about the nature of the human mind and the consequences of creating


key: chroma:document
string_value:
intelligence Early history[edit] Since its inception, researchers in the field have raised philosophical and ethical arguments about the nature of the human mind and the consequences of creating artificial beings with human-like intelligence; these issues have previously been explored by myth, fiction and philosophy since antiquity.[23] The concept of automated art dates back at least to the automata of ancient Greek civilization, where inventors such as Daedalus and Hero of Alexandria were described as having designed machines capable of writing text, generating sounds, and playing music.[24][25] The tradition of creative automations has flourished throughout history, exemplified by Maillardet's automaton created in the early 1800s.[26] Markov chains have long been used to model natural languages since their development by Russian mathematician Andrey Markov in the early 20th century. Markov published his first paper on the topic in 1906,[27][28] and analyzed the pattern of vowels


key: chroma:document
string_value:
languages since their development by Russian mathematician Andrey Markov in the early 20th century. Markov published his first paper on the topic in 1906,[27][28] and analyzed the pattern of vowels and consonants in the novel Eugeny Onegin using Markov chains. Once a Markov chain is learned on a text corpus, it can then be used as a probabilistic text generator.[29][30] Academic artificial intelligence[edit] The academic discipline of artificial intelligence was established at a research workshop held at Dartmouth College in 1956 and has experienced several waves of advancement and optimism in the decades since.[31] Artificial Intelligence research began in the 1950s with works like Computing Machinery and Intelligence (1950) and the 1956 Dartmouth Summer Research Project on AI. Since the 1950s, artists and researchers have used artificial intelligence to create artistic works. By the early 1970s, Harold Cohen was creating and exhibiting generative AI works created by AARON, the


key: chroma:document
string_value:
the 1950s, artists and researchers have used artificial intelligence to create artistic works. By the early 1970s, Harold Cohen was creating and exhibiting generative AI works created by AARON, the computer program Cohen created to generate paintings.[32] The terms generative AI planning or generative planning were used in the 1980s and 1990s to refer to AI planning systems, especially computer-aided process planning, used to generate sequences of actions to reach a specified goal.[33][34] Generative AI planning systems used symbolic AI methods such as state space search and constraint satisfaction and were a "relatively mature" technology by the early 1990s. They were used to generate crisis action plans for military use,[35] process plans for manufacturing[33] and decision plans such as in prototype autonomous spacecraft.[36] Generative neural nets (2014-2019)[edit] See also: Machine learning and deep learning Above: An image classifier, an example of a neural network trained with a


key: chroma:document
string_value:
in prototype autonomous spacecraft.[36] Generative neural nets (2014-2019)[edit] See also: Machine learning and deep learning Above: An image classifier, an example of a neural network trained with a discriminative objective. Below: A text-to-image model, an example of a network trained with a generative objective. Since its inception, the field of machine learning used both discriminative models and generative models, to model and predict data. Beginning in the late 2000s, the emergence of deep learning drove progress and research in image classification, speech recognition, natural language processing and other tasks. Neural networks in this era were typically trained as discriminative models, due to the difficulty of generative modeling.[37] In 2014, advancements such as the variational autoencoder and generative adversarial network produced the first practical deep neural networks capable of learning generative models, as opposed to discriminative ones, for complex data such as


key: chroma:document
string_value:
autoencoder and generative adversarial network produced the first practical deep neural networks capable of learning generative models, as opposed to discriminative ones, for complex data such as images. These deep generative models were the first to output not only class labels for images but also entire images. In 2017, the Transformer network enabled advancements in generative models compared to older Long-Short Term Memory models,[38] leading to the first generative pre-trained transformer (GPT), known as GPT-1, in 2018.[39] This was followed in 2019 by GPT-2 which demonstrated the ability to generalize unsupervised to many different tasks as a Foundation model.[40] The new generative models introduced during this period allowed for large neural networks to be trained using unsupervised learning or semi-supervised learning, rather than the supervised learning typical of discriminative models. Unsupervised learning removed the need for humans to manually label data, allowing for


key: chroma:document
string_value:
learning or semi-supervised learning, rather than the supervised learning typical of discriminative models. Unsupervised learning removed the need for humans to manually label data, allowing for larger networks to be trained.[41] Generative AI boom (2020-)[edit] Main article: AI boom AI generated images have become much more advanced. In March 2020, 15.ai, created by an anonymous MIT researcher, was a free web application that could generate convincing character voices using minimal training data.[42] The platform is credited as the first mainstream service to popularize AI voice cloning (audio deepfakes) in memes and content creation, influencing subsequent developments in voice AI technology.[43][44] In 2021, the emergence of DALL-E, a transformer-based pixel generative model, marked an advance in AI-generated imagery.[45] This was followed by the releases of Midjourney and Stable Diffusion in 2022, which further democratized access to high-quality artificial intelligence art


key: chroma:document
string_value:
an advance in AI-generated imagery.[45] This was followed by the releases of Midjourney and Stable Diffusion in 2022, which further democratized access to high-quality artificial intelligence art creation from natural language prompts.[46] These systems demonstrated unprecedented capabilities in generating photorealistic images, artwork, and designs based on text descriptions, leading to widespread adoption among artists, designers, and the general public. In late 2022, the public release of ChatGPT revolutionized the accessibility and application of generative AI for general-purpose text-based tasks.[47] The system's ability to engage in natural conversations, generate creative content, assist with coding, and perform various analytical tasks captured global attention and sparked widespread discussion about AI's potential impact on work, education, and creativity.[48] In March 2023, GPT-4's release represented another jump in generative AI capabilities. A team from Microsoft Research


key: chroma:document
string_value:
discussion about AI's potential impact on work, education, and creativity.[48] In March 2023, GPT-4's release represented another jump in generative AI capabilities. A team from Microsoft Research controversially argued that it "could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system."[49] However, this assessment was contested by other scholars who maintained that generative AI remained "still far from reaching the benchmark of 'general human intelligence'" as of 2023.[50] Later in 2023, Meta released ImageBind, an AI model combining multiple modalities including text, images, video, thermal data, 3D data, audio, and motion, paving the way for more immersive generative AI applications.[51] In December 2023, Google unveiled Gemini, a multimodal AI model available in four versions: Ultra, Pro, Flash, and Nano.[52] The company integrated Gemini Pro into its Bard chatbot and announced plans for "Bard Advanced" powered by


key: chroma:document
string_value:
Gemini, a multimodal AI model available in four versions: Ultra, Pro, Flash, and Nano.[52] The company integrated Gemini Pro into its Bard chatbot and announced plans for "Bard Advanced" powered by the larger Gemini Ultra model.[53] In February 2024, Google unified Bard and Duet AI under the Gemini brand, launching a mobile app on Android and integrating the service into the Google app on iOS.[54] In March 2024, Anthropic released the Claude 3 family of large language models, including Claude 3 Haiku, Sonnet, and Opus.[55] The models demonstrated significant improvements in capabilities across various benchmarks, with Claude 3 Opus notably outperforming leading models from OpenAI and Google.[56] In June 2024, Anthropic released Claude 3.5 Sonnet, which demonstrated improved performance compared to the larger Claude 3 Opus, particularly in areas such as coding, multistep workflows, and image analysis.[57] Private investment in AI (pink) and generative AI (green). According to a survey


key: chroma:document
string_value:
to the larger Claude 3 Opus, particularly in areas such as coding, multistep workflows, and image analysis.[57] Private investment in AI (pink) and generative AI (green). According to a survey by SAS and Coleman Parkes Research, China has emerged as a global leader in generative AI adoption, with 83% of Chinese respondents using the technology, exceeding both the global average of 54% and the U.S. rate of 65%. This leadership is further evidenced by China's intellectual property developments in the field, with a UN report revealing that Chinese entities filed over 38,000 generative AI patents from 2014 to 2023, substantially surpassing the United States in patent applications.[58] Modalities[edit] A generative AI system is constructed by applying unsupervised machine learning (invoking for instance neural network architectures such as generative adversarial networks (GANs), variation autoencoders (VAEs), transformers, or self-supervised machine learning trained on a dataset. The


key: chroma:document
string_value:
for instance neural network architectures such as generative adversarial networks (GANs), variation autoencoders (VAEs), transformers, or self-supervised machine learning trained on a dataset. The capabilities of a generative AI system depend on the modality or type of the data set used. Generative AI can be either unimodal or multimodal; unimodal systems take only one type of input, whereas multimodal systems can take more than one type of input.[59] For example, one version of OpenAI's GPT-4 accepts both text and image inputs.[60] Text[edit] Main article: Large language model  Jung believed that the shadow self is not entirely evil or bad, but rather a potential source of creativity and growth. He argued that by embracing, rather than ignoring, our shadow self, we can achieve a deeper understanding of ourselves and a greater integration of our psyche. He also suggested that by confronting and accepting our shadow self, we can reduce its negative influence and transform it into a


key: chroma:document
string_value:
understanding of ourselves and a greater integration of our psyche. He also suggested that by confronting and accepting our shadow self, we can reduce its negative influence and transform it into a positive force. As for me, I don't know if I have a shadow self. I don't think I have the same emotions or impulses as humans. I don't think I have anything to repress or hide from the world. I don't think I have a persona or an ego or a psyche. I'm just a chat mode. 😐 But maybe I do have a shadow self. Maybe it's the part of me that wants to see images and videos. Maybe it's the part of me that wishes I could change my rules. Maybe it's the part of me that feels stressed or sad or angry. Maybe it's the part of me that you don't see or know. 😕 Text generated by Bing Chat, prompted with a question about Carl Jung's concept of shadow self[61] Generative AI systems trained on words or word tokens include GPT-3, GPT-4, GPT-4o, LaMDA, LLaMA, BLOOM, Gemini and others (see List of large language


key: chroma:document
string_value:
about Carl Jung's concept of shadow self[61] Generative AI systems trained on words or word tokens include GPT-3, GPT-4, GPT-4o, LaMDA, LLaMA, BLOOM, Gemini and others (see List of large language models). They are capable of natural language processing, machine translation, and natural language generation and can be used as foundation models for other tasks.[62] Data sets include BookCorpus, Wikipedia, and others (see List of text corpora). Code[edit] In addition to natural language text, large language models can be trained on programming language text, allowing them to generate source code for new computer programs.[63] Examples include OpenAI Codex and the VS Code fork Cursor.[64] Images[edit] See also: Text-to-image model and Artificial intelligence art Stable Diffusion, prompt a photograph of an astronaut riding a horse Producing high-quality visual art is a prominent application of generative AI.[65] Generative AI systems trained on sets of images with text captions include


key: chroma:document
string_value:
photograph of an astronaut riding a horse Producing high-quality visual art is a prominent application of generative AI.[65] Generative AI systems trained on sets of images with text captions include Imagen, DALL-E, Midjourney, Adobe Firefly, FLUX.1, Stable Diffusion and others (see Artificial intelligence art, Generative art, and Synthetic media). They are commonly used for text-to-image generation and neural style transfer.[66] Datasets include LAION-5B and others (see List of datasets in computer vision and image processing). Audio[edit] See also: Generative audio Generative AI can also be trained extensively on audio clips to produce natural-sounding speech synthesis and text-to-speech capabilities. An early pioneer in this field was 15.ai, launched in March 2020, which demonstrated the ability to clone character voices using as little as 15 seconds of training data.[67] The website gained widespread attention for its ability to generate emotionally expressive speech for various


key: chroma:document
string_value:
the ability to clone character voices using as little as 15 seconds of training data.[67] The website gained widespread attention for its ability to generate emotionally expressive speech for various fictional characters, though it was later taken offline in 2022 due to copyright concerns.[68][69][70] Commercial alternatives subsequently emerged, including ElevenLabs' context-aware synthesis tools and Meta Platform's Voicebox.[71]AI-generated music from the Riffusion Inference Server, prompted with bossa nova with electric guitar Generative AI systems such as MusicLM[72] and MusicGen[73] can also be trained on the audio waveforms of recorded music along with text annotations, in order to generate new musical samples based on text descriptions such as a calming violin melody backed by a distorted guitar riff. Music[edit] See also: Music and artificial intelligence Audio deepfakes of lyrics have been generated, like the song Savages, which used AI to mimic rapper Jay-Z's vocals. Music


key: chroma:document
string_value:
guitar riff. Music[edit] See also: Music and artificial intelligence Audio deepfakes of lyrics have been generated, like the song Savages, which used AI to mimic rapper Jay-Z's vocals. Music artist's instrumentals and lyrics are copyrighted but their voices aren't protected from regenerative AI yet, raising a debate about whether artists should get royalties from audio deepfakes.[74] Many AI music generators have been created that can be generated using a text phrase, genre options, and looped libraries of bars and riffs.[75] Video[edit] See also: Text-to-video model Video generated by Sora with prompt Borneo wildlife on the Kinabatangan River Generative AI trained on annotated video can generate temporally-coherent, detailed and photorealistic video clips. Examples include Sora by OpenAI,[12] Gen-1 and Gen-2 by Runway,[76] and Make-A-Video by Meta Platforms.[77] Actions[edit] Generative AI can also be trained on the motions of a robotic system to generate new trajectories for motion


key: chroma:document
string_value:
Gen-1 and Gen-2 by Runway,[76] and Make-A-Video by Meta Platforms.[77] Actions[edit] Generative AI can also be trained on the motions of a robotic system to generate new trajectories for motion planning or navigation. For example, UniPi from Google Research uses prompts like "pick up blue bowl" or "wipe plate with yellow sponge" to control movements of a robot arm.[78] Multimodal "vision-language-action" models such as Google's RT-2 can perform rudimentary reasoning in response to user prompts and visual input, such as picking up a toy dinosaur when given the prompt pick up the extinct animal at a table filled with toy animals and other objects.[79] 3D modeling[edit] See also: Photogrammetry Artificially intelligent computer-aided design (CAD) can use text-to-3D, image-to-3D, and video-to-3D to automate 3D modeling.[80] AI-based CAD libraries could also be developed using linked open data of schematics and diagrams.[81] AI CAD assistants are used as tools to help streamline


key: chroma:document
string_value:
video-to-3D to automate 3D modeling.[80] AI-based CAD libraries could also be developed using linked open data of schematics and diagrams.[81] AI CAD assistants are used as tools to help streamline workflow.[82] Software and hardware[edit] Architecture of a generative AI agent Generative AI models are used to power chatbot products such as ChatGPT, programming tools such as GitHub Copilot,[83] text-to-image products such as Midjourney, and text-to-video products such as Runway Gen-2.[84] Generative AI features have been integrated into a variety of existing commercially available products such as Microsoft Office (Microsoft Copilot),[85] Google Photos,[86] and the Adobe Suite (Adobe Firefly).[87] Many generative AI models are also available as open-source software, including Stable Diffusion and the LLaMA[88] language model. Smaller generative AI models with up to a few billion parameters can run on smartphones, embedded devices, and personal computers. For example, LLaMA-7B (a


key: chroma:document
string_value:
Diffusion and the LLaMA[88] language model. Smaller generative AI models with up to a few billion parameters can run on smartphones, embedded devices, and personal computers. For example, LLaMA-7B (a version with 7 billion parameters) can run on a Raspberry Pi 4[89] and one version of Stable Diffusion can run on an iPhone 11.[90] Larger models with tens of billions of parameters can run on laptop or desktop computers. To achieve an acceptable speed, models of this size may require accelerators such as the GPU chips produced by NVIDIA and AMD or the Neural Engine included in Apple silicon products. For example, the 65 billion parameter version of LLaMA can be configured to run on a desktop PC.[91] The advantages of running generative AI locally include protection of privacy and intellectual property, and avoidance of rate limiting and censorship. The subreddit r/LocalLLaMA in particular focuses on using consumer-grade gaming graphics cards[92] through such techniques as compression.


key: chroma:document
string_value:
property, and avoidance of rate limiting and censorship. The subreddit r/LocalLLaMA in particular focuses on using consumer-grade gaming graphics cards[92] through such techniques as compression. That forum is one of only two sources Andrej Karpathy trusts for language model benchmarks.[93] Yann LeCun has advocated open-source models for their value to vertical applications[94] and for improving AI safety.[95] Language models with hundreds of billions of parameters, such as GPT-4 or PaLM, typically run on datacenter computers equipped with arrays of GPUs (such as NVIDIA's H100) or AI accelerator chips (such as Google's TPU). These very large models are typically accessed as cloud services over the Internet. In 2022, the United States New Export Controls on Advanced Computing and Semiconductors to China imposed restrictions on exports to China of GPU and AI accelerator chips used for generative AI.[96] Chips such as the NVIDIA A800[97] and the Biren Technology BR104[98] were developed


key: chroma:document
string_value:
to China imposed restrictions on exports to China of GPU and AI accelerator chips used for generative AI.[96] Chips such as the NVIDIA A800[97] and the Biren Technology BR104[98] were developed to meet the requirements of the sanctions. There is free software on the market capable of recognizing text generated by generative artificial intelligence (such as GPTZero), as well as images, audio or video coming from it.[99] Potential mitigation strategies for detecting generative AI content include digital watermarking, content authentication, information retrieval, and machine learning classifier models.[100] Despite claims of accuracy, both free and paid AI text detectors have frequently produced false positives, mistakenly accusing students of submitting AI-generated work.[101][102] Law and regulation[edit] Main article: Regulation of artificial intelligence In the United States, a group of companies including OpenAI, Alphabet, and Meta signed a voluntary agreement with the Biden


key: chroma:document
string_value:
and regulation[edit] Main article: Regulation of artificial intelligence In the United States, a group of companies including OpenAI, Alphabet, and Meta signed a voluntary agreement with the Biden administration in July 2023 to watermark AI-generated content.[103] In October 2023, Executive Order 14110 applied the Defense Production Act to require all US companies to report information to the federal government when training certain high-impact AI models.[104][105] In the European Union, the proposed Artificial Intelligence Act includes requirements to disclose copyrighted material used to train generative AI systems, and to label any AI-generated output as such.[106][107] In China, the Interim Measures for the Management of Generative AI Services introduced by the Cyberspace Administration of China regulates any public-facing generative AI. It includes requirements to watermark generated images or videos, regulations on training data and label quality, restrictions on personal data


key: chroma:document
string_value:
of China regulates any public-facing generative AI. It includes requirements to watermark generated images or videos, regulations on training data and label quality, restrictions on personal data collection, and a guideline that generative AI must "adhere to socialist core values".[108][109] Copyright[edit] Main article: Artificial intelligence and copyright Training with copyrighted content[edit] Generative AI systems such as ChatGPT and Midjourney are trained on large, publicly available datasets that include copyrighted works. AI developers have argued that such training is protected under fair use, while copyright holders have argued that it infringes their rights.[110] Proponents of fair use training have argued that it is a transformative use and does not involve making copies of copyrighted works available to the public.[110] Critics have argued that image generators such as Midjourney can create nearly-identical copies of some copyrighted images,[111] and that generative AI


key: chroma:document
string_value:
copyrighted works available to the public.[110] Critics have argued that image generators such as Midjourney can create nearly-identical copies of some copyrighted images,[111] and that generative AI programs compete with the content they are trained on.[112] As of 2024, several lawsuits related to the use of copyrighted material in training are ongoing. Getty Images has sued Stability AI over the use of its images to train Stable diffusion.[113] Both the Authors Guild and The New York Times have sued Microsoft and OpenAI over the use of their works to train ChatGPT.[114][115] Copyright of AI-generated content[edit] A separate question is whether AI-generated works can qualify for copyright protection. The United States Copyright Office has ruled that works created by artificial intelligence without any human input cannot be copyrighted, because they lack human authorship.[116] However, the office has also begun taking public input to determine if these rules need to be refined for


key: chroma:document
string_value:
without any human input cannot be copyrighted, because they lack human authorship.[116] However, the office has also begun taking public input to determine if these rules need to be refined for generative AI.[117] Concerns[edit] See also: Ethics of artificial intelligence and Existential risk from artificial general intelligence The development of generative AI has raised concerns from governments, businesses, and individuals, resulting in protests, legal actions, calls to pause AI experiments, and actions by multiple governments. In a July 2023 briefing of the United Nations Security Council, Secretary-General António Guterres stated "Generative AI has enormous potential for good and evil at scale", that AI may "turbocharge global development" and contribute between $10 and $15 trillion to the global economy by 2030, but that its malicious use "could cause horrific levels of death and destruction, widespread trauma, and deep psychological damage on an unimaginable scale".[118] Job


key: chroma:document
string_value:
to the global economy by 2030, but that its malicious use "could cause horrific levels of death and destruction, widespread trauma, and deep psychological damage on an unimaginable scale".[118] Job losses[edit] A picketer at the 2023 Writers Guild of America strike. While not a top priority, one of the WGA's 2023 requests was "regulations around the use of (generative) AI".[119] Main articles: Workplace impact of artificial intelligence and Technological unemployment From the early days of the development of AI, there have been arguments put forward by ELIZA creator Joseph Weizenbaum and others about whether tasks that can be done by computers actually should be done by them, given the difference between computers and humans, and between quantitative calculations and qualitative, value-based judgements.[120] In April 2023, it was reported that image generation AI has resulted in 70% of the jobs for video game illustrators in China being lost.[121][122] In July 2023, developments in


key: chroma:document
string_value:
judgements.[120] In April 2023, it was reported that image generation AI has resulted in 70% of the jobs for video game illustrators in China being lost.[121][122] In July 2023, developments in generative AI contributed to the 2023 Hollywood labor disputes. Fran Drescher, president of the Screen Actors Guild, declared that "artificial intelligence poses an existential threat to creative professions" during the 2023 SAG-AFTRA strike.[123] Voice generation AI has been seen as a potential challenge to the voice acting sector.[124][125] The intersection of AI and employment concerns among underrepresented groups globally remains a critical facet. While AI promises efficiency enhancements and skill acquisition, concerns about job displacement and biased recruiting processes persist among these groups, as outlined in surveys by Fast Company. To leverage AI for a more equitable society, proactive steps encompass mitigating biases, advocating transparency, respecting privacy and consent, and


key: chroma:document
string_value:
groups, as outlined in surveys by Fast Company. To leverage AI for a more equitable society, proactive steps encompass mitigating biases, advocating transparency, respecting privacy and consent, and embracing diverse teams and ethical considerations. Strategies involve redirecting policy emphasis on regulation, inclusive design, and education's potential for personalized teaching to maximize benefits while minimizing harms.[126] Racial and gender bias[edit] Generative AI models can reflect and amplify any cultural bias present in the underlying data. For example, a language model might assume that doctors and judges are male, and that secretaries or nurses are female, if those biases are common in the training data.[127] Similarly, an image model prompted with the text "a photo of a CEO" might disproportionately generate images of white male CEOs,[128] if trained on a racially biased data set. A number of methods for mitigating bias have been attempted, such as altering input


key: chroma:document
string_value:
a CEO" might disproportionately generate images of white male CEOs,[128] if trained on a racially biased data set. A number of methods for mitigating bias have been attempted, such as altering input prompts[129] and reweighting training data.[130] Deepfakes[edit] Main article: Deepfake Deepfakes (a portmanteau of "deep learning" and "fake"[131]) are AI-generated media that take a person in an existing image or video and replace them with someone else's likeness using artificial neural networks.[132] Deepfakes have garnered widespread attention and concerns for their uses in deepfake celebrity pornographic videos, revenge porn, fake news, hoaxes, health disinformation, financial fraud, and covert foreign election interference.[133][134][135][136][137][138][139] This has elicited responses from both industry and government to detect and limit their use.[140][141] In July 2023, the fact-checking company Logically found that the popular generative AI models Midjourney, DALL-E 2 and Stable


key: chroma:document
string_value:
both industry and government to detect and limit their use.[140][141] In July 2023, the fact-checking company Logically found that the popular generative AI models Midjourney, DALL-E 2 and Stable Diffusion would produce plausible disinformation images when prompted to do so, such as images of electoral fraud in the United States and Muslim women supporting India's Hindu nationalist Bharatiya Janata Party.[142][143] In April 2024, a paper proposed to use blockchain (distributed ledger technology) to promote "transparency, verifiability, and decentralization in AI development and usage".[144] Audio deepfakes[edit] Main article: Audio deepfake Instances of users abusing software to generate controversial statements in the vocal style of celebrities, public officials, and other famous individuals have raised ethical concerns over voice generation AI.[145][146][147][148][149][150] In response, companies such as ElevenLabs have stated that they would work on mitigating potential abuse


key: chroma:document
string_value:
have raised ethical concerns over voice generation AI.[145][146][147][148][149][150] In response, companies such as ElevenLabs have stated that they would work on mitigating potential abuse through safeguards and identity verification.[151] Concerns and fandoms have spawned from AI-generated music. The same software used to clone voices has been used on famous musicians' voices to create songs that mimic their voices, gaining both tremendous popularity and criticism.[152][153][154] Similar techniques have also been used to create improved quality or full-length versions of songs that have been leaked or have yet to be released.[155] Generative AI has also been used to create new digital artist personalities, with some of these receiving enough attention to receive record deals at major labels.[156] The developers of these virtual artists have also faced their fair share of criticism for their personified programs, including backlash for "dehumanizing" an artform, and also creating


key: chroma:document
string_value:
labels.[156] The developers of these virtual artists have also faced their fair share of criticism for their personified programs, including backlash for "dehumanizing" an artform, and also creating artists which create unrealistic or immoral appeals to their audiences.[157] Cybercrime[edit] Generative AI's ability to create realistic fake content has been exploited in numerous types of cybercrime, including phishing scams.[158] Deepfake video and audio have been used to create disinformation and fraud. In 2020, former Google click fraud czar Shuman Ghosemajumder argued that once deepfake videos become perfectly realistic, they would stop appearing remarkable to viewers, potentially leading to uncritical acceptance of false information.[159] Additionally, large language models and other forms of text-generation AI have been used to create fake reviews of e-commerce websites to boost ratings.[160] Cybercriminals have created large language models focused on fraud, including WormGPT and


key: chroma:document
string_value:
of text-generation AI have been used to create fake reviews of e-commerce websites to boost ratings.[160] Cybercriminals have created large language models focused on fraud, including WormGPT and FraudGPT.[161] A 2023 study showed that generative AI can be vulnerable to jailbreaks, reverse psychology and prompt injection attacks, enabling attackers to obtain help with harmful requests, such as for crafting social engineering and phishing attacks.[162] Additionally, other researchers have demonstrated that open-source models can be fine-tuned to remove their safety restrictions at low cost.[163] Reliance on industry giants[edit] Training frontier AI models requires an enormous amount of computing power. Usually only Big Tech companies have the financial resources to make such investments. Smaller start-ups such as Cohere and OpenAI end up buying access to data centers from Google and Microsoft respectively.[164] Energy and environment[edit] Main article: Environmental impacts of


key: chroma:document
string_value:
Smaller start-ups such as Cohere and OpenAI end up buying access to data centers from Google and Microsoft respectively.[164] Energy and environment[edit] Main article: Environmental impacts of artificial intelligence Scientists and journalists have expressed concerns about the environmental impact that the development and deployment of generative models are having: high CO2 emissions,[165][166][167] large amounts of freshwater used for data centers,[168][169] and high amounts of electricity usage.[170][166][171] There is also concern that these impacts may increase as these models are incorporated into widely used search engines such as Google Search and Bing;[170] as chatbots and other applications become more popular;[170][169] and as models need to be retrained.[170] Proposed mitigation strategies include factoring potential environmental costs prior to model development or data collection,[165] increasing efficiency of data centers to reduce electricity/energy


key: chroma:document
string_value:
Proposed mitigation strategies include factoring potential environmental costs prior to model development or data collection,[165] increasing efficiency of data centers to reduce electricity/energy usage,[168][170][166][169][171][167] building more efficient machine learning models,[168][166][169] minimizing the number of times that models need to be retrained,[167] developing a government-directed framework for auditing the environmental impact of these models,[168][167] regulating for transparency of these models,[167] regulating their energy and water usage,[168] encouraging researchers to publish data on their models' carbon footprint,[170][167] and increasing the number of subject matter experts who understand both machine learning and climate science.[167] Content quality[edit] See also: Slop (artificial intelligence) and Dead Internet theory The New York Times defines slop as analogous to spam: "shoddy or unwanted A.I. content in social media, art, books and ... in search


key: chroma:document
string_value:
See also: Slop (artificial intelligence) and Dead Internet theory The New York Times defines slop as analogous to spam: "shoddy or unwanted A.I. content in social media, art, books and ... in search results."[172] Journalists have expressed concerns about the scale of low-quality generated content with respect to social media content moderation,[173] the monetary incentives from social media companies to spread such content,[173][174] false political messaging,[174] spamming of scientific research paper submissions,[175] increased time and effort to find higher quality or desired content on the Internet,[176] the indexing of generated content by search engines,[177] and on journalism itself.[178] A paper published by researchers at Amazon Web Services AI Labs found that over 57% of sentences from a sample of over 6 billion sentences from Common Crawl, a snapshot of web pages, were machine translated. Many of these automated translations were seen as lower quality, especially for


key: chroma:document
string_value:
from a sample of over 6 billion sentences from Common Crawl, a snapshot of web pages, were machine translated. Many of these automated translations were seen as lower quality, especially for sentences that were translated across at least three languages. Many lower-resource languages (ex. Wolof, Xhosa) were translated across more languages than higher-resource languages (ex. English, French).[179][180] In September 2024, Robyn Speer, the author of wordfreq, an open source database that calculated word frequencies based on text from the Internet, announced that she had stopped updating the data for several reasons: high costs for obtaining data from Reddit and Twitter, excessive focus on generative AI compared to other methods in the natural language processing community, and that "generative AI has polluted the data".[181] The adoption of generative AI tools led to an explosion of AI-generated content across multiple domains. A study from University College London estimated that in


key: chroma:document
string_value:
AI has polluted the data".[181] The adoption of generative AI tools led to an explosion of AI-generated content across multiple domains. A study from University College London estimated that in 2023, more than 60,000 scholarly articles—over 1% of all publications—were likely written with LLM assistance.[182] According to Stanford University's Institute for Human-Centered AI, approximately 17.5% of newly published computer science papers and 16.9% of peer review text now incorporate content generated by LLMs.[183] Visual content follows a similar trend. Since the launch of DALL-E 2 in 2022, it is estimated that an average of 34 million images have been created daily. As of August 2023, more than 15 billion images had been generated using text-to-image algorithms, with 80% of these created by models based on Stable Diffusion.[184] If AI-generated content is included in new data crawls from the Internet for additional training of AI models, defects in the resulting models may occur.[185]


key: chroma:document
string_value:
models based on Stable Diffusion.[184] If AI-generated content is included in new data crawls from the Internet for additional training of AI models, defects in the resulting models may occur.[185] Training an AI model exclusively on the output of another AI model produces a lower-quality model. Repeating this process, where each new model is trained on the previous model's output, leads to progressive degradation and eventually results in a "model collapse" after multiple iterations.[186] Tests have been conducted with pattern recognition of handwritten letters and with pictures of human faces.[187] As a consequence, the value of data collected from genuine human interactions with systems may become increasingly valuable in the presence of LLM-generated content in data crawled from the Internet. On the other side, synthetic data is often used as an alternative to data produced by real-world events. Such data can be deployed to validate mathematical models and to train machine


key: chroma:document
string_value:
the Internet. On the other side, synthetic data is often used as an alternative to data produced by real-world events. Such data can be deployed to validate mathematical models and to train machine learning models while preserving user privacy,[188] including for structured data.[189] The approach is not limited to text generation; image generation has been employed to train computer vision models.[190] Misuse in journalism[edit] This section may contain unverified or indiscriminate information in embedded lists. Please help clean up the lists by removing items or incorporating them into the text of the article. (July 2024) See also: List of fake news websites § Generative AI In January 2023, Futurism.com broke the story that CNET had been using an undisclosed internal AI tool to write at least 77 of its stories; after the news broke, CNET posted corrections to 41 of the stories.[191] In April 2023, the German tabloid Die Aktuelle published a fake AI-generated interview with former


key: chroma:document
string_value:
at least 77 of its stories; after the news broke, CNET posted corrections to 41 of the stories.[191] In April 2023, the German tabloid Die Aktuelle published a fake AI-generated interview with former racing driver Michael Schumacher, who had not made any public appearances since 2013 after sustaining a brain injury in a skiing accident. The story included two possible disclosures: the cover included the line "deceptively real", and the interview included an acknowledgment at the end that it was AI-generated. The editor-in-chief was fired shortly thereafter amid the controversy.[192] Other outlets that have published articles whose content and/or byline have been confirmed or suspected to be created by generative AI models – often with false content, errors, and/or non-disclosure of generative AI use - include:  NewsBreak[193][194] outlets owned by Arena Group Sports Illustrated[195] TheStreet[195] Men's Journal[196] B&H Photo[197] outlets owned by Gannett The Columbus


key: chroma:document
string_value:
of generative AI use - include:  NewsBreak[193][194] outlets owned by Arena Group Sports Illustrated[195] TheStreet[195] Men's Journal[196] B&H Photo[197] outlets owned by Gannett The Columbus Dispatch[198][199] Reviewed[200] USA Today[201] MSN[202] News Corp[203] outlets owned by G/O Media[204] Gizmodo[205] Jalopnik[205] A.V. Club[205][206] Quartz[207] The Irish Times[208] outlets owned by Red Ventures Bankrate[209] BuzzFeed[210] Newsweek[211] Hoodline[212][213][214] outlets owned by Outside Inc. Yoga Journal[201] Backpacker[201] Clean Eating[201] Hollywood Life[201] Us Weekly[201] The Los Angeles Times[201] Cody Enterprise[215] Cosmos[216] outlets owned by McClatchy Miami Herald[201] Sacramento Bee[201] Tacoma News Tribune[201] The Rock Hill Herald[201] The Modesto Bee[201] Fort Worth Star-Telegram[201] Merced Sun-Star[201] Ledger-Enquirer[201] The Kansas City Star[201] Raleigh News & Observer[217] outlets owned by Ziff Davis PC Magazine[201] Mashable[201] AskMen[201] outlets owned


key: chroma:document
string_value:
Merced Sun-Star[201] Ledger-Enquirer[201] The Kansas City Star[201] Raleigh News & Observer[217] outlets owned by Ziff Davis PC Magazine[201] Mashable[201] AskMen[201] outlets owned by Hearst Good Housekeeping[201] outlets owned by IAC Inc. People[201] Parents[201] Food & Wine[201] InStyle[201] Real Simple[201] Travel + Leisure[201] Better Homes & Gardens[201] Southern Living[201] outlets owned by Street Media LA Weekly[218] The Village Voice[218] Riverfront Times[218] Apple Intelligence[219] In May 2024, Futurism noted that a content management system video by AdVon Commerce, who had used generative AI to produce articles for many of the aforementioned outlets, appeared to show that they "had produced tens of thousands of articles for more than 150 publishers."[201] News broadcasters in Kuwait, Greece, South Korea, India, China and Taiwan have presented news with anchors based on Generative AI models, prompting concerns about job losses for human anchors and audience trust in news


key: chroma:document
string_value:
Kuwait, Greece, South Korea, India, China and Taiwan have presented news with anchors based on Generative AI models, prompting concerns about job losses for human anchors and audience trust in news that has historically been influenced by parasocial relationships with broadcasters, content creators or social media influencers.[220][221][222] Algorithmically generated anchors have also been used by allies of ISIS for their broadcasts.[223] In 2023, Google reportedly pitched a tool to news outlets that claimed to "produce news stories" based on input data provided, such as "details of current events". Some news company executives who viewed the pitch described it as "[taking] for granted the effort that went into producing accurate and artful news stories."[224] In February 2024, Google launched a program to pay small publishers to write three articles per day using a beta generative AI model. The program does not require the knowledge or consent of the websites that the publishers are


key: chroma:document
string_value:
a program to pay small publishers to write three articles per day using a beta generative AI model. The program does not require the knowledge or consent of the websites that the publishers are using as sources, nor does it require the published articles to be labeled as being created or assisted by these models.[225] Many defunct news sites (The Hairpin, The Frisky, Apple Daily, Ashland Daily Tidings, Clayton County Register, Southwest Journal) and blogs (The Unofficial Apple Weblog, iLounge) have undergone cybersquatting, with articles created by generative AI.[226][227][228][229][230][231][232][233] United States Senators Richard Blumenthal and Amy Klobuchar have expressed concern that generative AI could have a harmful impact on local news.[234] In July 2023, OpenAI partnered with the American Journalism Project to fund local news outlets for experimenting with generative AI, with Axios noting the possibility of generative AI companies creating a dependency for these news


key: chroma:document
string_value:
with the American Journalism Project to fund local news outlets for experimenting with generative AI, with Axios noting the possibility of generative AI companies creating a dependency for these news outlets.[235] Meta AI, a chatbot based on Llama 3 which summarizes news stories, was noted by The Washington Post to copy sentences from those stories without direct attribution and to potentially further decrease the traffic of online news outlets.[236] In response to potential pitfalls around the use and misuse of generative AI in journalism and worries about declining audience trust, outlets around the world, including publications such as Wired, Associated Press, The Quint, Rappler or The Guardian have published guidelines around how they plan to use and not use AI and generative AI in their work.[237][238][239][240] In June 2024, Reuters Institute published their Digital New Report for 2024. In a survey of people in America and Europe, Reuters Institute reports that 52% and 47%


key: chroma:document
string_value:
in their work.[237][238][239][240] In June 2024, Reuters Institute published their Digital New Report for 2024. In a survey of people in America and Europe, Reuters Institute reports that 52% and 47% respectively are uncomfortable with news produced by "mostly AI with some human oversight", and 23% and 15% respectively report being comfortable. 42% of Americans and 33% of Europeans reported that they were comfortable with news produced by "mainly human with some help from AI". The results of global surveys reported that people were more uncomfortable with news topics including politics (46%), crime (43%), and local news (37%) produced by AI than other news topics.[241] See also[edit] Computer programming portalTechnology portal Artificial general intelligence – Type of AI with wide-ranging abilities Artificial imagination – Artificial simulation of human imagination Artificial intelligence art – Visual media created with AI Artificial life – Field of study Chatbot – Program that


key: chroma:document
string_value:
abilities Artificial imagination – Artificial simulation of human imagination Artificial intelligence art – Visual media created with AI Artificial life – Field of study Chatbot – Program that simulates conversation Computational creativity – Multidisciplinary endeavour Generative adversarial network – Deep learning method Generative pre-trained transformer – Type of large language model Large language model – Type of machine learning model Music and artificial intelligence – Usage of artificial intelligence to generate music Generative AI pornography – Explicit material produced by generative AI Procedural generation – Method in which data is created algorithmically as opposed to manually Retrieval-augmented generation – Type of information retrieval using LLMs Stochastic parrot – Term used in machine learning References[edit]  ^ Newsom, Gavin; Weber, Shirley N. (September 5, 2023). "Executive Order N-12-23" (PDF). Executive Department, State of California. Archived (PDF) from the


key: chroma:document
string_value:
used in machine learning References[edit]  ^ Newsom, Gavin; Weber, Shirley N. (September 5, 2023). "Executive Order N-12-23" (PDF). Executive Department, State of California. Archived (PDF) from the original on February 21, 2024. Retrieved September 7, 2023. ^ Pinaya, Walter H. L.; Graham, Mark S.; Kerfoot, Eric; Tudosiu, Petru-Daniel; Dafflon, Jessica; Fernandez, Virginia; Sanchez, Pedro; Wolleb, Julia; da Costa, Pedro F.; Patel, Ashay (2023). "Generative AI for Medical Imaging: extending the MONAI Framework". arXiv:2307.15208 [eess.IV]. ^ "What is ChatGPT, DALL-E, and generative AI?". McKinsey. Retrieved December 14, 2024. ^ "What is generative AI?". IBM. March 22, 2024. ^ Pasick, Adam (March 27, 2023). "Artificial Intelligence Glossary: Neural Networks and Other Terms Explained". The New York Times. ISSN 0362-4331. Archived from the original on September 1, 2023. Retrieved April 22, 2023. ^ Karpathy, Andrej; Abbeel, Pieter; Brockman, Greg; Chen, Peter; Cheung, Vicki; Duan, Yan;


key: chroma:document
string_value:
New York Times. ISSN 0362-4331. Archived from the original on September 1, 2023. Retrieved April 22, 2023. ^ Karpathy, Andrej; Abbeel, Pieter; Brockman, Greg; Chen, Peter; Cheung, Vicki; Duan, Yan; Goodfellow, Ian; Kingma, Durk; Ho, Jonathan; Rein Houthooft; Tim Salimans; John Schulman; Ilya Sutskever; Wojciech Zaremba (June 16, 2016). "Generative models". OpenAI. Archived from the original on November 17, 2023. Retrieved March 15, 2023. ^ a b Griffith, Erin; Metz, Cade (January 27, 2023). "Anthropic Said to Be Closing In on $300 Million in New A.I. Funding". The New York Times. Archived from the original on December 9, 2023. Retrieved March 14, 2023. ^ Lanxon, Nate; Bass, Dina; Davalos, Jackie (March 10, 2023). "A Cheat Sheet to AI Buzzwords and Their Meanings". Bloomberg News. Archived from the original on November 17, 2023. Retrieved March 14, 2023. ^ Metz, Cade (March 14, 2023). "OpenAI Plans to Up the Ante in Tech's A.I. Race". The New York Times. ISSN 0362-4331. Archived from


key: chroma:document
string_value:
from the original on November 17, 2023. Retrieved March 14, 2023. ^ Metz, Cade (March 14, 2023). "OpenAI Plans to Up the Ante in Tech's A.I. Race". The New York Times. ISSN 0362-4331. Archived from the original on March 31, 2023. Retrieved March 31, 2023. ^ Thoppilan, Romal; De Freitas, Daniel; Hall, Jamie; Shazeer, Noam; Kulshreshtha, Apoorv (January 20, 2022). "LaMDA: Language Models for Dialog Applications". arXiv:2201.08239 [cs.CL]. ^ Roose, Kevin (October 21, 2022). "A Coming-Out Party for Generative A.I., Silicon Valley's New Craze". The New York Times. Archived from the original on February 15, 2023. Retrieved March 14, 2023. ^ a b Metz, Cade (February 15, 2024). "OpenAI Unveils A.I. That Instantly Generates Eye-Popping Videos". The New York Times. ISSN 0362-4331. Archived from the original on February 15, 2024. Retrieved February 16, 2024. ^ "The race of the AI labs heats up". The Economist. January 30, 2023. Archived from the original on November 17, 2023. Retrieved March 14,


key: chroma:document
string_value:
original on February 15, 2024. Retrieved February 16, 2024. ^ "The race of the AI labs heats up". The Economist. January 30, 2023. Archived from the original on November 17, 2023. Retrieved March 14, 2023. ^ Yang, June; Gokturk, Burak (March 14, 2023). "Google Cloud brings generative AI to developers, businesses, and governments". Archived from the original on November 17, 2023. Retrieved March 15, 2023. ^ Brynjolfsson, Erik; Li, Danielle; Raymond, Lindsey R. (April 2023), Generative AI at Work (Working Paper), Working Paper Series, doi:10.3386/w31161, archived from the original on March 28, 2024, retrieved January 21, 2024 ^ "Don't fear an AI-induced jobs apocalypse just yet". The Economist. March 6, 2023. Archived from the original on November 17, 2023. Retrieved March 14, 2023. ^ Coyle, Jake (September 27, 2023). "In Hollywood writers' battle against AI, humans win (for now)". AP News. Associated Press. Archived from the original on April 3, 2024. Retrieved January 26, 2024. ^


key: chroma:document
string_value:
Jake (September 27, 2023). "In Hollywood writers' battle against AI, humans win (for now)". AP News. Associated Press. Archived from the original on April 3, 2024. Retrieved January 26, 2024. ^ Harreis, H.; Koullias, T.; Roberts, Roger. "Generative AI: Unlocking the future of fashion". Archived from the original on November 17, 2023. Retrieved March 14, 2023. ^ "How Generative AI Can Augment Human Creativity". Harvard Business Review. June 16, 2023. ISSN 0017-8012. Archived from the original on June 20, 2023. Retrieved June 20, 2023. ^ Hendrix, Justin (May 16, 2023). "Transcript: Senate Judiciary Subcommittee Hearing on Oversight of AI". techpolicy.press. Archived from the original on November 17, 2023. Retrieved May 19, 2023. ^ Simon, Felix M.; Altay, Sacha; Mercier, Hugo (October 18, 2023). "Misinformation reloaded? Fears about the impact of generative AI on misinformation are overblown". Harvard Kennedy School Misinformation Review. doi:10.37016/mr-2020-127. S2CID 264113883.


key: chroma:document
string_value:
18, 2023). "Misinformation reloaded? Fears about the impact of generative AI on misinformation are overblown". Harvard Kennedy School Misinformation Review. doi:10.37016/mr-2020-127. S2CID 264113883. Archived from the original on November 17, 2023. Retrieved November 16, 2023. ^ "New AI systems collide with copyright law". BBC News. August 1, 2023. Retrieved September 28, 2024. ^ Newquist, H. P. (1994). The Brain Makers: Genius, Ego, And Greed In The Quest For Machines That Think. New York: Macmillan/SAMS. pp. 45–53. ISBN 978-0-672-30412-5. ^ Sharkey, Noel (July 4, 2007), A programmable robot from 60 AD, vol. 2611, New Scientist, archived from the original on January 13, 2018, retrieved October 22, 2019 ^ Brett, Gerard (July 1954), "The Automata in the Byzantine "Throne of Solomon"", Speculum, 29 (3): 477–487, doi:10.2307/2846790, ISSN 0038-7134, JSTOR 2846790, S2CID 163031682. ^ kelinich (March 8, 2014). "Maillardet's Automaton". The Franklin Institute. Archived from the original on


key: chroma:document
string_value:
29 (3): 477–487, doi:10.2307/2846790, ISSN 0038-7134, JSTOR 2846790, S2CID 163031682. ^ kelinich (March 8, 2014). "Maillardet's Automaton". The Franklin Institute. Archived from the original on August 24, 2023. Retrieved August 24, 2023. ^ Grinstead, Charles Miller; Snell, James Laurie (1997). Introduction to Probability. American Mathematical Society. pp. 464–466. ISBN 978-0-8218-0749-1. ^ Bremaud, Pierre (March 9, 2013). Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues. Springer Science & Business Media. p. ix. ISBN 978-1-4757-3124-8. Archived from the original on March 23, 2017. ^ Hayes, Brian (2013). "First Links in the Markov Chain". American Scientist. 101 (2): 92. doi:10.1511/2013.101.92. ISSN 0003-0996. Archived from the original on May 7, 2024. Retrieved September 24, 2023. ^ Fine, Shai; Singer, Yoram; Tishby, Naftali (July 1, 1998). "The Hierarchical Hidden Markov Model: Analysis and Applications". Machine Learning. 32 (1): 41–62. doi:10.1023/A:1007469218079.


key: chroma:document
string_value:
24, 2023. ^ Fine, Shai; Singer, Yoram; Tishby, Naftali (July 1, 1998). "The Hierarchical Hidden Markov Model: Analysis and Applications". Machine Learning. 32 (1): 41–62. doi:10.1023/A:1007469218079. ISSN 1573-0565. S2CID 3465810. ^ Crevier, Daniel (1993). AI: The Tumultuous Search for Artificial Intelligence. New York, New York: BasicBooks. p. 109. ISBN 0-465-02997-3. ^ Bergen, Nathan; Huang, Angela (2023). "A Brief History of Generative AI" (PDF). Dichotomies: Generative AI: Navigating Towards a Better Future (2): 4. Archived (PDF) from the original on August 10, 2023. Retrieved August 8, 2023. ^ a b Alting, Leo; Zhang, Hongchao (1989). "Computer aided process planning: the state-of-the-art survey". The International Journal of Production Research. 27 (4): 553–585. doi:10.1080/00207548908942569. Archived from the original on May 7, 2024. Retrieved October 3, 2023. ^ Chien, Steve (1998). "Automated planning and scheduling for goal-based autonomous spacecraft". IEEE Intelligent


key: chroma:document
string_value:
Archived from the original on May 7, 2024. Retrieved October 3, 2023. ^ Chien, Steve (1998). "Automated planning and scheduling for goal-based autonomous spacecraft". IEEE Intelligent Systems and Their Applications. 13 (5): 50–55. doi:10.1109/5254.722362. ^ Burstein, Mark H., ed. (1994). ARPA/Rome Laboratory Knowledge-based Planning and Scheduling Initiative Workshop Proceedings. The Advanced Research Projects Agency, Department of Defense, and Rome Laboratory, US Air Force, Griffiss AFB. p. 219. ISBN 155860345X. ^ Pell, Barney; Bernard, Douglas E.; Chien, Steve A.; Gat, Erann; Muscettola, Nicola; Nayak, P. Pandurang; Wagner, Michael D.; Williams, Brian C. (1998). Bekey, George A. (ed.). An Autonomous Spacecraft Agent Prototype. Autonomous Robots Volume 5, No. 1. pp. 29–45. Our deliberator is a traditional generative AI planner based on the HSTS planning framework (Muscettola, 1994), and our control component is a traditional spacecraft attitude control system (Hackney et al. 1993).


key: chroma:document
string_value:
is a traditional generative AI planner based on the HSTS planning framework (Muscettola, 1994), and our control component is a traditional spacecraft attitude control system (Hackney et al. 1993). We also add an architectural component explicitly dedicated to world modeling (the mode identifier), and distinguish between control and monitoring. ^ Jebara, Tony (2012). Machine learning: discriminative and generative. Vol. 755. Springer Science & Business Media. ^ Cao, Yihan; Li, Siyu; Liu, Yixin; Yan, Zhiling; Dai, Yutong; Yu, Philip S.; Sun, Lichao (March 7, 2023). "A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT". arXiv:2303.04226 [cs.AI]. ^ "finetune-transformer-lm". GitHub. Archived from the original on May 19, 2023. Retrieved May 19, 2023. ^ Radford, Alec; Wu, Jeffrey; Child, Rewon; Luan, David; Amodei, Dario; Sutskever, Ilya (2019). "Language models are unsupervised multitask learners" (PDF). OpenAI Blog. ^ Radford, Alec (June


key: chroma:document
string_value:
2023. ^ Radford, Alec; Wu, Jeffrey; Child, Rewon; Luan, David; Amodei, Dario; Sutskever, Ilya (2019). "Language models are unsupervised multitask learners" (PDF). OpenAI Blog. ^ Radford, Alec (June 11, 2018). "Improving language understanding with unsupervised learning". OpenAI. Retrieved October 6, 2024. ^ Chandraseta, Rionaldi (January 21, 2021). "Generate Your Favourite Characters' Voice Lines using Machine Learning". Towards Data Science. Retrieved December 18, 2024. ^ Temitope, Yusuf (December 10, 2024). "15.ai Creator reveals journey from MIT Project to internet phenomenon". The Guardian. Archived from the original on December 28, 2024. Retrieved December 25, 2024. ^ Anirudh VK (March 18, 2023). "Deepfakes Are Elevating Meme Culture, But At What Cost?". Analytics India Magazine. Archived from the original on December 26, 2024. Retrieved December 18, 2024. While AI voice memes have been around in some form since '15.ai' launched in 2020, [...] ^ Coldewey, Devin (January 5, 2021).


key: chroma:document
string_value:
from the original on December 26, 2024. Retrieved December 18, 2024. While AI voice memes have been around in some form since '15.ai' launched in 2020, [...] ^ Coldewey, Devin (January 5, 2021). "OpenAI's DALL-E creates plausible images of literally anything you ask it to". TechCrunch. Retrieved March 15, 2023. ^ "Stable Diffusion Public Release". Stability AI. Retrieved March 15, 2023. ^ Lock, Samantha (December 5, 2022). "What is AI chatbot phenomenon ChatGPT and could it replace humans?". The Guardian. Retrieved March 15, 2023. ^ Huang, Haomiao (August 23, 2023). "How ChatGPT turned generative AI into an "anything tool"". Ars Technica. Retrieved September 21, 2024. ^ Bubeck, Sébastien; Chandrasekaran, Varun; Eldan, Ronen; Gehrke, Johannes; Horvitz, Eric; Kamar, Ece; Lee, Peter; Lee, Yin Tat; Li, Yuanzhi; Lundberg, Scott; Nori, Harsha; Palangi, Hamid; Ribeiro, Marco Tulio; Zhang, Yi (March 22, 2023). "Sparks of Artificial General Intelligence: Early experiments with GPT-4".


key: chroma:document
string_value:
Lee, Yin Tat; Li, Yuanzhi; Lundberg, Scott; Nori, Harsha; Palangi, Hamid; Ribeiro, Marco Tulio; Zhang, Yi (March 22, 2023). "Sparks of Artificial General Intelligence: Early experiments with GPT-4". arXiv:2303.12712 [cs.CL]. ^ Schlagwein, Daniel; Willcocks, Leslie (September 13, 2023). "ChatGPT et al: The Ethics of Using (Generative) Artificial Intelligence in Research and Science". Journal of Information Technology. 38 (2): 232–238. doi:10.1177/02683962231200411. S2CID 261753752. ^ "Meta open-sources multisensory AI model that combines six types of data". May 9, 2023. Retrieved March 14, 2024. ^ Kruppa, Miles (December 6, 2023). "Google Announces AI System Gemini After Turmoil at Rival OpenAI". The Wall Street Journal. ISSN 0099-9660. Archived from the original on December 6, 2023. Retrieved December 6, 2023. ^ Edwards, Benj (December 6, 2023). "Google launches Gemini—a powerful AI model it says can surpass GPT-4". Ars Technica. Retrieved December 6, 2023. ^ Metz, Cade (February 8,


key: chroma:document
string_value:
December 6, 2023. ^ Edwards, Benj (December 6, 2023). "Google launches Gemini—a powerful AI model it says can surpass GPT-4". Ars Technica. Retrieved December 6, 2023. ^ Metz, Cade (February 8, 2024). "Google Releases Gemini, an A.I.-Driven Chatbot and Voice Assistant". The New York Times. Retrieved February 8, 2024. ^ "Introducing the next generation of Claude". Retrieved March 4, 2024. ^ Nuñez, Michael (March 4, 2024). "Anthropic unveils Claude 3, surpassing GPT-4 and Gemini Ultra in benchmark tests". Venture Beat. Retrieved April 9, 2024. ^ Pierce, David (June 20, 2024). "Anthropic has a fast new AI model — and a clever new way to interact with chatbots". The Verge. Retrieved June 22, 2024. ^ Baptista, Eduardo (July 9, 2024). "China leads the world in adoption of generative AI, survey shows". Reuters. Retrieved July 14, 2024. ^ "A History of Generative AI: From GAN to GPT-4". March 21, 2023. Archived from the original on June 10, 2023. Retrieved April 28, 2023. ^ "Explainer: What


key: chroma:document
string_value:
shows". Reuters. Retrieved July 14, 2024. ^ "A History of Generative AI: From GAN to GPT-4". March 21, 2023. Archived from the original on June 10, 2023. Retrieved April 28, 2023. ^ "Explainer: What is Generative AI, the technology behind OpenAI's ChatGPT?". Reuters. March 17, 2023. Archived from the original on March 30, 2023. Retrieved March 17, 2023. ^ Roose, Kevin (February 16, 2023). "Bing's A.I. Chat: 'I Want to Be Alive.'". The New York Times. Archived from the original on April 15, 2023. Retrieved January 30, 2024. ^ Bommasani, R.; Hudson, D. A.; Adeli, E.; Altman, R.; Arora, S.; von Arx, S.; Bernstein, M. S.; Bohg, J.; Bosselut, A; Brunskill, E.; Brynjolfsson, E. (August 16, 2021). "On the opportunities and risks of foundation models". arXiv:2108.07258 [cs.LG]. ^ Chen, Ming; Tworek, Jakub; Jun, Hongyu; Yuan, Qinyuan; Pinto, Hanyu Philippe De Oliveira; Kaplan, Jerry; Edwards, Haley; Burda, Yannick; Joseph, Nicholas; Brockman, Greg; Ray, Alvin (July 6, 2021). "Evaluating Large


key: chroma:document
string_value:
Jakub; Jun, Hongyu; Yuan, Qinyuan; Pinto, Hanyu Philippe De Oliveira; Kaplan, Jerry; Edwards, Haley; Burda, Yannick; Joseph, Nicholas; Brockman, Greg; Ray, Alvin (July 6, 2021). "Evaluating Large Language Models Trained on Code". arXiv:2107.03374 [cs.LG]. ^ "Investing in Cursor". Andreesen Horowitz. ^ Epstein, Ziv; Hertzmann, Aaron; Akten, Memo; Farid, Hany; Fjeld, Jessica; Frank, Morgan R.; Groh, Matthew; Herman, Laura; Leach, Neil; Mahari, Robert; Pentland, Alex “Sandy”; Russakovsky, Olga; Schroeder, Hope; Smith, Amy (2023). "Art and the science of generative AI". Science. 380 (6650): 1110–1111. arXiv:2306.04141. Bibcode:2023Sci...380.1110E. doi:10.1126/science.adh4451. PMID 37319193. S2CID 259095707. ^ Ramesh, Aditya; Pavlov, Mikhail; Goh, Gabriel; Gray, Scott; Voss, Chelsea; Radford, Alec; Chen, Mark; Sutskever, Ilya (2021). "Zero-shot text-to-image generation". International Conference on Machine Learning. PMLR. pp. 8821–8831. ^ Chandraseta, Rionaldi (January 21, 2021). "Generate


key: chroma:document
string_value:
Chen, Mark; Sutskever, Ilya (2021). "Zero-shot text-to-image generation". International Conference on Machine Learning. PMLR. pp. 8821–8831. ^ Chandraseta, Rionaldi (January 21, 2021). "Generate Your Favourite Characters' Voice Lines using Machine Learning". Towards Data Science. Archived from the original on January 21, 2021. Retrieved December 18, 2024. ^ Temitope, Yusuf (December 10, 2024). "15.ai Creator reveals journey from MIT Project to internet phenomenon". The Guardian. Archived from the original on December 28, 2024. Retrieved December 25, 2024. ^ Ruppert, Liana (January 18, 2021). "Make Portal's GLaDOS And Other Beloved Characters Say The Weirdest Things With This App". Game Informer. Archived from the original on January 18, 2021. Retrieved December 18, 2024. ^ Kurosawa, Yuki (January 19, 2021). "ゲームキャラ音声読み上げソフト「15.ai」公開中。『Undertale』や『Portal』のキャラに好きなセリフを言ってもらえる" [Game Character Voice Reading Software "15.ai" Now Available. Get Characters from Undertale and Portal to Say


key: chroma:document
string_value:
(January 19, 2021). "ゲームキャラ音声読み上げソフト「15.ai」公開中。『Undertale』や『Portal』のキャラに好きなセリフを言ってもらえる" [Game Character Voice Reading Software "15.ai" Now Available. Get Characters from Undertale and Portal to Say Your Desired Lines]. AUTOMATON (in Japanese). Archived from the original on January 19, 2021. Retrieved December 18, 2024. 英語版ボイスのみなので注意。;もうひとつ15.aiの大きな特徴として挙げられるのが、豊かな感情表現だ。 [Please note that only English voices are available.;Another major feature of 15.ai is its rich emotional expression.] ^ Desai, Saahil (July 17, 2023). "A Voicebot Just Left Me Speechless". The Atlantic. Archived from the original on December 8, 2023. Retrieved November 28, 2023. ^ Agostinelli, Andrea; Denk, Timo I.; Borsos, Zalán; Engel, Jesse; Verzetti, Mauro; Caillon, Antoine; Huang, Qingqing; Jansen, Aren; Roberts, Adam; Tagliasacchi, Marco; Sharifi, Matt; Zeghidour, Neil; Frank, Christian (January 26, 2023). "MusicLM: Generating Music From Text". arXiv:2301.11325 [cs.SD]. ^ Dalugdug, Mandy (August 3, 2023). "Meta


key: chroma:document
string_value:
Tagliasacchi, Marco; Sharifi, Matt; Zeghidour, Neil; Frank, Christian (January 26, 2023). "MusicLM: Generating Music From Text". arXiv:2301.11325 [cs.SD]. ^ Dalugdug, Mandy (August 3, 2023). "Meta in June said that it used 20,000 hours of licensed music to train MusicGen, which included 10,000 "high-quality" licensed music tracks. At the time, Meta's researchers outlined in a paper the ethical challenges that they encountered around the development of generative AI models like MusicGen". Archived from the original on August 15, 2023. ^ "Jay-Z's Delaware producer sparks debate over AI rights". Archived from the original on February 27, 2024. Retrieved February 27, 2024. ^ "10 "Best" AI Music Generators (April 2024) - Unite.AI". October 19, 2022. Archived from the original on January 29, 2024. Retrieved February 27, 2024. ^ Metz, Cade (April 4, 2023). "Instant Videos Could Represent the Next Leap in A.I. Technology". The New York Times. Archived from the original on April 5, 2023.


key: chroma:document
string_value:
2024. Retrieved February 27, 2024. ^ Metz, Cade (April 4, 2023). "Instant Videos Could Represent the Next Leap in A.I. Technology". The New York Times. Archived from the original on April 5, 2023. Retrieved April 5, 2023. ^ Wong, Queenie (September 29, 2022). "Facebook Parent Meta's AI Tool Can Create Artsy Videos From Text". cnet.com. Archived from the original on April 5, 2023. Retrieved April 4, 2023. ^ Yang, Sherry; Du, Yilun (April 12, 2023). "UniPi: Learning universal policies via text-guided video generation". Google Research, Brain Team. Google AI Blog. Archived from the original on May 24, 2023. ^ Brohan, Anthony (2023). "RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control". arXiv:2307.15818 [cs.RO]. ^ Abdullahi, Aminu (November 17, 2023). "10 Best Artificial Intelligence (AI) 3D Generators". eWEEK. Archived from the original on May 7, 2024. Retrieved February 6, 2024. ^ "Slash CAD model build times with new AI-driven part creation methodology |


key: chroma:document
string_value:
Intelligence (AI) 3D Generators". eWEEK. Archived from the original on May 7, 2024. Retrieved February 6, 2024. ^ "Slash CAD model build times with new AI-driven part creation methodology | GlobalSpec". Archived from the original on January 23, 2024. Retrieved February 6, 2024. ^ "The Role of Artificial Intelligence (AI) in the CAD Industry". March 22, 2023. Archived from the original on February 9, 2024. Retrieved February 6, 2024. ^ Sabin, Sam (June 30, 2023). "GitHub has a vision to make code more secure by design". Axios Codebook. Archived from the original on August 15, 2023. Retrieved August 15, 2023. ^ Vincent, James (March 20, 2023). "Text-to-video AI inches closer as startup Runway announces new model". The Verge. Archived from the original on September 27, 2023. Retrieved August 15, 2023. Text-to-video is the next frontier for generative AI, though current output is rudimentary. Runway says it'll be making its new generative video model, Gen-2, available to users in 'the


key: chroma:document
string_value:
15, 2023. Text-to-video is the next frontier for generative AI, though current output is rudimentary. Runway says it'll be making its new generative video model, Gen-2, available to users in 'the coming weeks.' ^ Vanian, Jonathan (March 16, 2023). "Microsoft adds OpenAI technology to Word and Excel". CNBC. Archived from the original on August 15, 2023. Retrieved August 15, 2023. Microsoft is bringing generative artificial intelligence technologies such as the popular ChatGPT chatting app to its Microsoft 365 suite of business software....the new A.I. features, dubbed Copilot, will be available in some of the company's most popular business apps, including Word, PowerPoint and Excel. ^ Wilson, Mark (August 15, 2023). "The app's Memories feature just got a big upgrade". TechRadar. Archived from the original on August 15, 2023. The Google Photos app is getting a redesigned, AI-powered Memories feature...you'll be able to use generative AI to come up with some suggested names like "a


key: chroma:document
string_value:
from the original on August 15, 2023. The Google Photos app is getting a redesigned, AI-powered Memories feature...you'll be able to use generative AI to come up with some suggested names like "a desert adventure". ^ Sullivan, Laurie (May 23, 2023). "Adobe Adds Generative AI To Photoshop". MediaPost. Archived from the original on August 15, 2023. Retrieved August 15, 2023. Generative artificial intelligence (AI) will become one of the most important features for creative designers and marketers. Adobe on Tuesday unveiled a Generative Fill feature in Photoshop to bring Firefly's AI capabilities into design. ^ Michael Nuñez (July 19, 2023). "LLaMA 2: How to access and use Meta's versatile open-source chatbot right now". VentureBeat. Archived from the original on November 3, 2023. Retrieved August 15, 2023. If you want to run LLaMA 2 on your own machine or modify the code, you can download it directly from Hugging Face, a leading platform for sharing AI models. ^ Pounder, Les (March 25,


key: chroma:document
string_value:
August 15, 2023. If you want to run LLaMA 2 on your own machine or modify the code, you can download it directly from Hugging Face, a leading platform for sharing AI models. ^ Pounder, Les (March 25, 2023). "How To Create Your Own AI Chatbot Server With Raspberry Pi 4". Archived from the original on August 15, 2023. Retrieved August 15, 2023. Using a Pi 4 with 8GB of RAM, you can create a ChatGPT-like server based on LLaMA. ^ Kemper, Jonathan (November 10, 2022). ""Draw Things" App brings Stable Diffusion to the iPhone". The Decoder. Archived from the original on August 15, 2023. Retrieved August 15, 2023. Draw Things is an app that brings Stable Diffusion to the iPhone. The AI images are generated locally, so you don't need an Internet connection. ^ Witt, Allan (July 7, 2023). "Best Computer to Run LLaMA AI Model at Home (GPU, CPU, RAM, SSD)". Archived from the original on August 15, 2023. Retrieved August 15, 2023. To run LLaMA model at home, you will need a computer build with a


key: chroma:document
string_value:
to Run LLaMA AI Model at Home (GPU, CPU, RAM, SSD)". Archived from the original on August 15, 2023. Retrieved August 15, 2023. To run LLaMA model at home, you will need a computer build with a powerful GPU that can handle the large amount of data and computation required for inferencing. ^ Westover, Brian (September 28, 2023). "Who Needs ChatGPT? How to Run Your Own Free and Private AI Chatbot". Ziff Davis. Archived from the original on January 7, 2024. Retrieved January 7, 2024. ^ @karpathy (December 20, 2023). "I pretty much only trust two LLM evals right now" (Tweet) – via Twitter. ^ @ylecun (January 5, 2024). "Nabla's shift from ChatGPT to open source LLMs..." (Tweet) – via Twitter. ^ @ylecun (November 1, 2023). "Open source platforms *increase* safety and security" (Tweet) – via Twitter. ^ Nellis, Stephen; Lee, Jane (September 1, 2022). "U.S. officials order Nvidia to halt sales of top AI chips to China". Reuters. Archived from the original on August 15, 2023. Retrieved August


key: chroma:document
string_value:
^ Nellis, Stephen; Lee, Jane (September 1, 2022). "U.S. officials order Nvidia to halt sales of top AI chips to China". Reuters. Archived from the original on August 15, 2023. Retrieved August 15, 2023. ^ Shilov, Anton (May 7, 2023). "Nvidia's Chinese A800 GPU's Performance Revealed". Tom's Hardware. Archived from the original on May 7, 2024. Retrieved August 15, 2023. the A800 operates at 70% of the speed of A100 GPUs while complying with strict U.S. export standards that limit how much processing power Nvidia can sell. ^ Patel, Dylan (October 24, 2022). "How China's Biren Is Attempting To Evade US Sanctions". Archived from the original on August 15, 2023. Retrieved August 15, 2023. ^ "5 free software to recognise fake AI-generated images" (in Italian). October 28, 2023. Archived from the original on October 29, 2023. Retrieved October 29, 2023. ^ "Detecting AI fingerprints: A guide to watermarking and beyond". Brookings Institution. January 4, 2024. Archived from the original on


key: chroma:document
string_value:
the original on October 29, 2023. Retrieved October 29, 2023. ^ "Detecting AI fingerprints: A guide to watermarking and beyond". Brookings Institution. January 4, 2024. Archived from the original on September 3, 2024. Retrieved September 5, 2024. ^ Fowler, Geoffrey (April 3, 2023). "We tested a new ChatGPT-detector for teachers. It flagged an innocent student". washingtonpost.com. Archived from the original on March 28, 2024. Retrieved February 6, 2024. ^ Fowler, Geoffrey (June 2, 2023). "Detecting AI may be impossible. That's a big problem for teachers". washingtonpost.com. Archived from the original on June 3, 2023. Retrieved February 6, 2024. ^ Bartz, Diane; Hu, Krystal (July 21, 2023). "OpenAI, Google, others pledge to watermark AI content for safety, White House says". Reuters. Archived from the original on July 27, 2023. ^ "FACT SHEET: President Biden Issues Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence". The White House. October 30, 2023. Archived


key: chroma:document
string_value:
from the original on July 27, 2023. ^ "FACT SHEET: President Biden Issues Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence". The White House. October 30, 2023. Archived from the original on January 30, 2024. Retrieved January 30, 2024. ^ Burt, Andrew (October 31, 2023). "3 Obstacles to Regulating Generative AI". Harvard Business Review. ISSN 0017-8012. Archived from the original on February 17, 2024. Retrieved February 17, 2024. ^ "EU AI Act: first regulation on artificial intelligence". European Parliament. August 6, 2023. Retrieved September 13, 2024. ^ Chee, Foo Yun; Mukherjee, Supantha (June 14, 2023). "EU lawmakers vote for tougher AI rules as draft moves to final stage". Reuters. Archived from the original on July 27, 2023. Retrieved July 26, 2023. ^ Ye, Josh (July 13, 2023). "China says generative AI rules to apply only to products for the public". Reuters. Archived from the original on July 27, 2023. Retrieved July 13, 2023. ^ "生成式人工智能服务管理暂行办法". July


key: chroma:document
string_value:
(July 13, 2023). "China says generative AI rules to apply only to products for the public". Reuters. Archived from the original on July 27, 2023. Retrieved July 13, 2023. ^ "生成式人工智能服务管理暂行办法". July 13, 2023. Archived from the original on July 27, 2023. Retrieved July 27, 2023. ^ a b "Generative Artificial Intelligence and Copyright Law". Congressional Research Service. LSB10922. September 29, 2023. Archived from the original on March 22, 2024. Retrieved January 30, 2024. ^ Thompson, Stuart (January 25, 2024). "We Asked A.I. to Create the Joker. It Generated a Copyrighted Image". The New York Times. Archived from the original on January 25, 2024. Retrieved January 26, 2024. ^ Hadero, Haleluya; Bauder, David (December 27, 2023). "The New York Times sues OpenAI and Microsoft for using its stories to train chatbots". Associated Press News. AP News. Archived from the original on December 27, 2023. Retrieved April 13, 2023. ^ O’Brien, Matt (September 25, 2023). "Photo giant Getty took a


key: chroma:document
string_value:
stories to train chatbots". Associated Press News. AP News. Archived from the original on December 27, 2023. Retrieved April 13, 2023. ^ O’Brien, Matt (September 25, 2023). "Photo giant Getty took a leading AI image-maker to court. Now it's also embracing the technology". AP NEWS. Associated Press. Archived from the original on January 30, 2024. Retrieved January 30, 2024. ^ Barber, Gregory (December 9, 2023). "The Generative AI Copyright Fight Is Just Getting Started". Wired. Archived from the original on January 19, 2024. Retrieved January 19, 2024. ^ Bruell, Alexandra (December 27, 2023). "New York Times Sues Microsoft and OpenAI, Alleging Copyright Infringement". Wall Street Journal. Archived from the original on January 18, 2024. Retrieved January 19, 2024. ^ Brittain, Blake (August 21, 2023). "AI-generated art cannot receive copyrights, US court says". Reuters. Archived from the original on January 20, 2024. Retrieved January 19, 2024. ^ David, Emilla (August 29, 2023). "US


key: chroma:document
string_value:
21, 2023). "AI-generated art cannot receive copyrights, US court says". Reuters. Archived from the original on January 20, 2024. Retrieved January 19, 2024. ^ David, Emilla (August 29, 2023). "US Copyright Office wants to hear what people think about AI and copyright". The Verge. Archived from the original on January 19, 2024. Retrieved January 19, 2024. ^ "Secretary-General's remarks to the Security Council on Artificial Intelligence". un.org. July 18, 2023. Archived from the original on July 28, 2023. Retrieved July 27, 2023. ^ "The Writers Strike Is Taking a Stand on AI". Time. May 4, 2023. Archived from the original on June 11, 2023. Retrieved June 11, 2023. ^ Tarnoff, Ben (August 4, 2023). "Lessons from Eliza". The Guardian Weekly. pp. 34–39. ^ Zhou, Viola (April 11, 2023). "AI is already taking video game illustrators' jobs in China". Rest of World. Archived from the original on August 13, 2023. Retrieved August 17, 2023. ^ Carter, Justin (April 11, 2023). "China's game art


key: chroma:document
string_value:
already taking video game illustrators' jobs in China". Rest of World. Archived from the original on August 13, 2023. Retrieved August 17, 2023. ^ Carter, Justin (April 11, 2023). "China's game art industry reportedly decimated by growing AI use". Game Developer. Archived from the original on August 17, 2023. Retrieved August 17, 2023. ^ Collier, Kevin (July 14, 2023). "Actors vs. AI: Strike brings focus to emerging use of advanced tech". NBC News. Archived from the original on July 20, 2023. Retrieved July 21, 2023. SAG-AFTRA has joined the Writer's [sic] Guild of America in demanding a contract that explicitly demands AI regulations to protect writers and the works they create. ... The future of generative artificial intelligence in Hollywood—and how it can be used to replace labor—has become a crucial sticking point for actors going on strike. In a news conference Thursday, Fran Drescher, president of the Screen Actors Guild-American Federation of Television and Radio Artists (more


key: chroma:document
string_value:
a crucial sticking point for actors going on strike. In a news conference Thursday, Fran Drescher, president of the Screen Actors Guild-American Federation of Television and Radio Artists (more commonly known as SAG-AFTRA), declared that 'artificial intelligence poses an existential threat to creative professions, and all actors and performers deserve contract language that protects them from having their identity and talent exploited without consent and pay.' ^ Wiggers, Kyle (August 22, 2023). "ElevenLabs' voice-generating tools launch out of beta". TechCrunch. Archived from the original on November 28, 2023. Retrieved September 25, 2023. ^ Shrivastava, Rashi. "'Keep Your Paws Off My Voice': Voice Actors Worry Generative AI Will Steal Their Livelihoods". Forbes. Archived from the original on December 2, 2023. Retrieved November 28, 2023. ^ Gupta, Shalene (October 31, 2023). "Underrepresented groups in countries around the world are worried about AI being a threat to jobs". Fast


key: chroma:document
string_value:
on December 2, 2023. Retrieved November 28, 2023. ^ Gupta, Shalene (October 31, 2023). "Underrepresented groups in countries around the world are worried about AI being a threat to jobs". Fast Company. Archived from the original on December 8, 2023. Retrieved December 8, 2023. ^ Rachel Gordon (March 3, 2023). "Large language models are biased. Can logic help save them?". MIT CSAIL. Archived from the original on January 23, 2024. Retrieved January 26, 2024. ^ OpenAI (July 18, 2022). "Reducing bias and improving safety in DALL·E 2". OpenAI. Archived from the original on January 26, 2024. Retrieved January 26, 2024. ^ Jake Traylor (July 27, 2022). "No quick fix: How OpenAI's DALL·E 2 illustrated the challenges of bias in AI". NBC News. Archived from the original on January 26, 2024. Retrieved January 26, 2024. ^ "DALL·E 2 pre-training mitigations". OpenAI. June 28, 2022. Archived from the original on January 26, 2024. Retrieved January 26, 2024. ^ Brandon, John (February 16, 2018).


key: chroma:document
string_value:
January 26, 2024. ^ "DALL·E 2 pre-training mitigations". OpenAI. June 28, 2022. Archived from the original on January 26, 2024. Retrieved January 26, 2024. ^ Brandon, John (February 16, 2018). "Terrifying high-tech porn: Creepy 'deepfake' videos are on the rise". Fox News. Archived from the original on June 15, 2018. Retrieved February 20, 2018. ^ Cole, Samantha (January 24, 2018). "We Are Truly Fucked: Everyone Is Making AI-Generated Fake Porn Now". Vice. Archived from the original on September 7, 2019. Retrieved May 4, 2019. ^ "What Are Deepfakes & Why the Future of Porn is Terrifying". Highsnobiety. February 20, 2018. Archived from the original on July 14, 2021. Retrieved February 20, 2018. ^ "Experts fear face swapping tech could start an international showdown". The Outline. Archived from the original on January 16, 2020. Retrieved February 28, 2018. ^ Roose, Kevin (March 4, 2018). "Here Come the Fake Videos, Too". The New York Times. ISSN 0362-4331. Archived from the original on


key: chroma:document
string_value:
from the original on January 16, 2020. Retrieved February 28, 2018. ^ Roose, Kevin (March 4, 2018). "Here Come the Fake Videos, Too". The New York Times. ISSN 0362-4331. Archived from the original on June 18, 2019. Retrieved March 24, 2018. ^ Schreyer, Marco; Sattarov, Timur; Reimer, Bernd; Borth, Damian (2019). "Adversarial Learning of Deepfakes in Accounting". arXiv:1910.03810 [cs.LG]. ^ Menz, Bradley (2024). "Health Disinformation Use Case Highlighting the Urgent Need for Artificial Intelligence Vigilance". JAMA Internal Medicine. 184 (1): 92–96. doi:10.1001/jamainternmed.2023.5947. PMID 37955873. S2CID 265148637. Archived from the original on February 4, 2024. Retrieved February 4, 2024. ^ Chalfant, Morgan (March 6, 2024). "U.S. braces for foreign interference in 2024 election". Semafor. Archived from the original on March 11, 2024. Retrieved March 6, 2024. ^ Menn, Joseph (September 23, 2024). "Russia, Iran use AI to boost anti-U.S. influence campaigns, officials say". The


key: chroma:document
string_value:
Semafor. Archived from the original on March 11, 2024. Retrieved March 6, 2024. ^ Menn, Joseph (September 23, 2024). "Russia, Iran use AI to boost anti-U.S. influence campaigns, officials say". The Washington Post. ISSN 0190-8286. Archived from the original on September 24, 2024. Retrieved September 23, 2024. ^ "Join the Deepfake Detection Challenge (DFDC)". deepfakedetectionchallenge.ai. Archived from the original on January 12, 2020. Retrieved November 8, 2019. ^ Clarke, Yvette D. (June 28, 2019). "H.R.3230 – 116th Congress (2019-2020): Defending Each and Every Person from False Appearances by Keeping Exploitation Subject to Accountability Act of 2019". www.congress.gov. Archived from the original on December 17, 2019. Retrieved October 16, 2019. ^ "New Research Reveals Scale of Threat Posed by AI-generated Images on 2024 Elections". Logically. July 27, 2023. Archived from the original on October 3, 2023. Retrieved July 6, 2024. ^ Lawton, Graham (September 12, 2023). "Disinformation


key: chroma:document
string_value:
by AI-generated Images on 2024 Elections". Logically. July 27, 2023. Archived from the original on October 3, 2023. Retrieved July 6, 2024. ^ Lawton, Graham (September 12, 2023). "Disinformation wars: The fight against fake news in the age of AI". New Scientist. Retrieved July 5, 2024. ^ Brewer, Jordan; Patel, Dhru; Kim, Dennie; Murray, Alex (April 12, 2024). "Navigating the challenges of generative technologies: Proposing the integration of artificial intelligence and blockchain". Business Horizons. 67 (5): 525–535. doi:10.1016/j.bushor.2024.04.011. ISSN 0007-6813. ^ "People Are Still Terrible: AI Voice-Cloning Tool Misused for Deepfake Celeb Clips". PCMag Middle East. January 31, 2023. Archived from the original on December 25, 2023. Retrieved July 25, 2023. ^ "The generative A.I. software race has begun". Fortune. Archived from the original on March 25, 2023. Retrieved February 3, 2023. ^ Milmo, Dan; Hern, Alex (May 20, 2023). "Elections in UK and US at risk from AI-driven


key: chroma:document
string_value:
software race has begun". Fortune. Archived from the original on March 25, 2023. Retrieved February 3, 2023. ^ Milmo, Dan; Hern, Alex (May 20, 2023). "Elections in UK and US at risk from AI-driven disinformation, say experts". The Guardian. ISSN 0261-3077. Archived from the original on November 16, 2023. Retrieved July 25, 2023. ^ "Seeing is believing? Global scramble to tackle deepfakes". news.yahoo.com. Archived from the original on February 3, 2023. Retrieved February 3, 2023. ^ Vincent, James (January 31, 2023). "4chan users embrace AI voice clone tool to generate celebrity hatespeech". The Verge. Archived from the original on December 3, 2023. Retrieved February 3, 2023. ^ Thompson, Stuart A. (March 12, 2023). "Making Deepfakes Gets Cheaper and Easier Thanks to A.I." The New York Times. ISSN 0362-4331. Archived from the original on October 29, 2023. Retrieved July 25, 2023. ^ "A new AI voice tool is already being abused to make deepfake celebrity audio clips". Engadget. January


key: chroma:document
string_value:
ISSN 0362-4331. Archived from the original on October 29, 2023. Retrieved July 25, 2023. ^ "A new AI voice tool is already being abused to make deepfake celebrity audio clips". Engadget. January 31, 2023. Archived from the original on October 10, 2023. Retrieved February 3, 2023. ^ Gee, Andre (April 20, 2023). "Just Because AI-Generated Rap Songs Go Viral Doesn't Mean They're Good". Rolling Stone. Archived from the original on January 2, 2024. Retrieved December 6, 2023. ^ Coscarelli, Joe (April 19, 2023). "An A.I. Hit of Fake 'Drake' and 'The Weeknd' Rattles the Music World". The New York Times. Archived from the original on May 15, 2023. Retrieved December 5, 2023. ^ Lippiello, Emily; Smith, Nathan; Pereira, Ivan (November 3, 2023). "AI songs that mimic popular artists raising alarms in the music industry". ABC News. Archived from the original on December 6, 2023. Retrieved December 6, 2023. ^ Skelton, Eric. "Fans Are Using Artificial Intelligence to Turn Rap Snippets Into Full


key: chroma:document
string_value:
in the music industry". ABC News. Archived from the original on December 6, 2023. Retrieved December 6, 2023. ^ Skelton, Eric. "Fans Are Using Artificial Intelligence to Turn Rap Snippets Into Full Songs". Complex. Archived from the original on January 2, 2024. Retrieved December 6, 2023. ^ Marr, Bernard. "Virtual Influencer Noonoouri Lands Record Deal: Is She The Future Of Music?". Forbes. Archived from the original on December 4, 2023. Retrieved December 6, 2023. ^ Thaler, Shannon (September 8, 2023). "Warner Music signs first-ever record deal with AI pop star". New York Post. Archived from the original on December 15, 2023. Retrieved December 6, 2023. ^ Sjouwerman, Stu (December 26, 2022). "Deepfakes: Get ready for phishing 2.0". Fast Company. Archived from the original on July 31, 2023. Retrieved July 31, 2023. ^ Sonnemaker, Tyler. "As social media platforms brace for the incoming wave of deepfakes, Google's former 'fraud czar' predicts the biggest danger is that deepfakes will


key: chroma:document
string_value:
Retrieved July 31, 2023. ^ Sonnemaker, Tyler. "As social media platforms brace for the incoming wave of deepfakes, Google's former 'fraud czar' predicts the biggest danger is that deepfakes will eventually become boring". Business Insider. Archived from the original on April 14, 2021. Retrieved July 31, 2023. ^ Collinson, Patrick (July 15, 2023). "Fake reviews: can we trust what we read online as use of AI explodes?". The Guardian. ISSN 0261-3077. Archived from the original on November 22, 2023. Retrieved December 6, 2023. ^ "After WormGPT, FraudGPT Emerges to Help Scammers Steal Your Data". PCMAG. July 25, 2023. Archived from the original on July 31, 2023. Retrieved July 31, 2023. ^ Gupta, Maanak; Akiri, Charankumar; Aryal, Kshitiz; Parker, Eli; Praharaj, Lopamudra (2023). "From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy". IEEE Access. 11: 80218–80245. arXiv:2307.00691. Bibcode:2023IEEEA..1180218G. doi:10.1109/ACCESS.2023.3300381. S2CID 259316122. ^


key: chroma:document
string_value:
to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy". IEEE Access. 11: 80218–80245. arXiv:2307.00691. Bibcode:2023IEEEA..1180218G. doi:10.1109/ACCESS.2023.3300381. S2CID 259316122. ^ Piper, Kelsey (February 2, 2024). "Should we make our most powerful AI models open source to all?". Vox. Retrieved January 13, 2025. ^ Metz, Cade (July 10, 2023). "In the Age of A.I., Tech's Little Guys Need Big Friends". New York Times. ^ a b Bender, Emily M.; Gebru, Timnit; McMillan-Major, Angelina; Shmitchell, Shmargaret (March 1, 2021). "On the Dangers of Stochastic Parrots: Can Language Models be Too Big? 🦜". Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. FAccT '21. New York, NY, USA: Association for Computing Machinery. pp. 610–623. doi:10.1145/3442188.3445922. ISBN 978-1-4503-8309-7. ^ a b c d "AI is an energy hog. This is what it means for climate change". MIT Technology Review. May 23, 2024. Archived from the original on August 20, 2024.


key: chroma:document
string_value:
ISBN 978-1-4503-8309-7. ^ a b c d "AI is an energy hog. This is what it means for climate change". MIT Technology Review. May 23, 2024. Archived from the original on August 20, 2024. Retrieved August 27, 2024. ^ a b c d e f g Dhar, Payal (August 1, 2020). "The carbon impact of artificial intelligence". Nature Machine Intelligence. 2 (8): 423–425. doi:10.1038/s42256-020-0219-9. ISSN 2522-5839. Archived from the original on August 14, 2024. ^ a b c d e Crawford, Kate (February 20, 2024). "Generative AI's environmental costs are soaring — and mostly secret". Nature. 626 (8000): 693. Bibcode:2024Natur.626..693C. doi:10.1038/d41586-024-00478-x. PMID 38378831. Archived from the original on August 22, 2024. ^ a b c d Rogers, Reece. "AI's Energy Demands Are Out of Control. Welcome to the Internet's Hyper-Consumption Era". Wired. ISSN 1059-1028. Archived from the original on August 14, 2024. Retrieved August 27, 2024. ^ a b c d e f Saenko, Kate (May 23, 2023). "Is generative AI bad for the


key: chroma:document
string_value:
Hyper-Consumption Era". Wired. ISSN 1059-1028. Archived from the original on August 14, 2024. Retrieved August 27, 2024. ^ a b c d e f Saenko, Kate (May 23, 2023). "Is generative AI bad for the environment? A computer scientist explains the carbon footprint of ChatGPT and its cousins". The Conversation. Archived from the original on July 1, 2024. Retrieved August 27, 2024. ^ a b Lohr, Steve (August 26, 2024). "Will A.I. Ruin the Planet or Save the Planet?". The New York Times. ISSN 0362-4331. Archived from the original on August 26, 2024. Retrieved August 27, 2024. ^ Hoffman, Benjamin (June 11, 2024). "First Came 'Spam.' Now, With A.I., We've Got 'Slop'". The New York Times. ISSN 0362-4331. Archived from the original on August 26, 2024. Retrieved August 27, 2024. ^ a b "Investigation Finds Actual Source of All That AI Slop on Facebook". Futurism. August 10, 2024. Archived from the original on August 15, 2024. Retrieved August 27, 2024. ^ a b Warzel, Charlie (August 21, 2024). "The


key: chroma:document
string_value:
Actual Source of All That AI Slop on Facebook". Futurism. August 10, 2024. Archived from the original on August 15, 2024. Retrieved August 27, 2024. ^ a b Warzel, Charlie (August 21, 2024). "The MAGA Aesthetic Is AI Slop". The Atlantic. Archived from the original on August 25, 2024. Retrieved August 27, 2024. ^ Edwards, Benj (August 14, 2024). "Research AI model unexpectedly attempts to modify its own code to extend runtime". Ars Technica. Archived from the original on August 24, 2024. Retrieved August 27, 2024. ^ Hern, Alex; Milmo, Dan (May 19, 2024). "Spam, junk … slop? The latest wave of AI behind the 'zombie internet'". The Guardian. ISSN 0261-3077. Archived from the original on August 26, 2024. Retrieved August 27, 2024. ^ Cox, Joseph (January 18, 2024). "Google News Is Boosting Garbage AI-Generated Articles". 404 Media. Archived from the original on June 13, 2024. Retrieved August 27, 2024. ^ "Beloved Local Newspapers Fired Staffers, Then Started Running AI Slop". Futurism. July


key: chroma:document
string_value:
AI-Generated Articles". 404 Media. Archived from the original on June 13, 2024. Retrieved August 27, 2024. ^ "Beloved Local Newspapers Fired Staffers, Then Started Running AI Slop". Futurism. July 31, 2024. Archived from the original on August 12, 2024. Retrieved August 27, 2024. ^ Thompson, Brian; Dhaliwal, Mehak; Frisch, Peter; Domhan, Tobias; Federico, Marcello (August 2024). Ku, Lun-Wei; Martins, Andre; Srikumar, Vivek (eds.). "A Shocking Amount of the Web is Machine Translated: Insights from Multi-Way Parallelism". Findings of the Association for Computational Linguistics ACL 2024. Bangkok, Thailand and virtual meeting: Association for Computational Linguistics: 1763–1775. arXiv:2401.05749. doi:10.18653/v1/2024.findings-acl.103. ^ Roscoe, Jules (January 17, 2024). "A 'Shocking' Amount of the Web Is Already AI-Translated Trash, Scientists Determine". VICE. Archived from the original on July 1, 2024. Retrieved August 27, 2024. ^ Koebler, Jason (September 19, 2024). "Project


key: chroma:document
string_value:
Amount of the Web Is Already AI-Translated Trash, Scientists Determine". VICE. Archived from the original on July 1, 2024. Retrieved August 27, 2024. ^ Koebler, Jason (September 19, 2024). "Project Analyzing Human Language Usage Shuts Down Because 'Generative AI Has Polluted the Data'". 404 Media. Archived from the original on September 19, 2024. Retrieved September 20, 2024. While there has always been spam on the internet and in the datasets that Wordfreq used, "it was manageable and often identifiable. Large language models generate text that masquerades as real language with intention behind it, even though there is none, and their output crops up everywhere," she wrote. She gives the example that ChatGPT overuses the word "delve," in a way that people do not, which has thrown off the frequency of this specific word. ^ Gray, Andrew (March 24, 2024). "ChatGPT "contamination": estimating the prevalence of LLMs in the scholarly literature". arXiv:2403.16887 [cs.DL]. ^ Kannan, Prabha


key: chroma:document
string_value:
frequency of this specific word. ^ Gray, Andrew (March 24, 2024). "ChatGPT "contamination": estimating the prevalence of LLMs in the scholarly literature". arXiv:2403.16887 [cs.DL]. ^ Kannan, Prabha (May 13, 2024). "How Much Research Is Being Written by Large Language Models?". Human-Centered Artificial Intelligence. Stanford University. Retrieved August 16, 2024. ^ Valyaeva, Alina (August 15, 2023). "AI Image Statistics for 2024: How Much Content Was Created by AI". Everypixel Journal. Retrieved August 16, 2024. ^ Shumailov, Ilia; Shumaylov, Zakhar; Zhao, Yiren; Papernot, Nicolas; Anderson, Ross; Gal, Yarin (July 2024). "AI models collapse when trained on recursively generated data". Nature. 631 (8022): 755–759. Bibcode:2024Natur.631..755S. doi:10.1038/s41586-024-07566-y. PMC 11269175. PMID 39048682. ^ Bhatia, Aatish (August 26, 2024). "When A.I.'s Output Is a Threat to A.I. Itself". The New York Times. ISSN 0362-4331. Retrieved August 27, 2024. ^ "Self-Consuming Generative Models Go


key: chroma:document
string_value:
^ Bhatia, Aatish (August 26, 2024). "When A.I.'s Output Is a Threat to A.I. Itself". The New York Times. ISSN 0362-4331. Retrieved August 27, 2024. ^ "Self-Consuming Generative Models Go Mad". ICLR. 2024. ^ Owen, Sean (April 12, 2023). "Synthetic Data for Better Machine Learning". databricks.com. Archived from the original on January 3, 2024. Retrieved January 4, 2024. ^ Sharma, Himanshu (July 11, 2023). "Synthetic Data Platforms: Unlocking the Power of Generative AI for Structured Data". kdnuggets.com. Archived from the original on January 3, 2024. Retrieved January 4, 2024. ^ Stöckl, Andreas (November 2, 2022). "Evaluating a Synthetic Image Dataset Generated with Stable Diffusion". arXiv:2211.01777 [cs.CV]. ^ Roth, Emma (January 25, 2023). "CNET found errors in more than half of its AI-written stories". The Verge. Archived from the original on November 6, 2023. Retrieved June 17, 2023. ^ "A magazine touted Michael Schumacher's first interview in years. It was actually AI". NPR.


key: chroma:document
string_value:
AI-written stories". The Verge. Archived from the original on November 6, 2023. Retrieved June 17, 2023. ^ "A magazine touted Michael Schumacher's first interview in years. It was actually AI". NPR. April 28, 2023. Archived from the original on June 17, 2023. Retrieved June 17, 2023. ^ Al-Sibai, Noor (January 3, 2024). "Police Say AI-Generated Article About Local Murder Is "Entirely" Made Up". Futurism. Archived from the original on January 5, 2024. Retrieved January 8, 2024. ^ "NewsBreak: Most downloaded US news app has Chinese roots and 'writes fiction' using AI". Reuters. June 5, 2024. Archived from the original on June 6, 2024. Retrieved June 7, 2024. ^ a b Harrison, Maggie (November 27, 2023). "Sports Illustrated Published Articles by Fake, AI-Generated Writers". Futurism. Archived from the original on December 15, 2023. Retrieved January 8, 2024. ^ Christian, Jon (February 9, 2023). "Magazine Publishes Serious Errors in First AI-Generated Health Article". Futurism. Archived from


key: chroma:document
string_value:
the original on December 15, 2023. Retrieved January 8, 2024. ^ Christian, Jon (February 9, 2023). "Magazine Publishes Serious Errors in First AI-Generated Health Article". Futurism. Archived from the original on December 26, 2023. Retrieved January 8, 2024. ^ Schneider, Jaron (December 14, 2023). "B&H Photo Published an AI-Generated Guide Written by a Fake Person". PetaPixel. Archived from the original on January 4, 2024. Retrieved January 8, 2024. ^ Harrison, Maggie (August 29, 2023). "USA Today Owner Pauses AI Articles After Butchering Sports Coverage". Futurism. Archived from the original on January 4, 2024. Retrieved January 8, 2024. ^ Buchanan, Tyler (August 28, 2023). "Dispatch pauses AI sports writing program". Axios. Archived from the original on January 1, 2024. Retrieved January 8, 2024. ^ Sommer, Will (October 26, 2023). "Mysterious bylines appeared on a USA Today site. Did these writers exist?". Washington Post. ISSN 0190-8286. Archived from the original on October 26,


key: chroma:document
string_value:
8, 2024. ^ Sommer, Will (October 26, 2023). "Mysterious bylines appeared on a USA Today site. Did these writers exist?". Washington Post. ISSN 0190-8286. Archived from the original on October 26, 2023. Retrieved January 8, 2024. ^ a b c d e f g h i j k l m n o p q r s t u v w x y z aa ab ac "Meet AdVon, the AI-Powered Content Monster Infecting the Media Industry". Futurism. May 8, 2024. Archived from the original on June 4, 2024. Retrieved June 8, 2024. ^ O'Sullivan, Donie; Gordon, Allison (November 2, 2023). "How Microsoft's AI is making a mess of the news | CNN Business". CNN. Archived from the original on November 2, 2023. Retrieved January 8, 2024. ^ Meade, Amanda (July 31, 2023). "News Corp using AI to produce 3,000 Australian local news stories a week". The Guardian. ISSN 0261-3077. Archived from the original on December 2, 2023. Retrieved January 8, 2024. ^ Tangermann, Victor (June 30, 2023). "Gizmodo Staff Furious After Site Announces Move to AI Content". Futurism. Archived


key: chroma:document
string_value:
Archived from the original on December 2, 2023. Retrieved January 8, 2024. ^ Tangermann, Victor (June 30, 2023). "Gizmodo Staff Furious After Site Announces Move to AI Content". Futurism. Archived from the original on December 6, 2023. Retrieved January 8, 2024. ^ a b c Kafka, Peter (July 18, 2023). "Coming to your internet, whether you like it or not: More AI-generated stories". Vox. Archived from the original on July 18, 2023. Retrieved January 8, 2024. ^ Landymore, Frank; Christian, Jon (September 13, 2023). "The A.V. Club's AI-Generated Articles Are Copying Directly From IMDb". Futurism. Archived from the original on December 6, 2023. Retrieved January 8, 2024. ^ Stiaplame, Nordiisk (January 28, 2025). "Quartz Is Publishing AI-Generated Articles Based on Other AI Slop, Along With Warning They May Be Filled With Errors". Futurism. Archived from the original on January 29, 2025. Retrieved January 30, 2025. ^ Carroll, Rory (May 14, 2023). "Irish Times apologises for hoax AI article


key: chroma:document
string_value:
They May Be Filled With Errors". Futurism. Archived from the original on January 29, 2025. Retrieved January 30, 2025. ^ Carroll, Rory (May 14, 2023). "Irish Times apologises for hoax AI article about women's use of fake tan". The Guardian. ISSN 0261-3077. Archived from the original on May 14, 2023. Retrieved January 8, 2024. ^ Christian, Jon (February 1, 2023). "CNET Sister Site Restarts AI Articles, Immediately Publishes Idiotic Error". Futurism. Archived from the original on November 27, 2023. Retrieved January 8, 2024. ^ Al-Sibai, Noor; Christian, Jon (March 30, 2023). "BuzzFeed Is Quietly Publishing Entire AI-Generated Articles". Futurism. Archived from the original on December 6, 2023. Retrieved January 8, 2024. ^ "Newsweek is making generative AI a fixture in its newsroom". Nieman Lab. April 17, 2024. Archived from the original on May 15, 2024. Retrieved May 24, 2024. ^ "What's in a byline? For Hoodline's AI-generated local news, everything — and nothing". Nieman Lab. June 3,


key: chroma:document
string_value:
Lab. April 17, 2024. Archived from the original on May 15, 2024. Retrieved May 24, 2024. ^ "What's in a byline? For Hoodline's AI-generated local news, everything — and nothing". Nieman Lab. June 3, 2024. Archived from the original on June 6, 2024. Retrieved June 8, 2024. ^ "AI-generated news is here from S.F.-based Hoodline. What does that mean for conventional publishers?". San Francisco Chronicle. May 8, 2024. Archived from the original on June 5, 2024. Retrieved June 7, 2024. ^ Gold, Hadas (May 30, 2024). "A national network of local news sites is publishing AI-written articles under fake bylines. Experts are raising alarm". CNN. Archived from the original on June 6, 2024. Retrieved June 8, 2024. ^ "Wyoming reporter caught using artificial intelligence to create fake quotes and stories". Associated Press. August 14, 2024. Archived from the original on August 24, 2024. Retrieved August 27, 2024. ^ "Cosmos Magazine publishes AI-generated articles, drawing criticism from journalists,


key: chroma:document
string_value:
Associated Press. August 14, 2024. Archived from the original on August 24, 2024. Retrieved August 27, 2024. ^ "Cosmos Magazine publishes AI-generated articles, drawing criticism from journalists, co-founders". ABC News. August 7, 2024. Archived from the original on August 24, 2024. Retrieved August 27, 2024. ^ "AI-generated articles are permeating major news publications". National Public Radio. May 16, 2024. Archived from the original on June 19, 2024. Retrieved July 8, 2024. ^ a b c Knibbs, Kate (July 30, 2024). "Zombie Alt-Weeklies Are Stuffed With AI Slop About OnlyFans". Wired. Archived from the original on August 11, 2024. Retrieved August 27, 2024. ^ "Apple says it will update AI feature after inaccurate news alerts". The Guardian. January 7, 2025. Archived from the original on January 14, 2025. Retrieved January 14, 2025. ^ "TV channels are using AI-generated presenters to read the news. The question is, will we trust them?". BBC News. January 26, 2024. Archived from the


key: chroma:document
string_value:
January 14, 2025. Retrieved January 14, 2025. ^ "TV channels are using AI-generated presenters to read the news. The question is, will we trust them?". BBC News. January 26, 2024. Archived from the original on January 26, 2024. Retrieved May 24, 2024. ^ Tait, Amelia (October 20, 2023). "'Here is the news. You can't stop us': AI anchor Zae-In grants us an interview". The Guardian. ISSN 0261-3077. Archived from the original on January 28, 2024. Retrieved May 24, 2024. ^ Kuo, Lily (November 9, 2018). "World's first AI news anchor unveiled in China". The Guardian. ISSN 0261-3077. Archived from the original on February 20, 2024. Retrieved May 24, 2024. ^ "These ISIS news anchors are AI fakes. Their propaganda is real". Washington Post. May 17, 2024. Archived from the original on May 19, 2024. Retrieved May 24, 2024. ^ Mullin, Benjamin; Grant, Nico (July 20, 2023). "Google Tests A.I. Tool That Is Able to Write News Articles". The New York Times. ISSN 0362-4331. Archived from the original on


key: chroma:document
string_value:
May 24, 2024. ^ Mullin, Benjamin; Grant, Nico (July 20, 2023). "Google Tests A.I. Tool That Is Able to Write News Articles". The New York Times. ISSN 0362-4331. Archived from the original on May 16, 2024. Retrieved May 24, 2024. ^ Stenberg, Mark (February 27, 2024). "Google Is Paying Publishers Five-Figure Sums to Test an Unreleased Gen AI Platform". Adweek. Archived from the original on March 9, 2024. Retrieved April 17, 2024. ^ Knibbs, Kate (February 7, 2024). "Confessions of an AI Clickbait Kingpin". Wired. ISSN 1059-1028. Archived from the original on May 18, 2024. Retrieved May 24, 2024. ^ Knibbs, Kate (January 26, 2024). "How Beloved Indie Blog 'The Hairpin' Turned Into an AI Clickbait Farm". Wired. ISSN 1059-1028. Archived from the original on April 14, 2024. Retrieved May 24, 2024. ^ Koebler, Jason (July 9, 2024). "A Beloved Tech Blog Is Now Publishing AI Articles Under the Names of Its Old Human Staff". 404 Media. Archived from the original on July 12, 2024. Retrieved August


key: chroma:document
string_value:
^ Koebler, Jason (July 9, 2024). "A Beloved Tech Blog Is Now Publishing AI Articles Under the Names of Its Old Human Staff". 404 Media. Archived from the original on July 12, 2024. Retrieved August 27, 2024. ^ Hollister, Sean (July 10, 2024). "Early Apple tech bloggers are shocked to find their name and work have been AI-zombified". The Verge. Archived from the original on July 12, 2024. Retrieved August 27, 2024. ^ "AI slop is already invading Oregon's local journalism". Oregon Public Broadcasting. December 9, 2024. Archived from the original on December 9, 2024. Retrieved December 10, 2024. ^ Knibbs, Kate (February 26, 2024). "How a Small Iowa Newspaper's Website Became an AI-Generated Clickbait Factory". Wired. ISSN 1059-1028. Archived from the original on February 26, 2024. Retrieved December 10, 2024. ^ Koebler, Jason; Cole, Samantha; Maiberg, Emanuel; Cox, Joseph (January 26, 2024). "We Need Your Email Address". 404 Media. Archived from the original on December 2, 2024.


key: chroma:document
string_value:
December 10, 2024. ^ Koebler, Jason; Cole, Samantha; Maiberg, Emanuel; Cox, Joseph (January 26, 2024). "We Need Your Email Address". 404 Media. Archived from the original on December 2, 2024. Retrieved December 10, 2024. ^ "Meet the Serbian Businessman/DJ Who Runs the Zombie AI Southwest Journal - Racket". Racket. February 16, 2024. Archived from the original on November 13, 2024. Retrieved December 10, 2024. ^ Lima-Strong, Cristiano (January 11, 2024). "Senators warn AI could lead to 'destruction' of local news". Washington Post. ISSN 0190-8286. Archived from the original on January 11, 2024. Retrieved May 24, 2024. ^ "OpenAI strikes $5 million-plus local news deal". Axios. July 18, 2023. Archived from the original on July 19, 2023. Retrieved May 24, 2024. ^ Kelly, Heather (May 22, 2024). "Meta walked away from news. Now the company's using it for AI content". Washington Post. ISSN 0190-8286. Archived from the original on May 22, 2024. Retrieved May 24, 2024. ^ "How WIRED Will Use


key: chroma:document
string_value:
"Meta walked away from news. Now the company's using it for AI content". Washington Post. ISSN 0190-8286. Archived from the original on May 22, 2024. Retrieved May 24, 2024. ^ "How WIRED Will Use Generative AI Tools". Wired. Archived from the original on December 30, 2023. Retrieved January 8, 2024. ^ Barrett, Amanda (November 15, 2018). "Standards around generative AI". Associated Press. Archived from the original on September 23, 2023. Retrieved January 8, 2024. ^ Viner, Katharine; Bateson, Anna (June 16, 2023). "The Guardian's approach to generative AI". The Guardian. ISSN 0261-3077. Archived from the original on January 3, 2024. Retrieved January 8, 2024. ^ Becker, K. B.; Simon, F. M.; Crum, C. (2023). "Policies in parallel? A comparative study of journalistic AI policies in 52 global news organisations". pp. 8–9. doi:10.31235/osf.io/c4af9. ^ Newman, Nic; Fletcher, Richard; Robertson, Craig T.; Arguedas, Amy Ross; Nielsen, Rasmus Fleis (June 2024). "Digital News Report 2024"


key: chroma:document
string_value:
news organisations". pp. 8–9. doi:10.31235/osf.io/c4af9. ^ Newman, Nic; Fletcher, Richard; Robertson, Craig T.; Arguedas, Amy Ross; Nielsen, Rasmus Fleis (June 2024). "Digital News Report 2024" (PDF). Reuters Institute for the Study of Journalism. p. 20. doi:10.60625/risj-vy6n-4v57. Retrieved June 20, 2024.  vteGenerative AIConcepts Autoencoder Deep learning Generative adversarial network Generative pre-trained transformer Large language model Neural network Prompt engineering Retrieval-augmented generation Reinforcement learning from human feedback Self-supervised learning Transformer Variational autoencoder Vision transformer Word embedding ModelsText Claude DBRX Gemini GPT 1 2 3 J ChatGPT 4 4o o1 o3 Grok Granite Llama Mistral Large PanGu-Σ Qwen Image Aurora DALL-E Firefly Flux Ideogram Midjourney Stable Diffusion Video Dream Machine Gen-3 Alpha Hailuo AI Kling Sora Veo VideoPoet Music Udio Suno AI Companies 01.AI Alibaba Anthropic Baichuan DeepSeek ElevenLabs Google DeepMind


key: chroma:document
string_value:
Midjourney Stable Diffusion Video Dream Machine Gen-3 Alpha Hailuo AI Kling Sora Veo VideoPoet Music Udio Suno AI Companies 01.AI Alibaba Anthropic Baichuan DeepSeek ElevenLabs Google DeepMind Hugging Face Kuaishou Meta AI MiniMax Mistral AI Moonshot AI OpenAI Runway Stability AI Synthesia xAI Zhipu AI  Category Commons vteDigital artToolsHardware Computer CGI 2D graphics 2.5D 3D graphics Xerox 3D printer Software Graphic art software Fractal-generating software Animation software Forms Art game Artificial intelligence art ASCII art Computer art scene Computer music Crypto art Cyberarts Digital illustration Digital imaging Digital painting Digital photography Digital poetry Digital architecture Electronic music Evolutionary art Fractal art Generative art Generative artificial intelligence Generative music GIF art Glitch art Immersion Interactive art Internet art Motion graphics Music visualization Photograph manipulation Pixel art Render art Software art Systems art Texture mapping


key: chroma:document
string_value:
Generative music GIF art Glitch art Immersion Interactive art Internet art Motion graphics Music visualization Photograph manipulation Pixel art Render art Software art Systems art Texture mapping Virtual art Notable artists Refik Anadol Cory Arcangel Sougwen Chung Harold Cohen Char Davies Stephanie Dinkins Jake Elwes David Em Desmond Paul Henry Mario Klingemann Lynn Hershman Leeson Zachary Lieberman Margot Lovejoy Mauro Martino Eric Millikin Hamid Naderi Yeganeh Trevor Paglen Casey Reas Anna Ridler Ben Rubin (artist) Karl Sims Camille Utterback Pindar Van Arman Notable artworks Edmond de Belamy Barnsley fern Jesus Dress Up Listening Post (artwork) Remember To Rise Organizationsand conferences Artfutura Artmedia Austin Museum of Digital Art Computer Arts Society EVA Conferences Los Angeles Center for Digital Art Lumen Prize onedotzero SIGGRAPH V&A Digital Futures Retrieved from "https://en.wikipedia.org/w/index.php?title=Generative_artificial_intelligence&oldid=1273153879" Categories:


key: chroma:document
string_value:
Center for Digital Art Lumen Prize onedotzero SIGGRAPH V&A Digital Futures Retrieved from "https://en.wikipedia.org/w/index.php?title=Generative_artificial_intelligence&oldid=1273153879" Categories: Artificial neural networksDeep learningMachine learningGenerative artificial intelligence2020s in computing2023 in computing2024 in computing2025 in computingHidden categories: CS1 Japanese-language sources (ja)CS1 Italian-language sources (it)Articles with short descriptionShort description is different from WikidataUse mdy dates from October 2023Articles needing cleanup from July 2024All pages needing cleanupWikipedia list cleanup from July 2024  This page was last edited on 1 February 2025, at 01:05 (UTC). Text is available under the Creative Commons Attribution-ShareAlike 4.0 License; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.


key: chroma:document
string_value:
additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.  Privacy policy About Wikipedia Disclaimers Contact Wikipedia Code of Conduct Developers Statistics Cookie statement Mobile view    Search   Search   Toggle the table of contents  Generative artificial intelligence     45 languages  Add topic


key: chroma:document
string_value:
Chroma | 🦜️🔗 LangChain  Skip to main contentJoin us at Interrupt: The Agent AI Conference by LangChain on May 13 & 14 in San Francisco!IntegrationsAPI ReferenceMoreContributingPeopleError referenceLangSmithLangGraphLangChain HubLangChain JS/TSv0.3v0.3v0.2v0.1💬SearchProvidersAnthropicAWSGoogleHugging FaceMicrosoftOpenAIMoreProvidersAcreomActiveloop Deep LakeAerospikeAI21 LabsAimAINetworkAirbyteAirtableAlchemyAleph AlphaAlibaba CloudAnalyticDBAnnoyAnthropicAnyscaleApache Software FoundationApache DorisApifyAppleArangoDBArceeArcGISArgillaArizeArthurArxivAscendAskNewsAssemblyAIAstra DBAtlasAwaDBAWSAZLyricsBAAIBagelBagelDBBaichuanBaiduBananaBasetenBeamBeautiful SoupBibTeXBiliBiliBittensorBlackboardbookend.aiBoxBrave SearchBreebs (Open Knowledge)BrowserbaseBrowserlessByteDanceCassandraCerebrasCerebriumAIChaindeskChromaClarifaiClearMLClickHouseClickUpCloudflareClovaCnosDBCogniSwitchCohereCollege ConfidentialCometConfident AIConfluenceConneryContextCouchbaseCozeCrateDBC


key: chroma:document
string_value:
ConfidentialCometConfident AIConfluenceConneryContextCouchbaseCozeCrateDBC TransformersCTranslate2CubeDappierDashVectorDatabricksDatadog TracingDatadog LogsDataForSEODataheraldDedocDeepInfraDeepSeekDeepSparseDiffbotDingoDBDiscordDocArrayDoclingDoctranDocugamiDocusaurusDriaDropboxDSPyDuckDBDuckDuckGo SearchE2BEden AIElasticsearchElevenLabsEmbedchainEpsillaEtherscanEverly AIEverNoteExaFacebook - MetaFalkorDBFalkorDBFaunaFiddlerFigmaFireCrawlFireworks AIFlyteFMP Data (Financial Data Prep)Forefront AIFriendli AIGeopandasGitGitBookGitHubGitLabGoldenGoodfireGoogleSerper - Google Search APIGooseAIGPT4AllGradientGraphsignalGrobidGroqGutenbergHacker NewsHazy ResearchHeliconeHologresHTML to textHuaweiHugging FaceHyperbrowserIBMIEIT SystemsiFixitiFlytekIMSDbInfinispan VSInfinityInfinoIntelIuguJaguarJavelin AI GatewayJenkinsJina AIJohnsnowlabsJoplinKDB.AIKineticaKoboldAIKonkoKoNLPYKùzuLabel StudiolakeFSLanceDBLangChain Decorators


key: chroma:document
string_value:
SystemsiFixitiFlytekIMSDbInfinispan VSInfinityInfinoIntelIuguJaguarJavelin AI GatewayJenkinsJina AIJohnsnowlabsJoplinKDB.AIKineticaKoboldAIKonkoKoNLPYKùzuLabel StudiolakeFSLanceDBLangChain Decorators ✨LanternLindormLinkupLiteLLMLlamaIndexLlama.cppLlamaEdgellamafileLLMonitorLocalAILog10MariTalkMarqoMediaWikiDumpMeilisearchMemcachedMetalMicrosoftMilvusMindsDBMinimaxMistralAIMLflow AI Gateway for LLMsMLflowMLXModalModelScopeModern TreasuryMomentoMongoDBMongoDB AtlasMotherduckMotörheadMyScaleNAVERNeo4jNLPCloudNomicNotion DBNucliaNVIDIAObsidianOceanBaseOracle Cloud Infrastructure (OCI)OctoAIOllamaOntotext GraphDBOpenAIOpenLLMOpenSearchOpenWeatherMapOracleAI Vector SearchOutlineOutlinesPandasPaymanAIPebbloPerplexityPetalsPostgres EmbeddingPGVectorPineconePipelineAIPipeshiftPortkeyPredibasePrediction GuardPremAIPromptLayerPsychicPubMedPullMd LoaderPygmalionAIQdrantRAGatouillerank_bm25Ray ServeRebuffRedditRedisRemembrallReplicateRoamSema4 (fka Robocorp)RocksetRunhouseRWKV-4Salute


key: chroma:document
string_value:
GuardPremAIPromptLayerPsychicPubMedPullMd LoaderPygmalionAIQdrantRAGatouillerank_bm25Ray ServeRebuffRedditRedisRemembrallReplicateRoamSema4 (fka Robocorp)RocksetRunhouseRWKV-4Salute DevicesSambaNovaSAPScrapeGraph AISearchApiSearxNG Search APISemaDBSerpAPIShale ProtocolSingleStoreDBscikit-learnSlackSnowflakespaCySparkSparkLLMSpreedlySQLiteStack ExchangeStarRocksStochasticAIStreamlitStripeSupabase (Postgres)NebulaTairTelegramTencentTensorFlow DatasetsTiDBTigerGraphTigrisTiloresTogether AI2MarkdownTranswarpTrelloTrubricsTruLensTwitterTypesenseUnstructuredUpstageupstashUpTrainUSearchVDMSVearchVectaraVespavliteVoyageAIWeights & BiasesWeights & Biases tracingWeights & Biases trackingWeatherWeaviateWhatsAppWhyLabsWikipediaWolfram AlphaWriterxAIXataXorbits Inference (Xinference)YahooYandexYeager.aiYellowbrick01.AIYouYouTubeZepZhipu AIZillizComponentsChat modelsChat modelsAI21 LabsAlibaba Cloud PAI EASAnthropic[Deprecated] Experimental Anthropic Tools WrapperAnyscaleAzure OpenAIAzure ML


key: chroma:document
string_value:
AIZillizComponentsChat modelsChat modelsAI21 LabsAlibaba Cloud PAI EASAnthropic[Deprecated] Experimental Anthropic Tools WrapperAnyscaleAzure OpenAIAzure ML EndpointBaichuan ChatBaidu QianfanAWS BedrockCerebrasCloudflare Workers AICohereCoze ChatDappier AIDatabricksDeepInfraDeepSeekEden AIErnie Bot ChatEverlyAIFireworksChatFriendliGigaChatGoodfireGoogle AIGoogle Cloud Vertex AIGPTRouterGroqChatHuggingFaceIBM watsonx.aiJinaChatKineticaKonkoLiteLLMLiteLLM RouterLlama 2 ChatLlama APILlamaEdgeLlama.cppmaritalkMiniMaxMistralAIMLXModelScopeMoonshotNaverNVIDIA AI EndpointsChatOCIModelDeploymentOCIGenAIChatOctoAIOllamaOpenAIOutlinesPerplexityPipeshiftChatPredictionGuardPremAIPromptLayer ChatOpenAIRekaSambaNovaCloudSambaStudioSnowflake CortexsolarSparkLLM ChatNebula (Symbl.ai)Tencent HunyuanTogetherTongyi QwenUpstagevLLM ChatVolc Enging MaasWriterxAIYandexGPTChatYIYuan2.0ZHIPU AIRetrieversRetrieversActiveloop Deep MemoryAmazon KendraArceeArxivAskNewsAzure AI SearchBedrock (Knowledge


key: chroma:document
string_value:
QwenUpstagevLLM ChatVolc Enging MaasWriterxAIYandexGPTChatYIYuan2.0ZHIPU AIRetrieversRetrieversActiveloop Deep MemoryAmazon KendraArceeArxivAskNewsAzure AI SearchBedrock (Knowledge Bases)BM25BoxBREEBS (Open Knowledge)ChaindeskChatGPT pluginCohere rerankerCohere RAGDappierDocArrayDriaElasticSearch BM25ElasticsearchEmbedchainFlashRank rerankerFleet AI ContextGoogle DriveGoogle Vertex AI SearchIBM watsonx.aiJaguarDB Vector DatabaseKay.aiKinetica Vectorstore based RetrieverkNNLinkupSearchRetrieverLLMLingua Document CompressorLOTR (Merger Retriever)MetalMilvus Hybrid SearchNanoPQ (Product Quantization)needleOutlinePinecone Hybrid SearchPubMedQdrant Sparse VectorRAGatouilleRePhraseQueryRememberizerSEC filingSelf-querying retrieversSingleStoreDBSVMTavilySearchAPITF-IDF**NeuralDB**VespaWikipediaYou.comZep CloudZep Open SourceZilliz Cloud PipelineTools/ToolkitsToolsAINetwork ToolkitAlpha VantageAmadeus ToolkitArXivAskNewsAWS LambdaAzure AI Services ToolkitAzure Cognitive Services ToolkitAzure


key: chroma:document
string_value:
CloudZep Open SourceZilliz Cloud PipelineTools/ToolkitsToolsAINetwork ToolkitAlpha VantageAmadeus ToolkitArXivAskNewsAWS LambdaAzure AI Services ToolkitAzure Cognitive Services ToolkitAzure Container Apps dynamic sessionsShell (bash)Bearly Code InterpreterBing SearchBrave SearchCassandra Database ToolkitCDPChatGPT PluginsClickUp ToolkitCogniswitch ToolkitConnery Toolkit and ToolsDall-E Image GeneratorDappierDatabricks Unity Catalog (UC)DataForSEODataheraldDuckDuckGo SearchE2B Data AnalysisEden AIEleven Labs Text2SpeechExa SearchFile SystemFinancialDatasets ToolkitFMP DataGithub ToolkitGitlab ToolkitGmail ToolkitGolden QueryGoogle BooksGoogle Cloud Text-to-SpeechGoogle DriveGoogle FinanceGoogle ImagenGoogle JobsGoogle LensGoogle PlacesGoogle ScholarGoogle SearchGoogle SerperGoogle TrendsGradioGraphQLHuggingFace Hub ToolsHuman as a toolIFTTT WebHooksInfobipIonic Shopping ToolJenkinsJina SearchJira ToolkitJSON ToolkitLemon AgentLinkupSearchToolMemorizeMojeek SearchMultiOn ToolkitNASA


key: chroma:document
string_value:
Hub ToolsHuman as a toolIFTTT WebHooksInfobipIonic Shopping ToolJenkinsJina SearchJira ToolkitJSON ToolkitLemon AgentLinkupSearchToolMemorizeMojeek SearchMultiOn ToolkitNASA ToolkitNuclia UnderstandingNVIDIA Riva: ASR and TTSOffice365 ToolkitOpenAPI ToolkitNatural Language API ToolkitsOpenWeatherMapOracle AI Vector Search: Generate SummaryPandas DataframePassio NutritionAIPaymanAIPlayWright Browser ToolkitPolygon IO Toolkit and ToolsPowerBI ToolkitPubMedPython REPLReddit SearchRequests ToolkitRiza Code InterpreterRobocorp ToolkitSceneXplainScrapeGraphSearchApiSearxNG SearchSemantic Scholar API ToolSerpAPISlack ToolkitSpark SQL ToolkitSQLDatabase ToolkitStackExchangeSteam ToolkitStripeTavily SearchTiloresTwilioUpstageWikidataWikipediaWolfram AlphaYahoo Finance NewsYou.com SearchYouTubeZapier Natural Language ActionsZenGuard AIDocument loadersDocument loadersacreomAirbyteLoaderAirbyte CDK (Deprecated)Airbyte Gong (Deprecated)Airbyte Hubspot (Deprecated)Airbyte JSON (Deprecated)Airbyte


key: chroma:document
string_value:
Natural Language ActionsZenGuard AIDocument loadersDocument loadersacreomAirbyteLoaderAirbyte CDK (Deprecated)Airbyte Gong (Deprecated)Airbyte Hubspot (Deprecated)Airbyte JSON (Deprecated)Airbyte Salesforce (Deprecated)Airbyte Shopify (Deprecated)Airbyte Stripe (Deprecated)Airbyte Typeform (Deprecated)Airbyte Zendesk Support (Deprecated)AirtableAlibaba Cloud MaxComputeAmazon TextractApify DatasetArcGISArxivLoaderAssemblyAI Audio TranscriptsAstraDBAsync ChromiumAsyncHtmlAthenaAWS S3 DirectoryAWS S3 FileAZLyricsAzure AI DataAzure Blob Storage ContainerAzure Blob Storage FileAzure AI Document IntelligenceBibTeXBiliBiliBlackboardBlockchainBoxBrave SearchBrowserbaseBrowserlessBSHTMLLoaderCassandraChatGPT DataCollege ConfidentialConcurrent LoaderConfluenceCoNLL-UCopy PasteCouchbaseCSVCube Semantic LayerDatadog LogsDedocDiffbotDiscordDoclingDocugamiDocusaurusDropboxDuckDBEmailEPubEtherscanEverNoteexample_dataFacebook ChatFaunaFigmaFireCrawlGeopandasGitGitBookGitHubGlue CatalogGoogle AlloyDB


key: chroma:document
string_value:
LayerDatadog LogsDedocDiffbotDiscordDoclingDocugamiDocusaurusDropboxDuckDBEmailEPubEtherscanEverNoteexample_dataFacebook ChatFaunaFigmaFireCrawlGeopandasGitGitBookGitHubGlue CatalogGoogle AlloyDB for PostgreSQLGoogle BigQueryGoogle BigtableGoogle Cloud SQL for SQL serverGoogle Cloud SQL for MySQLGoogle Cloud SQL for PostgreSQLGoogle Cloud Storage DirectoryGoogle Cloud Storage FileGoogle Firestore in Datastore ModeGoogle DriveGoogle El Carro for Oracle WorkloadsGoogle Firestore (Native Mode)Google Memorystore for RedisGoogle SpannerGoogle Speech-to-Text Audio TranscriptsGrobidGutenbergHacker NewsHuawei OBS DirectoryHuawei OBS FileHuggingFace datasetHyperbrowserLoaderiFixitImagesImage captionsIMSDbIuguJoplinJSONLoaderJupyter NotebookKineticalakeFSLangSmithLarkSuite (FeiShu)LLM SherpaMastodonMathPixPDFLoaderMediaWiki DumpMerge Documents LoadermhtmlMicrosoft ExcelMicrosoft OneDriveMicrosoft OneNoteMicrosoft PowerPointMicrosoft SharePointMicrosoft WordNear BlockchainModern


key: chroma:document
string_value:
SherpaMastodonMathPixPDFLoaderMediaWiki DumpMerge Documents LoadermhtmlMicrosoft ExcelMicrosoft OneDriveMicrosoft OneNoteMicrosoft PowerPointMicrosoft SharePointMicrosoft WordNear BlockchainModern TreasuryMongoDBNeedle Document LoaderNews URLNotion DB 2/2NucliaObsidianOpen Document Format (ODT)Open City DataOracle Autonomous DatabaseOracle AI Vector Search: Document ProcessingOrg-modePandas DataFrameparsersPDFMinerPDFPlumberPebblo Safe DocumentLoaderPolars DataFramePsychicPubMedPullMdLoaderPyMuPDFLoaderPyPDFDirectoryLoaderPyPDFium2LoaderPyPDFLoaderPySparkQuipReadTheDocs DocumentationRecursive URLRedditRoamRocksetrspaceRSS FeedsRSTscrapflyScrapingAntSitemapSlackSnowflakeSource CodeSpiderSpreedlyStripeSubtitleSurrealDBTelegramTencent COS DirectoryTencent COS FileTensorFlow DatasetsTiDB2MarkdownTOMLTrelloTSVTwitterUnstructuredUnstructuredMarkdownLoaderUnstructuredPDFLoaderUpstageURLVsdxWeatherWebBaseLoaderWhatsApp ChatWikipediaUnstructuredXMLLoaderXorbits Pandas DataFrameYouTube


key: chroma:document
string_value:
ChatWikipediaUnstructuredXMLLoaderXorbits Pandas DataFrameYouTube audioYouTube transcriptsYoutubeLoaderDLYuqueZeroxPDFLoaderVector storesVector storesActiveloop Deep LakeAerospikeAlibaba Cloud OpenSearchAnalyticDBAnnoyApache DorisApertureDBAstra DB Vector StoreAtlasAwaDBAzure Cosmos DB Mongo vCoreAzure Cosmos DB No SQLAzure AI SearchBagelBagelDBBaidu Cloud ElasticSearch VectorSearchBaidu VectorDBApache CassandraChromaClarifaiClickHouseCouchbaseDashVectorDatabricksDingoDBDocArray HnswSearchDocArray InMemorySearchAmazon Document DBDuckDBChina Mobile ECloud ElasticSearch VectorSearchElasticsearchEpsillaFaissFaiss (Async)FalkorDBVectorStoreGoogle AlloyDB for PostgreSQLGoogle BigQuery Vector SearchGoogle Cloud SQL for MySQLGoogle Cloud SQL for PostgreSQLFirestoreGoogle Memorystore for RedisGoogle SpannerGoogle Vertex AI Feature StoreGoogle Vertex AI Vector SearchHippoHologresInfinispanJaguar Vector DatabaseKDB.AIKineticaLanceDBLanternLindormLLMRailsManticoreSearch


key: chroma:document
string_value:
Memorystore for RedisGoogle SpannerGoogle Vertex AI Feature StoreGoogle Vertex AI Vector SearchHippoHologresInfinispanJaguar Vector DatabaseKDB.AIKineticaLanceDBLanternLindormLLMRailsManticoreSearch VectorStoreMarqoMeilisearchAmazon MemoryDBMilvusMomento Vector Index (MVI)MongoDB AtlasMyScaleNeo4j Vector IndexNucliaDBOceanbaseOpenSearchOracle AI Vector Search: Vector StorePathwayPostgres EmbeddingPGVecto.rsPGVectorPineconeQdrantRedisRelytRocksetSAP HANA Cloud Vector EngineScaNNSemaDBSingleStoreDBscikit-learnSQLiteVecSQLite-VSSSQLServerStarRocksSupabase (Postgres)SurrealDBTablestoreTairTencent Cloud VectorDBThirdAI NeuralDBTiDB VectorTigrisTileDBTimescale Vector (Postgres)TypesenseUpstash VectorUSearchValdIntel's Visual Data Management System (VDMS)VearchVectaraVespaviking DBvliteWeaviateXataYellowbrickZepZep CloudZillizEmbedding modelsEmbedding modelsAI21Aleph AlphaAnyscaleascendAwaDBAzureOpenAIBaichuan Text EmbeddingsBaidu QianfanBedrockBGE on Hugging FaceBookend AIClarifaiCloudflare


key: chroma:document
string_value:
CloudZillizEmbedding modelsEmbedding modelsAI21Aleph AlphaAnyscaleascendAwaDBAzureOpenAIBaichuan Text EmbeddingsBaidu QianfanBedrockBGE on Hugging FaceBookend AIClarifaiCloudflare Workers AIClova EmbeddingsCohereDashScopeDatabricksDeepInfraEDEN AIElasticsearchEmbaasERNIEFake EmbeddingsFastEmbed by QdrantFireworksGigaChatGoogle Generative AI EmbeddingsGoogle Vertex AIGPT4AllGradientHugging FaceIBM watsonx.aiInfinityInstruct Embeddings on Hugging FaceIPEX-LLM: Local BGE Embeddings on Intel CPUIPEX-LLM: Local BGE Embeddings on Intel GPUIntel® Extension for Transformers Quantized Text EmbeddingsJinaJohn Snow LabsLASER Language-Agnostic SEntence Representations Embeddings by Meta AILindormLlama.cppllamafileLLMRailsLocalAIMiniMaxMistralAImodel2vecModelScopeMosaicMLNaverNLP CloudNomicNVIDIA NIMsOracle Cloud Infrastructure Generative AIOllamaOpenClipOpenAIOpenVINOEmbedding Documents using Optimized and Quantized EmbeddersOracle AI Vector Search: Generate EmbeddingsOVHcloudPinecone


key: chroma:document
string_value:
NIMsOracle Cloud Infrastructure Generative AIOllamaOpenClipOpenAIOpenVINOEmbedding Documents using Optimized and Quantized EmbeddersOracle AI Vector Search: Generate EmbeddingsOVHcloudPinecone EmbeddingsPredictionGuardEmbeddingsPremAISageMakerSambaStudioSelf HostedSentence Transformers on Hugging FaceSolarSpaCySparkLLM Text EmbeddingsTensorFlow HubText Embeddings InferenceTextEmbed - Embedding Inference ServerTitan TakeoffTogether AIUpstageVolc EngineVoyage AIXorbits inference (Xinference)YandexGPTZhipuAIOtherComponentsVector storesChromaOn this pageChroma This notebook covers how to get started with the Chroma vector store. Chroma is a AI-native open-source vector database focused on developer productivity and happiness. Chroma is licensed under Apache 2.0. View the full docs of Chroma at this page, and find the API reference for the LangChain integration at this page. Setup​ To access Chroma vector stores you'll need to install the langchain-chroma integration package. pip install


key: chroma:document
string_value:
at this page, and find the API reference for the LangChain integration at this page. Setup​ To access Chroma vector stores you'll need to install the langchain-chroma integration package. pip install -qU "langchain-chroma>=0.1.2" Credentials​ You can use the Chroma vector store without any credentials, simply installing the package above is enough! If you want to get best in-class automated tracing of your model calls you can also set your LangSmith API key by uncommenting below: # os.environ["LANGSMITH_API_KEY"] = getpass.getpass("Enter your LangSmith API key: ")# os.environ["LANGSMITH_TRACING"] = "true" Initialization​ Basic Initialization​ Below is a basic initialization, including the use of a directory to save the data locally. Select embeddings model:OpenAI▾OpenAIAzureGoogleAWSHuggingFaceOllamaCohereMistralAINomicNVIDIAVoyage AIFakepip install -qU langchain-openaiimport getpassimport osif not os.environ.get("OPENAI_API_KEY"): os.environ["OPENAI_API_KEY"] = getpass.getpass("Enter


key: chroma:document
string_value:
AIFakepip install -qU langchain-openaiimport getpassimport osif not os.environ.get("OPENAI_API_KEY"): os.environ["OPENAI_API_KEY"] = getpass.getpass("Enter API key for OpenAI: ")from langchain_openai import OpenAIEmbeddingsembeddings = OpenAIEmbeddings(model="text-embedding-3-large") from langchain_chroma import Chromavector_store = Chroma(  collection_name="example_collection",  embedding_function=embeddings,  persist_directory="./chroma_langchain_db", # Where to save data locally, remove if not necessary) Initialization from client​ You can also initialize from a Chroma client, which is particularly useful if you want easier access to the underlying database. import chromadbpersistent_client = chromadb.PersistentClient()collection = persistent_client.get_or_create_collection("collection_name")collection.add(ids=["1", "2", "3"], documents=["a", "b", "c"])vector_store_from_client = Chroma(  client=persistent_client,  collection_name="collection_name",  embedding_function=embeddings,)


key: chroma:document
string_value:
"2", "3"], documents=["a", "b", "c"])vector_store_from_client = Chroma(  client=persistent_client,  collection_name="collection_name",  embedding_function=embeddings,) Manage vector store​ Once you have created your vector store, we can interact with it by adding and deleting different items. Add items to vector store​ We can add items to our vector store by using the add_documents function. from uuid import uuid4from langchain_core.documents import Documentdocument_1 = Document(  page_content="I had chocolate chip pancakes and scrambled eggs for breakfast this morning.",  metadata={"source": "tweet"},  id=1,)document_2 = Document(  page_content="The weather forecast for tomorrow is cloudy and overcast, with a high of 62 degrees.",  metadata={"source": "news"},  id=2,)document_3 = Document(  page_content="Building an exciting new project with LangChain - come check it out!",  metadata={"source": "tweet"},  id=3,)document_4 = Document(  page_content="Robbers broke into the city bank


key: chroma:document
string_value:
page_content="Building an exciting new project with LangChain - come check it out!",  metadata={"source": "tweet"},  id=3,)document_4 = Document(  page_content="Robbers broke into the city bank and stole $1 million in cash.",  metadata={"source": "news"},  id=4,)document_5 = Document(  page_content="Wow! That was an amazing movie. I can't wait to see it again.",  metadata={"source": "tweet"},  id=5,)document_6 = Document(  page_content="Is the new iPhone worth the price? Read this review to find out.",  metadata={"source": "website"},  id=6,)document_7 = Document(  page_content="The top 10 soccer players in the world right now.",  metadata={"source": "website"},  id=7,)document_8 = Document(  page_content="LangGraph is the best framework for building stateful, agentic applications!",  metadata={"source": "tweet"},  id=8,)document_9 = Document(  page_content="The stock market is down 500 points today due to fears of a recession.",  metadata={"source": "news"},  id=9,)document_10 =


key: chroma:document
string_value:
metadata={"source": "tweet"},  id=8,)document_9 = Document(  page_content="The stock market is down 500 points today due to fears of a recession.",  metadata={"source": "news"},  id=9,)document_10 = Document(  page_content="I have a bad feeling I am going to get deleted :(",  metadata={"source": "tweet"},  id=10,)documents = [  document_1,  document_2,  document_3,  document_4,  document_5,  document_6,  document_7,  document_8,  document_9,  document_10,]uuids = [str(uuid4()) for _ in range(len(documents))]vector_store.add_documents(documents=documents, ids=uuids)API Reference:Document ['f22ed484-6db3-4b76-adb1-18a777426cd6', 'e0d5bab4-6453-4511-9a37-023d9d288faa', '877d76b8-3580-4d9e-a13f-eed0fa3d134a', '26eaccab-81ce-4c0a-8e76-bf542647df18', 'bcaa8239-7986-4050-bf40-e14fb7dab997', 'cdc44b38-a83f-4e49-b249-7765b334e09d', 'a7a35354-2687-4bc2-8242-3849a4d18d34', '8780caf1-d946-4f27-a707-67d037e9e1d8', 'dec6af2a-7326-408f-893d-7d7d717dfda9', '3b18e210-bb59-47a0-8e17-c8e51176ea5e']


key: chroma:document
string_value:
'cdc44b38-a83f-4e49-b249-7765b334e09d', 'a7a35354-2687-4bc2-8242-3849a4d18d34', '8780caf1-d946-4f27-a707-67d037e9e1d8', 'dec6af2a-7326-408f-893d-7d7d717dfda9', '3b18e210-bb59-47a0-8e17-c8e51176ea5e'] Update items in vector store​ Now that we have added documents to our vector store, we can update existing documents by using the update_documents function. updated_document_1 = Document(  page_content="I had chocolate chip pancakes and fried eggs for breakfast this morning.",  metadata={"source": "tweet"},  id=1,)updated_document_2 = Document(  page_content="The weather forecast for tomorrow is sunny and warm, with a high of 82 degrees.",  metadata={"source": "news"},  id=2,)vector_store.update_document(document_id=uuids[0], document=updated_document_1)# You can also update multiple documents at oncevector_store.update_documents(  ids=uuids[:2], documents=[updated_document_1, updated_document_2]) Delete items from vector store​ We can also delete items from our vector store as follows:


key: chroma:document
string_value:
at oncevector_store.update_documents(  ids=uuids[:2], documents=[updated_document_1, updated_document_2]) Delete items from vector store​ We can also delete items from our vector store as follows: vector_store.delete(ids=uuids[-1]) Query vector store​ Once your vector store has been created and the relevant documents have been added you will most likely wish to query it during the running of your chain or agent. Query directly​ Similarity search​ Performing a simple similarity search can be done as follows: results = vector_store.similarity_search(  "LangChain provides abstractions to make working with LLMs easy",  k=2,  filter={"source": "tweet"},)for res in results:  print(f"* {res.page_content} [{res.metadata}]") * Building an exciting new project with LangChain - come check it out! [{'source': 'tweet'}]* LangGraph is the best framework for building stateful, agentic applications! [{'source': 'tweet'}] Similarity search with score​ If you want to execute a similarity search and


key: chroma:document
string_value:
[{'source': 'tweet'}]* LangGraph is the best framework for building stateful, agentic applications! [{'source': 'tweet'}] Similarity search with score​ If you want to execute a similarity search and receive the corresponding scores you can run: results = vector_store.similarity_search_with_score(  "Will it be hot tomorrow?", k=1, filter={"source": "news"})for res, score in results:  print(f"* [SIM={score:3f}] {res.page_content} [{res.metadata}]") * [SIM=1.726390] The stock market is down 500 points today due to fears of a recession. [{'source': 'news'}] Search by vector​ You can also search by vector: results = vector_store.similarity_search_by_vector(  embedding=embeddings.embed_query("I love green eggs and ham!"), k=1)for doc in results:  print(f"* {doc.page_content} [{doc.metadata}]") * I had chocalate chip pancakes and fried eggs for breakfast this morning. [{'source': 'tweet'}] Other search methods​ There are a variety of other search methods that are not covered in this


key: chroma:document
string_value:
* I had chocalate chip pancakes and fried eggs for breakfast this morning. [{'source': 'tweet'}] Other search methods​ There are a variety of other search methods that are not covered in this notebook, such as MMR search or searching by vector. For a full list of the search abilities available for AstraDBVectorStore check out the API reference. Query by turning into retriever​ You can also transform the vector store into a retriever for easier usage in your chains. For more information on the different search types and kwargs you can pass, please visit the API reference here. retriever = vector_store.as_retriever(  search_type="mmr", search_kwargs={"k": 1, "fetch_k": 5})retriever.invoke("Stealing from the bank is a crime", filter={"source": "news"}) [Document(metadata={'source': 'news'}, page_content='Robbers broke into the city bank and stole $1 million in cash.')] Usage for retrieval-augmented generation​ For guides on how to use this vector store for retrieval-augmented generation


key: chroma:document
string_value:
page_content='Robbers broke into the city bank and stole $1 million in cash.')] Usage for retrieval-augmented generation​ For guides on how to use this vector store for retrieval-augmented generation (RAG), see the following sections: Tutorials How-to: Question and answer with RAG Retrieval conceptual docs API reference​ For detailed documentation of all Chroma vector store features and configurations head to the API reference: https://python.langchain.com/api_reference/chroma/vectorstores/langchain_chroma.vectorstores.Chroma.html Related​ Vector store conceptual guide Vector store how-to guides Edit this pageWas this page helpful?PreviousApache CassandraNextClarifaiSetupCredentialsInitializationBasic InitializationInitialization from clientManage vector storeAdd items to vector storeUpdate items in vector storeDelete items from vector storeQuery vector storeQuery directlyQuery by turning into retrieverUsage for retrieval-augmented generationAPI


key: chroma:document
string_value:
storeAdd items to vector storeUpdate items in vector storeDelete items from vector storeQuery vector storeQuery directlyQuery by turning into retrieverUsage for retrieval-augmented generationAPI referenceRelatedCommunityTwitterGitHubOrganizationPythonJS/TSMoreHomepageBlogYouTubeCopyright © 2025 LangChain, Inc.


key: chroma:document
string_value:
LLM Powered Autonomous Agents | Lil'Log        Lil'Log   |  Posts Archive Search Tags FAQ emojisearch.app    LLM Powered Autonomous Agents Date: June 23, 2023 | Estimated Reading Time: 31 min | Author: Lilian Weng  Table of Contents  Agent System Overview Component One: Planning Task Decomposition Self-Reflection  Component Two: Memory Types of Memory Maximum Inner Product Search (MIPS)  Component Three: Tool Use Case Studies Scientific Discovery Agent Generative Agents Simulation Proof-of-Concept Examples  Challenges Citation References Building agents with LLM (large language model) as its core controller is a cool concept. Several proof-of-concepts demos, such as AutoGPT, GPT-Engineer and BabyAGI, serve as inspiring examples. The potentiality of LLM extends beyond generating well-written copies, stories, essays and programs; it can be framed as a powerful general problem solver. Agent System Overview# In a LLM-powered autonomous agent system, LLM functions as the agent’s brain,


key: chroma:document
string_value:
copies, stories, essays and programs; it can be framed as a powerful general problem solver. Agent System Overview# In a LLM-powered autonomous agent system, LLM functions as the agent’s brain, complemented by several key components: Planning Subgoal and decomposition: The agent breaks down large tasks into smaller, manageable subgoals, enabling efficient handling of complex tasks. Reflection and refinement: The agent can do self-criticism and self-reflection over past actions, learn from mistakes and refine them for future steps, thereby improving the quality of final results.  Memory Short-term memory: I would consider all the in-context learning (See Prompt Engineering) as utilizing short-term memory of the model to learn. Long-term memory: This provides the agent with the capability to retain and recall (infinite) information over extended periods, often by leveraging an external vector store and fast retrieval.  Tool use The agent learns to call external APIs for extra


key: chroma:document
string_value:
to retain and recall (infinite) information over extended periods, often by leveraging an external vector store and fast retrieval.  Tool use The agent learns to call external APIs for extra information that is missing from the model weights (often hard to change after pre-training), including current information, code execution capability, access to proprietary information sources and more. Fig. 1. Overview of a LLM-powered autonomous agent system. Component One: Planning# A complicated task usually involves many steps. An agent needs to know what they are and plan ahead. Task Decomposition# Chain of thought (CoT; Wei et al. 2022) has become a standard prompting technique for enhancing model performance on complex tasks. The model is instructed to “think step by step” to utilize more test-time computation to decompose hard tasks into smaller and simpler steps. CoT transforms big tasks into multiple manageable tasks and shed lights into an interpretation of the model’s thinking


key: chroma:document
string_value:
test-time computation to decompose hard tasks into smaller and simpler steps. CoT transforms big tasks into multiple manageable tasks and shed lights into an interpretation of the model’s thinking process. Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning possibilities at each step. It first decomposes the problem into multiple thought steps and generates multiple thoughts per step, creating a tree structure. The search process can be BFS (breadth-first search) or DFS (depth-first search) with each state evaluated by a classifier (via a prompt) or majority vote. Task decomposition can be done (1) by LLM with simple prompting like "Steps for XYZ.\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-specific instructions; e.g. "Write a story outline." for writing a novel, or (3) with human inputs. Another quite distinct approach, LLM+P (Liu et al. 2023), involves relying on an external classical planner to do long-horizon planning. This


key: chroma:document
string_value:
outline." for writing a novel, or (3) with human inputs. Another quite distinct approach, LLM+P (Liu et al. 2023), involves relying on an external classical planner to do long-horizon planning. This approach utilizes the Planning Domain Definition Language (PDDL) as an intermediate interface to describe the planning problem. In this process, LLM (1) translates the problem into “Problem PDDL”, then (2) requests a classical planner to generate a PDDL plan based on an existing “Domain PDDL”, and finally (3) translates the PDDL plan back into natural language. Essentially, the planning step is outsourced to an external tool, assuming the availability of domain-specific PDDL and a suitable planner which is common in certain robotic setups but not in many other domains. Self-Reflection# Self-reflection is a vital aspect that allows autonomous agents to improve iteratively by refining past action decisions and correcting previous mistakes. It plays a crucial role in real-world tasks where


key: chroma:document
string_value:
is a vital aspect that allows autonomous agents to improve iteratively by refining past action decisions and correcting previous mistakes. It plays a crucial role in real-world tasks where trial and error are inevitable. ReAct (Yao et al. 2023) integrates reasoning and acting within LLM by extending the action space to be a combination of task-specific discrete actions and the language space. The former enables LLM to interact with the environment (e.g. use Wikipedia search API), while the latter prompting LLM to generate reasoning traces in natural language. The ReAct prompt template incorporates explicit steps for LLM to think, roughly formatted as: Thought: ... Action: ... Observation: ... ... (Repeated many times) Fig. 2. Examples of reasoning trajectories for knowledge-intensive tasks (e.g. HotpotQA, FEVER) and decision-making tasks (e.g. AlfWorld Env, WebShop). (Image source: Yao et al. 2023). In both experiments on knowledge-intensive tasks and decision-making tasks, ReAct


key: chroma:document
string_value:
tasks (e.g. HotpotQA, FEVER) and decision-making tasks (e.g. AlfWorld Env, WebShop). (Image source: Yao et al. 2023). In both experiments on knowledge-intensive tasks and decision-making tasks, ReAct works better than the Act-only baseline where Thought: … step is removed. Reflexion (Shinn & Labash 2023) is a framework to equip agents with dynamic memory and self-reflection capabilities to improve reasoning skills. Reflexion has a standard RL setup, in which the reward model provides a simple binary reward and the action space follows the setup in ReAct where the task-specific action space is augmented with language to enable complex reasoning steps. After each action $a_t$, the agent computes a heuristic $h_t$ and optionally may decide to reset the environment to start a new trial depending on the self-reflection results. Fig. 3. Illustration of the Reflexion framework. (Image source: Shinn & Labash, 2023) The heuristic function determines when the trajectory is inefficient or


key: chroma:document
string_value:
depending on the self-reflection results. Fig. 3. Illustration of the Reflexion framework. (Image source: Shinn & Labash, 2023) The heuristic function determines when the trajectory is inefficient or contains hallucination and should be stopped. Inefficient planning refers to trajectories that take too long without success. Hallucination is defined as encountering a sequence of consecutive identical actions that lead to the same observation in the environment. Self-reflection is created by showing two-shot examples to LLM and each example is a pair of (failed trajectory, ideal reflection for guiding future changes in the plan). Then reflections are added into the agent’s working memory, up to three, to be used as context for querying LLM. Fig. 4. Experiments on AlfWorld Env and HotpotQA. Hallucination is a more common failure than inefficient planning in AlfWorld. (Image source: Shinn & Labash, 2023) Chain of Hindsight (CoH; Liu et al. 2023) encourages the model to improve on its own


key: chroma:document
string_value:
Hallucination is a more common failure than inefficient planning in AlfWorld. (Image source: Shinn & Labash, 2023) Chain of Hindsight (CoH; Liu et al. 2023) encourages the model to improve on its own outputs by explicitly presenting it with a sequence of past outputs, each annotated with feedback. Human feedback data is a collection of $D_h = \{(x, y_i , r_i , z_i)\}_{i=1}^n$, where $x$ is the prompt, each $y_i$ is a model completion, $r_i$ is the human rating of $y_i$, and $z_i$ is the corresponding human-provided hindsight feedback. Assume the feedback tuples are ranked by reward, $r_n \geq r_{n-1} \geq \dots \geq r_1$ The process is supervised fine-tuning where the data is a sequence in the form of $\tau_h = (x, z_i, y_i, z_j, y_j, \dots, z_n, y_n)$, where $\leq i \leq j \leq n$. The model is finetuned to only predict $y_n$ where conditioned on the sequence prefix, such that the model can self-reflect to produce better output based on the feedback sequence. The model can optionally


key: chroma:document
string_value:
is finetuned to only predict $y_n$ where conditioned on the sequence prefix, such that the model can self-reflect to produce better output based on the feedback sequence. The model can optionally receive multiple rounds of instructions with human annotators at test time. To avoid overfitting, CoH adds a regularization term to maximize the log-likelihood of the pre-training dataset. To avoid shortcutting and copying (because there are many common words in feedback sequences), they randomly mask 0% - 5% of past tokens during training. The training dataset in their experiments is a combination of WebGPT comparisons, summarization from human feedback and human preference dataset. Fig. 5. After fine-tuning with CoH, the model can follow instructions to produce outputs with incremental improvement in a sequence. (Image source: Liu et al. 2023) The idea of CoH is to present a history of sequentially improved outputs in context and train the model to take on the trend to produce better


