Notes on AI, search, and building software.https://joanfihu.com/https://joanfihu.com/can-llms-find-knowledge-gaps-in-the-internet/https://joanfihu.com/can-llms-find-knowledge-gaps-in-the-internet/🚀 Excited to share new work on “ Harnessing Retrieval-Augmented Generation (RAG) for Uncovering Knowledge Gaps “. In this paper, I simulate how users search the Internet but instea…Thu, 21 Dec 2023 00:00:00 GMThttps://joanfihu.com/colpali-a-breakthrough-in-efficient-document-retrieval/https://joanfihu.com/colpali-a-breakthrough-in-efficient-document-retrieval/Introduction to ColPali ColPali is a novel document retrieval model that significantly enhances the efficiency and accuracy of matching user queries to relevant documents. It lever…Mon, 15 Jul 2024 00:00:00 GMThttps://joanfihu.com/constraining-the-ai-agent-action-space-increases-reliability/https://joanfihu.com/constraining-the-ai-agent-action-space-increases-reliability/Working with LLMs is shifting from human-machine interactions to human-machine and machine-machine interactions. This allows LLMs to do ever more complex tasks. This new interactiv…Mon, 22 Jan 2024 00:00:00 GMThttps://joanfihu.com/data-quality-over-capacity-internalizing-documents-into-lora-adapters-for-closed-book-qa/https://joanfihu.com/data-quality-over-capacity-internalizing-documents-into-lora-adapters-for-closed-book-qa/[New paper] https://arxiv.org/abs/2607.21861Mon, 27 Jul 2026 00:00:00 GMThttps://joanfihu.com/document-chunking-for-rag-retrieval-augmented-generation/https://joanfihu.com/document-chunking-for-rag-retrieval-augmented-generation/LLMs have a limited input they can generate and output. In retrieval augmented generation (RAG) applications, a set of documents is first retrieved and added to the input alongside…Sat, 13 Jan 2024 00:00:00 GMThttps://joanfihu.com/evaluating-adepts-fuyu-model-for-ui-navigation/https://joanfihu.com/evaluating-adepts-fuyu-model-for-ui-navigation/Adept's mission to enable computers to interact with UIs will enhance our productivity and save time. I have been eagerly awaiting their ACT-1 model for quite some time. However, w…Thu, 19 Oct 2023 00:00:00 GMThttps://joanfihu.com/evolution-of-media-from-ancient-communications-to-ai-generated-content/https://joanfihu.com/evolution-of-media-from-ancient-communications-to-ai-generated-content/Ancient Media Sharing stories and messages with an audience has been a fundamental human need since ancient times. From smoke signals and hieroglyphs to theatre acts and folklore s…Mon, 07 Oct 2024 00:00:00 GMThttps://joanfihu.com/introduction-to-building-llm-applications/https://joanfihu.com/introduction-to-building-llm-applications/I've put together this presentation to help anyone with a technical background to build their own LLM powered applications. The topic is quite broad but it covers all the basic mov…Fri, 17 May 2024 00:00:00 GMThttps://joanfihu.com/kaggle-docs-qa-with-gemma-data-collection-dataset-generation-fine-tuning-rag-colbert-re-ranker-evaluation/https://joanfihu.com/kaggle-docs-qa-with-gemma-data-collection-dataset-generation-fine-tuning-rag-colbert-re-ranker-evaluation/Sharing my my notebook for the Google AI Assistants For Data Task With Gemma competition. The notebook (link at the end) covers the basic building blocks to adapt LLMs for your own…Mon, 01 Apr 2024 00:00:00 GMThttps://joanfihu.com/memgpt-enhancing-llm-capabilities-through-operating-system-principles/https://joanfihu.com/memgpt-enhancing-llm-capabilities-through-operating-system-principles/Large Language Models (LLMs) have transformed artificial intelligence, leading to significant advancements in conversational AI and applications. However, these models face limitat…Wed, 17 Jul 2024 00:00:00 GMThttps://joanfihu.com/simplifying-language-image-pre-training-with-sigmoid-loss/https://joanfihu.com/simplifying-language-image-pre-training-with-sigmoid-loss/Introduction to Language-Image Pre-Training Language-Image Pre-Training (LIP) has become a popular approach to obtain robust visual and textual representations. It involves alignin…Wed, 17 Jul 2024 00:00:00 GMThttps://joanfihu.com/software-engineering-has-expanded-choosing-between-rules-prompts-chains-and-agents/https://joanfihu.com/software-engineering-has-expanded-choosing-between-rules-prompts-chains-and-agents/Software engineering has expanded to solve new kinds of problems that weren’t practical before. There are 4 ways to solve problems , depending on how certain we are about the path …Thu, 01 Jan 2026 00:00:00 GMThttps://joanfihu.com/the-generative-internet/https://joanfihu.com/the-generative-internet/Generative AI is opening the doors to new products. It's great to witness in real time how the Internet is evolving and to imagine what it might become in the coming years. Search,…Wed, 14 Feb 2024 00:00:00 GMThttps://joanfihu.com/the-platform-for-the-universal-personal-assistant/https://joanfihu.com/the-platform-for-the-universal-personal-assistant/The quest to build the universal personal assistant (UPA) has been reignited, with both OpenAI and Google going head to head. But what would the ultimate personal assistant look li…Sat, 25 Jan 2025 00:00:00 GMThttps://joanfihu.com/vqa-and-image-chunking-for-gpt-4v-and-gemini/https://joanfihu.com/vqa-and-image-chunking-for-gpt-4v-and-gemini/MLLMs (Multi Modal Large Language Models) such as GPT-4V and Gemini are able to ingest data in multiple modalities such as: text, video, sound and images. Personally, one of the mo…Tue, 20 Feb 2024 00:00:00 GMT