Joan's Tech Stuff

  • 🚧 Constraining AI agents increases reliability

    Working with LLMs is shifting from human-machine interactions to human-machine and machine-machine interactions. This allows LLMs to do ever more complex tasks. This new interactivity has been coined as AI agent. Threaded conversations lack structure to complete complex tasks. Therefore, objective divergence is a common issue with AI agents. Objective divergence is the equivalent of

    Read more

  • Document Chunking for RAG (Retrieval Augmented Generation)

    LLMs have a limited input they can generate and output. In retrieval augmented generation (RAG) applications, a set of documents is first retrieved and added to the input alongside an instruction, thus creating the prompt. This is referred to as in-context learning. We can draw an analogy with computer architectures here: the LLM is the

    Read more

  • 📖 Can LLMs find knowledge gaps in the Internet?

    🚀 Excited to share new work on “Harnessing Retrieval-Augmented Generation (RAG) for Uncovering Knowledge Gaps“. In this paper, I simulate how users search the Internet but instead of searching for content that exists through traditional information retrieval methods, we search for the most relevant content, even if it doesn’t exist.  Therefore, information retrieval shifts from

    Read more

  • How can Adept’s FUYU be used for UI Navigation?

    Adept’s mission to enable computers to interact with UIs will enhance our productivity and save time. I have been eagerly awaiting their ACT-1 model for quite some time. However, while that is being developed, they have released FUYU, a multimodal LLM or MLMM. The term ‘multimodal’ implies its ability to process both text and images.

    Read more