I enjoyed this essay on X about RAG and learned a lot. 

Part of LLMs advancing is a technology called RAG (retrieval augmentation generation). RAG is an evolving technology in the world of information retrieval for LLMs but it has problems. 

1. For large models, freshness of source data for time sensitive queries is a limiter right now. A LLM needs to update it’s data source and put it back through the AI model and know to site the new source when a user writes a prompt or query. 
2. RAGs has a similar problem for smaller models too. Amongst a corpus of internal documents with perhaps conflicting information because there are drafts how does the RAG know which once to use? 

I think one important quote from Aaron that ties to a post from Google is this: “The AI’s answer is only as good as the underlying information that you serve it in the prompt.” Google’s Liz Reid wrote: “With our custom Gemini model’s multi-step reasoning capabilities, AI Overviews will help with increasingly complex questions. Rather than breaking your question into multiple searches, you can ask your most complex questions, with all the nuances and caveats you have in mind, all in one go.” 

This is something we started talking about 12+ months ago at this point where the better the prompt the better the answer. It’s clear search engines and LLMs are trying to nudge humans to change search behavior in this way to improve answers and reduce hallucinations. 

The question on my mind is, will consumers change their search behavior to more complex searches or will tech companies need to find other ways to improve source identification during RAG to improve output? 

One hard problem with AI right now is retrieval augmented generation (RAG) with wide-ranging heterogeneous information.  A common architecture pattern in AI right now is that you connect up a large amount of data to an AI model, and when a user or machine sends in a query, you…

— Aaron Levie (@levie) May 24, 2024

I find it human nature to chase the shiny new thing. Or make something overly complex and over-engineer it. I was reminded last week that sometimes the least sexiest thing is most useful. In my case, it was a flat priority list of projects for Q2. From highest to lowest priority, I was meeting with the Engineering Manager and Product Manager on my team to discuss the prior and upcoming sprint. It wasn’t a fancy strategy image or something complicated in Jira. 

Our discussion was effective because we were able to see and discuss accomplishments & learning, where we’re going next, and what is delayed, which is exactly what you need to find ways to discuss throughout a quarter. 

It was a simple flat priority list of ~15 projects in buckets from P1, P2, and P3. Sometimes the least sexiest thing is most useful.

It is good to know I am not alone. Recently, I have read or listened to strong thinkers in their respective domains question if AI can be used in their day to day. I too have had these observations in my own day to day. To be really clear, I see the power of it and how it can be used to solve certain problems, mainly jumping steps from start to finish. 

That said, in my day to day as an SEO leader, I am struggling to adopt it as a power user. Despite this, I remain open and excited to the new technology so that I can adapt my skills and experience to stay relevant in the field of SEO. Here’s a few tactical ways I’ve been reading about using AI:

• 10 AI Prompts to Level Up Your Product Work
• How The Best AI Products Grow | Brian Balfour (Reforge)

Here’s a few AI ideas I am most excited about: 

Generative AI and Retrieval Augmentation Generation (RAG) have a chance to completely change the point and click experience that the graphical user interface (GUI) brought us. The GUI was an unlock in technology that opened up computers to more industries. Prior to that, workers in certain industries couldn’t see value in computers.

My Take: I totally see how this concept applies to desktop software applications specifically. I’d love to chat with software versus trying to figure out how to use it with a series of clicks and sometimes waiting each click. The best use case I can think of is chatting with something that is hooked into data I need to get at. So instead of knowing SQL or point and click steps in Tableau or Looker, I can tell the computer what I want. This is great and something I’d use. My thought is this use case is very specific to desktop experiences and not mobile. The tap, type, scroll, read experience on mobile is totally different. I am not entirely convinced that AI chat is integral to the mobile experience as so much of mobile use is browsing articles, social media, and search. Maybe it does catch on via mobile or simply is complementary to existing search, social, and other products? 

The best way to familiarize yourself with Generative AI is to identify a problem you have and try solving it with AI. That is where the magic will be felt.

My Take: About 6 months ago, I came across this experience. There was something I was dying to do for about a year but was trying to solve it with someone who had expertise in a certain area. The problem was I could not get the internal or external help. Enter AI. I began trying to solve the problem myself with AI and it proved to be useful. I compared against other LLMs and learned about each and then shopped it around to others on my team who supported the use case. We then worked to productionalize it. This felt to me like magic and the use case fit AI perfectly.  

I remain excited and interested to see where this new technology takes consumer behavior. 

I found this image provided by the UK’s Competition and Markets Authority (CMA) very helpful for understanding the current LLM value chain. In the image they use FM which stands for Foundation Models. Specifically you can see where the big tech incumbents are currently playing. Some interesting pieces to me are:

• The FM release will be how other enterprise and start up companies are able to leverage these FMs at scale for proprietary data. Perhaps startup FMs will run in tandem with these as well. I am excited about innovation here because it opens up the door for a lot of improvements to SEO and CRO executional improvements and changes. 
• It will be interesting to follow how the bottom row continues to play out since that is where the end users. I am excited to see how AI makes its way into productivity software like Gsuite, Android, Microsoft Office, or  iOS. 
• Notice where Apple is playing? They’re developing models and with its mobile ecosystem of users has a chance at shaking things up. 
• When people talk about GenAI favoring incumbents, this image makes that clear.