Learnings & Thoughts on RAG

Posted on by Brendan

I enjoyed this essay on X about RAG and learned a lot. 

Part of LLMs advancing is a technology called RAG (retrieval augmentation generation). RAG is an evolving technology in the world of information retrieval for LLMs but it has problems. 

  1. For large models, freshness of source data for time sensitive queries is a limiter right now. A LLM needs to update it’s data source and put it back through the AI model and know to site the new source when a user writes a prompt or query. 
  2. RAGs has a similar problem for smaller models too. Amongst a corpus of internal documents with perhaps conflicting information because there are drafts how does the RAG know which once to use? 

I think one important quote from Aaron that ties to a post from Google is this: “The AI’s answer is only as good as the underlying information that you serve it in the prompt.” Google’s Liz Reid wrote: “With our custom Gemini model’s multi-step reasoning capabilities, AI Overviews will help with increasingly complex questions. Rather than breaking your question into multiple searches, you can ask your most complex questions, with all the nuances and caveats you have in mind, all in one go.” 

This is something we started talking about 12+ months ago at this point where the better the prompt the better the answer. It’s clear search engines and LLMs are trying to nudge humans to change search behavior in this way to improve answers and reduce hallucinations. 

The question on my mind is, will consumers change their search behavior to more complex searches or will tech companies need to find other ways to improve source identification during RAG to improve output? 

Mortgage Interest Rates: Existing vs New

Posted on by Brendan

It feels like the talk of interest rates and how high they are is everywhere. I don’t think I go a day without reading or hearing someone mention how high rates are. I do it myself as well. As a homeowner, I talk and think about the rate I got versus the rate I would get today. I found this chart put together by Axios with info from NYT and Federal Housing Finance Agency puts into perspective why it is such a prominent topic. Dating back to 1998, the gap has never been as wide as it is now.  

When I shared this chart with someone who has bought homes since the 80s, I learned more about this delta further back in time. He told me about an assumable mortgage he used to finance a house in 1981. The market mortgage rate he said was 18% and he picked up the seller’s existing mortgage at 9% which is a wider delta than we have today. This gives me hope that America can get through this. We’ve done it before.

existing and new mortgage rates since '98

The Least Sexiest Thing

Posted on by Brendan

I find it human nature to chase the shiny new thing. Or make something overly complex and over-engineer it. I was reminded last week that sometimes the least sexiest thing is most useful. In my case, it was a flat priority list of projects for Q2. From highest to lowest priority, I was meeting with the Engineering Manager and Product Manager on my team to discuss the prior and upcoming sprint. It wasn’t a fancy strategy image or something complicated in Jira. 

Our discussion was effective because we were able to see and discuss accomplishments & learning, where we’re going next, and what is delayed, which is exactly what you need to find ways to discuss throughout a quarter. 

It was a simple flat priority list of ~15 projects in buckets from P1, P2, and P3. Sometimes the least sexiest thing is most useful.

Weekly Listen #3

Posted on by Brendan

A listen I found valuable this week is a podcast from Cal Newport. He goes through the history of productivity over the last 20 years with important themes and popular books.

One book he talks about is Steven Covey’s 7 Habits of Highly Effective People. I remember I had a hockey coach teach me about this book in the early 2000s. I took notes on each as he walked through them. I was an early teenager at the time and thinking back I remember enjoying the classroom part of the hockey camp where I learned about this. It was good to get the mind going as well as the body physically moving.

Also posting the Spotify link here.

Comprehending Pitches

Posted on by Brendan

As my career progresses, I seem to get more outside pitches. This is obviously common for most people. As one becomes either more of a decision maker or someone who influences a decision, people responsible for sales will target you with emails and try to get you on the phone. 

As a senior in-house SEO professional, I’ve learned the importance of taking information from outside company walls to develop an outside-in perspective to SEO strategy. When it comes to a software or tool vendor, it can be tricky to digest if you need exactly what they’re selling because their pitch is often a solution to a problem they’re assuming you have. You might not have that problem. 

A few weeks ago, I took a call to understand more about a product. The rep jumped right to the problem and solution their company thinks is the one that resonates with people in my role. In reality, I don’t have that problem and don’t need that solution. So I moved on and decided for now, I don’t need this company to help reach my goals. 

I get why software companies lean this way. As it scales, a company needs to standardize and have streamlined pitches and processes so maybe it isn’t so much that reps should move away from having the problem / solution ready but that reps need to know when their problem or solution needs to be refined since it isn’t resonating. Or put together more evidence to support why that person has your problem.

Whatever side you might be on, I believe it is important to stay on your toes. If you lead in-house, know what problems you need help with. If you sell, listen and be open to hearing new problems. Not just the one your company is convinced is the most important one to solve.