Interesting Stuff from the Internet
Ask HN: Is RAG the Future of LLMs? | Hacker News (ycombinator.com)
The Hacker News thread titled "Ask HN: Is RAG the Future of LLMs?" discusses the potential of Retrieval-Augmented Generation (RAG) in the future of large language models (LLMs). Contributors to the thread explore various aspects of RAG, including its ability to mitigate the problem of hallucinations in LLMs by incorporating real-time, domain-specific data into responses. Several users highlight the advantages of RAG in providing context-specific answers that are not solely reliant on an LLM's pre-existing knowledge.
Key points from the discussion include:
Utility of RAG: RAG is praised for its ability to integrate specific, real-time data, improving the relevance and accuracy of LLM outputs.
Cost and Complexity: Concerns are raised about the cost and complexity of implementing RAG, especially when dealing with large context windows, which can be expensive and slow.
Alternatives and Enhancements: Various alternatives and enhancements to RAG are discussed, such as fine-tuning models on specific domains and using lightweight, flexible overlays akin to Docker containers for models.
Future of RAG: There is debate about whether advancements in model context windows and computational efficiencies might reduce the need for RAG. However, many believe that RAG or similar mechanisms will continue to be relevant due to their ability to handle specific user needs and real-time data integration.
The discussion reflects a consensus that while RAG is currently a valuable tool for enhancing LLMs, its necessity and form may evolve as technology advances, particularly as models become capable of handling larger contexts more efficiently.
One of the most common concerns about AI is the risk that it takes a meaningful portion of jobs that humans currently do, leading to major economic dislocation. Often these headlines come out of economic studies that look at various job functions and estimate the impact that AI could have on these roles, and then extrapolates the resulting labor impact. What these reports generally get wrong is the analysis is done in a vacuum, explicitly ignoring the decisions that companies actually make when presented with productivity gains introduced by a new technology -- especially given the competitive nature of most industries.
The thinking generally goes that if a company could, say, be 50% more productive in a particular function, it would mean a commensurate reduction of jobs in that area. For instance, if a certain function (like engineering or sales) required 10 units of labor before, then with a 50% gain in productivity, in the future that same function would now only need ~7 units of labor. The challenge with this type of thinking is that it assumes that companies have maximized the amount of labor they wish they had for a particular function, when in reality many functions are only staffed at the level the company can afford. Further, it assumes that a company is not in a competitive field, and that the company would be complacent and happy about generating the same output as before, just with less costs. Finally, it ignores the fact that productivity gains in a market will lead to increased response from competition, which companies equally have to respond to with more productivity not necessarily more profit. Time and time again this is the type of flawed thinking that we tend to get out of broad economic studies on the labor needs in the economy.
To break this down and make it practical, I thought I'd illustrate the point with the example of an engineering function -- one that already is seeing the benefits of AI starting to roll out. The numbers will all be kept simple, but you can change almost any variable and the point will remain the same. The key to thinking through job impacts is to think through what happens a step or two *after* the productivity gain of AI is experienced.
So, imagine you're a software company that can afford to employee 10 engineers based on your current revenue. By default, those 10 engineers produce a certain amount of output of product that you then sell to customers. If you're like almost any company on the planet, the list of things your customers want from your product far exceeds your ability to deliver those features any time soon with those 10 engineers. But the challenge, again, is that you can only afford those 10 engineers at today's revenue level. So, you decide to implement AI, and the absolute best case scenario happens: each engineer becomes magically 50% more productive. Overnight, you now have the equivalent of 15 engineers working in your company, for the previous cost of 10.
Finally, you can now build the next set of things on your product roadmap that your customers have been asking for. We can't assume it will be 50% more because there are new points of friction and coordination tax that emerge as you have 15 equivalent engineers, but let's say your output goes up meaningfully. Assuming you're acting in your best interests as a company, the features you build make your product that much more compelling, which means at some point (sooner or later) they should result in an incremental gain in revenue. Let's be somewhat conservative on what impact these new features will have on your product, but let's say they generate an incremental 10% of revenue over time or keep customers retained at a 10% greater rate (roughly the same financial benefit).
Now let's assess the downstream impact. Firstly, any growth of revenue will often lead to some functions in the business growing as well to support these new customers, which will directly create new jobs. But further, the company now has to decide whether it remains satisfied with its 10 engineers that have the output of 15, or with their incremental revenue should they hire even more engineers to build the *next* set of features that will make them even more compelling to customers. Unless this company is in some rare monopoly position, they likely will want to build the next set of features even faster than the last set to grow even more quickly. This then means AI has caused the company --counterintuitively-- to hire more engineers than before, because the productivity of each engineer is much higher, allowing them to generate more return per engineer, and thus more revenue.
What's interesting is this analogy works similarly for most functions in a business. In sales, if you could make sales reps 10% more productive (i.e. they sell 10% more of your products/services for the same cost), almost every company in the world would prefer to hire even more sales reps, instead of merely banking the incremental profit. That incremental sales productivity again would lead to downstream implications, like the need to deliver more features to customers, and thus more R&D hiring! Even back-office functions that don't as directly tie to revenue growth, often are a bottleneck to growth . If you can reduce the bottleneck -- say lawyers reviewing contracts, or people processing invoices-- cycle time in businesses accelerates, which almost always lets you serve more customers faster or grow more quickly, again letting a company reinvest those dollars.
In the end, when you step out of the vacuum of just the specific productivity gain of a particular job function, and look at how the whole system will adapt and improve due to that productivity gain, a very different picture of AI's impact on jobs will emerge. Yes there will absolutely be changes to what jobs become more or less in demand in the future, but the competitive nature of companies inevitably ensures that across the whole system companies will be focused on leveraging AI to become more productive.
Language Modeling Reading List (to Start Your Paper Club) (eugeneyan.com)
Snippets from the Newsletters/ Newspapers/ Books
“The fact that millions of people share the same vices does not make these vices virtues, the fact that they share so many errors does not make the errors to be truths, and the fact that millions of people share the same mental pathology does not make these people sane.” — Erich Fromm
“Is it so bad, then, to be misunderstood? Pythagoras was misunderstood, and Socrates, and Jesus, and Luther, and Copernicus, and Galileo, and Newton, and every pure and wise spirit that ever took flesh. To be great is to be misunderstood.” - Ralph Waldo Emerson
“Whatever you do, you need courage. Whatever course you decide upon, there is always someone to tell you that you are wrong. There are always difficulties arising that tempt you to believe your critics are right. To map out a course of action and follow it to an end requires some of the same courage that a soldier needs. Peace has its victories, but it takes brave men and women to win them.” - Ralph Waldo Emerson
“Foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines. With consistency a great soul has simply nothing to do. He may as well concern himself with his shadow on the wall. Speak what you think now in hard words, and tomorrow speak what tomorrow thinks in hard words again, though it contradict every thing you said today.” - Ralph Waldo Emerson
30 Ideas from Margin of Safety (safalniveshak.com)
Here are the 30 ideas from Seth Klarman’s "Margin of Safety" as outlined in the document, each briefly described:
Value Investing isn't Easy: Emphasizes the hard work, discipline, and long-term horizon required in value investing.
Being a Value Investor: Describes value investing as a risk-averse approach often involving going against market trends.
An Investor’s Worst Enemy: Discusses how value investing can be disadvantageous in rising markets but benefits investors when the market falls.
It’s All About the Mindset: This book stresses the importance of having the right mindset for investing, focusing on investment fundamentals over speculation.
Don’t Seek Mr. Market’s Advice: Warns against using market price movements to guide investment decisions.
Stock Price vs. Business Reality highlights the need to differentiate between a company's stock price movements and its actual business performance.
Price vs. Value urges investors to focus on the relationship between price and value, not just price alone.
Emotions Play Havoc in Investing: Notes how emotional reactions can lead to poor investment decisions.
Stock Market ≠ Quick Money: Criticizes the view of the stock market as a source of quick, easy money.
Stock Market Cycles: Points out that markets are cyclical and that fads in investment come and go.
How Big Investors Misbehave and Why They Underperform: Discusses the pitfalls of following institutional investors who often focus on short-term gains.
Short-Term, Relative-Performance Derby: Critiques the focus on short-term performance that prevails among institutional investors.
First, Avoid Losses: Emphasizes the importance of avoiding losses as a primary investment goal.
Relevance of Temporary Price Fluctuations: Discusses how temporary market price fluctuations should not be confused with real investment risks.
Reasonable & Consistent Returns > Spectacular & Volatile Returns: Argues that consistent, moderate returns are preferable to volatile, sometimes spectacular returns.
Prepare for the Worst: Advises preparing for adverse conditions to ensure survival and success in investing.
Focus on Process, Not the Outcome: Encourages focusing on a disciplined investment process rather than specific return goals.
Wait for the Right Pitch: Uses a baseball analogy to describe the disciplined approach of waiting for the right investment opportunity.
Complexity of Business Valuation: Acknowledges the challenges and uncertainties involved in valuing businesses.
Expecting Precision in Valuation: Warns against expecting too much precision in investment valuation, highlighting its inherent imprecision.
Why Margin of Safety: Explains the concept of 'margin of safety' and its importance in protecting against errors and unforeseen issues.
How Much Margin of Safety: Discusses the individual nature of deciding the appropriate margin of safety based on personal risk tolerance.
Three Elements of Value Investing: Outlines the three key elements: bottom-up analysis, absolute-performance orientation, and focusing on risk and return.
Overpaying for Growth: Warns against the risks associated with paying too much for projected growth.
Shenanigans of Growth: Critiques the overly optimistic assumptions that can lead investors to overvalue growth prospects.
Conservatism and Growth Investing: Advises on the need for conservative estimates in growth projections.
How Much Research and Analysis Are Sufficient?: Discusses the balance between thorough research and the diminishing returns of excessive information gathering.
Value Investing and Contrarian Thinking: Describes value investing as inherently contrarian, often going against popular market trends.
Fallacy of Indexing: Criticizes indexing as a passive investment strategy that may lead to inefficiencies and missed opportunities.
It’s a Dangerous Place: Reminds investors to be wary of Wall Street’s short-term focus and self-interest.
Framework to assess your friendships:
Leaves, Branches, & Roots
Actor Tyler Perry—portraying his wise Madea alter ego—once shared a brilliant framing for thinking about your relationships.
There are three types of people in your life:
Leaves: These are the people that are only around from time to time when the weather is good. They blow around as the winds change. They provide shade during the summer, but as soon as winter comes, they fall off the tree and disappear.
Branches: These are the people who are more present and stable than the leaves, but they aren't permanent. They look strong, but if you try to stand on them or pull yourself up from them, they may break under your weight.
Roots: These are the people who are permanent. They are deep and wide. They are there in the summer and the winter, they are unperturbed by the changing seasons. The leaves and branches may come and go, but the roots are there forever.
“Imagine an onion,” he tells me. “The center is the people. The layer outside is the product. The external layer is the process. The process makes sense when you have a product, and it’s only through the motivation of the people that you make a product. This is the way I see the company. And if you see the company this way, you have to start with the people.”
“Suppose that on January 31, 2006, you knew with certainty that the price of gold would more than triple over the next five years, from $569 per ounce to $1,900 per ounce. Knowing that, might you have invested in Newmont Mining, the world’s second-largest gold producer? Odds are high that you would have gladly bought Newmont Shares. What that kind of surge in the price of gold, buying in one of the world’s largest gold companies would seem like a no-brainer. Yet, the share price of Newmont fell 5% over that time.”
Frank Slootman on the difference between skills and behaviors:
“Performance is something that we will give more time; behavior we won’t. And that’s because behavior is a choice, not a skill set. When you come in as a new leader, everybody’s watching not just what you’re doing but [also] what you’re not doing. So if you’re not moving on things that everybody is seeing, your leadership brand is already in question because apparently you’re blind and apparently you’re hesitating or you’re tolerant of behavior that you shouldn’t be tolerant of."
Billy Oppenheimer
Character is Fate
There is an old Greek saying: Character is fate. “Your character is creating what happens to you in life,” Robert Greene elaborates. And other people’s character creates what happens to them, “so you want to find people who have a strong character to associate with.” If there’s a choice to make, go humanity, character, a good past, over talent, every time.