Note: Not a strictly Elixir question but relates to us as well.
A year or so ago I asked Grok (or ChatGPT, can’t remember) what would happen when itself and other models lose access to up to date information due to the diminishing incentive for such sources of information to continue to exist, or exist outside of a paywall (e.g. websites, forums, ..)?
It didn’t have a finite answer. Actually, it asked me back the same question.
To what extent have your personal habits changed when looking for answers on the web? Do you still Google or do you just Grok/ChatGPT, ..? Yes, Google added their own LLM in the top left corner, but it changes nothing because the effect is the same if the user is no longer looking at the offered search results (or ads).
Do you share the view that this may substantially affect the incentives for sites to provide access to new information free of charge (i.e. free for access to the LLM scrapers)?
It sure would especially for content that is publicly available on the internet where the main source of monetisation is through advertising and the main source of traffic source is search engines. Because monetisation required eye balls which would most likely move to using LLMs and less traffic source as search engines are already incorporating those LLMs so it is a double whammy !
Just moving it behind a paywall doesn’t solve anything because a single paying user can end up scraping all the content as well if there are no other limiting measures in place.
It’s a real breath of fresh air to see how good web search can be when you’re not the product — when you’re paying for it. I think that LLMs are basically the same thing. Amazing web search without the ads.
I have no intention of dropping my Kagi subscription, although I now also subscribe to ChatGPT and Claude.
To give you a perspective on “losing access to information”, my startup has 500,000+ pages of accurate legal content. I’m working on making it easy for AI to access. Am I stupid or naive? Maybe, time will tell. But bottom line, some content producers are happy to share with AI.
Not sure I get your point here. If a user is identified as a paying customer accepting then violating the TOS, it’s easy to sue or even fine them by charging their credit card if it’s the provider’s policy.
Also, I wasn’t referring to individual bloggers but the sites they blog on.
So your business model is to create content which you then encourage LLMs to learn from without any compensation while the users don’t even need to know where the knowledge came from let alone visit your site?
So let’s assume open ai uses a single account to scrape all the hosted sites.
You then want to charge their credit card 100’000 x some amount ? Come on…
My point is that whatever license you can dream up, the big ones will simply ignore it because it’s too profitable for them to do so, and sueing them is borderline impossible as you are going up against Goliath.
It’s a bit like copyright or having a patent. Yeah, you can have it on something but unless you have the means to actually enforce it, it’s meaningless.
Maybe I’m a special case because I’m publishing legal info. It and medical advice are the two areas that LLMs tend to shy away from. (For liability reasons, I’d guess.)
My monetization strategy is:
Ad-driven public content - always up-to-date access to the law
Subscriptions for legal researchers - hide ads, other features the general public wouldn’t care about.
API access for other content publishers.
So, I dunno, when I see AI give an answer, there’s usually links to the sources. So I’m kind of expecting those to bring traffic.
But yeah, this being legal content, it’s kind of its own walled garden, by nature.
