Well, I finally got access to New Bing. In my very first search query it (as far as I can tell) made up a plot element of the game I was searching for, then injected one of its suggested search queries into my followup question and unsurprisingly did not find any results matching the made-up thing it had added. I love chatbots but it’s kind of like asking a distractible teenager to google things for you.
There’s a truism that AI only seems smart as long as you don’t know too much about the subject it’s covering, and after like 20 years I’m approaching the level of search engine mind-meld you’d find in a character from a William Gibson novel. I’m fascinated by the conversational aspect, but for actually running down complicated queries to dig up information, it feels like amateur hour.
In terms of actually providing factual answers, chatbots will almost certainly get better! But their format hides a bunch of the metadata that helps me figure out how to weigh a given piece of information quickly or explore the source text to glean stuff the search results don’t pull out. Until they get good enough to compensate for that by synthesizing data better than my brain can, it’s in sort of a “one step forward, two steps back” situation.