Tom Stafford

It's past 9pm on a Wednesday and I'm ready to try AI/ML powered academic literature search. I've got a very specific question which may have been answered somewhere in the voluminous scholarly literature. What shall I do? Just hit up chatGPT?

Suggestions welcome

#AcademicChatter #Research

1/

May 29, 2024 at 8:41pmWeb
Show threadTom StaffordMay 30, 2024

Update: I've tried

elicit.com - I like the autogenerated summaries of papers, but didn't find it helped me navigate the graph of papers (maybe my failing to discover the functionality)

researchrabbit.ai - which I like a lot, for exploring the literature using specific papers as a jumping off point. The functions are things I'd do (e..g with google scholar) like reading the references, looking at citing papers, but interface makes this much easier

2/

Show threadTom StaffordMay 30, 2024

3/

chatGPT4o - which was okay for a conversational introduction to the topic I was interested in, and the answers were reasonable if not particularly insightful, but having to check that the references weren't hallucinations was a drag (they were all real as far as I could tell)

Overall winner: researchrabbit.ai Something close to the raw material (the Universe of papers), but which adds an intelligent, usable interface

Show threadDebora Weber-WulffMay 30, 2024
@tomstafford
Do watch out for Research Rabbit, however. It also makes up things and is not always on the top of its game. It is based on Semanticscholar, which is in itself flawed. It cannot differentiate authors with the same name, pulls too many "similar" papers that are not actually similar, and only works if you give it references that are then of course tagged by the name of the collection you are creating. The interface is okay, large graphs unreadable though.
Show threadJames KingMay 30, 2024
@tomstafford not heard of researchrabbit.ai - will check it out, thanks! I like the horizontal workflow they have in the video.
Show threadBen WaberMay 29, 2024
@tomstafford Google Scholar is really your only bet. Using an LLM is probably worse than a waste of your time
Show threadTom StaffordMay 29, 2024

@bwaber so much for my Wednesday night desire for novelty!

Part of my mental model for LLMs is that they are good (best) at fuzzy matching, so maybe they would help me find relevant literature for my poorly articulated research question

Even the right keywords would be some progress

Show threadBen WaberMay 29, 2024
@tomstafford They can't really do this though, much more likely to point you to things that don't exist, aren't rigorous work, etc. You'll spend tons of time chasing your tail, better off just talking to some researchers in this vague area
Show threadTom StaffordMay 29, 2024

@bwaber but those researchers aren't necessarily here right now!

LLMs are the biro of the cognitive world - cheap & nasty, but have a certain utility

Anyway, chatGPT recommended some sensible things, but I the most relevant book is not in my university library so I'm stuck at your suggestion - I think I need to find some researchers in the area (another day!)

Show threadquantixedMay 29, 2024
@tomstafford I've tried doing this (using GPT-4 in Copilot app) and the results were disappointing. It reported back something that sounded plausible, with references, but none of them checked out.
Show threadTom StaffordMay 29, 2024

@quantixed just tried with chatGPT4o. Got some ballpark references (which most (all?) were real) but nothing super useful....

Will push it and see where I get...

Show threadMarek McGannMay 29, 2024

@tomstafford @quantixed It's probably already on your radar, and is probably more effort than your looking for for a casual quick search, but I've been curious about Research Rabbit, whether it makes that first dive into a new literature a bit easier.

https://www.researchrabbit.ai/

I'm pretty cynical about most ML claims though, so haven't bothered yet myself.

ResearchRabbit: AI Tool for Smarter, Faster Literature Reviews

Save hours on your literature review. Use ResearchRabbit to find related papers, build citation maps, and track research trends — powered by AI.

Show threadTom StaffordMay 30, 2024
@MarekMcGann @quantixed thanks for the encouragement. I was put off by the sign up, but using it this morning it seems pretty nifty. Probably would use again
Show threadDebora Weber-WulffMay 29, 2024
@tomstafford
Ask a librarian! They know all the coolplaces to search and how to formulate queries!
Show threadTom StaffordMay 30, 2024
@WiseWoman always good advice, thanks
Show threadJames KingMay 29, 2024
@tomstafford have you tried elicit?
Show threadTom StaffordMay 30, 2024
@jamesking no! But only because I forgot it existed - thanks for the suggestion. I'll have a look now
Show threadOliver D. ReithmaierMay 30, 2024
@tomstafford for graphs there's scite.ai and connectedpapers (domain eludes me). You'd need a subscription for both to work efficiently. The results are good for me, but might depend on your field.