Five maps so far. Is someone doing this by hand, the hard way? I figured it was an AI someone programmed, but if it's an individual or small team, big respect. Very neat project.

[–] CatsPajamas@lemmy.dbzer0.com -3 points 3 days ago (9 children)

AI would probably be pretty useful for this. You'd have to assume most of the "answers" are in the abstract, so you could just build one to scrape academic texts. Use an RAG so it doesn't hallucinate, maybe. Idk if that violates some T&C nonsense that doing it by hand doesn't though.

[–] entropicdrift@lemmy.sdf.org 8 points 3 days ago (8 children)

This is a bad idea. It's extremely likely to hallucinate at one point or another no matter how many tools you equip it with, and humans will eventually miss some fully made up citation or completely misrepresented conclusion.

[–] CatsPajamas@lemmy.dbzer0.com -4 points 3 days ago* (last edited 3 days ago) (2 children)

Google RAG

There are tons of AIs that are not auto regressive LLMs

[–] entropicdrift@lemmy.sdf.org 4 points 3 days ago

I'm a professional software engineer and I've used RAG. It doesn't prevent all hallucinations. Nothing can. The "hallucinations" are a fundamental part of the LLM architecture.

[–] obsoleteacct@lemmy.zip 3 points 3 days ago (1 children)

Are the down votes because people genuinely think this is an incorrect answer, or because they dislike anything remotely pro-AI?

[–] CatsPajamas@lemmy.dbzer0.com 2 points 3 days ago (1 children)

Both probably. Thought terminating cliches and all that. The most useful tool maybe ever. Wild.

[–] entropicdrift@lemmy.sdf.org 5 points 3 days ago* (last edited 3 days ago) (2 children)

I use LLMs daily as a professional software engineer. I didn't downvote you and I'm not disengaging my thinking here. RAGs don't solve everything, and it's better not to sacrifice scientific credibility to the altar of convenience.

It's always been easier to lie quickly than to dig for the truth. AIs are not consistent, regardless of the additional appendages you give them. They have no internal consistency by their very nature.

[–] porksnort@slrpnk.net 3 points 2 days ago (1 children)

And this isn’t even really a great application for RAG. Papermaps just goes off of references and citations. Perhaps a sentiment analysis would be marginally useful, but since you need a human to verify all LLM outputs it would be a dubious time savings.

The system scores review papers very favorably and the “yes/no/maybe” conclusion is right in the abstract, usually the last sentence or two of it. This is not a prime candidate for any LLM, it’s simple database operations on srtuctured data that already exists. There’s no use case here.

[–] entropicdrift@lemmy.sdf.org 3 points 2 days ago* (last edited 2 days ago)

Perhaps a sentiment analysis would be marginally useful, but since you need a human to verify all LLM outputs it would be a dubious time savings.

Thank you, yes. That's exactly my point. You'd need a human to verify all of the outputs anyways, and these are literally machines that exclusively make text that humans find believable, so you're likely adding to the problem of humans messing stuff up moreso than speeding anything up. Being wrong fast has always been easy, so it's no help here.

[–] CatsPajamas@lemmy.dbzer0.com 1 points 2 days ago

What would the failure rate on this be? What would the rate have to be to actually matter? Literally it would just poll the abstract and spit out yes no undecided. That is in the abstract. There is very little chance of there being any hallucinations that are meaningful at a degree large enough to vary literally anything.

Have you never had it organize things or analyze sentiments? I understand if that's not your use case but this is pretty fundamentally an easy application of AI.

load more comments (5 replies)

load more comments (8 replies)