Martin EscardoApr 29, 2024

Why do most papers in CS conferences say "novel" repeatedly now a days?

I find this extremely irritating.

It is up to me, as the reader, to judge whether this is novel or not.

Please tell me what you have done, rather than telling me that (you think) it is novel.

Don't tell me that your results are important, or surprising, or anything like that. I will judge the importance or surprise or novelty.

It is not like if you didn't tell me it is important (if indeed it is) I wouldn't notice.

And if it is not novel, why would you be submitting it to the conference, anyway?

Show threadDan PiponiApr 29, 2024

@MartinEscardo I'll give one justification: often papers contain a lot of background material, or simply build on earlier work, and I'd like the authors to draw my attention to the bit that they think is important and novel. There may be better ways to do this than claiming novelty but there should be some way of doing it that we can all agree isn't just puffery.

The one I don't like is when someone claims something is simple. That really is for the reader, not author, to judge.

Show threadTom_Drummond

@dpiponi @MartinEscardo I teach my students to make a catalog of claims (typically the last thing on page 1) as a set of bullet points. There it can help a reviewer to know exactly what is being claimed. Including if you’re claiming something is new vs a new application of a method that already existed.

I also tell them to put forward references to the point in the paper where the claim is fulfilled.

The goal of all this is to help time-poor reviewers do their job more easily. In my experience many reviewers don’t take the time to think about whether something is novel or not (and check the literature to verify).

Apr 29, 2024 at 10:28pmWeb
Show threadTom_DrummondApr 29, 2024

@dpiponi @MartinEscardo

Separate comment is that the quality of reviewing is plummeting. submissions in machine learning have been growing at Moore’s law rates which means that 50% of the papers are being submitted by people who have been in the field for less than 2 years. Sadly, these people are also reviewers because it’s the only way that conferences can get enough reviewing resources. This means that most reviews are conducted by looking for bold numbers and most theoretical considerations fall by the wayside.

I even had a reviewer complain that we made reference to the ‘condition number of a matrix’ without explaining what that was…

Show threadTom_DrummondApr 29, 2024

@dpiponi @MartinEscardo Tertiary comment: Conferences don’t scale. The algorithm for running a conference is something like: sort all papers by quality and select top-n. But as we know sort is n log(n) and reviewing resources grow like n.

Worse, with a large pool of reviewers, increasing effort needs to be made to ensure some level of consistency, so I suspect workload really grows as \(n \sqrt{n}\).