Why do most papers in CS conferences say "novel" repeatedly now a days?

I find this extremely irritating.

It is up to me, as the reader, to judge whether this is novel or not.

Please tell me what you have done, rather than telling me that (you think) it is novel.

Don't tell me that your results are important, or surprising, or anything like that. I will judge the importance or surprise or novelty.

It is not like if you didn't tell me it is important (if indeed it is) I wouldn't notice.

And if it is not novel, why would you be submitting it to the conference, anyway?

I will delete this post tomorrow morning. For the moment, I will keep it in protest for the irritation this causes to me.

I said "I will delete this post tomorrow morning. For the moment, I will keep it in protest for the irritation this causes to me."

Should I keep my promise?

Yes
21.7%
No
78.3%
Poll ended at .
Regardless of the result of this poll, I won't be able to keep the promise, because "tomorrow morning" has already passed.

@MartinEscardo

if it's for the reviewer, there's no reason it needs to be in the published version

if the reviewer thinks it should remain in the published version, it says something darker about the whole ecosystem

@MartinEscardo No - because you have a good point - “novel” is a rather overused adjective. And I’ve enjoyed the ensuing conversation…
It's fine to say, in your paper, that (in your view) *somebody else's* results are interesting, novel, surprising, important, etc. But such statements about your own work carry no weight.

"But such statements about your own work carry no weight."

I would say more: they make your paper seem suspicious.

@MartinEscardo There is a great deal of protesting too much, to be sure. And papers are not where the worst of it is to be found.

The other extreme, "My monomania has yielded theorems. Why you should value them is none of my business.", is also problematic, but cheaper.

@pigworker

It is perfectly possible, and this is what most people do, to write a paper to induce people to believe it is novel, important, surprising and interesting.

These are, indeed, the reasons why a paper is written and submitted and refereed and published and read.

A paper doesn't become important because the author says it is important. Or novel. Or surprising. Or interesting.

@MartinEscardo Congratulations on achieving true Britishness!

The author has a delicate line to walk. They must lead the reviewer to conclude their work is awesome without saying it is awesome. Many people find such subtlety difficult. I find it difficult to write at all, because I've been so dead for so long.

When Neil Ghani rewrites your work, he finds the sentence written between the lines and puts it in plain sight. Unevidenced puff is worthless. Evidenced puff is what makes the reviewers pay attention to the evidence.

@pigworker

I strongly oppose the idea that the paper is written for the reviewer.

The paper is written for the readers.

@MartinEscardo You may notice that I have not bothered writing for reviewers for some considerable time, at the expense of not publishing a damn thing.

Your remark makes me feel physically sick.

@MartinEscardo @pigworker The reviewer is the gatekeeper. If the paper is not written for the reviewer, then it will necessarily not be written for the reader, who will never get to see it.

Source: A reader who has been told many times that a paper I would have loved to see was rejected.

@zanzi @MartinEscardo @pigworker This discussion has made me think harder about writing one paper for the reviewers, and a (slightly) different one for readers.
@zanzi @MartinEscardo @pigworker I have recently discussed including a "note to reviewers" in a draft under review, specifically to direct the reviewers towards questions on which I would value an opinion. Of course that gets deleted between acceptance and publication.
@zanzi @MartinEscardo @pigworker I could go further down this path...

@jer_gib @zanzi @MartinEscardo @pigworker based on repeated reviews complaining that I don’t explain Idris syntax in sufficient details, I have always thought about including a ‘readers guide & assumptions guide’ in the last bit of the introduction. The guide gives pointers to documents that provide extra information, my assumptions on readers, and guide about digesting the work. (Peppering this information in the intro and body is not obvious enough) Similar to what some people do in their thesis.

Sadly there is never enough room, after the standard words on semantic highlighting…

@jfdm @jer_gib @zanzi @pigworker

Yes. Conferences have a page limit, and at the same time want you to explain everything.

Department of Computer Science, University of Oxford: Publication - Profunctor Optics: Modular Data Accessors

Data accessors allow one to read and write components of a data structure, such as the fields of a record, the variants of a union, or the elements of a container. These data accessors are collectively known as optics; they are fundamental to programs that manipulate complex data. Individual data accessors for simple data structures are easy to write, for example as pairs of "getter" and "setter" methods. However, it is not obvious how to combine data accessors, in such a way that data accessors for a compound data structure are composed out of smaller data accessors for the parts of that structure. Generally, one has to write a sequence of statements or declarations that navigate step by step through the data structure, accessing one level at a time—which is to say, data accessors are traditionally not first-class citizens, combinable in their own right. We present a framework for modular data access, in which individual data accessors for simple data structures may be freely combined to obtain more complex data accessors for compound data structures. Data accessors become first-class citizens. The framework is based around the notion of profunctors, a flexible generalization of functions. The language features required are higher-order functions ("lambdas" or "closures"), parametrized types ("generics" or "abstract types") of higher kind, and some mechanism for separating interfaces from implementations ("abstract classes" or "modules"). We use Haskell as a vehicle in which to present our constructions, but other languages such as Scala that provide the necessary features should work just as well. We provide implementations of all our constructions, in the form of a literate program: the manuscript file for the paper is also the source code for the program, and the extracted code is available separately for evaluation. We also prove the essential properties, demonstrating that our profunctor-based representations are precisely equivalent to the more familiar concrete representations. Our results should pave the way to simpler ways of writing programs that access the components of compound data structures.

Department of Computer Science

@jer_gib @zanzi @MartinEscardo @pigworker

that's nice

From the looks of it SIGPLAN does allow appendices of unknown length at submission time. However, how these translate to camera ready copy is not immediately clear to me.

@jfdm @zanzi @MartinEscardo @pigworker "Authors of accepted papers may provide additional online-only material, and a brief description thereof, that will be available from the paper's webpage in the ACM Digital Library. See Additional Online-only Material for more details." https://dl.acm.org/journal/pacmpl/author-guidelines
PROCEEDINGS OF THE ACM ON PROGRAMMING LANGUAGES Author Guidelines | ACM Digital Library

Proceedings of the ACM on Programming Languages (PACMPL) is a Gold Open Access journal publishing research on all aspects of programming languages, from design to implementation and from mathematical formalisms to empirical studies. Each issue of the journal is devoted to a particular subject area within programming languages and will be announced through publicized Calls for Papers. All accepted papers receive two rounds of reviewing and authors can expect initial decisions regarding submissions in under 3 months. The journal operates in close collaboration with the Special Interest Group on Programming Languages (SIGPLAN) and is committed to making high-quality peer-reviewed scientific research in programming languages free of restrictions on both access and use.

Proceedings of the ACM on Programming Languages
@jfdm @zanzi @MartinEscardo @pigworker That's PACMPL specifically, but I think it applies across ACM. Springer LNCS also allows online appendices.

@jer_gib @jfdm @zanzi @pigworker

Mathematical Structures in Computer Science also allows that. It also allows you to upload code as additional material, which I have done in a publication there.

@MartinEscardo Sometimes, that stuff is genuine motivation. But when it's the paragraph of the paper the supervisor wrote, it's usually just hype.

@pigworker

Not when I am the supervisor. On the contrary, I have deleted anything like this in any student's writings.

@MartinEscardo I certainly wasn't thinking of you.

I think it's ok to express the importance of a problem you're addressing, and thus why your work might help people. It's important to keep that stuff focused on stakeholders other than oneself.

@MartinEscardo But but but .... isn't that every grant application ever? And job application?

My preferred approach is something like the following.

The abstract is written for the experts. You don't need to say anything about originality, how clever you are, and how important your results are. Just tell what you've done.

The introduction should put your work in context. What has been done before, and what you have done to advance knowledge.

The experts will be happy with just that.

But then, as people discussed below, there are the reviewers.

And, more importantly, there are the intended readers that are not expert.

Mostly, these readers are PhD students, that is, the next generation of researchers. Nobody mentioned this in the discussions in this thread.

But these readers may also include people from neighbouring fields whose work either (1) could benefit from your work, or (2) could potentially contribute to your work. Such people must be addressed too.

At the end of the introduction, however you chose to write it, it is nice to have a quick summary of what you think your contributions are.

(Anecdote. Recently I was in a position to help the authors of a paper, as a journal editor, to add more to what they thought their contributions were. They didn't realize that there was something significant in the technical development that was worth adding to the introduction.)

Also, don't conflate writing a conference paper with writing a journal paper, or a grant application or a job application.

The last two require a completely different approach, and I don't have any good advice to give regarding them.

Also, write your paper so that it survives the passage of time. If you are young, 20 years may seem like a long time. Still, write your paper so that in 20 years time, when you will still be not old, makes sense, is understandable, and pleases you.

@MartinEscardo I'll give one justification: often papers contain a lot of background material, or simply build on earlier work, and I'd like the authors to draw my attention to the bit that they think is important and novel. There may be better ways to do this than claiming novelty but there should be some way of doing it that we can all agree isn't just puffery.

The one I don't like is when someone claims something is simple. That really is for the reader, not author, to judge.

@dpiponi "Novel" in a paper is 0% information and 100% irritation.

@dpiponi @MartinEscardo
Maybe just use subjects?

_A. and B. showed that... It was shown already...

VS

We show that... We design/present..._

@dpiponi @MartinEscardo I teach my students to make a catalog of claims (typically the last thing on page 1) as a set of bullet points. There it can help a reviewer to know exactly what is being claimed. Including if you’re claiming something is new vs a new application of a method that already existed.

I also tell them to put forward references to the point in the paper where the claim is fulfilled.

The goal of all this is to help time-poor reviewers do their job more easily. In my experience many reviewers don’t take the time to think about whether something is novel or not (and check the literature to verify).

@dpiponi @MartinEscardo

Separate comment is that the quality of reviewing is plummeting. submissions in machine learning have been growing at Moore’s law rates which means that 50% of the papers are being submitted by people who have been in the field for less than 2 years. Sadly, these people are also reviewers because it’s the only way that conferences can get enough reviewing resources. This means that most reviews are conducted by looking for bold numbers and most theoretical considerations fall by the wayside.

I even had a reviewer complain that we made reference to the ‘condition number of a matrix’ without explaining what that was…

@dpiponi @MartinEscardo Tertiary comment: Conferences don’t scale. The algorithm for running a conference is something like: sort all papers by quality and select top-n. But as we know sort is n log(n) and reviewing resources grow like n.

Worse, with a large pool of reviewers, increasing effort needs to be made to ensure some level of consistency, so I suspect workload really grows as \(n \sqrt{n}\).

@MartinEscardo I think we’re trained to do it by stupid reviews. Trying to break the habit now…

@MartinEscardo @zyang and @jonmsterling said it: it's written for the reviewers. It's no longer the case that a specialist paper is guaranteed, nor even likely, to get 2 or more reviewers who have tried something like this before and will form an opinion about novelty from an expert position.

So when writing A is novel, and B is novel, and C is novel, the expert says 'yes, yes, yes', and the non expert says 'I'm not an expert, but A,B, and C seem novel' and feel comfortable accepting the submission.

@ohad @zyang @jonmsterling

Is that supposed to justify how papers should be written, or that the conference publication system in CS is rotten?

@MartinEscardo @ohad @zyang @jonmsterling Jesus, Martin, you have to let people live in the moment with the system as it is. That doesn't mean not criticising the system. But we should understand when people not in positions of power do what they have to do.

@MartinEscardo @zyang @jonmsterling I'm not justifying any aspect of the peer review system, whether it's conference CS, journal CS, journal philosophy, etc.

I'm offering an explanation for why it might be like so, and you can form your opinion as to whether it explains the phenomena you observe better than other hypotheses, parsimonious, etc, maybe challenge it, etc.

@MartinEscardo @ohad @zyang @jonmsterling It's much wider than just CS conferences. The academic publishing system has huge problems. Overuse of the word 'novel' is very mild as symptoms go.

@MartinEscardo @zyang @jonmsterling other mechanisms that degrade the paper, but ease refereeing:

1. Repeatedly referring back to the paper structure:

end of section A: ... we have shown X, and next we will show Y.
\section B having shown X, let's turn to Y.

(Rationale: referee is not really interested in the paper, so might lose the plot, or has a stack of papers to read between meetings, on the bus, etc and loses the bug structure.)

2. Not citing related work during the technical development. Instead, cite it in related work section.

Rationale: makes technical development seem more novel to nonexpert while an expert will likely not complain if the citation exists. Bulks up related work section and makes it seem in-depth because it is more technical.

@ohad @MartinEscardo @zyang @jonmsterling "It was all my idea except for almost all of it!"

@MartinEscardo This complaint reminds me of the biscuit man that roves my neighborhood selling his biscuits. He has a recording that says: "Get your hot delicious biscuits here! I recommend them!", and I'm used to food vendors telling me their food is delicious, but the enthusiastic "I recommend them!" at the end always cracks me up.

I guess the moral of my biscuit man story is that you should be grateful that these articles you are reviewing stop at calling stuff novel and important, and don't explicitly contain the phrase "I strongly recommend this paper for publication!".

@MartinEscardo : As a reader, I appreciate when the abstract says ‘We consider a novel approach to problem X’ or ‘We consider a novel variation of Y's approach to problem X’. The only thing that bothers me about ‘novel’ here is that they should have just said ‘new’. But the term can be overused; there's no point saying ‘We make novel contributions’ without context.
@MartinEscardo I don't really agree. I think understanding why authors think their work is important is good information to have. As a reader I am happy to read this opinion, then agree, disagree, or reserve judgement as I see fit. Claims of novelty are to help the reader differentiate quickly what in the paper was already known from what the author believes is new. These sorts of claims are helpful to reviewers but even more helpful to non-expert readers (sometimes the same thing!)

@RanaldClouston

The point is that you won't convince me that your work is important if you say it is important. You need to say something else to convince me it is important.

@MartinEscardo do people really claim their work is important without giving reasons?
@RanaldClouston @MartinEscardo Yes. Politicians say they "refute" propositions without giving disproofs. Academia is full of politicians.
@RanaldClouston The reasons alone should be enough!
@MartinEscardo Isn't novelty the odd one out of the three properties you listed (novelty, importance, and surprisingness)? Importance always has an attached “to whom”, and an ideal paper will contain no surprising results given enough time, as experts start to use that paper as part of their basis for their heuristics. Both are subjective because future readers are likely to be very different to reviewers. On the other hand, novelty is fairly objective. As long as the reviewers collectively know enough about the field, they can verify or falsify claims of novelty by comparing the new work with existing approaches, which they're going to do anyway as part of reviewing. Furthermore, as results become accepted wisdom, it becomes harder for readers to tell the new parts from the background, so it's good to have that checked at reviewing time and then remain there as a historical record.