In the past year, there was an upturn in some seasonal infections, and we hear this now again. Terms of immune debt and immune gap are mentioned. What is this?

First note the upturn of only some and seasonal, infections. This is not a general thing.

Side note; I am personally not a fan of "immune debt", which implies something has been on loan. Immune gap is an existing term, also denoting the period of increased risk of infection between maternal antibodies running low and the baby's own immune system not at strength yet.

Some infectious agents are selected to invade us quickly and move out again. There is an arms race ongoing. We try and prevent them from infecting, but don´t fully manage.

They try and evade our immune system, but don´t manage. A stall mate of regular, mild infections ensures.
These are the seasonal infections, RSV and Influenza famous examples, and SARS-CoV-2, like the other human coronaviruses will be one of those. Their mode of operation depends on infecting our airways. These are relatively easy to access and to leave again.
Important aspects are to be quick and stealth-like to ensure just enough time to propagate, and leave the host again before the immune response clears you, ready to infect someone else. A balancing act that works best when people are together in rooms with poor ventilation.
An important ability is to mutate their receptor(s) enough to avoid the bulk of neutralizing antibodies by at least a proportion of us.
This ensures there are new people to infect each year. In addition, our antibodies wane after a few months. We have more pathogens to deal with.
The waning of antibody levels ensures there is space for new responses against the many pathogens we encounter every day. This also creates a weakness, our protection against fast pathogens is temporally reduced, but protection against severe disease remains in place.

There is an added element in young children. Their initial antiviral response is enhanced. This can reduce their susceptibility to disease, but also slow down the generation of memory cells. This means several rounds of infection are needed to get a good memory response in place.

In adults there is a good memory response, but, the pathogen mutates. Also in adults, re-infections ensure an updated response against new variants.

You often do not notice, unless a pathogen has suddenly changed a lot, like a new influenza strain/variant causing an epidemic.
This is the concept of the immune gap. We have been much more aware of infections and have taken precautions to reduce them. This does not mean infections are good, they never are. But reducing their spread, and then lifting the measures that have reduced them has consequences.
These are 2-fold. The youngest have not experienced some annual infections and will have to catch up. Some infections at an older age can be more symptomatic. Others have not built up sufficient immune memory yet, together with waning, this may cause more symptomatic disease.
For adults, the effects are probably less noticeable.

We may get more infections, since our antibodies have waned for longer, but, the pathogens have not undergone many rounds of infections and the selective pressure on mutations has been less.

Most infections are settling into their annual pattern. However, these timings and amplitudes always differ a bit from year to year. Bacterial infections are slower and may still have some catching up to do. Besides, new variants and antibiotic resistance are additional problems.

@marc_veld Sounds like a new equilibrium needs to settle in. Meanwhile we'll all suffer more