My first post from NSDI is up! I'm talking about spot instances and skypilot!

https://blog.appliedcomputing.io/p/nsdi-recap-part-i-spot-instances

#nsdi24 #aws #spotinstances #skypilot

Sorry I got tired and didn't liveblog the last two talks. One was on dynamic compression algorithms for lora networks in low bandwidth settings, which was really cool. They got to dump a bunch of little computers on a farm in Wisconsin.

The last talk was on domain hijacking via forgotten/dangling dns records, which was absolutely fascinating. Almost 18k (detected!) cases of this on the internet over a 3 year study. A big chunk of these were used for Indonesian gambling sites.

#nsdi24

Resource allocation is a very general problem -- now let's talk about cluster scheduling. There's a general approach/framework that we can use to solve all of these types of problems.

In cluster scheduling, we have resources (cpu, gpu, mem), demands (jobs), and the paths now correspond to the resources that group together.

#nsdi24

To get even faster, they approximately sort the demands (instead of exactly sorting) using a binning technique.

#nsdi24

Instead of solving an iterated optimization problem, they try to solve a single fixed optimization problem. The key observation here is to incentivize the solver to solve the problem once in the same order as it "would" solve it iteratively.

#nsdi24

Their solution is a new \alpha-approximate technique that is faster than the previous best with the same guarantee. They also have even faster heuristics that still provide competitive fairness.

#nsdi24

The fairness metric they use is "max min fairness", e.g., we want to maximize the minimum allocation among the remaining demands, and fix the demands that cannot receive more.

There are two types of allocators, single-path (fast, but unfair and inefficient) and multi-path (iterative optimization-based, fair and efficient, but slow)

#nsdi24