@alexcengler ... and, *that* argument would have far-reaching implications beyond the realm of auditing.

@alexcengler .... i.e. that selling powerful machine learning systems for important applications _without_ accompanying expertise is a recipe for trouble...

@alexcengler My primary concern would not be the availability of compute, but of (1) expertise and (2) test data. Many downstream users (e.g., police departments with respect to facial recognition) do indeed have "no idea how their system works" -- they lack and won't develop that expertise in house. Part of the pitch from platform vendors is that the end-using business doesn't need to. One might say that such a business model is per se irresponsible...

@geomblog true but on the other hand.... legs are coming!

@alexcengler I consider an analogous question about automated decision-making systems in Voices in the Code

@alexcengler Interesting idea. I think one thing this argument misses is that it's costly to audit large models. Parties that are down the value chain from a large-model provider are located there partly _because_ they lack the resources to scratch cook resource intensive AI applications. It is likely to follow that they also lack the resources to conduct this type of analysis themselves.