State media control shapes LLM behaviour by influencing training data
최근 연구에 따르면, 국가가 통제하는 미디어가 대규모 언어 모델(LLM)의 학습 데이터에 영향을 미쳐 모델의 행동과 출력에 편향을 유발할 수 있음이 밝혀졌다. 이는 LLM 개발 시 데이터 출처와 편향 문제를 신중히 고려해야 함을 시사하며, AI 모델의 공정성과 신뢰성 확보에 중요한 시사점을 제공한다. 특히, 정치적 통제와 정보 환경이 AI 모델의 응답에 미치는 영향을 이해하는 것이 AI 서비스 개발자에게 필수적이다.
#State #media #control influences #LargeLanguageModels (#LLMs)
"Millions of people around the world query LLMs for information. Although several studies have compellingly documented the persuasive potential of these models, there is limited evidence of who or what influences the models themselves, leading to a flurry of concerns about which companies and governments build and regulate the models. Here we show through six studies that government control of the media across the world already influences the output of LLMs via their #TrainingData. We use a cross-national audit to show that LLMs exhibit a #stronger #ProGovernment valence in the languages of countries with #LowerMediaFreedom than in those with higher media freedom. The combination of influence and persuasive potential across languages suggests the troubling conclusion that states and powerful institutions have increased strategic incentives to leverage media control in the hopes of shaping LLM output."
Government-controlled media influences the output of large language models via their training data, and models queried in the languages of countries with lower media freedom show a stronger pro-regime valence than models queried in the languages of countries with higher media freedom.
Show HN: Abliteration – made-to-order training data for classifiers and evals
Abliteration은 OpenAI 호환 API를 통해 거부 없이 맞춤형 합성 학습 및 평가 데이터를 생성하는 서비스입니다. 안전성 분류기 훈련, 평가 세트 생성, 적대적 데이터 구축 등 다양한 ML 파이프라인에 필요한 데이터를 정책 기반으로 통제하며, JSONL 구조화 출력과 프로젝트별 할당량 관리, 결정 메타데이터 기록을 지원해 데이터 거버넌스와 재현성을 보장합니다. 데이터는 저장하지 않고 SOC 2 인증 준비 중으로, 기업 감사 요건에 부합하는 합성 데이터 생성 솔루션입니다.
https://abliteration.ai/use-cases/synthetic-data
#syntheticdata #trainingdata #mlpipeline #datagovernance #api
Extracting alignment data in open models
이 논문은 사후 학습된 오픈 모델에서 상당한 양의 정렬(alignment) 훈련 데이터를 추출할 수 있음을 보여준다. 특히, 문자열 매칭 대신 고품질 임베딩 모델을 활용해 의미적 유사성을 측정함으로써 기존 방식보다 10배 이상 많은 데이터를 식별할 수 있음을 입증했다. 또한, SFT나 RL과 같은 사후 훈련 단계에서 사용된 데이터가 모델에 의해 쉽게 재생산되며, 이를 활용해 원본 성능을 회복하는 베이스 모델 훈련도 가능함을 확인했다. 이 연구는 정렬 데이터 추출의 잠재적 위험성을 드러내고, 증류(distillation) 과정이 사실상 원본 데이터에 간접적으로 재학습하는 효과를 가질 수 있음을 시사한다.
https://arxiv.org/abs/2510.18554
#alignment #openmodels #embedding #trainingdata #distillation
In this work, we show that it is possible to extract significant amounts of alignment training data from a post-trained model -- useful to steer the model to improve certain capabilities such as long-context reasoning, safety, instruction following, and maths. While the majority of related work on memorisation has focused on measuring success of training data extraction through string matching, we argue that embedding models are better suited for our specific goals. Distances measured through a high quality embedding model can identify semantic similarities between strings that a different metric such as edit distance will struggle to capture. In fact, in our investigation, approximate string matching would have severely undercounted (by a conservative estimate of $10\times$) the amount of data that can be extracted due to trivial artifacts that deflate the metric. Interestingly, we find that models readily regurgitate training data that was used in post-training phases such as SFT or RL. We show that this data can be then used to train a base model, recovering a meaningful amount of the original performance. We believe our work exposes a possibly overlooked risk towards extracting alignment data. Finally, our work opens up an interesting discussion on the downstream effects of distillation practices: since models seem to be regurgitating aspects of their training set, distillation can therefore be thought of as indirectly training on the model's original dataset.
PS An actual IR transmitter / receiver #hexpansion was created over two years ago... #TrainingData 
Feeding the Machine
AI 선도 기업들이 AGI 개발을 위해 방대한 데이터와 소프트웨어 엔지니어를 필요로 하는 가운데, 스타트업 Mercor는 자동화된 인력 중개 플랫폼으로 빠르게 성장하며 연간 5억 달러 매출을 달성했다. 특히 Scale AI가 1,200명의 소프트웨어 엔지니어를 요구하면서 데이터 라벨링 및 AI 학습 데이터 생산 분야의 인력 수요가 급증하고 있다. Mercor는 중간 플랫폼 문제를 해결하며 AI 데이터 작업자들의 임금 문제를 개선하는 방향으로 사업을 확장 중이다.
https://www.theverge.com/cs/features/831818/ai-mercor-handshake-scale-surge-staffing-companies
Anthropic (@AnthropicAI)
Anthropic 연구에서 불쾌한 행동을 줄이기 위해 학습 데이터에 무관한 도구와 시스템 프롬프트를 추가하는 간단한 조정만으로도 효과가 있었다는 점을 제시했다. 하찮아 보이는 데이터 다양화가 모델 안전성 개선에 실질적으로 기여할 수 있음을 보여주는 결과다.
fly51fly (@fly51fly)
훈련 데이터 필터링을 적응적으로 수행하는 새로운 방법인 CRAFT를 소개하는 연구입니다. Google과 BITS Pilani 연구진이 클러스터 기반 회귀를 활용해 학습 데이터 품질을 개선하는 접근을 제안했습니다.
https://x.com/fly51fly/status/2048879299584004474
#google #trainingdata #datafiltering #machinelearning #research
When the Radiologist Becomes the Expense
On March 25, 2026, at a Crain’s New York Business panel discussion of the city’s hospital sector, Mitchell H. Katz, MD, president and CEO of NYC Health + Hospitals, told the assembled executives what cost-cutting now sounds like in the largest public hospital system in the United States. “We could replace a great deal of radiologists with AI at this moment, if we are ready to do the regulatory challenge.” Sandra Scott, MD, who runs One Brooklyn Health, one of the city’s safety-net institutions operating on tight margins, replied that the move would be “a game-changer.” The exchange appeared in Crain’s coverage of the panel and was picked up by the radiology trade press within forty-eight hours.
The proposal reads as the second move in a strategy whose first move has been documented for fifteen years. American hospital systems built imaging volume on the back of a preventative-medicine apparatus that the American College of Cardiology’s own Choosing Wisely campaign identified in 2012 as substantially overused, with up to 45% of stress cardiac imaging in low-risk asymptomatic patients flagged as inappropriate by the ACC’s own appropriate-use criteria. That volume produced revenue. The same hospital systems now propose to automate away the labor cost of interpreting the revenue-producing volume. Imaging continues, billing continues, the radiologist disappears from the ledger, and the patient pays the same copay for a scan whose ordering was already questionable, now read by an algorithm whose performance varies by manufacturer, training data, patient population, and deployment context.
The strongest evidence base for AI in radiology supports a use case that the Katz proposal does not describe. The Mammography Screening with Artificial Intelligence trial, called MASAI, randomized over 100,000 Swedish women to either standard double reading by two radiologists or AI-supported single reading by one radiologist with the Transpara system from ScreenPoint Medical. Lead author Kristina Lång and colleagues at Lund University reported in The Lancet Oncology in 2023 that the AI-supported arm reduced radiologist workload by 44% while modestly increasing cancer detection. Follow-up data published in The Lancet in 2026 showed a 12% reduction in interval cancers, meaning cancers that emerge between screenings and that carry worse prognosis, with AI-supported screening compared to standard double reading. First author Jessie Gommers of Radboud University Medical Centre was direct in the press release: “Our study does not support replacing healthcare professionals with AI as the AI-supported mammography screening still requires at least one human radiologist to perform the screen reading.”
That distinction matters. AI-assisted reading, where a human radiologist works alongside an algorithm that flags suspicious findings and triages low-risk cases for single rather than double review, has been validated in randomized trials with hard outcome measures. The validation extends to AI as a triage and detection support, where one human radiologist remains in the loop. AI-only reading, where no human reviews the image unless the algorithm flags an abnormality, has not been tested to the same standard. A Stanford working paper on so-called “AI mirages” in medical imaging, which describes algorithms that perform well on benchmark datasets and fail in clinical deployment because the training distribution does not match the deployment distribution, was circulating at the time of the Katz panel and was awaiting peer review. Mohammed Suhail, MD, a radiologist at North Coast Imaging quoted in coverage of the Katz statement, said that any attempt to implement AI-only reads “would immediately result in patient harm and death, and only someone with zero understanding of radiology would say something so naive.” That is a strong claim from a working radiologist, but the structural point underneath it is conservative. The trial that would justify AI-only reading on a population basis has not been run. The trial that would justify AI-assisted reading has been run, and it requires the radiologist.
Set the safety question aside for a moment and consider what the proposal does to the labor market. The radiologist has been a high-margin specialist for the same reason all specialists are high-margin: the supply is constrained by the length of training and the licensing apparatus, and the demand is set by imaging volume. Katz’s proposal substitutes capital for labor. If New York State relaxes the regulation requiring radiologist review, NYC Health + Hospitals saves the salary of every radiologist whose reads can be displaced to the algorithm; the imaging machine still runs, still bills, still produces a chargeable encounter on the patient’s account. Generalized, the same logic applies to dermatology, where machine-learning skin lesion classifiers have shown strong retrospective performance, and to pathology, ophthalmology, and any imaging-heavy specialty whose work product is a classification task on a digital image. A worsening shortage of breast imaging specialists, particularly in rural and underserved markets, is the legitimate operational pressure Katz is responding to, and the American College of Radiology has documented this shortage at length. Using that pressure to license a deployment model the trial evidence has not endorsed is the illegitimate response.
Two profits accrue to the hospital system. The first is the original imaging revenue, generated by the appropriate-use-violating ordering patterns that produced the screening volume in the first place. The second is elimination of the labor cost of reading the imaging. A patient pays the copay, the insurer pays the technical fee, and the AI vendor takes a per-read or subscription fee that comes in well below the radiologist’s salary equivalent. Vendor and hospital split the gain. Radiologists are unemployed or shifted to abnormality review only, which substantially compresses earnings since the volume of abnormal reads is a fraction of total reads. Care delivered to the patient may be equivalent in accuracy under the AI-supported model and inferior in accuracy under the AI-only model, with no individual professional license held responsible for the read.
The regulatory politics will determine which model gets deployed. Katz himself flagged the regulatory challenge at the Crain’s panel, asking the assembled CEOs whether there was any reason they should not be lobbying New York State to permit AI-only reads. Lobbying for the relaxation is the hospital system facing margin pressure. Lobbying against is the American College of Radiology and the radiologists themselves, organized through their professional society. New York State legislators will decide. Patients do not have a seat at this table. A patient learns about the change when the mammogram comes back from the screening center read by Transpara version whatever and the bill arrives in the mail with no indication of who, if anyone, looked at the image.
Liability shifts. Under current regulation, a missed cancer on a mammogram exposes the reading radiologist to malpractice litigation, which is why the radiologist carries professional liability insurance and why the radiologist’s professional license is on the line for every read. Under a proposed AI-only model with radiologist confirmation only of flagged abnormalities, the missed cancer that occurred when the algorithm scored the image low and no human looked at it produces a liability question with no individual defendant. The plaintiff sues the institution, the institution sues the AI vendor, the AI vendor sues the training data licensor or invokes the FDA clearance as a shield. Many degrees of separation now sit between the patient and the party with deep pockets. The structural change resembles the shift from the family doctor to the corporate practice in primary care: personal accountability disappears into the institutional defendant, and the patient learns that the system is the system.
The preventative-medicine apparatus that produced excess imaging volume and the AI-radiology apparatus that proposes to read it without human review are two faces of the same financial logic. Both extract value from patient bodies through technical interventions whose individual benefit is small or unproven on a population basis, both produce steady recurring revenue, and both depend on the patient being a passive substrate rather than an active agent in the care chain. One creates the imaging. The other eliminates the labor cost of reading it. The hospital system, which is the only party that crosses both moves, captures the margin on both.
AI-assisted radiology is a real technology with real performance data. The MASAI trial demonstrated that the right deployment, with the right oversight, in the right population, produces better cancer detection at lower radiologist workload. That is a legitimate technological gain and the trial is one of the cleaner pieces of clinical evidence for AI in medicine to date. The question is who controls deployment, under what oversight, and to what end. If AI becomes a tool that radiologists use to read more imaging more accurately at lower cost per read, with patient outcomes that match or exceed the current standard, that is medicine. If AI becomes a license to eliminate the radiologist altogether, with the institutional savings flowing to hospital margins and the patient losing the only party in the imaging chain whose individual professional license is on the line for the read, that is bookkeeping. Mitchell Katz proposed the second model at a panel in March. The trial evidence supports the first. The next move belongs to the New York State legislature, which is to say, to whoever lobbies hardest in Albany.
#ai #health #healthcare #living #medicine #patient #radiologists #radiology #revenue #safety #screening #tech #trainingData
Ars Technica (@arstechnica)
Meta가 직원 추적 소프트웨어를 활용해 AI 에이전트 학습에 나선다는 보도입니다. 기업 내부 업무 추적 데이터를 AI 에이전트 훈련에 적용하는 사례로, AI 학습 데이터 수집·활용 방식의 변화라는 점에서 주목됩니다.
sayzard
Christophe Bousquet
Martin Hamilton
David Boles