Nano Banana Pro. Реальное применение, а не мемные картинки
Когда данных мало, а домен сильно отличается, предобученные модели перестают работать. Я попробовал вместо сбора и ручной разметки генерировать дорожные дефекты поверх реальных кадров. Что получилось, где работает, где нет и сколько это стоит - в статье.
https://habr.com/ru/articles/993968/
#ai #nano_banana_pro #synthetic_data #computer_vision #segmentation
[Перевод] Humans-in-the-loop vs synthetic data: за что идёт борьба на рынке AaaS
Scale зарабатывает более $750 млн в год на продаже данных для RLHF. Кто собирается их потеснить? Scale AI — стартап, ранее известный своими контрактами на разметку данных для беспилотных автомобилей и военных проектов, приближается к годовому обороту в $1 млрд благодаря своим дата-сервисам, используемым в техниках вроде reinforcement learning from human feedback (RLHF). Я давно слышал слухи об их масштабах, о том, что они работают буквально со всеми крупными AI-лабораториями — от Meta до OpenAI, но увидеть подтверждение этого в публичных отчетах ощущается совсем иначе.
https://habr.com/ru/articles/944768/
#llm #ai #scale_ai #genai #synthetic_data #finetuning #mlops #rlhf #human_in_the_loop #alignment
GAPartManip: A Large-scale Part-centric Dataset for Material-Agnostic Articulated Object Manipulation
Authors: Wenbo Cui, Chengyang Zhao, Songlin Wei, Jiazhao Zhang, Haoran Geng, Yaran Chen, He Wang
pre-print -> https://arxiv.org/abs/2411.18276
#robotics #articulated_objects #manipulation #dataset #synthetic_data #data_generation
Effectively manipulating articulated objects in household scenarios is a crucial step toward achieving general embodied artificial intelligence. Mainstream research in 3D vision has primarily focused on manipulation through depth perception and pose detection. However, in real-world environments, these methods often face challenges due to imperfect depth perception, such as with transparent lids and reflective handles. Moreover, they generally lack the diversity in part-based interactions required for flexible and adaptable manipulation. To address these challenges, we introduced a large-scale part-centric dataset for articulated object manipulation that features both photo-realistic material randomization and detailed annotations of part-oriented, scene-level actionable interaction poses. We evaluated the effectiveness of our dataset by integrating it with several state-of-the-art methods for depth estimation and interaction pose prediction. Additionally, we proposed a novel modular framework that delivers superior and robust performance for generalizable articulated object manipulation. Our extensive experiments demonstrate that our dataset significantly improves the performance of depth perception and actionable interaction pose prediction in both simulation and real-world scenarios. More information and demos can be found at: https://pku-epic.github.io/GAPartManip/.
GARField: Addressing the visual Sim-to-Real gap in garment manipulation with mesh-attached radiance fields
Authors: Donatien Delehelle, Darwin G. Caldwell, Fei Chen
pre-print -> https://arxiv.org/abs/2410.05038
website -> https://ddonatien.github.io/garfield-website/
#robotics #deformable_manipulation #garment_manipulation #NeRF #deep_learning #synthetic_data #data_generation #real2sim
While humans intuitively manipulate garments and other textile items swiftly and accurately, it is a significant challenge for robots. A factor crucial to human performance is the ability to imagine, a priori, the intended result of the manipulation intents and hence develop predictions on the garment pose. That ability allows us to plan from highly obstructed states, adapt our plans as we collect more information and react swiftly to unforeseen circumstances. Conversely, robots struggle to establish such intuitions and form tight links between plans and observations. We can partly attribute this to the high cost of obtaining densely labelled data for textile manipulation, both in quality and quantity. The problem of data collection is a long-standing issue in data-based approaches to garment manipulation. As of today, generating high-quality and labelled garment manipulation data is mainly attempted through advanced data capture procedures that create simplified state estimations from real-world observations. However, this work proposes a novel approach to the problem by generating real-world observations from object states. To achieve this, we present GARField (Garment Attached Radiance Field), the first differentiable rendering architecture, to our knowledge, for data generation from simulated states stored as triangle meshes. Code is available on https://ddonatien.github.io/garfield-website/
DataSynthTool Repository https://slott56.github.io/2024-07-01-datasynthtool_repository.html
Habr
Robotics papers
S. Lott -- No Kings!