If you are building an application that requires search, I recommend using Elasticsearch early on. In addition to the usual full-text search, Elasticsearch allows you to perform a hybrid search: combine the results of text and vector search.
Of course, for small amounts of data, you can use PostgreSQL tsvector with the pgvector extension, but in the long term, Elasticsearch will provide good performance.
#Elasticsearch #Search #tsvector #pgvector #KNN #Embedding #SentenceTransformers #AI
Given how vocal I am against the "AI" industry, some of my followers might be surprised to learn that I'm now a co-author on a machine learning paper.
That paper has been submitted to the proceedings of an upcoming conference under their "Responsible AI" track, but it has nothing to do with LLMs or really anything that has recently been pushed by the industry's hype-machine. A pre-print is available on arxiv.org ("Tiny, Hardware-Independent, Compression-based Classification") while its formal review is pending.
Our paper expands on a technique I've been using to classify my emails for more than two years called "NCD-KNN" (Normalized Compression Distance with K-Nearest Neighbours). This method uses commonly available compression utilities like GZIP to estimate the relative "distance" between an input and a set of labeled examples, ultimately categorizing that input according to the labels of the K-nearest examples.
We solved some fundamental problems which could result in negative distances under specific circumstances which we identified, addressed some other theoretical limitations which prevented its broader use, and extended NCD to applications using Support-Vector-Machines (SVMs) for non-linear classification.
My co-authors are not on Fedi, but if any of this interests you then feel free to Ask Me Anything
The recent developments in machine learning have highlighted a conflict between online platforms and their users in terms of privacy. The importance of user privacy and the struggle for power over user data has been intensified as regulators and operators attempt to police online platforms. As users have become increasingly aware of privacy issues, client-side data storage, management, and analysis have become a favoured approach to large-scale centralised machine learning. However, state-of-the-art machine learning methods require vast amounts of labelled user data, making them unsuitable for models that reside client-side and only have access to a single user's data. State-of-the-art methods are also computationally expensive, which degrades the user experience on compute-limited hardware and also reduces battery life. A recent alternative approach has proven remarkably successful in classification tasks across a wide variety of data -- using a compression-based distance measure (called normalised compression distance) to measure the distance between generic objects in classical distance-based machine learning methods. In this work, we demonstrate that the normalised compression distance is actually not a metric; develop it for the wider context of kernel methods to allow modelling of complex data; and present techniques to improve the training time of models that use this distance measure. We demonstrate that the normalised compression distance works as well as and sometimes better than other metrics and kernels -- while requiring only marginally more computational costs and in spite of the lack of formal metric properties. The end results is a simple model with remarkable accuracy even when trained on a very small number of samples allowing for models that are small and effective enough to run entirely on a client device using only user-supplied data.
Как написать собственные классы классификации для маленьких
В прошлый раз я уже рассказывала о том, как в ходе обучения в "Школе 21" создавала класс линейной регресии , на этот раз будем рассматривать реализацию LogisticRegression, GaussianNB, KNN. Как и в прошлый раз, минимум теории, максимум практики.
High Performance GPU Implementation of KNN Algorithm: A Review
#Zoomposium with Dr. #Gabriele #Scheler: “The #language of the #brain - or how #AI can learn from #biological #language #models”
There is a #paradigmshift away from the purely information-technological-mechanistic, purely data-driven #Big #Data concept of #LLMs towards increasingly information-biological-polycontextural, structure-driven #artificial, #neural #networks (#KNN) concepts.
More at: https://philosophies.de/index.php/2024/11/18/sprache-des-gehirns/
#Zoomposium mit Dr. #Gabriele #Scheler: "Die #Sprache des #Gehirns - oder wie #KI von #biologischen #Sprachmodellen lernen kann"
Es gibt einen #Paradigmenwechsel weg vom rein informationstechnologischen-mechanistischen, rein daten-getriebenen #Big #Data-Konzept der #LLMs hin zu immer stärker informationsbiologische-polykontexturalen, struktur-getriebenen #Künstliche, #Neuronale #Netzwerke (#KNN)-Konzepten.
Mehr auf: https://philosophies.de/index.php/2024/11/18/sprache-des-gehirns/
#Zoomposium with Dr. #Gabriele #Scheler: “The #language of the #brain - or how #AI can learn from #biological #language #models”
There is a #paradigmshift away from the purely information-technological-mechanistic, purely data-driven #Big #Data concept of #LLMs towards increasingly information-biological-polycontextural, structure-driven #artificial, #neural #networks (#KNN) concepts.
More at: https://philosophies.de/index.php/2024/11/18/sprache-des-gehirns/
#Zoomposium mit Dr. #Gabriele #Scheler: "Die #Sprache des #Gehirns - oder wie #KI von #biologischen #Sprachmodellen lernen kann"
Es gibt einen #Paradigmenwechsel weg vom rein informationstechnologischen-mechanistischen, rein daten-getriebenen #Big #Data-Konzept der #LLMs hin zu immer stärker informationsbiologische-polykontexturalen, struktur-getriebenen #Künstliche, #Neuronale #Netzwerke (#KNN)-Konzepten.
Mehr auf: https://philosophies.de/index.php/2024/11/18/sprache-des-gehirns/
Сравниваем быстродействие новой функциональности ClickHouse по поиску ближайших векторов с другими решениями
Всем привет! Меня зовут Диана Бутько, я студентка 3 курса, изучаю информационные системы и программирование. В InfoWatch я пришла на практику, и одной из моих задач стал сравнительный анализ различных методов поиска похожих векторов. Это один из ключевых аспектов машинного обучения и анализа данных, используемых в рекомендательных системах, кластеризации, семантическом поиске и других областях. Но чем больше объем данных, тем важнее становится выбор инструментов: полный перебор векторов требует больших вычислительных ресурсов, а в других алгоритмах порой необходимо балансировать между точностью и скоростью поиска. В этой статье я сравниваю пять методов поиска похожих векторов: — полный перебор по евклидову расстоянию с реализацией в Python; — FAISS с индексами IndexFlatL2 (полный перебор, евклидово расстояние) и IndexIVFFlat (сегментирование по ячейкам, евклидово расстояние); — векторный поиск в ClickHouse с индексом HNSW и метриками расстояния L2Distance (евклидово расстояние) и cosineDistance (косинусное сходство).
yegorov
ansuz
Habr
GripNews
HGPU group
Philo Sophies
Philo Sophies