Mastodawn

Paul Jacobson

@kjr still my preferred messaging app even though I only use it with my wife, kids and a friend.

@paul have you developed an app?

@kjr no, I'm more of a dabbler with code and tend to do more data processing and analysis lately.

@paul data processing and analysis... I like it. What kind of analysis do you do?
I am quite active in text data analysis.

@kjr I recently started doing some data analysis with sales data, mostly using Looker and Superset. I also have a set of personal workflows written in Python that process text notes in Obsidian, and generate some of my routine reports at work

@paul I am currently finishing two projects in Hebrew archival text analysis.
One is to detect entities (people, places, locations, dates and organizations) from archival collections and the relationships between them, so we can extract biographic information about the people: when and where born, name of the parents, organizations the person belonged, etc.
The second one is just to extract the entities, link them to bibliographic entity files and improve the search of the data and the linking between documents of different archives.

@paul I run a project symilar to the first one for English data, but there I take care only of the relations. I use existing open source models for the entity detection.

@kjr that sounds both amazing and way above the kind of analysis I'm doing. I love the sort of work you're doing. It fascinates me.

@paul Above... I don't know... let us say that they are different work areas. For me what you do, business intelligence, it is a different world with its own challenges.

@kjr perhaps. I'm still pretty new to this sort of work so I have a lot to learn. I've spent about a year and a half learning SQL and a bit longer learning Python.

The challenge is now taking all that theory and making good use of it.

@paul the best is to start working in projects, if possible with payment - not only for the money, but for the pressure of achieving success.

@kjr how old is the information you're working with?

@paul
The first project is Holocaust data, the customer is Yad Vashem.
The second one goes from old to brand new data. The customer is the Archive Network Israel
https://www.nli.org.il/he/discover/archives/ian

רשת ארכיוני ישראל (רא"י)

שימור ההיסטוריה של מדינת ישראל באמצעות ארכיונים

@kjr amazing! It must be fascinating work.

@paul
Yes, I love it.
Outside of archives, I have worked a lot with bio-medical data and with customer care data, that in the private sector.
Are you currently working for a company in your area?

@kjr no, working for a US company. I've been in customer/technical support for about 5 years. Decided to start moving more into data analysis to lay the groundwork for a possible role change next year.

@paul I am starting to look for new challenges in the private sector, since the public sector is going dry with the war efforts, and currently I have too few work.

@kjr I can imagine! You must have a pretty valuable skillset though. 🤔

@paul we will see, I have started looking for new challenges today.

@kjr בהצלחה!

@paul
תודה רבה!!
אבל לעמן האמת לא התחלתי באופן רציני.. פשוט יום קשה מאוד היום.
בוא נראה מה קורה בשבועות הבאות.
מתי אתה חושב שתוכל להתחיל עבודה בתחום ניתוח נתונים?

@kjr I'm sort of working my way into it. I'm working on a couple of projects involving data analysis as part of my regular working hours. My focus is on learning as much as I can and then take advantage of future opportunities to shift into a data role. 🤞🤞

@paul sounds good. Good luck man!!

Ramesh #NotGoingBack Nov 14, 2023

@kjr

Have you or do you plan to use Relation Ontology in your projects? ➡️ http://obofoundry.org/ontology/ro.html

@paul

OBO Foundry

Paul Jacobson Nov 15, 2023

@rameshgupta @kjr I haven't come across this at all. 🤔 I suspect I'm not the intended audience for the tool though.

Ramesh #NotGoingBack Nov 15, 2023

@paul @kjr

This tech is rooted in #graphDB rather than #relationalDB, although some implementations use relational db for storing triples for each edge of graph (2 vertices + edge relation). You then use #SPARQL instead of #SQL to find information.

What caught my eye was https://babka.social/@kjr/111381934379266332
& https://babka.social/@kjr/111381988654130074

#OBOFoundry is used extensively in bio sciences & research.

A schema-less graph repo could work well here, especially with new standardization around https://www.nist.gov/system/files/documents/2021/10/14/nist-ai-rfi-cubrc_inc_004.pdf

KjR (@[email protected])

@[email protected] I am currently finishing two projects in Hebrew archival text analysis. One is to detect entities (people, places, locations, dates and organizations) from archival collections and the relationships between them, so we can extract biographic information about the people: when and where born, name of the parents, organizations the person belonged, etc. The second one is just to extract the entities, link them to bibliographic entity files and improve the search of the data and the linking between documents of different archives.

Babka Social

Ramesh #NotGoingBack Nov 15, 2023

@paul @kjr

A big part of this technology is automated reasoning (not AI) with #OWL Direct Entailments & more.

In #relationalDB, you may only retrieve what you've explicitly put into the database.

In #graphDB with OWL reasoning, the system INFERS additional info beyond what you put in, based on strict definitions of Descriptive & First Order Logic.

So, you can glean additional inferences, sometimes subtle, based on what you actually input into the DB.

#CommonCoreOntology #RelationsOntology