McDonald Observatory: HETDEX Opens Massive Cosmic Dataset to Scientists, Novices, and AI. “Today, the Hobby-Eberly Telescope Dark Energy Experiment (HETDEX) – which recently completed the largest survey ever taken of the early universe – has released all of its immense, information-rich database to the public.”

FedTech Magazine: How NIH Is Translating 70 Years of Health Data to Speak the Same Language. “The National Institutes of Health sits on one of the largest collections of biomedical research data in the world. Decades of federally funded studies on health issues such as heart disease, lung conditions, sleep disorders and genomics have generated petabytes of information. But for most of that […]

Modern scraping problem:

Your parser is fine.
Your response is blocked 😅

I tested Bright Data Web Unlocker API with Python + BeautifulSoup to fetch protected, JS-rendered pages without managing proxies.

Full article 👇https://medium.com/gitconnected/how-i-scraped-modern-protected-websites-in-python-without-managing-a-single-proxy-2e0f07d30208

#DataEngineering
#WebScraping
#Datasets
#MachineLearning
#RAG

I spent 3 days building a LinkedIn scraper.

Then I found the dataset already existed 😅

Sometimes the best engineering decision is not to scrape more — but to check whether structured data is already available.

Full article 👇
https://medium.com/gitconnected/i-spent-3-days-building-a-linkedin-scraper-then-i-found-the-dataset-already-existed-9e9093504ca1

#DataEngineering
#WebScraping
#Datasets
#MachineLearning
#RAG

#gov #BigData #datasets #FederalData

'This Federal Data Field Guide includes more than eighty specific examples of federal datasets across more than 50 federal agencies.'

https://www.ischool.berkeley.edu/news/2026/new-federal-data-field-guide-helps-americans-navigate-rich-diversity-our-federal-data

UC Berkeley: New Federal Data Field Guide Helps Americans Navigate the Rich Diversity of Our Federal Data Ecosystem. “Denice Ross, who served as the nation’s second U.S. Chief Data Scientist, and her former White House colleague Christopher Marcum have launched the Federal Data Field Guide, a free, plain-language resource designed to help Americans understand, use, and advocate for the full […]

I am always surprised to see how, against all odds, CSV files remain the backbone of #data ingestion. What is even more surprising is how difficult it is to simply look at or quickly edit them without breaking something. So I built:

https://github.com/CedricBonjour/nanocell-csv

Coded with ❤️

I want to shape it around actual #dataengineering workflows. What feature would make this an instant bookmark for you?

like it ? Feel free to drop a star or break it with some of your worst #datasets and open an issue!

"The open-source database of sanctions, watchlists, and politically exposed persons — aggregating hundreds of sources and relied on by compliance teams, investigators, and journalists. OpenSanctions is a financial crime data provider. "

#datasets #compliance #regulation

https://www.opensanctions.org/docs/about/