4.24

Important note 4.23 is not safe when there are multiple disks configured and erasure coding(EC) is using the worker. The worker added a capability to distribute EC shards to different disks to ensure proper shard distribution. However, the volume...

https://github.com/seaweedfs/seaweedfs/releases/tag/4.24

#seaweedfs #s3 #objectstorage

Release 4.24 · seaweedfs/seaweedfs

Important note 4.23 is not safe when there are multiple disks configured and erasure coding(EC) is using the worker. The worker added a capability to distribute EC shards to different disks to ensu...

GitHub

4.23 (Not safe for erasure coding and multi-disk volume servers)

What's Changed Shell fix(shell): correct volume.list -writable filter unit and comparison by @qzhello in #9231 shell: expand ~ in local file path arguments by @chrislusf in #9265 fix(shell): use exact match for volume.balance -racks/-nodes...

https://github.com/seaweedfs/seaweedfs/releases/tag/4.23

#seaweedfs #s3 #objectstorage

Release 4.23 · seaweedfs/seaweedfs

What's Changed Shell fix(shell): correct volume.list -writable filter unit and comparison by @qzhello in #9231 shell: expand ~ in local file path arguments by @chrislusf in #9265 fix(shell): use...

GitHub
What object storage services would you recommend? I tried Hetzner's object storage, but it felt like they kept raising the prices month after month. I usually use Backblaze and haven't been able to find anything better so far. #s3 #objectstorage #object #storage #backblaze #hetzner #b2

s4 – Super Simple Storage Service

S4는 경량화된 자체 포함형 S3 호환 스토리지 솔루션으로, Ceph RADOS Gateway와 표준 파일시스템을 결합하여 단일 컨테이너로 배포할 수 있습니다. 웹 기반 관리 UI를 제공하며, POC, 개발 환경, 데모 및 간단한 배포에 적합합니다. 완전한 S3 API 호환성을 지원하고, 버킷 관리, 객체 조작, 스토리지 간 파일 전송, 웹훅 알림, HuggingFace 모델 직접 임포트 등의 기능을 갖추고 있습니다. Kubernetes 및 OpenShift 환경에서 Helm 차트로 쉽게 설치할 수 있습니다.

https://github.com/rh-aiservices-bu/s4

#s3 #objectstorage #ceph #kubernetes #helm

GitHub - rh-aiservices-bu/s4: Super Simple Storage Service

Super Simple Storage Service. Contribute to rh-aiservices-bu/s4 development by creating an account on GitHub.

GitHub

OpenAI's Data Agent and the S3 Gap

OpenAI가 600PB 규모의 비정형 데이터(S3, GCS, Azure 등) 처리를 위해 구축한 데이터 에이전트 스택은 기존 구조화 데이터 웨어하우스와 달리 스키마, 데이터셋, 파일 참조, 계보(lineage)라는 네 가지 기본 요소를 새롭게 정의하고 구현했다. 특히, Pydantic을 활용해 파일 기반 데이터의 스키마를 코드로 표현하고, 파일 참조를 타입화하여 원본 바이트를 직접 가리키는 방식을 도입했다. 또한, 데이터셋을 이름과 버전으로 관리해 협업과 에이전트 활용을 용이하게 하는 점이 핵심이다. 이 접근법은 멀티모달 AI, 신경과학, 물리 AI 등 대규모 비정형 데이터 처리에 필수적인 인프라로 평가된다.

https://datachain.ai/blog/openai-data-agent-s3-gap

#openai #dataagent #objectstorage #pydantic #multimodal

OpenAI's Data Agent and the S3 Gap

OpenAI built their in-house data agent for structured warehouse data, where schema, lineage, and queries come for free. Files in S3, GCS, or Azure - videos, sensor logs, image corpora, PDFs - have none of that, and the problems get a lot more interesting. Here is how we built the four foundations that close the gap.

DataChain

"There is a new release of SeaweedFS. Happy updating your instance and don't forget to check the changelog."

# 4.23

What's Changed Shell fix(shell): correct volume.list -writable filter unit and comparison by @qzhello in #9231 shell: expand ~ in local file path arguments by @chrislusf in #9265 fix(shell): use exact match for volume.balance -racks/-nodes...

https://github.com/seaweedfs/seaweedfs/releases/tag/4.23

#seaweedfs #s3 #objectstorage

Release 4.23 · seaweedfs/seaweedfs

What's Changed Shell fix(shell): correct volume.list -writable filter unit and comparison by @qzhello in #9231 shell: expand ~ in local file path arguments by @chrislusf in #9265 fix(shell): use...

GitHub

#MinIO Is Done With #OpenSource, What Are Your Options?
MinIO is one of the most widely used #selfhosted #objectstorage solutions out there. It is #S3-compatible, lightweight, and runs as a single binary, integrating with pretty much everything in the #cloudnative stack.

Here are the three open source alternatives worth looking at. 👇
#SeaweedFS
#Garage
#RustFS

I have heard good things about Garage but haven't kicked the tires yet.
https://itsfoss.com/news/minio-moves-away-from-open-source/

MinIO Is Done With Open Source, What Are Your Options?

Archived, unarchived, and archived again, the repo's status may keep changing, but MinIO's direction hasn't.

It's FOSS

Garage Self-hosted S3 object storage - Tutorial - Set up on fresh VPS & Sync Obsidian Notes

https://northtube.ca/w/89ProXBYfKfp9W3EfYUUPg

Garage Self-hosted S3 object storage - Tutorial - Set up on fresh VPS & Sync Obsidian Notes

PeerTube

"There is a new release of SeaweedFS. Happy updating your instance and don't forget to check the changelog."

# 4.22

> Comments For clusters with multiple volume servers with multiple disks, and if Erasure Coding is used, please upgrade! #9219 fixed a problem when the volume server is restarted, some EC shards may be reported as missing while actually they are...

https://github.com/seaweedfs/seaweedfs/releases/tag/4.22

#seaweedfs #s3 #objectstorage

Release 4.22 · seaweedfs/seaweedfs

Comments For clusters with multiple volume servers with multiple disks, and if Erasure Coding is used, please upgrade! #9219 fixed a problem when the volume server is restarted, some EC shards may ...

GitHub
So, dann lade ich mal was hoch und schau, ob es im S3-Bucket landet. Zum Beispiel das Bild, das zeigt, das die Hühnereier bald zurückschlagen werden.
#Mastodon #ObjectStorage #Test