Advice for moving my files (or if I should even bother) - sh.itjust.works
Hey Folks, I have a bit of a conundrum that I’m hoping the hive mind can assist
with. I am in the process of learning docker to prep for my migration to Linux,
but I have some questions about my filesystem structure. Currently my media
files of all types live on a single file-based iSCSI LUN hosted on a QNAP which
I connect to from a Windows machine. In my research to see if this would be
consistent with best practice, I came to the conclusion that I should create
independent NFS shares that the docker containers would connect to individually,
rather than serving the files to the containers through the host and it’s iSCSI
connection. This leads to my problem. I can’t seem to find any way to directly
copy data from the LUN to one of my newly created NFS shares. With the volume of
data I’ll need to copy I’m trying to avoid as much overhead as possible, and
using my Windows machine to connect to the new NFS share, then transferring the
files from the iSCSI share, would be ludicrously inefficient. As I’m able to SSH
into my NAS, my first thought was to try and mount the iSCSI file locally and
rsync the contents directly to the NFS share. After finding the home of the
iSCSI file in the NAS filesystem, I discovered that it is not stored as a
single, mountable file, but broken up into 1TB chunks. This leaves me unable to
mount it, even in part, as each of the files lack an identifiable filesystem.
Further, this is my largest partition, and so I don’t (currently) have the space
to attempt to concatenate the files into a single file (assuming that would even
work, no idea). After giving up on this approach, I decided to try and log into
it’s own external iSCSI target (from the NAS), then mount the LUN as I would
from an external client. I thought I might be in the clear, as the login was
successful, and both iscsiadm and the NAS GUI showed the active session to
itself. But no matter where I looked I could see no evidence of a newly
available partition, only those that were there from before I connected to the
iSCSI target. At this point the next step seems to be shrinking the partition
and trying to concatenate the iSCSI files as I mentioned earlier. I have the
space to play with, but I’ll need to convert the volume to thin-provisioned,
then shrink the volume, which would likely take foreverrrrrrr. But really, even
this option sucks, because I’d prefer to avoid jeopardizing my primary storage
volume in changing the provisioning style. So anyway, after banging my head on
it for the last few hours, I decided to step away and do some “rubber ducky
debugging” with you guys. So here are my questions: Is migrating to NFS worth
the effort? Would the file concatenation method even work? COULD the loopback
iSCSI method work if I do something differently? Any other tricks, or maybe
something in the QNAP App Marketplace? Any assistance welcome, thanks for
reading!
