Michael DexterDid UNIX/Unix tapes have live file systems to dump in or tree of resources to pull from? Or something else!
cynicalsecurity 
@dexter I used to boot AIX on a 1st gen RS/6000 off tape :)
Michael Galassi@dexter In the Unix world storage (tape/disk/whatever) is just a series of bytes, you can put anything you like there. The most common use of tapes (in the Unix world) is tar archives which are documented in
man 5 tar. Many more formats are documented in man 5 libarchive-formats. man 8 dump is also relevant.@michaelgalassi Well yeah but what about the famous BSD distribution tapes or recently unearthed UNIX ones? When “distributing”, what form did that take in relationship to, if any, booting the target system?
Wintermute_BBS@dexter @michaelgalassi they sent you a tape for your system (e.g. PDP-11) with instructions to issue at the hardware boot console + switches. then you booted from tape the way @erikarn described. then you went on to install (from tape to disk).
@dexter Historically tape drives are character devices, not block devices. File systems require block devices. That's not to say that you could have a special boot loader which read in a tape and treated it like a disk but early Unix systems ran in 64k words of memory or less, not enough to run from a memory disk. I can however imagine an equivalent of dd from the tape drive to the disk device followed by a regular boot from the disk drive.
Adrian Chadd <verified.png>@dexter so yeah that's what 'tar' and 'pax' and 'cpio' were for. And for booting they'd typically boot the first stretch of files to an EOF/EOR marker on the tape, run that as the kernel; the kernel would then suck the next stretch until EOF/EOR into swap and boot that.
And yeah, I think sunos had mini-root that ran in RAM to format/partition a drive so you DID have swap. THen you'd boot it again from tape to do the install.
Garrett Wollman@erikarn @dexter Worth noting that very few disk devices were actually supported in early Unix and the partition layouts were compiled into the kernel, so there was no need for (or possibility of) labeling. I believe the original BSD disklabel was invented by Robert Elz to deal with the proliferation of disks, especially on workstations with SCSI.
@dexter @erikarn Oh, I also forgot about /stand. The purpose of the stuff in /stand was to have a set of executables you could load *instead of a kernel* (hence "standalone") to fix stuff. That would often include utilities for handling device-to-device copies, so on some systems, a boot tape would have a boot loader in the first file, the standalone copy utility in the second file, and then a filesystem image in the third file. That way you wouldn't have to input a copy utility from paper tape.
@dexter @erikarn For what it's worth, all my FreeBSD installs use a live filesystem, never the installer. On the filesystem image I have a filesystem image (it's literally the same filesystem, modulo a couple changes to the configuration so it boots without network) which is then `zfs recv`d to the newly initialized boot pool. Some day I will write a proper unattended installer, but I've been cloning this way for nearly 15 years.
@dexter Well, you couldn't use any Unix filesystem on a sequential medium like tape. But I do recall that (in the pre-disklabel days) it was common to use a low-level monitor routine to copy an installer filesystem (containing just a kernel and specialized init) from tape to the future swap partition of the boot disk, then manually boot from that to complete the install from the next "file" on the tape which was in tar/cpio format.
Tom Lyon ✅@wollman @dexter Lots of such tape images here: http://www.bitsavers.org/bits/Sun/
But, even older, the original DECtapes could be randomly read & written, so in hours (not moments) of desperation they could host filesystems.
Poul-Henning Kamp
Philip Guenther@dexter When I was a student sysadmin at St Olaf college, Craig Rice, the senior admin dug out the v6 and v7 tapes and had us take a stab at extracting them, pointing me at Mike Haertel (an Olaf graduate) for help. That was, uh, 1993?
Ahah! I found my notes from talk(1)ing with him!
-----v6----
Greetings from the distant past...
So, you've gotten interested in those old tapes, eh?
You're gonna love this...
the v6 tape is 12100 512-byte blocks.
The first 100 blocks are a "tp" format archive.
Yeah, tp. Dig up the old v6 manuals. :-)
The remaining 12000 blocks are evenly split into 3 4000 block
file system images, the first being root, the second /usr/source,
and the third /usr/docs.
(They had different names in those days.)
So, to read those tapes, you're going to have to understand the v7
(I mean v6) file system format. The stuff in the tp archive at
the beginning doesn't relly matter--it is a boot block, and then
a few programs meant to be booted by the boot block. Its main
purpose is to extract the rest of the tape.
Anyway, I went through all this a few years ago. It turned out
to be remarkably simple to write a simple program to read the file
system image, but I no longer have it.
----v7----
Now, about the v7 tape. The format of that tape is as follows:
There are ... 5 files consisting of 512 byte records,
and 2 files consisting of 10240 byte records.
The first 5 files are basically boot stuff again.
The second or third is just a list of the tape contents,
the others are standalone "cat", standalone "mkfs", and standalone "restor",
and the boot block itself.
Then, the last two files are v7 dumps of root and usr.
I have a program I wrote to extract those.
And I still have the sources for that one. So I'll just email it to you.
@dexter Finally, Mike's comments on filesystem image vs dump format
----
Incidentally, dumps are much more of a pain than file system images
to extract.
It's hard to describe.
No, it's not that. It's just that...(looking at v7restore source)
Ah, that's right. In order to extract a dump, you need to first
build an in-core picture of the hierarchy.
Because the dump is just done of inode numbers in order.
Then, once you know what name to associate with each inode, you
can extract things.
Traversing a file system image is much simpler--you just start
at the root directory inode, and do the moral equivalent of a recursive
walk on the file system, extracting the files as you visit them.
The problem is that dumps are in linear order, and don't contain
convenient seek addresses for traversing in logical order.
----
Incidentally, dumps are much more of a pain than file system images
to extract.
It's hard to describe.
No, it's not that. It's just that...(looking at v7restore source)
Ah, that's right. In order to extract a dump, you need to first
build an in-core picture of the hierarchy.
Because the dump is just done of inode numbers in order.
Then, once you know what name to associate with each inode, you
can extract things.
Traversing a file system image is much simpler--you just start
at the root directory inode, and do the moral equivalent of a recursive
walk on the file system, extracting the files as you visit them.
The problem is that dumps are in linear order, and don't contain
convenient seek addresses for traversing in logical order.
ティージェーグレェSt. Olaf? Hah! I was assigned a student sysadmin job when I attended Carleton College!
Dave Diehl, told me I couldn't work the job I was assigned, because it was: "reserved for upperclassmen" and I was only a freshman. Perhaps the first time he was a thorn in my side, not the last. ;( He also seemed to think I was a "hacker" because I had listed a Commodore Amiga on my résumé, but he clearly didn't mean it in a complimentary fashion. ;(
Worse: I was told by the MathCS department: I couldn't use the SGIs, because I was also "not an upperclassmen" even though I had already been toiling on such systems (and nicer ones, some of them, down to their microcode) at nps.navy.mil (now nps.edu) before I was even at Carleton and had toured SGI's campus more than once. Indeed, seeing the SGIs in Carleton's newly renovated (this was circa 1994) MathCS department was one of the reasons I decided to attend. (A gorgeous red head I met during the prospective student week was another reason, but alas, she also wasn't interested in me beyond friendship.)
Instead? I ended up slogging away on their awful NeXT systems and horrid VMS running Vax and such. I think they later had one DEC Alpha that ran Tru64 that only professors used, I got to see one in person, but wasn't allowed "time" on it.
sigh I did eventually (winter of my first year) get a different student job after someone in my dorm was tipped off to me being a nerd, so I ended up working as a "Divisional Computing Consultant" for Humanities faculty and was granted all the elevated privileges I presumably would have received if I hadn't been thrown under a bus by Dave Diehl (whom I later learned: never even graduated, he just kind of kept hanging around the geek houses and became a fixture on campus).
So much grief, for such backwater systems, and minimum wage student job pay at that! IIRC, Carleton's uplink for the entire campus was a 56K leased line when I started there (my 14.4 modem presumably ate up about a quarter of the bandwidth before they upgraded). Oh yeah, and whether it was UPS or the mail room (I suspect the latter) my Amiga 2000 was smashed en route to college. I didn't end up getting a replacement (thanks to someone selling a used A1200 on comp.sys.amiga.marketplace for about what the UPS insurance payout was) until my second trimester. Which meant that my first trimester I was mostly sitting in front of their dumb terminals and those awful NeXT systems. Eventually the MathCS department upgraded to a 486dx2 66MHz NeXTStep lab and that super slow OS became barely usable. Omniweb, almost made the World Wide Web seem, bearable. I still think it's mostly awful. (Not that I was a proponent of gopher).
I know older UNIX sources exist (I helped restore some of them).
Do any of you know Bob Fabry? It was his birthday on the 12th, apparently he turned 86!
"In 1971, Berkeley Professor Bob Fabry buys a $99 copy of UNIX and provides it to a group of students, including Bill Joy, who modify the original code to include a number of new features." (via https://www.berkeley.edu/about/history-discoveries/ click on Contributions/Discoveries and then the 1977: "Berkeley UNIX and the birth of open-source software" as a citation for that quote)
I haven't crossed paths with him personally, despite doubtlessly knowing many in common (I work in Berkeley these days too); but I am guessing he has some memories in such realms?
CC: @[email protected]
Dave Diehl, told me I couldn't work the job I was assigned, because it was: "reserved for upperclassmen" and I was only a freshman. Perhaps the first time he was a thorn in my side, not the last. ;( He also seemed to think I was a "hacker" because I had listed a Commodore Amiga on my résumé, but he clearly didn't mean it in a complimentary fashion. ;(
Worse: I was told by the MathCS department: I couldn't use the SGIs, because I was also "not an upperclassmen" even though I had already been toiling on such systems (and nicer ones, some of them, down to their microcode) at nps.navy.mil (now nps.edu) before I was even at Carleton and had toured SGI's campus more than once. Indeed, seeing the SGIs in Carleton's newly renovated (this was circa 1994) MathCS department was one of the reasons I decided to attend. (A gorgeous red head I met during the prospective student week was another reason, but alas, she also wasn't interested in me beyond friendship.)
Instead? I ended up slogging away on their awful NeXT systems and horrid VMS running Vax and such. I think they later had one DEC Alpha that ran Tru64 that only professors used, I got to see one in person, but wasn't allowed "time" on it.
sigh I did eventually (winter of my first year) get a different student job after someone in my dorm was tipped off to me being a nerd, so I ended up working as a "Divisional Computing Consultant" for Humanities faculty and was granted all the elevated privileges I presumably would have received if I hadn't been thrown under a bus by Dave Diehl (whom I later learned: never even graduated, he just kind of kept hanging around the geek houses and became a fixture on campus).
So much grief, for such backwater systems, and minimum wage student job pay at that! IIRC, Carleton's uplink for the entire campus was a 56K leased line when I started there (my 14.4 modem presumably ate up about a quarter of the bandwidth before they upgraded). Oh yeah, and whether it was UPS or the mail room (I suspect the latter) my Amiga 2000 was smashed en route to college. I didn't end up getting a replacement (thanks to someone selling a used A1200 on comp.sys.amiga.marketplace for about what the UPS insurance payout was) until my second trimester. Which meant that my first trimester I was mostly sitting in front of their dumb terminals and those awful NeXT systems. Eventually the MathCS department upgraded to a 486dx2 66MHz NeXTStep lab and that super slow OS became barely usable. Omniweb, almost made the World Wide Web seem, bearable. I still think it's mostly awful. (Not that I was a proponent of gopher).
I know older UNIX sources exist (I helped restore some of them).
Do any of you know Bob Fabry? It was his birthday on the 12th, apparently he turned 86!
"In 1971, Berkeley Professor Bob Fabry buys a $99 copy of UNIX and provides it to a group of students, including Bill Joy, who modify the original code to include a number of new features." (via https://www.berkeley.edu/about/history-discoveries/ click on Contributions/Discoveries and then the 1977: "Berkeley UNIX and the birth of open-source software" as a citation for that quote)
I haven't crossed paths with him personally, despite doubtlessly knowing many in common (I work in Berkeley these days too); but I am guessing he has some memories in such realms?
CC: @[email protected]