David W. Jones

Any GUI Linux directory comparison tools that work?

I tried KDIFF3, it crashes partway through comparison. I understand that v1.10 has a bug and crashes when a directory has symlinks in it, as my home folder does.

I tried KDIFF3 v1.12 via the abomination called Flatpak, but it also crashes partway through scanning.

I tried MELD, it just hangs there "scanning" but uses no CPU and is doing nothing...

Ideas?

#Linux

Feb 4 at 6:55amWeb
Show threadARGVMI~1.PIFFeb 4

@dancingtreefrog

I usually use Meld, but yeah, I've seen it crash and burn sometimes. Not sure what would be better. If anybody has any ideas, I'd like to hear them!

Show threadDavid W. JonesFeb 4
@argv_minus_one I can't figure out how to do anything with Meld. Pick the two folders, it just hangs doing nothing... Ideas?
Show threadAldarch IIIFeb 4
@dancingtreefrog @argv_minus_one How much data?
Any filters set?
Comparing hashes for all files in a big data set can take a while...
Show threadDavid W. JonesFeb 4

@aldarch3 @argv_minus_one Data:

My home folder:
333.4 GiB, 200300 items (180046 files, 20253 folders)
comparing to the mirror folder on an external SSD connected via USB-C:
272.5 GiB, 137587 items (118302 files, 19284 folders)

There are items in my home folder that aren't mirrored to the external drive, that's why the difference.

Show threadDavid W. JonesFeb 4
@aldarch3 @argv_minus_one Filters? I assume you're referring to Meld? I've never set or changed any filters in it, don't know how to do that, anyway.
Show threadAldarch IIIFeb 4
@dancingtreefrog @argv_minus_one
I'd imagine that taking a while.
Not in front of it, but poke around in the settings - I think you can set it to just compare size/date and not contents.
Show threadAldarch IIIFeb 4
@dancingtreefrog @argv_minus_one
There is also rmlint (and gui).
https://rmlint.readthedocs.io/en/latest/gui.html
Graphical user interface — rmlint (2.10.3 Ludicrous Lemur) documentation

Show threadDavid W. JonesFeb 4
@aldarch3 @argv_minus_one Hmm, interesting. Kind of like a GUI tool I already have, CZKawka's fork of fslint... Except I'm not looking for file duplicates between the directories, I'm looking for files that exist only in one of them.
Show threaddirk dierickxFeb 4
@dancingtreefrog no gui tool, but good old cli diff?
Show threadDavid W. JonesFeb 4
@dirkdierickx How can I use CLI diff conveniently remove files in one of the compared directories?
Show threaddirk dierickxFeb 4

@dancingtreefrog use the command;

diff <(ls /directory_abc) <(ls /directory_xyz)

to compare two directories.

Show threadDavid W. JonesFeb 4
@dirkdierickx Thanks.
Show threaddiesUndDasMitTassen 🇺🇦Feb 4

@dancingtreefrog beyondcompare from scootersoftware, but it is not for free and not opensource, I am using it since 1997, it is very reliable and synchronizes even large files (> 2 GByte) or many files (> 100.000) without any problems, even over poor internet connections e.g. via ftp. Most tools crash with those requests. It has a simple scripting engine also.

#filecomparison #foldercomparison #textcomparison #synchronisation #backup

Show threadDavid W. JonesFeb 4
@Andreas_Sturm Thanks, never heard of it, but I'll pass.
Show threadAcorn SquashbucklerFeb 4

@dancingtreefrog Would it be worthwhile in your case to print the directory structures to files and then compare the results? It's definitely not as good, but it could be more stable.

Including file sizes could be informative. You might also be able to include file hashes to represent the content… Depends on what you're trying to do though

Show threadDavid W. JonesFeb 4
@AcornSquashbuckler Well, CLI diff can give me a list of files that are ONLY in the second folder by feeding to grep and looking only for the pathnames referring to the external drive...Might be an option.
Show threadBenAvelingFeb 4
I’ve written some perl scripts to do that. If that sounds useful, hit me up and I’ll share a copy. @AcornSquashbuckler @dancingtreefrog
Show threadpetur 😶🇺🇦🇵🇸🇹🇼Feb 4

@dancingtreefrog don't laugh, but I use WinMerge with Wine since I haven't found any Linux GUI tool that comes close....

If you only need compare, there's also FreeFileSync but for multi-threading they want a donation

edit to clarify: FreeFileSync works in 2 steps, a compare and a sync step, so you can use it for compare only

Show threadThe ObservationsFeb 4
@peturdainn @dancingtreefrog I’ll second FreeFileSync. Have been using it for years (donation edition).
Show threadDavid W. JonesFeb 4
@peturdainn Thanks. I haven't been able to make Wine work with any Windows app on my system, so I didn't even think of trying a Windows app for this.
Show threadpetur 😶🇺🇦🇵🇸🇹🇼Feb 4
@dancingtreefrog oh weird... I just get the portable version, not the installer, no special settings at all.
Show threadDavid W. JonesFeb 4
@peturdainn That's the way Wine has been for me for many years. That's OK, I don't need any Windows apps!
Show threadTriddleFeb 4

@dancingtreefrog I use both kdiff3 and meld on a regular basis on Linux and BSD with no issues, but not with directories as large as yours, only a few gigabytes and tens of thousands of files per side. Both work well at that level, the initial scan taking maybe fifteen seconds.

It's possible meld is working, but just needs more time.

It's also possible there are more files than either can handle; I don't know their upper limits.

Show threadDavid W. JonesFeb 4
@Triddle Update: I left meld running over night, it finished about 8-10 hours later and seemed to find everything.
Show threadTriddleFeb 4
@dancingtreefrog ok, that's good. Pity it takes so long, but I guess it will do what you need. Hopefully you don't need to do it too often...
Show threadDavid W. JonesFeb 4

@Triddle Thanks, the discussions inspired me to check out other options and think over my situation.

I mirror my home directory each day to two other locations, so I can get files back if needed.

When no longer needed, I need to clean up orphaned files at the other locations.

I use rsync, which can delete destination files that are missing from the source, so I put together a script to run each month that removes orphans after my regular monthly backup.

But nice to have options to KDIFF3!

Show threadHerz++Feb 4
@dancingtreefrog Kompare.
Show threadDavid W. JonesFeb 4
@herzenschein Ah, thanks, checking that one out.
Show threadMarcos DioneFeb 4
@dancingtreefrog `xxdiff`. It's TK, ugly, but for me it works better than the other two.
Show threadDavid W. JonesFeb 4
@mdione Thanks. I tried it, not particularly easy to use on my 4K display. Turning on its recursive option makes it really slow!
Show threadMarcos DioneFeb 4
@dancingtreefrog maybe it's not the recursiveness but the code size, or the diff size, or the screen size (I doubt TK is in any way optimized for modern displays or rendering library). TBH since git, it's been a long time I hadn't used it.
Show threadDavid W. JonesFeb 4

@mdione Well, TK has been improved for HIRez displays, I use a synthesizer app built on TK and it handles just fine.

Oh, well!