10's millions of files should have been the ideal use case for XFS, that's why I installed it in the first place. This was for the 'reocities.com' project and by the time I realized what the problem was most of the import had already been done so I let it run to completion but it makes updating the project a real PITA.
There's so much that can go wrong setting up a Linux server that it's impossible to give much advice with something like this.
I guess the general stuff is: the easy default partitioning setup you get from a Linux distro is total bs, you need more RAM than you think you do, the way you're serving files or accessing the system (NFS!) has plenty of ways to screw things up as well, and tens or hundreds of millions of files is not any filesystem's ideal use case. The classic IRIX workload would be guaranteed-rate streaming of large media files, and the Linux port of the filesystem obviously inherited a lot of that system's traits (without the GRIO).
XFS has received some very serious performance improvements in the past couple of years to address indexing, large volumes of metadata, and so on, so that'd be one very relevant thing. Dave Chinner's talks are worth the time to watch if you're interested. You would be giving bad advice if you steered people one way or the other with regard to filesystems based on a seven-year old project (unless you've refreshed that system much more recently, of course).
> XFS has received some very serious performance improvements in the past couple of years to address indexing, large volumes of metadata, and so on, so that'd be one very relevant thing.
That's probably the difference right there. Thanks for pointing that out.
Sure, but the issue could be configuration, drive, interface, etc. It's impossible to speculate in, but what we know is you have trouble with one machine, and it's the only one that has used XFS. It's unfortunate, but likely a coincidence, or at least unrelated to XFS at its core.
I've been using XFS for 10 years without the issues you seem to be having.
