Can You Do More Than 255?

In my role here at Datadobi, talking about moving, replicating, and syncing unstructured data all the time, I cannot begin to tell you how often I hear people ask if we can handle long file names or long path names.  They of course ask the question for a valid reason.  Many legacy tools cannot handle paths over 255 characters.

The short answer?

Our DobiMiner Suite doesn’t have a limit as long as the source system, target system, or the protocol in-use doesn’t have a limit.

The longer answer: with step-by-step-proof

Step 1: Create some sample data

Create a 2G filesystem on a VNX:

Mount the filesystem:

Create an export of it:

Now write a simple shell script/loop to create a dataset: (60 directories wide by 60 directories deep).  The file name for the curious is the first line of the first page from War and Peace by Leo Tolstoy.

Run the script:

Files successfully created:

Step 2: In DobiMigrate, setup a migration of this filesystem

Create a Migration from (in this case, the VNX, to Isilon):

Perform the migration (with NFS in this case, but SMB works the same for this test):

Step 3: Evaluate the results

Look at the structure and data integrity

So here is a file that is 61+ levels deep in the source VNX structure:

Here is the MD5 hash of that file on the source:

Here is the preserved structure on the target:

Here is the same file on the target:

Notice the MD5 hashes match exactly, as do the paths relative to what we picked as the path on the source and path on the target.

Look at the Character Count

Frequently in migrations many legacy tools run up against a character limit of 255 characters for the entire path.

So, let’s just take a quick view on how many characters are actually in the path we migrated above (excluding the hostname).  From the root of the VNX control station to the root of the filesystem there are 29 characters.

I picked a single path to go down and here is the full structure:

So, that’s 600 characters of folder names.  629-29 (how the control station sees it) = 600.  Plus another 86 characters for the filename within that folder.

In Summary

We successfully migrated a file that was 60 levels deep with a file with 86 characters in the filename.  That’s a total of 686 characters in the path, not including the server name or domain name, and we handled it without a problem.  As you can see above the file checksum is also identical source-to-target.  So to the question of our title above: “Can you do more than 255?”  with DobiMigrate (and our whole DobiMiner suite) the answer is unequivocally YES!

Appendix: Limits on Different NAS Systems & Protocols

Dell EMC VNX

VNX has a limit of 255 bytes over NFS but 255 characters over SMB/CIFS. (See page 9 of this guide for more information): https://support.emc.com/docu32528_NAS-Support-Matrix.pdf

Dell EMC Isilon

Isilon’s OneFS has a limit (currently) of 255 bytes in a filename. (See page 13 of this guide for more information): https://www.emc.com/collateral/TechnicalDocument/docu84267.pdf

NetApp ONTAP9

ONTAP has a limit of 255 characters in a filename regardless of protocol:

https://kb.netapp.com/app/answers/answer_view/a_id/1000976

Another helpful page on NetApp ONTAP 9 that explains a number of these concepts very well:

https://library.netapp.com/ecm/ecm_get_file/ECMLP2494081

Protocol Limitations

Ultimately these limits are all bound by both the storage system and the protocol.

SMB

http://msdn.microsoft.com/en-us/library/aa365247%28v=vs.85%29.aspx#maxpath

“A pathname MUST be no more than 32,760 characters in length.”

“Each pathname component MUST be no more than 255 characters in length.”

https://docs.microsoft.com/en-us/windows/desktop/fileio/naming-a-file

Talks about all the different standards and reasons for them such as historical 8.3 format, etc.

NFS

http://community.netapp.com/t5/Network-Storage-Protocols-Discussions/NetAPP-Data-ONTAP-length-of-filenames-restricted-to-255-characters/td-p/27447

https://tools.ietf.org/html/rfc1813

“Any POSIX limitations in the remote server filesystem will be passed through to the client. For example, the length of filenames may vary across servers from different operating systems. NFS (versions 2 and 3) limits filenames to 255 characters; mountd (versions 1 and 3) limits pathnames to 1024 characters.”