Enterprise level NAS questions

[Mahaguru]

I have two questions about enterprise level NAS (multiple 100 TB+, Netapp, EMC, HP, etc with millions of files).

1) How to discover what is actually on the NAS? I have seen instances where departments kept multiple copies of the same data in different folders and then lost track. Not only does this make the content difficult to manage, it makes backups take longer, and space utilization goes up. Netapp and EMC always have these "discovery" services, but seems like they are one time and cost a lot of money. Any way to keep an ongoing track?

2) Any enterprise level migration software, other than Robocopy? Yeah I am being sarcastic. Is there software that can keep two shares (with millions of files) in sync automatically so that we can do a graceful cut over?

 

[6string]

If users access the NAS through a windows share I know windows 2008 server has some pretty cool reporting tools. I use this often to report on duplicate files, large files, old files. I'll then email users and ask them to clean them up. As for backing up multiple copies you can use a De-duplication backup solution. It looks at a file at the bit level so no bit is backed up twice. I'm currently using ArcServe and EMC Avamar for dedup backups.

 

[ImpulsE69]

NAS are not designed to care about what is actually on their disks. That is left up to outside applications.

Where I work some of the groups use Kazeon and/or Volscan to scan their file systems and run reports that are on our NAS. Won't say either are the most effective, but they work.

Most tend to use Robocopy for any kind of external migrations that need to happen where multiple shares need to be combined/moved into one or more or from one type of storage system to another, but for what you are talking about most NAS have built in mirroring solutions varying by need using point in time technologies. (Trying to stay away from speaking to any one vendor) that allow for multiple mirrors to other locations of the same data.

Also, just something to consider (not sure of your level of involvement) when you say shares with millions of files remember to use a wide and shallow folder structure with 5-10k files each because Windows tends to choke when it needs to read lots of files in one directory and NAS have limited memory and must enumerate an entire folder path every time you make a request. This means if you have a deep structure path and are accessing a file at the bottom, it must load every file in every folder in that path into memory.