February 18th, 2008
What is Lustre? It is a cluster file system developed by Cluster File Systems, Inc., which was recently purchased by Sun. We do not have much experience with Lustre but we have played around a bit with it in house and I have talked a lot with other centers who do use it (like Sanger). The main advantages of Lustre are that it is very scalable and can sustain very high performance. The two main problems with Lustre is that it is extremely difficult to get working and it is very difficult to back up your data on a Lustre file system. The difficulty in getting it to work seems to be a business strategy of Cluster File Systems. It is hard to get paid for installation and configuration consulting if you have a well-documented, easy to perform process. It will be interesting to see if this will change given Sun CEO Jonathan Schwartz‘s well-publicized commitment to open source software. The second difficulty, back up, may be a bit of a red herring. Do you really want to put archive-worthy data on a high-performance, clustered, 20 TB file systems that depends on a metadata server for file system integrity? Lustre file systems should really be used for high-performance scratch space. Once the calculations are done, move the data somewhere else. Of course, if you have a lot of data, this may take a while. So you may want to move it two places: on-line and off-line storage. Because if you have a lot of data to move, it will also be a lot of data to back up. So you may just want to dump it to tape one time while you are dumping it to your on-line storage.