RocketDivision.Com RDS - software that rocks!    FAQ                       Target Disk Fragmentation Goto page 1, 2  Next          RocketDivision.Com Forum Index -> StarWind View previous topic :: View next topic   Author Message Brandon Joined: 21 Jun 2005 Posts: 14 Location: Olympia, WA - USA Posted: T    Post subject: Target Disk Fragmentation I'm experiencing a pretty severe issue with fragmentation during large/long file copies from an initiator. Scenario: Target System - Starwind 2.6.3 3.6TB RAID5 - (11) 400GB - 3ware 9500S-12 1.5TB Sparse Image Initiator System - MS Initiator 1.06 Copying from external 250GB HDDs through the iSCSI connection to the 1.5TB volume. The copy starts fine but gradually slows down and eventually stops all together. The target array fragments severely as the file copy progresses. It appears to be a file system issue on the target but I would like to hear other people’s impressions. I'm going to be trying different strip size and cluster sizes on the array. Has anyone else experienced these types of issues? Thanks, Brandon Back to top Brandon Joined: 21 Jun 2005 Posts: 14 Location: Olympia, WA - USA Posted: T    Post subject: I should note that some colleagues have been testing a similar target setup but have been using WinTarget (String Bean) instead of StarWind. They have reported very little fragmentation during long data writes to an image file on the array. I have not witnessed every detail personally but it sounds like something software related during the data writes. Any ideas? Brandon Back to top DJ_Datte Joined: 18 Jul 2005 Posts: 27 Posted: T    Post subject: Hello! I will have to sign this one too! But, I have found a way around it, dont use sparse image. With your writing performace, you should be able to make a flat image in about a hour, and be done with it. I've had similiar problems, with fragmentation running in thousands parts of a file, making performance become like using raided floppydrives I belive its a windows issue tho, not a starwind issue. It might be correctable by starwind tho, if for example, when starwind knows a 100mb file is incomming, it will allocate all 100mb of the sparse file, before it begins the write to disk! /Damir Back to top anton (staff) Joined: 18 Jun 2004 Posts: 1128 Location: Ukraine Posted: T    Post subject: Absolutely correct! StarWind can work with non-allocated images, other cannot. This (sparse files) leaves you with comparably free disk but you have a penalty of extra allocation write per every physical iSCSI write. Another thing - it's a good idea to have cluster size of 64KB+ on volume holding iSCSI image. _________________ Regards, Anton Kolomyeytsev Rocket Division Software Back to top Tim Joined: 18 Jul 2005 Posts: 54 Posted: W    Post subject: How do I create a multi-gigabyte zero filled image file? Not sparse but fully allocated for maximum performance. Back to top valery (staff) Joined: 29 Jun 2004 Posts: 280 Posted: W    Post subject: Tim wrote: How do I create a multi-gigabyte zero filled image file? Not sparse but fully allocated for maximum performance. Tim, You could use our mksparse.exe with '-o' parameter. F.e. mksparse.exe -o file.img 1024G _________________ Best regards, Valery Back to top Tim Joined: 18 Jul 2005 Posts: 54 Posted: W    Post subject: Quote: You could use our mksparse.exe with '-o' parameter. Ah, so that's what the '-o' parameter means. A quick look around didn't reveal any documentation for the mksparse program. Thanks. Back to top valery (staff) Joined: 29 Jun 2004 Posts: 280 Posted: W    Post subject: Tim wrote: Quote: You could use our mksparse.exe with '-o' parameter. Ah, so that's what the '-o' parameter means. A quick look around didn't reveal any documentation for the mksparse program. Thanks. Just start the utility without parameters and you will see: Quote: mksparse: Image file creation utility Version 1.1 Usage: mksparse [-o|-extend] <filename> size[k|M|G] Examples: mksparse file.img 10M -- create a new 10MB image (sparse file) mksparse -o file.img 10M -- create a new 10MB image (flat file) mksparse -extend c:\temp\image.dat 100M -- extend an existing image by 100MB Note: size will be aligned on sector boundary. _________________ Best regards, Valery Back to top Brandon Joined: 21 Jun 2005 Posts: 14 Location: Olympia, WA - USA Posted: W    Post subject: DJ_Datte: Thanks for the suggestion. The flat file fixed the problem. I wrote 300GB of data and there was practically no fragmentation on the target. I made sure to defrag the empty flat file prior to writing any data. Brandon, Back to top DJ_Datte Joined: 18 Jul 2005 Posts: 27 Posted: W    Post subject: Brandon: You are welcome Dont forget that you can also defragment the actual drive on the initiator too from time to time, just like a normal drive =). Hey, I got a question for you. When you defragmented, did you also notice that windows writes a part of the file on the start of the disk, then makes a gap thats about 20% of the capacity, then continues writing the file, so there is no way making so you dont have the flat image atleast in 2 parts (which isnt bad, just interesting ntfs behaviour). /Damir Back to top Tim Joined: 18 Jul 2005 Posts: 54 Posted: W    Post subject: Quote: Hey, I got a question for you. When you defragmented, did you also notice that windows writes a part of the file on the start of the disk, then makes a gap thats about 20% of the capacity, then continues writing the file, so there is no way making so you dont have the flat image atleast in 2 parts (which isnt bad, just interesting ntfs behaviour). That'll be the MFT getting in the way. It normally sits near the start of the partition. If you are only using the drive for large image file(s) then you can recover some space from the MFT (can't remember how ATM). You only need a 'normal' size MFT when you there are lots of files in the filesystem. Back to top DJ_Datte Joined: 18 Jul 2005 Posts: 27 Posted: W    Post subject: Uhm, a 200GB MFT ? /Damir Back to top Tim Joined: 18 Jul 2005 Posts: 54 Posted: W    Post subject: Quote: Uhm, a 200GB MFT ? Maybe, depends on the total partition size. Standard MFT size is 12.5% of total free space when formatted. There is a registry option to change this to 25%, 37.5 or 50%. Back to top anton (staff) Joined: 18 Jun 2004 Posts: 1128 Location: Ukraine Posted: T    Post subject: Just a hint: use raw volumes w/o file system at all. Would save some place and could avoid "piggybacks" with growing cache size on server machine. Tim wrote: Quote: Uhm, a 200GB MFT ? Maybe, depends on the total partition size. Standard MFT size is 12.5% of total free space when formatted. There is a registry option to change this to 25%, 37.5 or 50%. _________________ Regards, Anton Kolomyeytsev Rocket Division Software Back to top DJ_Datte Joined: 18 Jul 2005 Posts: 27 Posted: T    Post subject: I've tried raw volumes.. But do they work with the IBV plugin ? As I understand it, they dont ? Or am I wrong ? /Damir Back to top        RocketDivision.Com Forum Index -> StarWind Goto page 1, 2  Next Page 1 of 2