Network Attached Storage, strange slowness

[shmokes]

I have a Buffalo Terastation NAS device that I store all my media on.  Recently I gave my neighbor's access to it, but they've got a couple of kids so I want to separate out the rated R stuff and the TV shows that aren't appropriate for children, so that the parents can watch them, but the children can't.  This should be easy: just make an everybody folder and a password protected folder and sort all the files into them.  But for some reason, moving the files is taking FOREVER.  I tried to move my music folder, which is under 5 GB, and it was going to take at least all night, and that just ain't going to work when it comes time to move hundreds of gigabytes worth of TV shows and movies. 

I don't get it.  It wouldn't even make sense to take that long if I were copying the files, but cutting/pasting them (same drive, just moving from one folder to another) should be practically instantaneous.  The only thing I can think of is that, for some reason, it's copying the file over my network to a temporary location my desktop computer before copying it back over the network to the new location back on the NAS device.  The NAS is sitting on a network bridge so it can communicate wirelessly over a mediocre 802.11g signal, which would explain the slow transfer speed if each file is having to copy over a slow wireless network twice before settling in its new location. Does my hunch sound right?  If so, is there any way to tell it to stop using my PC to temporarily store the files, so that the entire copy operation takes place on the network drive?

Am I even making sense?

[ark_ader]

Yes you are making sense, but most of the nas drives out there use some kind of Linux OS.  I got the very same issue with my Airlink Nas (which is crap BTW avoid it).  Looks like the fetch is coming from your PC via the network instead from the unit directly.

With regards to the R rated stuff I would just reattach the drive or buy another format it to NTFS.  Then the NAS (most NAS drives) treat the drive read only.  Do your file movements locally via USB external, then pop it back into the NAS.

MAKE SURE YOU BACK IT UP!

I lost the entire drive to the Airlink.     It    me right off.

If it was me I would use an old PC laptop and have it sitting on the network, than those NAS drives...

[shmokes]

I don't know . . . it doesn't make sense.  It can't possibly be designed to work that way.  It's just so incredibly inefficient.  People wouldn't even use NAS if it doubled the time of every copy operationa and about 1,000,000x'ed the time of a typical cut/paste operation.  This has to be fixable.

[Samstag]

I don't know . . . it doesn't make sense.  It can't possibly be designed to work that way.  It's just so incredibly inefficient.  People wouldn't even use NAS if it doubled the time of every copy operationa and about 1,000,000x'ed the time of a typical cut/paste operation.  This has to be fixable.

I don't think the Windows filesharing protocol has any mechanism for local copy commands.  There's not much a NAS designer can do about that.  If your device can support telnet/SSH sessions you can log in and do local copies/moves which will be instantaneous.  That's what I do in these situations, but I'm running a linux PC.

[bishmasterb]

I believe samstag is correct.

If the same thing (file move on a remote server) was being done across a network to file targets on a Windows Server, you'd have the same issue.

The solution, as samstag pointed out, is to always initiate file copies/moves locally on the system hosting the data. This can be done via SSH or RDP on most systems, but I don't think the Buffalo NAS devices support either. If I'm wrong about Buffalo's support for remote control, please let me know.

[sirwoogie]

sam and bish are correct. The concept of a remote initiation of a file copy/move really isn't in commodity hardware. Thus, in your case, your local windows machine is reading the bits in over the wireless connection, then writing them back out over the wireless connection in very small chunks (typically XP/Vista with SMB copy is 4k chunks). So, the inefficiency of SMB/CIFS to work over a slow connection (your 802.11g connection) is definitely your cause. At best case, you'll probably eek about 1-2 Megabytes per second.

Remember also that the more people you have using wireless on the access point, the available bandwidth drops by half. It based simply on the total number of connected devices, not how much each user is pulling from it. Thus, on an 802.11g you'll get theoretical 54mbps with one user, 27mbps with two users, and so on. Then, just for overhead and inefficiencies, take 30-40% off that number, and that's how much bandwidth you have to play with.

The first-most option is to do the copy on the station itself if supported (either through a command line or a web based front-end). Your other solution is to go wired, and preferably gigabit between the NAS and the host doing the copy. Not knowing what Terastation model you have, and what it's capabilities are, I can't advise you further.


Edit: Eeek, you get the Satan post...

[shmokes]

God, how disappointing.  Gigabit is out of the question.  I have a Gigabit NIC on my PC, and the Terastation has gigabit, but I don't have a gigabit switch/router, nor a crossover (or regular cable) long enough to reach between the two.  If I do a wired connection it'll have to be with a laptop, cos unhooking the Terastation and bringing it to my PC is too big a hassle to deal with based on where it's located.

And, yeah, as near as I can tell, there is no local option.  The web interface allows me to create folders and manage their share properties, but that is all.  And there's no built in telnet or remote desktop as far as I can tell.  There's a hacked bios that adds some telnet support.  I haven't really looked into it, but I'm afraid of destroying my array (and the data on it) by going to those extremes.  It has a built-in FTP server, but it's wonky.  Only some of the folders show up and I can't create new folders from within the FTP client.

It's looking like hooking a laptop up locally, so I can at least get a dedicated 100 Mb/s connection going will be my best bet.  It'll still take forever, but at least it won't be quite so painful.

[SavannahLion]

It's looking like hooking a laptop up locally, so I can at least get a dedicated 100 Mb/s connection going will be my best bet.  It'll still take forever, but at least it won't be quite so painful.

Give it a shot, but I usually don't bother. I've consistently bumped my head against this exact same problem. ON TOP of the insane slowness, there appears to be some sort of cut off or time out that innitiates on some Windows installs. It never seems to trigger with huge files but triggers when you're moving around a large number of files on networked drives.

My laptop suffers from it, but not any of the desktops. 

I ---smurfing--- hate Microsoft's network protocols. Even at its most complex configurations, Linux gets it dead right. Windows, on the other hand, forces you to jump through hurdle after hurdle just to get simple ---steaming pile of meadow muffin--- going. Bleh.

[bishmasterb]

shmokes,

As a workaround, the Buffalo NAS devices do support "backup" to local USB devices. You can connect a USB HD directly to the Buffalo NAS's USB port, and then setup a one-time backup operation via the web admin. You'll get normal USB speeds using this method (100MB/sec or so on USB 2.0).

Hope this helps.

[shmokes]

I can't figure out how I can use that method to move files from one folder to another on the NAS . . .  Can you elaborate?

[bishmasterb]

Oops, that could have been more clear.

You might try doing a backup to USB, and then seeing if the restore function will let you specify a target folder. Now that I think about it, I'm not sure I recall seeing a "restore" option on the Buffalo's web admin utility, so perhaps the designers just figured you do a restore initiated from a workstation...and then of course, you're right back where you started.

[Spaz Monkey]

Let me state that I know nothing about how the NAS drive works, but could you just pull the disk out, attach it to your desk top, copy files, and re-install the driive?

[bishmasterb]

Let me state that I know nothing about how the NAS drive works, but could you just pull the disk out, attach it to your desk top, copy files, and re-install the driive?

If it's part of a RAID array, that won't work. If it isn't part of a RAID array, and it's an NTFS volume (which I assume it is with the Buffalo NAS units) that just might work.

[shmokes]

It's a four-disk RAID 5 array.

[boykster]

PEBKAC?

 

[HaRuMaN]

PEBKAC?

 

Yes!

[shmokes]

PEBKAC?

 

That was certainly my hope, but from the responses I've got in this thread it appears to not be the case.

[boykster]

If a roundtrip of the data for a copy command is true, then that's truly messed up design and implementation on Buffalo's part. 

 

[shmokes]

What's worse, I'm not trying to do a copy command.  I'm trying to do a move command.  There's no reason for the physical location of the data on the platters to move at all.

[boykster]

yeah, I meant move not copy....

It looks like the terastations run some flavor of linux and samba, so its a bit wierd that the move command is forcing a round trip of the data.  I use built linux boxes for my NAS storage running samba, and a "move" operation stays local to the linux box, even when initiated thru windows explorer from a client machine.  Works fine.

 

[shmokes]

It could be something else, I suppose.  The time just makes me think that the data is moving across the network.  It could be that the Terastation is just not functioning correctly.

[Grasshopper]

I'm not sure if this helps but according to this site you can open up an SSH session on the device:

link

[patrickl]

It could be something else, I suppose.  The time just makes me think that the data is moving across the network.  It could be that the Terastation is just not functioning correctly.

I have an Infrant ReadyNAS and moves (within the same drive) are instantaneous. So (like Boykster already confirmed) it sure can be done properly.

On the other hand when I map the folder to a different drive letter or different "share" on the NAS then it takes forever too. Even though technically it's on the same drive on the NAS. I created a new share with public access too and it took ages to copy all the files from the secure shares to the new public share. I have a gigabit network so I just left the move running during the night. Back then I didn't have SSH access to the NAS yet (now I do, so I tend to use that now)

[shmokes]

I tried connecting via SSH, but it didn't work.  I looked around a bit more and all the hacked firmwares say that they add SSH, so I'm pretty sure that it must not be included with the stock firmware.  I'm a little hesitant to mess around with unsupported firmwares with over a terabyte of data on the box that I'd rather not lose.  Since this is just a one-time thing, I might just suffer through the retarded slow procedure.

[patrickl]

Odd. If it has no SSH (or Telnet) then how would the Buffalo support people (or you) reach your unit if it has some sort of software problem? You would have to send the unit to them?

[sirwoogie]

It looks like the terastations run some flavor of linux and samba, so its a bit wierd that the move command is forcing a round trip of the data.  I use built linux boxes for my NAS storage running samba, and a "move" operation stays local to the linux box, even when initiated thru windows explorer from a client machine.  Works fine.

This is very true, a move should be instantaneous. My post above centered around a copy command. Crossing any partition boundary (e.g. moving things from drive c: to drive d:) is considered a copy. Perhaps the Terrastation assumes that with how the folder is created, a copy is required.

Not knowing how the Terrastation works internally is my gap.

[patrickl]

Perhaps the Terrastation assumes that with how the folder is created, a copy is required.

Not knowing how the Terrastation works internally is my gap.

Yes that was my question too.

I also created a public folder and (because I set up security on share level) I had to create this as a seperate share and thus the move turned into a "copy like move".

Shmokes, does a move work fast if you move a file into a subfolder of the folder it is in? (like P:\Folder\Bigfile.avi to P:\Folder\SubFolder\Bigfile.avi)

[boykster]

good points on crossing a partition boundary;  I have no clue how the terastation is carving up the space and presenting it.

[channelmaniac]

Ouch...

You guys should get some 802.11g or n gear and ditch the .11b.

In any case, wireless is like an old hub - shared medium, not dedicated like a switch. 802.11b is 11mb half duplex which means the max you'll get is 5.5mb. Subtract speed for distance away from the access point and for obstacles or interference from 2.4GHz cordless phones or other access points and you'll end up with 1 to 2Mb/s transfer rates.

[boykster]

I just setup my new 12 port 3ware 9500S-12 RAID card and seeded it with 3 WD 1Tb drives in RAID5.  3 drives to supplant an 8 drive array (250gig drives) and room to grow.  With 3ware's online array expansion and xfs on a linux box, I'm setup to expand this all the way to 11Tb of storage as the price of drives comes down.

I chose WD's new "green" drives over my favorite Seagate for 3 reasons: power, heat, and noise.  WD's drives are quiet, cool, and draw 1/2 the power.  Sure, they're a bit slower and have a 3year warranty vs a 5 year one, but I'll deal with it.

Oh, and moves, yeah, they're instantaneous 

[sirwoogie]

I just setup my new 12 port....

Thank you for the manhood measurement. 

[boykster]

I just setup my new 12 port....

Thank you for the manhood measurement. 

 

[bishmasterb]

shmokes,

You've probably already got this figured out, but I wanted to let you know that I was at a site that had a Terastation today and thought I'd test it out.

(BTW, I realize that, as others have pointed out, I didn't initially make the distinction between moving and copying, my bad. Yes, moves should be nearly instantaneous.)

File moves to the same share were nearly instantaneous.

Files "moves" (forcing Windows to "move" the data) to different shares (UNC paths) on the SAME Terastation volume behaved like file copies (i.e. slow).