Large and Fast Storage ...

Cosworth · September 2nd, 2008, 06:27 PM

Howdie folks,

I'm looking for a solution to a storage problem I'm having. I use a program called Lightroom to archive and sort all my photographs. LR basically catalog's all these images and you can access them remotely if need be. For example, I currently have all my photos stored on my desktop machine but I access them through my WiFi router and work on them from my laptop.

Obviously, there is considerable speed issues here as it becomes very, very slow when accessing vast amounts of images. Also, It's a PITA making sure my desktop machine is powered on and then checking it for its IP address so I can access it from the MacBook. I can't store the files locally on the MacBook as there is around 7-800GB's of images and even the Desktop is starting to reach it's capacity.

What I 'think' I need is some sort of Network storage device that I can stick in a corner and access via ethernet cables around the room. I don't really need WiFi access, although it is handy, I reckon it could be one of the main reasons the speeds are suffering? I'm also thinking something accessed via ethernet would be much faster than USB or Firewire (I've tried locating the files on a USB external HD but speeds were slow aswell)

Ideally, I'd need at least 2-3TB of storage to give me a year or two of not having to worry about storage space. Then I have to worry about backups but I could manage with a couple of external drives TBH.

Could anyone point me towards what I should be looking at / for ?

It'll be Mucho Appreciated !

the Interceptor · September 2nd, 2008, 06:32 PM

What you need is a NAS (Network Attached Storage), such as this one:http://www.dlink.ie/?go=jN7uAYLx/oIJ...8Qterab4S3GdW+

Speed is limited by netwerk capabilities. Unless you have a gigabit LAN, USB and Firewire will be faster.

Cobol74 · September 2nd, 2008, 06:36 PM

What you want is something like this.

http://www.amazon.co.uk/Iomega-StorC...376815&sr=1-18

jebjeb · September 2nd, 2008, 08:24 PM

Best thing would be to make your own NAS.
Go find an old p4 box add in a few 500GB hdds and yor can make a 2TB NAS for around $250 compaired to most retail ones that cost $999 and you can add on as you go.

Zuhaib · September 2nd, 2008, 08:31 PM

Well first I would say you dont need to explain what Lightroom is, i think its a pretty popular software =P

As for your comment that Ethernet would be faster then USB/Firewire is incorrect unless we are talking about gigabit ethernet and even then with the overhead you might be close to Firewire 800 speeds. Since you said you are using Wireless, that can be huge bottleneck and I would be willing to think that most of your speed issues is caused by that. But using your desktop to host that many files is not a good idea so going to a NAS is a good start.

But a network solution has a lot of benefits like being able to access it anywhere and not having to drag a hard drive with you everywhere. So like other mentioned a NAS would be a great idea, you can buy them per made like the ones people linked too.

The one that I like a lot from the reviews is Drobo, http://www.drobo.com/Products/Index.html, which is not a NAS but you can add a Drobo share which will allow it to be shared over the network. I am told its not the fastest solution, but, its the best in terms of reliability which might be more important with photos.

And if you have a PC that you are not using you can also build your own NAS, using a any *nix OS (Linux, FreeBSD and etc). There is also some prebuilt OS for NAS like freeNAS http://www.freenas.org/ or openfiler http://www.openfiler.com/ . I went the DYI route and used Ubuntu to make my NAS since I wanted more freedom.

the Interceptor · September 2nd, 2008, 08:43 PM

If you build a NAS based on a PC, does it have functions like auto-standby and auto-wake-on-LAN?

If so, I might be interested...

jebjeb · September 2nd, 2008, 09:06 PM

The Wake on lan is a motherboard thing so I assume if you are using one that has that then yes. Or might be worth getting the "green" hdds that auto power down after x amount of time since last use but that will cause lag on first use of the session.

But for the price of hdds $59-69 for a sata2 500GB its much cheaper to make one your self plus you can set do you want lan or wlan or what ever and have room to grow as you need or add redundancy.
I've made several for cleints and the DIY seems to be about 1/3 the price of a prebuilt systems. And you can tweak it to speed it up a bit like if you dont want a webinterface kill it and save a little bit of ram/cpu.

the Interceptor · September 2nd, 2008, 09:25 PM

It would be important for me that the whole PC goes on standby after a while and wakes on LAN, as the machine I'd use would use quite some power overall. I think I might be better of with getting a couple of HDDs and a low power Mini-ITX board.

smib · September 3rd, 2008, 02:30 PM

This plus two of these should work well enough I imagine, but I honestly don't know much about this stuff.

matt2000 · September 4th, 2008, 01:22 PM

I would have personally suggested one of these, but at £515 the price isn't exactly competitive with the one smib posted. images/smilies/rolleyes.gif

Cosworth · October 8th, 2008, 12:16 AM

And I'm back again ...

Thinking I'm going to make the jump to the FW800 Drobo, have been reading that its not as fast as it claims to be but surely its faster than trying to work on the files over WiFi on an external USB drive thats accessed through a desktop machine ?

Going to probably put 2TB (2x1TB) in it which will give me around a perfectly safe 1TB to bide me over ! Anybody here have any experiences with these ?

Viper007Bond · October 8th, 2008, 02:43 AM

USB is really fucking slow. Avoid it if you can.

Also, 1.5TB drives are cheaper per gig here at least ($190 vs. $140).

Cosworth · October 8th, 2008, 01:19 PM

Aye, the firewire800 should be quicker ? I do have an FW800 port on the Mac I think ... (Goes looking)

Thanks for the advice of 1.5TB > 1TB

matt2000 · October 8th, 2008, 01:36 PM

I would go for a NAS drive wherever possible. Apparently even firewire can't match 100mbps ethernet. Not my logic.

Cosworth · October 8th, 2008, 02:05 PM

Can always add a Droboshare in the future to turn it into a fairly decent NAS setup ?

matt2000 · October 8th, 2008, 02:08 PM

Eh?

Oh right, hadn't read the entire thread so I didn't realise you were on wireless. In that case Firewire is the best, or grab an eSATA card and use one of those drives, much, much faster than USB or firewire.

Cosworth · October 8th, 2008, 02:17 PM

Don't think I can add a eSata card to my laptop though and only have one PCI slot free on the XP machine ...

chaos386 · October 8th, 2008, 02:35 PM

There are plenty of eSATA cards for PCMCIA and ExpressCard laptop slots.

Cosworth · October 8th, 2008, 03:00 PM

Aye but going down the eSata route wouldn't provide me with any backup ? I've no interest in going through RAID configurations either, I just want something I can plug a cable into, get my data and not have to worry about it. I really don't think there is anything out there that can match the drobo purely for user friendliness. Fair enough, 6TB (4x1.5TB) in a Drobo only gives me 4TB of useable data but with a Droboshare, I can put together another Drobo for another 4TB, don't think I'd fill 8TB in a hurry ....

jebjeb · October 8th, 2008, 04:39 PM

Quote:

Originally Posted by Cosworth

Aye but going down the eSata route wouldn't provide me with any backup ? I've no interest in going through RAID configurations either, I just want something I can plug a cable into, get my data and not have to worry about it. I really don't think there is anything out there that can match the drobo purely for user friendliness. Fair enough, 6TB (4x1.5TB) in a Drobo only gives me 4TB of useable data but with a Droboshare, I can put together another Drobo for another 4TB, don't think I'd fill 8TB in a hurry ....

Would be amazed how fast stuff fills up...Got 7TB and growing at 500GB every 2 months. I would go with 100Mbit/Gbit lan over usb/firewire/esata. Its the most universal setup nothing else will need addon cards/spare cables.

That being said USB2 or firewire wont really show the speed issues since you are mainly loading pictures thats what an index at start then maybe 10MB. USB/fw have really bad sustainable speeds but its burst decently well and might be workable just depends on the price of the setup.

At work I set up 1-2 of the systems each month ranging from cheap no backup 500GB drives to 10TB raided SAS systems(big $). The key is finding out what speed is livable and at what cost.

September 2nd, 2008, 06:27 PM	#1
Cosworth Joined: Mar 24th, 2005 Last Online: 07:30 PM Location: Waterford, Rep of Ireland Age: 22 Posts: 1,711 LFS Status: Rep Power: 25	Large and Fast Storage ... Howdie folks, I'm looking for a solution to a storage problem I'm having. I use a program called Lightroom to archive and sort all my photographs. LR basically catalog's all these images and you can access them remotely if need be. For example, I currently have all my photos stored on my desktop machine but I access them through my WiFi router and work on them from my laptop. Obviously, there is considerable speed issues here as it becomes very, very slow when accessing vast amounts of images. Also, It's a PITA making sure my desktop machine is powered on and then checking it for its IP address so I can access it from the MacBook. I can't store the files locally on the MacBook as there is around 7-800GB's of images and even the Desktop is starting to reach it's capacity. What I 'think' I need is some sort of Network storage device that I can stick in a corner and access via ethernet cables around the room. I don't really need WiFi access, although it is handy, I reckon it could be one of the main reasons the speeds are suffering? I'm also thinking something accessed via ethernet would be much faster than USB or Firewire (I've tried locating the files on a USB external HD but speeds were slow aswell) Ideally, I'd need at least 2-3TB of storage to give me a year or two of not having to worry about storage space. Then I have to worry about backups but I could manage with a couple of external drives TBH. Could anyone point me towards what I should be looking at / for ? It'll be Mucho Appreciated ! __________________ Catch My Drift? W:www.pmcgphotos.com Powered by Canon.

September 2nd, 2008, 06:32 PM	#2
the Interceptor Joined: Feb 22nd, 2005 Last Online: 08:19 PM Location: Germany Posts: 1,964 Car: BMW E36 323i superleggera (-85 kg to date) Rep Power: 48	What you need is a NAS (Network Attached Storage), such as this one:http://www.dlink.ie/?go=jN7uAYLx/oIJ...8Qterab4S3GdW+ Speed is limited by netwerk capabilities. Unless you have a gigabit LAN, USB and Firewire will be faster. __________________ "We have to laugh at the Germans, because they're so riddled with guilt about their history, they can't laugh at themselves." - Jeremy Clarkson

September 2nd, 2008, 06:36 PM	#3
Cobol74 Joined: Mar 21st, 2006 Last Online: 07:44 PM Location: Southampton Posts: 5,806 Car: Vauxhall Zafira 1.9 TDI Slush box. Rep Power: 50	What you want is something like this. http://www.amazon.co.uk/Iomega-StorC...376815&sr=1-18 __________________

September 2nd, 2008, 08:24 PM	#4
jebjeb Joined: Nov 17th, 2006 Last Online: 06:09 PM Location: Dallas, TX Posts: 77 Car: 03 F150, 96 Celica Rep Power: 9	Best/cheapest thing Best thing would be to make your own NAS. Go find an old p4 box add in a few 500GB hdds and yor can make a 2TB NAS for around $250 compaired to most retail ones that cost $999 and you can add on as you go.

September 2nd, 2008, 08:31 PM	#5
Zuhaib Joined: Dec 11th, 2003 Last Online: 08:52 AM Location: San Francisco, CA Posts: 619 Car: 07 VW GTI, 06 S500, 05 HD Softail Rep Power: 25	Well first I would say you dont need to explain what Lightroom is, i think its a pretty popular software =P As for your comment that Ethernet would be faster then USB/Firewire is incorrect unless we are talking about gigabit ethernet and even then with the overhead you might be close to Firewire 800 speeds. Since you said you are using Wireless, that can be huge bottleneck and I would be willing to think that most of your speed issues is caused by that. But using your desktop to host that many files is not a good idea so going to a NAS is a good start. But a network solution has a lot of benefits like being able to access it anywhere and not having to drag a hard drive with you everywhere. So like other mentioned a NAS would be a great idea, you can buy them per made like the ones people linked too. The one that I like a lot from the reviews is Drobo, http://www.drobo.com/Products/Index.html, which is not a NAS but you can add a Drobo share which will allow it to be shared over the network. I am told its not the fastest solution, but, its the best in terms of reliability which might be more important with photos. And if you have a PC that you are not using you can also build your own NAS, using a any *nix OS (Linux, FreeBSD and etc). There is also some prebuilt OS for NAS like freeNAS http://www.freenas.org/ or openfiler http://www.openfiler.com/ . I went the DYI route and used Ubuntu to make my NAS since I wanted more freedom. __________________ I am Pakistani And No! I do not own a 7/11! And my NAME is not Apu! Half the wheels, but twice as much fun =D "MPG describes riding like biology describes Sex" -- Harley Davidson Ad

September 2nd, 2008, 08:43 PM	#6
the Interceptor Joined: Feb 22nd, 2005 Last Online: 08:19 PM Location: Germany Posts: 1,964 Car: BMW E36 323i superleggera (-85 kg to date) Rep Power: 48	If you build a NAS based on a PC, does it have functions like auto-standby and auto-wake-on-LAN? If so, I might be interested... __________________ "We have to laugh at the Germans, because they're so riddled with guilt about their history, they can't laugh at themselves." - Jeremy Clarkson

September 2nd, 2008, 09:06 PM	#7
jebjeb Joined: Nov 17th, 2006 Last Online: 06:09 PM Location: Dallas, TX Posts: 77 Car: 03 F150, 96 Celica Rep Power: 9	The Wake on lan is a motherboard thing so I assume if you are using one that has that then yes. Or might be worth getting the "green" hdds that auto power down after x amount of time since last use but that will cause lag on first use of the session. But for the price of hdds $59-69 for a sata2 500GB its much cheaper to make one your self plus you can set do you want lan or wlan or what ever and have room to grow as you need or add redundancy. I've made several for cleints and the DIY seems to be about 1/3 the price of a prebuilt systems. And you can tweak it to speed it up a bit like if you dont want a webinterface kill it and save a little bit of ram/cpu.

September 2nd, 2008, 09:25 PM	#8
the Interceptor Joined: Feb 22nd, 2005 Last Online: 08:19 PM Location: Germany Posts: 1,964 Car: BMW E36 323i superleggera (-85 kg to date) Rep Power: 48	It would be important for me that the whole PC goes on standby after a while and wakes on LAN, as the machine I'd use would use quite some power overall. I think I might be better of with getting a couple of HDDs and a low power Mini-ITX board. __________________ "We have to laugh at the Germans, because they're so riddled with guilt about their history, they can't laugh at themselves." - Jeremy Clarkson

September 3rd, 2008, 02:30 PM	#9
smib Joined: Aug 31st, 2007 Last Online: 06:59 PM Location: Bristol, Connecticut Age: 21 Posts: 1,083 Car: 2008 Scion tC LFS Status: Rep Power: 28	This plus two of these should work well enough I imagine, but I honestly don't know much about this stuff. __________________ PunisherBass: ANYONE with an IQ higher than a jar of mayo wouldn't be able to get through the scientology mumbojumbo without laughing their ass off. Clegg: PunisherBass, quit insulting mayo. "Sorry for the short replies, I can't use my left hand. In fact the whole left side of my body feels like it got hit by a planet. The right side is better, it only got hit by a truck." -Blind_Io

September 4th, 2008, 01:22 PM	#10
matt2000 Squirrel Mincer Joined: Feb 17th, 2006 Last Online: 07:04 PM Location: Market Harborough, UK Age: 18 Posts: 3,476 Car: '77 Land Rover Series III LFS Status: Rep Power: 54	I would have personally suggested one of these, but at £515 the price isn't exactly competitive with the one smib posted. __________________ Signature last updated 27/11/08. Photos last updated 25/10/08.

October 8th, 2008, 12:16 AM	#11
Cosworth Joined: Mar 24th, 2005 Last Online: 07:30 PM Location: Waterford, Rep of Ireland Age: 22 Posts: 1,711 LFS Status: Rep Power: 25	And I'm back again ... Thinking I'm going to make the jump to the FW800 Drobo, have been reading that its not as fast as it claims to be but surely its faster than trying to work on the files over WiFi on an external USB drive thats accessed through a desktop machine ? Going to probably put 2TB (2x1TB) in it which will give me around a perfectly safe 1TB to bide me over ! Anybody here have any experiences with these ? __________________ Catch My Drift? W:www.pmcgphotos.com Powered by Canon.

October 8th, 2008, 02:43 AM	#12
Viper007Bond Lazy Head Dude Joined: Sep 21st, 2003 Last Online: 01:29 PM Location: Portland, Oregon Age: 24 Posts: 20,988 Car: Dodge Viper (I wish!) Rep Power: 234	USB is really fucking slow. Avoid it if you can. Also, 1.5TB drives are cheaper per gig here at least ($190 vs. $140). __________________ Fight back against the evil Quiky by +1'ing this post! There is no replacement for displacement. - Wolfgang Bernhard, Chief Operating Officer, Chrysler Group talking about the Dodge Viper SRT-10 ... I ask Herb Helbig, vehicle synthesis manager for SRT and a member of the original Team Viper development group since day one, if they'd ever thought of adding traction control. "It comes with two," he says, pointing at my feet. "Learn to use them." Got it. - Motor Trend on the 2006 Dodge Viper Coupe, November 2005

October 8th, 2008, 01:19 PM	#13
Cosworth Joined: Mar 24th, 2005 Last Online: 07:30 PM Location: Waterford, Rep of Ireland Age: 22 Posts: 1,711 LFS Status: Rep Power: 25	Aye, the firewire800 should be quicker ? I do have an FW800 port on the Mac I think ... (Goes looking) Thanks for the advice of 1.5TB > 1TB __________________ Catch My Drift? W:www.pmcgphotos.com Powered by Canon.

October 8th, 2008, 01:36 PM	#14
matt2000 Squirrel Mincer Joined: Feb 17th, 2006 Last Online: 07:04 PM Location: Market Harborough, UK Age: 18 Posts: 3,476 Car: '77 Land Rover Series III LFS Status: Rep Power: 54	I would go for a NAS drive wherever possible. Apparently even firewire can't match 100mbps ethernet. Not my logic. __________________ Signature last updated 27/11/08. Photos last updated 25/10/08.

October 8th, 2008, 02:05 PM	#15
Cosworth Joined: Mar 24th, 2005 Last Online: 07:30 PM Location: Waterford, Rep of Ireland Age: 22 Posts: 1,711 LFS Status: Rep Power: 25	Can always add a Droboshare in the future to turn it into a fairly decent NAS setup ? __________________ Catch My Drift? W:www.pmcgphotos.com Powered by Canon.

October 8th, 2008, 02:08 PM	#16
matt2000 Squirrel Mincer Joined: Feb 17th, 2006 Last Online: 07:04 PM Location: Market Harborough, UK Age: 18 Posts: 3,476 Car: '77 Land Rover Series III LFS Status: Rep Power: 54	Eh? Oh right, hadn't read the entire thread so I didn't realise you were on wireless. In that case Firewire is the best, or grab an eSATA card and use one of those drives, much, much faster than USB or firewire. __________________ Signature last updated 27/11/08. Photos last updated 25/10/08.

October 8th, 2008, 02:17 PM	#17
Cosworth Joined: Mar 24th, 2005 Last Online: 07:30 PM Location: Waterford, Rep of Ireland Age: 22 Posts: 1,711 LFS Status: Rep Power: 25	Don't think I can add a eSata card to my laptop though and only have one PCI slot free on the XP machine ... __________________ Catch My Drift? W:www.pmcgphotos.com Powered by Canon.

October 8th, 2008, 02:35 PM	#18
chaos386 .sa = bad driver! Joined: Nov 8th, 2004 Last Online: Yesterday Location: Paris, France Age: 22 Posts: 3,443 LFS Status: Rep Power: 58	There are plenty of eSATA cards for PCMCIA and ExpressCard laptop slots. __________________ "My dog is worried about the economy because Alpo is up to $3.00 a can. That's almost $21.00 in dog money." -Joe Weinstein

October 8th, 2008, 03:00 PM	#19
Cosworth Joined: Mar 24th, 2005 Last Online: 07:30 PM Location: Waterford, Rep of Ireland Age: 22 Posts: 1,711 LFS Status: Rep Power: 25	Aye but going down the eSata route wouldn't provide me with any backup ? I've no interest in going through RAID configurations either, I just want something I can plug a cable into, get my data and not have to worry about it. I really don't think there is anything out there that can match the drobo purely for user friendliness. Fair enough, 6TB (4x1.5TB) in a Drobo only gives me 4TB of useable data but with a Droboshare, I can put together another Drobo for another 4TB, don't think I'd fill 8TB in a hurry .... __________________ Catch My Drift? W:www.pmcgphotos.com Powered by Canon.