FlexRAID-View or Storage Pool for XBMC and SickBeard?

Hi,
My current setup is:
I have a media server (Norco case) with 20 hard drives full of media (tv and movies), running win7. I run XBMC, Sabnzbd and SickBeard on my main pc, also win7. All new movies and eps are downloaded to the drives on the main pc. I have XBMC setup to update library on startup, but the sources on the server are not being scanned every time (because, theres no need - the files don't change often, if they do, I manually update that source). So XBMC scans only drives on the main pc, where sab and sickbeard downloads new stuff. Everything works just fine.
The problem I would like to eliminate, is with tv episodes being split between the drives. I could run sab and sb on server and have all the show's seasons in a same folder, but I don't want to do that, because I want to avoid spinning up various drives when new episodes being downloaded every day. And especially, I don't want to spin up and rescan all 20 drives every time I launch XBMC.

Question - could ALL this be achieved with a FlexRAID-View or Storage Pool and if yes, which one:
1. Have files separated as they are now, but combined in single view. I understand, it can be done, I'm just providing the example to be clear, what I want to achieve.
Example:

2. Is there a way to setup an option like "active hard drive", ie: all the new files/folders that will be created in the combined root folder "X:\" or any other combined path, eg: "X:\Archer\S04" will ALWAYS be created in the local drive (G:\), and never on the server drive (\\NORCO-SERVER\s-05\). Let's say, I point sickbeard to the combined "X:\Archer" folder. Can the combined view be setup that all the files sab downloads, and all the new folders/metafiles sickbeard creates be always moved to the local drive (G:\)?

3. Very important. Does combined view or storage pool cache the info or whatnot about files and directories? I mean, if I setup XBMC source as combined view, which contains folders and files from both local drives and servers drives, when xbmc scans the source for changes on library update, would it wake up all the drives on my server? That would be bad. I know, that this is probably not 100% flexraid related question, but I'm thinking if it's not an obvious answer for a flexraid veteran, then maybe someone already tried it and can answer from experience. This is probably the most important question, because if there is no way to achieve combined view + xbmc auto update library on startup + do NOT wake up server's drives (there are NO changes there anyway) - this whole thing is pretty much pointless.

4. Can I have files from "same name" folders combined and treated as being in the same folder? I would like to keep the video files from older seasons stored on the server's drives, but the metadata files, like nfos and tbns stored on the local drives. I mean, once I store the older seasons on the server, the video files rarely change - get moved/deleted/replaced, etc. The meta files, though, I change/update quite frequently. Further more, I'm not 100% sure, but XBMC does not cache all the fanart (eg: I think "extrafanart" does not get cached and I sometimes get "hickups" while browsing through gallery, when xbmc wants to slideshow extrafanart from a sleeping drive on the server. Like I said, not sure about this). Anyway, having video files separate from metafiles, would make my collection "cleaner" by itself.
Example:

5. Same as #2, but with combined files as in #4's example. Can I setup "active" drive, where all the new files/folders will be created, even though I have 2 "same name" folders combined from different hard drives?

6. Same as #3, but with combined files as in #4's example. Will the sleeping drives on the server be spinned up while browsing / xbmc library update?

7. I would like to be able to copy/move/delete file outside the combined view, without "breaking" the combined view.

8. I don't want to use any raid atm. Can FlexRAID-View or Storage Pool be used without FlexRaid's parity protection? I'm kinda on a fence with this snapshot raid. Methinks I'm gonna wait for a 100% stable FlexRAID-Live.

9. Where should I run the FlexRAID-View or Storage Pool? On the server, main pc or both?

10. Very important. Do I risk loosing my data using FlexRAID-View or Storage Pool? I'm kinda fond of my collection, and would hate to lose it

If there are any risks, what are they?

Hmm, I think that's all I can think of, for now. Sorry for long post, poor english and probably stupid questions.
Thanks in advance.

This message was edited 1 time. Last update was at 25/09/2011 03:05:31

cryodream wrote:
1. Have files separated as they are now, but combined in single view. I understand, it can be done, I'm just providing the example to be clear, what I want to achieve.
Example:
files on pc: G:\Archer G:\Archer\S03 files on server: \\NORCO-SERVER\s-05\Archer \\NORCO-SERVER\s-05\Archer\S01 \\NORCO-SERVER\s-05\Archer\S02 Combined view: X:\Archer X:\Archer\S01 X:\Archer\S02 X:\Archer\S03

Yes this can be done with the Storage View (new name of FlexRAID-View), and in fact I have a similar setup (Win7 based server, Win7 based front end with mediabrowser instead of XBMC).
To achieve what you're looking for, you need to setup the Storage View with:

cryodream wrote:
2. Is there a way to setup an option like "active hard drive", ie: all the new files/folders that will be created in the combined root folder "X:\" or any other combined path, eg: "X:\Archer\S04" will ALWAYS be created in the local drive (G:\), and never on the server drive (\\NORCO-SERVER\s-05\). Let's say, I point sickbeard to the combined "X:\Archer" folder. Can the combined view be setup that all the files sab downloads, and all the new folders/metafiles sickbeard creates be always moved to the local drive (G:\)?

You can setup the Storage View to read and combine all drives (including NW ones), but to write only on some of them, using the "Restricted Volumes".
In your case, by putting \\NORCO-SERVER\s-05\ in the "Restricted Voumes" part, nothing will be written there.

cryodream wrote:
3. Very important. Does combined view or storage pool cache the info or whatnot about files and directories? I mean, if I setup XBMC source as combined view, which contains folders and files from both local drives and servers drives, when xbmc scans the source for changes on library update, would it wake up all the drives on my server? That would be bad. I know, that this is probably not 100% flexraid related question, but I'm thinking if it's not an obvious answer for a flexraid veteran, then maybe someone already tried it and can answer from experience. This is probably the most important question, because if there is no way to achieve combined view + xbmc auto update library on startup + do NOT wake up server's drives (there are NO changes there anyway) - this whole thing is pretty much pointless.

I think not yet, but there is a feature request about this, so this could happen in the future.

cryodream wrote:
4. Can I have files from "same name" folders combined and treated as being in the same folder? I would like to keep the video files from older seasons stored on the server's drives, but the metadata files, like nfos and tbns stored on the local drives. I mean, once I store the older seasons on the server, the video files rarely change - get moved/deleted/replaced, etc. The meta files, though, I change/update quite frequently. Further more, I'm not 100% sure, but XBMC does not cache all the fanart (eg: I think "extrafanart" does not get cached and I sometimes get "hickups" while browsing through gallery, when xbmc wants to slideshow extrafanart from a sleeping drive on the server. Like I said, not sure about this). Anyway, having video files separate from metafiles, would make my collection "cleaner" by itself.
Example:
files on pc: G:\Archer G:\Archer\S01\Archer.S01E01.720p.BluRay.X264-REWARD.mkv files on server: H:\Archer \\NORCO-SERVER\s-05\Archer\Archer.S01E01.720p.BluRay.X264-REWARD.nfo \\NORCO-SERVER\s-05\Archer\Archer.S01E01.720p.BluRay.X264-REWARD.tbn Combined view: X:\Archer X:\Archer\S01\Archer.S01E01.720p.BluRay.X264-REWARD.nfo X:\Archer\S01\Archer.S01E01.720p.BluRay.X264-REWARD.mkv X:\Archer\S01\Archer.S01E01.720p.BluRay.X264-REWARD.tbn

No, I'm assuming you need a similar folder tree on all drives for them to be combined as you wish.
In your case, you'll get something like this:

X:\Archer
X:\Archer\Archer.S01E01.720p.BluRay.X264-REWARD.nfo
X:\Archer\S01\Archer.S01E01.720p.BluRay.X264-REWARD.mkv
X:\Archer\Archer.S01E01.720p.BluRay.X264-REWARD.tbn

Because Archer.S01E01.720p.BluRay.X264-REWARD.nfo and Archer.S01E01.720p.BluRay.X264-REWARD.tbn are at the root of Archer drive on your server.
If you move them, on your server, to a S01 folder, it'll work.

cryodream wrote:
5. Same as #2, but with combined files as in #4's example. Can I setup "active" drive, where all the new files/folders will be created, even though I have 2 "same name" folders combined from different hard drives?

Yes, see above, using Restricted Volume on your "non active" drives.

cryodream wrote:
6. Same as #3, but with combined files as in #4's example. Will the sleeping drives on the server be spinned up while browsing / xbmc library update?

This I don't know, as I don't know how XBMC scans the library. You may have to do some tests.

cryodream wrote:
7. I would like to be able to copy/move/delete file outside the combined view, without "breaking" the combined view.

You can do that, but it's better to temporarily stop the view to do it, otherwise you can get some file locks (not serious - restarting FlexRAID service or rebooting the PC will be enough to solve them), see http://wiki.flexraid.com/2011/06/05/tips-flexraid-storage-pooling-under-windows/

cryodream wrote:
8. I don't want to use any raid atm. Can FlexRAID-View or Storage Pool be used without FlexRaid's parity protection? I'm kinda on a fence with this snapshot raid. Methinks I'm gonna wait for a 100% stable FlexRAID-Live.

Yes, those can be used independently. But the snapshot view is great and will provide you some level of protection until the realtime RAID is stable.

cryodream wrote:
9. Where should I run the FlexRAID-View or Storage Pool? On the server, main pc or both?

On your case, I guess on the main PC since you've got some files there as well.
I guess you could set this up on the server, and from the server access shares on the main PC to re-share them to the main PC, but that's weird.

cryodream wrote:
10. Very important. Do I risk loosing my data using FlexRAID-View or Storage Pool? I'm kinda fond of my collection, and would hate to lose it If there are any risks, what are they?

No additional risk compared to accessing your data directly.

xliv wrote:
No, I'm assuming you need a similar folder tree on all drives for them to be combined as you wish.
In your case, you'll get something like this:

X:\Archer
X:\Archer\Archer.S01E01.720p.BluRay.X264-REWARD.nfo
X:\Archer\S01\Archer.S01E01.720p.BluRay.X264-REWARD.mkv
X:\Archer\Archer.S01E01.720p.BluRay.X264-REWARD.tbn

Because Archer.S01E01.720p.BluRay.X264-REWARD.nfo and Archer.S01E01.720p.BluRay.X264-REWARD.tbn are at the root of Archer drive on your server.
If you move them, on your server, to a S01 folder, it'll work.

Oops, sorry, that was my mistake in the example

My files would be in the appropriate (same) folders:

Thank you very much for the answers. I guess I need to do some tests with xbmc, thought if its gonna wake up all the drives to scan for changes every time, that's gonna be a deal breaker. Maybe I need to look into solving this problem from xbmc's side.

xliv wrote:

cryodream wrote:
7. I would like to be able to copy/move/delete file outside the combined view, without "breaking" the combined view.

You can do that, but it's better to temporarily stop the view to do it, otherwise you can get some file locks (not serious - restarting FlexRAID service or rebooting the PC will be enough to solve them), see http://wiki.flexraid.com/2011/06/05/tips-flexraid-storage-pooling-under-windows/

I'm a bit worried about this one: so do I basically forget about using my drives the "normal" way, if I have Storage View installed?

1. Sickbeard is running 24/7 and if I setup it to see/use Storage View, then I'm never 100% sure if any of the files are in use by sb at any given moment... Same with XBMC, my gf might be watching anything from the library (which would be set to Storage View) at any given time on livingroom or bedroom htpcs.

2. What about defrag? I don't use it on server's drives, as they are ok, but I run scheduled defrags on my pc.

3. I have sab and sickbeard run at startup. Now the question is, when is Storage View launched? I gather it must be running before sab and sickbeard try to use the Storage View. How do I solve that?

I would let Brahim confirm, but I think the issue can come when the same file is opened at the same time through the storage pool, and directly on the drive.
So I guess you can continue accessing your data, as soon as you can guanrantee that the above will not happen.

And again, consequences are not that bad, as far as I understood.

I don't know if the next implementation of the storage pool (under rework) will suffer the same restrictions.

6. Same as #3, but with combined files as in #4's example. Will the sleeping drives on the server be spinned up while browsing / xbmc library update?

Yes the sleeping drive will spin up when xbmc scans. I'm running 2 xbmc boxes and a server using flexraid for file serving.