We have moved permanently! Join us @ http://forum.flexraid.com
We have moved permanently! Join us @ http://forum.flexraid.com
We have moved permanently! Join us @ http://forum.flexraid.com
[Logo] (Closed - visit http://forum.flexraid.com)
  [Search] Search   [Recent Topics] Recent Topics   [Hottest Topics] Hottest Topics   [Members]  Member Listing   [Groups] Back to home page 
[Register] Register / 
[Login] Login 
Question about how Flex Raid works..  XML
Forum Index » General discussion
Author Message
nsainfreek


Joined: 23/12/2009 15:03:53
Messages: 25
Offline

My understanding is that Flex Raid looks across the hard drives (or folders) that you are protecting and adds the 1s and 0s up. This would be either odd or even for each block in the parity drive and Flex Raid would mark it accordingly in the parity drive. When a drive or folder fails and you recover, Flex Raid adds the remaining drives' 1s and 0s up and determines whether each block in the failed drive was a 1 or 0. Using that info, it rebuilds the failed drive. Is that correct so far?

My question is, if you make a change to a drive in the array (e.g. add a movie to it) and the another drive in the array fails before you can resynch, how would Flex Raid do the math correctly to rebuild the failed drive? Wouldn't the additional file in the non-failing hard drive create 1s where 0s used to be and throw of the FlexRaid calculation?

I'm not sure if that was clear. I haven't had a drive fail yet but am concerned if I add or change a file to a drive without resynching, taht it may throw off my rebuild.
NLS


Joined: 25/09/2009 05:57:23
Messages: 591
Location: GREECE
Offline

1) yes

2) no and yes - file changes (note: CHANGES) before resync will likely result in trashed files (on the recovered volume) - the good thing is that change means edit or delete and the trashing will only occur for the CHANGED bytes (so in a large volume will likely mean a small percentage of files trashed), change doesn't mean NEW files... new files as they were not included in the parity anyway, cannot mess the existing parity (and the recovery)... again: only editing a file (which usually means a few bytes or kilobytes) or deleting a file (which means parity change to the extend of the size of the missing file) will result in recovery corruption

flexraid (non-live) is meant for archive data - Brahim has said so, many times


---
NLS
(sorry cannot put my specs on the sig - testing under a few different VMs - will put specific specs when my home-SBS7 is ready)
nsainfreek


Joined: 23/12/2009 15:03:53
Messages: 25
Offline

Ok, I understand that changing a file will result in trashed files in the recovery volume. But couldnt' adding a files result in error if the drive that fails has the most data? For example let's say I have three drives that are 1TB, 750GB, and 500GB. The 1TB drive is full but the 500GB drive is only half full. When I run an rsynch, wouldn't the half the 500GB hard drive half 0s assigned to them? So those 0s would match up with the respective portions of the 1TB and 750GB drive.

Now let's say I add files to the 500GB drive, that would change some of those 0s to 1s. If the 1TB drive then crashes before I can do a Rsynch, wouldn't the recovered drive be incorrect to the extent of the files added to the 500GB drive since areas that were once 0s are now 1s?

BTW, what do you mean by trashed file? They won't show up in the recovery or they'll show up corrupted? Would Flexraid indicate which file has been trashed?
Brahim


Joined: 09/04/2008 23:28:33
Messages: 2883
Offline

@nsainfreek

FlexRAID will alert you if a recovered file is corrupted.

The scenario you presented poses no danger to the way that FlexRAID works.
FlexRAID works at the filesystem level and as such knows what 0's and 1's are meaningful and what aren't.

So, NLS is correct in his description of the risk associated with using FlexRAID Basic.

Server (VMware ESXi): dual Quad 8356@2.4Ghz | ASUS KFN5-D SLI | 16GB (4x 4GB) DDR2 667Mhz ECC REG w/Parity [Chipkill] | Radeon X300 | Intel 160GB SSD (VM datastore) | 6+ TB storage
File Server VM (running FlexRAID): 512MB RAM | 2 vCPUs | 6TB storage | Parity on 2TB NAS
nsainfreek


Joined: 23/12/2009 15:03:53
Messages: 25
Offline

Ok, that makes sense. Thanks for the reply!
umax


Joined: 22/02/2010 08:32:32
Messages: 38
Offline

Jut to recap this, I have WHS and Flexraid is only protecting My Movies, which consist only of 1:1 blu-ray rips (isos). Since I dont change/edit anything during daytime (except adding new movies) and flexraid is doing its thing during night, I would not get any corruption if one of the disk would failed before resynch?

This is really important to me as I dont want any missing bits/hiccups during playback and it would mean the same as I would lost that rip entirely. Is there any checksum/MD5 built into flexraid to check for consistency of the files?
Brahim


Joined: 09/04/2008 23:28:33
Messages: 2883
Offline

umax wrote:Jut to recap this, I have WHS and Flexraid is only protecting My Movies, which consist only of 1:1 blu-ray rips (isos). Since I dont change/edit anything during daytime (except adding new movies) and flexraid is doing its thing during night, I would not get any corruption if one of the disk would failed before resynch?

This is really important to me as I dont want any missing bits/hiccups during playback and it would mean the same as I would lost that rip entirely. Is there any checksum/MD5 built into flexraid to check for consistency of the files?


Yes to all questions.
Read on the validate and verify tasks.

If you want additional protection during a re-sync (for the rare case that a drive would fail during a rsynch operation), enable and configure the undo recovery unit.

Server (VMware ESXi): dual Quad 8356@2.4Ghz | ASUS KFN5-D SLI | 16GB (4x 4GB) DDR2 667Mhz ECC REG w/Parity [Chipkill] | Radeon X300 | Intel 160GB SSD (VM datastore) | 6+ TB storage
File Server VM (running FlexRAID): 512MB RAM | 2 vCPUs | 6TB storage | Parity on 2TB NAS
umax


Joined: 22/02/2010 08:32:32
Messages: 38
Offline

So adding data will not lead to corruption of random files on restore if its not rsynch in the meantime? You only risk data corruption if you edit/delete files?
Brahim


Joined: 09/04/2008 23:28:33
Messages: 2883
Offline

umax wrote:So adding data will not lead to corruption of random files on restore if its not rsynch in the meantime? You only risk data corruption if you edit/delete files?


Yes.

Server (VMware ESXi): dual Quad 8356@2.4Ghz | ASUS KFN5-D SLI | 16GB (4x 4GB) DDR2 667Mhz ECC REG w/Parity [Chipkill] | Radeon X300 | Intel 160GB SSD (VM datastore) | 6+ TB storage
File Server VM (running FlexRAID): 512MB RAM | 2 vCPUs | 6TB storage | Parity on 2TB NAS
jrwalte


Joined: 24/01/2010 11:20:01
Messages: 29
Offline

One thing to watch out for is metadata editing, which many programs will do automatically. I use MediaBrowser and it turns out it was randomly editing my meta data (to change backdrop, etc). It's a small amount of KB, but could easily cause corruption of a recovered movie file. I decided to turn off the mediabrowser metadata updater and manually update through metaBrowser.

This message was edited 2 times. Last update was at 25/02/2010 12:10:25

spork985


Joined: 18/10/2010 17:10:34
Messages: 13
Offline

I know bumping really old threads is generally looked down upon in forums, but I found this thread through google and had a question about it...

Help me with these two situations please...

1) I have a movie and I synced the drives, then I edited that movie and did not resync and the drive failed. When I recover it, I would get the origional version? or would it be corrupted?

2) I have a movie and I synced the drives, then edited it, then synced again. The drive dies and I recover it. Do I get the original, edited, or a corrupt file?

In both cases, it would make sense to me to have the last synced copy restored but I'm slightly confused after reading through this thread.
Brahim


Joined: 09/04/2008 23:28:33
Messages: 2883
Offline

spork985 wrote:I know bumping really old threads is generally looked down upon in forums, but I found this thread through google and had a question about it...

Help me with these two situations please...

1) I have a movie and I synced the drives, then I edited that movie and did not resync and the drive failed. When I recover it, I would get the origional version? or would it be corrupted?

2) I have a movie and I synced the drives, then edited it, then synced again. The drive dies and I recover it. Do I get the original, edited, or a corrupt file?

In both cases, it would make sense to me to have the last synced copy restored but I'm slightly confused after reading through this thread.


1. the version since you last synced (will lose the edits)
2. the version since you last synced (won't lose the edits)


Server (VMware ESXi): dual Quad 8356@2.4Ghz | ASUS KFN5-D SLI | 16GB (4x 4GB) DDR2 667Mhz ECC REG w/Parity [Chipkill] | Radeon X300 | Intel 160GB SSD (VM datastore) | 6+ TB storage
File Server VM (running FlexRAID): 512MB RAM | 2 vCPUs | 6TB storage | Parity on 2TB NAS
amieres


Joined: 23/02/2011 09:18:18
Messages: 8
Offline

I have a question regarding the same topic.

Let's assume I have 3 data units and 1 parity unit.
If I change 1 file in one unit, wouldn't then 3 files be at risk until the next resync?
The changed file and one file (or more) in each of the other units?

Thanks
dscline


Joined: 23/03/2009 14:39:38
Messages: 233
Offline

Depends on which disk failed. If you edit a file and the disk that hosts that file fails before an rsynch, you will be able to recover all your data, except the edited file will end up being the pre-edited version. If it's a different disk that failed, then an equal portion of data on the failed disk may be unrecoverable. So if you edited 10MB of data, 10MB of data on the failed disk is at risk of being unrecoverable. Which is precisely why FlexRAID basic is recommended only for data that isn't often edited or deleted. It coule be more (if the failed disk contains all 40GB blu-ray ISOs, 10MB of bad data could still ruin a whole movie), or it could be less (if the edited data was on a disk with more data than the others, it MIGHT not even be paired with data on the other disks).
jrwalte wrote:One thing to watch out for is metadata editing, which many programs will do automatically. I use MediaBrowser and it turns out it was randomly editing my meta data (to change backdrop, etc). It's a small amount of KB, but could easily cause corruption of a recovered movie file. I decided to turn off the mediabrowser metadata updater and manually update through metaBrowser.

It should be noted that FlexRAID now supports exclusions. I use MyMovies, and it also will automatically update metadata files, as well as cover scans. I use the following exclusion in my setup to exclude those files:

This is specific to a WHS setup (where all disks are represented as a folder in c:\fs\(drive letter)). But it can be edited for your specific setup. This causes FlexRAID to not include any .xml, .jpg, or .jpeg files in my movie storage paths, regardless of which disk they are on.

This message was edited 3 times. Last update was at 28/02/2011 07:41:14


WHS v1
FlexRAID 2.0b10 T2+ 19 DRUs 2PPUs
Supermicro C2SEA, E8400 (stock), 4GB
Supermicro AOC-SASLP-MV8
Sans Digital 4ESPCIE (SiI3124)
Generic SiI3132
 
Forum Index » General discussion
Go to:   
Powered by JForum 2.1.8 © JForum Team



Locations of visitors to this page