Messages posted by: gorman

Glad to help. One question. Considering what I described (basically rsynch being interrupted and my flx.meta file being with an older date than some of the .flxr files), do I have to rebuild from scratch or would a rsynch be enough, with the new version you uploaded?

That's the whole log, sorry.

But which file has been deleted? Certainly not the one indicated as Incomplete...

The inconsistent behaviour has been reported on the main forum before, under this account.

[2009-03-23 01:49:47,437] INFO : Total process size = 277973042809
[2009-03-23 02:54:42,156] ERROR:
java.lang.NullPointerException
at java.lang.Void.<unknown>(Unknown Source)
at java.lang.Void.<unknown>(Unknown Source)
at java.lang.Void.<unknown>(Unknown Source)
at java.lang.Void.<unknown>(Unknown Source)
[2009-03-23 02:54:42,250] WARN : Incomplete - G:\MyStuff\Onlymine.min - size:4236701427 read:2306589294

Now I have .flxr files with a 23 March date and a flx.meta file still with an older date. Do I have to rebuild everything?
I stopped rsynching when I discovered that rsynching, validate and quick validate were not producing consistent results. Hoped the issue had been solved with RC1.1

Now I really think that, in case of a drive failure, I'd be defenseless.

"Unfortunately" the .cbr and .xls files are not reported as corrupted anymore. The smallest file is the FLAC one. It's a DTS, FLAC encoded file. 256MB. If you wish to advise me through a PM where I could put it for you, I'd gladly do it.

Moved files, rsynched. Moved files back, rsynched. Validated.

10 files, all among the ones previously reported, are still reported as corrupted. One of them is the .rar, still testing out perfectly. Another one is one of the two .flac, again testing out perfectly against it's own CRC mechanism (and here I am sure there's no redundancy of data whatsoever).

I'm using the WebUI, if that matters.

I don't know what to say. I disabled the antivirus and got the same corruption reports.

I would even try to move one of the reported files and repair the DRU but... considering that something is messed up I'm *really* afraid that this will destroy some other files...

And yet again, the "corrupted" RAR tests perfectly fine and this time I've even extracted files from it. Everything perfect.

Edit: decided to try this approach. I'm gonna move all the "corrupted" files on a non FlexRAID drive. Rsynch. Move them back in position. Rsynch. Validate.
Let's see if the "false positives" stop.

It wasn't that. Still getting file corruption reported for files that are not corrupted. RAR files that test out correctly, FLAC files that test out correctly (against the internal checksum)...

I substituted two hard drives two weeks ago (1.5TB Seagates with 1TB WDs) but I copied all the files and FlexRAID should ignore system and hidden files, right?

Could different cluster size matter in this case?

As I mentioned, I run AntiVir, which I discovered allows process to be excluded from the Guard thing.

Good. Will rerun verify and report back. Thanks for all the good and quick info.

Thanks for the answer. I am running AntiVir, yes. I don't know if I can configure to ignore specifically FlexRAID activities. I'll have a look.

Again, thanks for the quick answer, I really appreciate it!

But in case files are corrupted, how do you recover them? Delete corrupted files and restore DRUs? Will FlexRAID rebuild only the missing files?

Also, let's say I run rsynch today. Tonight I add some files and FlexRAID has not rsynched yet. One DRU fails. Will I lose only the new files I have not synched or the whole array would be compromised? These two points are really unclear in the documentation.

I always do rsynch before running validate. Usually I run quick validate.

Yesterday I decided to to a more thorough validation and... excerpt from my log:

[2009-01-17 13:18:27,562] INFO : Total process size = 2479798683
[2009-01-19 01:36:43,843] INFO : Total process size = 18893623942
[2009-01-19 12:21:34,593] INFO : Total process size = 21995238206
[2009-01-20 02:10:06,750] INFO : Total process size = 19185784529
[2009-01-28 17:54:45,437] INFO : Total process size = 139918986994
[2009-01-29 11:39:18,468] ERROR: Corrupted: G:\mystuff.mkv
[2009-01-29 11:39:18,734] ERROR: Corrupted: G:\System Volume Information\_restore{D3A7F6F1-7F50-4C8D-8EC9-654579F44EF2}\RP99\change.log.1
[2009-01-29 12:00:27,468] ERROR: Corrupted: G:\mystuff.mkv
[2009-01-29 12:58:55,406] ERROR: Corrupted: M:\System Volume Information\_restore{D3A7F6F1-7F50-4C8D-8EC9-654579F44EF2}\RP98\change.log.4
[2009-01-29 13:02:20,062] ERROR: Corrupted: M:\mystuff.mkv
[2009-01-29 13:03:27,609] ERROR: Corrupted: M:\mystuff.mkv
[2009-01-29 13:20:38,265] ERROR: Corrupted: M:\mystuff.mkv
[2009-01-29 13:44:29,062] ERROR: Corrupted: M:\mystuff.mkv
[2009-01-29 13:44:34,515] ERROR: Corrupted: E:\mystuff.mkv
[2009-01-29 14:06:58,203] ERROR: Corrupted: K:\mystuff.rar
[2009-01-29 14:14:53,531] ERROR: Corrupted: K:\mystuff.flac
[2009-01-29 14:32:16,000] ERROR: Corrupted: M:\mystuff.mkv
[2009-01-29 14:36:38,078] ERROR: Corrupted: K:\mystuff.cbr
[2009-01-29 14:39:25,859] ERROR: Corrupted: E:\mystuff.mkv
[2009-01-29 15:24:55,281] ERROR: Corrupted: K:\mystuff.cbr
[2009-01-29 15:29:22,593] ERROR: Corrupted: K:\mystuff.cbr
[2009-01-29 15:31:54,171] ERROR: Corrupted: K:\mystuff.cbr
[2009-01-29 15:35:16,890] ERROR: Corrupted: K:\mystuff.cbr
[2009-01-29 15:37:35,156] ERROR: Corrupted: K:\mystuff.cbr
[2009-01-29 15:40:53,718] ERROR: Corrupted: K:\mystuff.cbr
[2009-01-29 15:42:52,015] ERROR: Corrupted: K:\mystuff.cbr
[2009-01-29 15:50:22,906] ERROR: Corrupted: K:\mystuff.cbr
[2009-01-29 16:00:04,421] ERROR: Corrupted: E:\mystuff.mkv
[2009-01-29 16:12:33,875] ERROR: Corrupted: K:\mystuff.cbr
[2009-01-29 17:55:15,671] ERROR: Corrupted: K:\mystuff.flac
[2009-01-29 17:58:31,859] ERROR: Corrupted: K:\mystuff.xls[b]

Now... at the beginning are all the rsynch processes run in the past few days. Then there's the corruption reports but... the video stuff plays with no problem. The RAR file (which is kinda big, over 400MB with multiple files inside) tests correctly (so no corruption)... the comics stuff (cbr) displays correctly. The XLS opens and works perfectly.

The corruption reports seem clearly wrong. I could understand movies having only a few frames corrupted and me not noticing them but Excel and RAR are picky when it comes to damaged stuff.

What could have happened? What is the right course of action in cases like these? Should I rebuild the whole array from scratch (I sure hope not)?

I had forgotten System restore on these drives, but those are hidden files, according to the documentation should be irrelevant.

Also, supposing one of the files was corrupted for real (it is not as far as I can tell), does one have to rebuild the whole DRU or you just delete what's reported as corrupted and rebuild the DRU). In case of corruption spread on multiple DRUs (like this case, even if the corruption is not true)?

Thanks for your answers, I'm kind of uneasy with this sort of errors creeping up. I use this solution to feel safe, not the opposite.

Hi Brahim, using Link Shell Extension (http://schinagl.priv.at/nt/hardlinkshellext/hardlinkshellext.html) here and I'm wondering...

I have 5 different movie folders on different harddrives and, so far, I've managed to create a Movie Aggregate folder that, inside, has junctions to the 5 different folders.
Is there a way to have Movie Aggregate containing directly all the files contained in the five different directories?

This "Movie Aggregate" is not part of the RAID, the hard drives containing the actual files are.

The blue screen... I don't know. I launched rsynch through the WebUI and when it started "crunching" everything stopped and BSOD with a message regarding NTFS if I'm not mistaken.

I don't know what you fixed but you fixed it

One thing, the first time I got a blue screen of death and I found out, to my delight, that Flexraid creates a backup of the metadata when it starts.

This is the error I get:

[2008-12-30 16:14:01,828] ERROR: [rsynch] Invalid state - workers greater than 2!
java.io.IOException: Invalid state - workers greater than 2!
at java.lang.Void.<unknown>(Unknown Source)
at java.lang.Void.<unknown>(Unknown Source)
at java.lang.Void.<unknown>(Unknown Source)
at java.lang.Void.<unknown>(Unknown Source)

Still on track for next week?

Hi! Today lots of questions...

I'm wondering: let's say tomorrow I go out and buy a new hard disk.
What steps should I then take to add it to my FlexRAID array? Would a rsynch be enough or would I need to recreate the array from scratch (supposing that initially the hard drive is empty, btw).

I'd like some clarification on the scheduling functions.

For them to work, obviously the FlexRAID Host Service has to be running in the background.

What I'm not sure about is: does the Tomcat thing have to run as well? Or, once set in the WebUI, the Host Service will "know" what to do and when?

Now I'm surprised.

The first few rsynch I ran added small (40KB, 50Kb, and so on) xxxxxx.flxr files.
Last night I added about 40GB of data and deleted about 10GB (biggest data unit didn't change in size) and I went from having:

896 1GB .flxr files

One 896MB .flxr file

Three 40-50KB .flxr files

(Total of 900 .flxr files)

I went back to:

896 1GB .flxr files

One 896MB .flxr file

One single 40KB .flxr file

(Total of 898 .flxr files)

Is this normal behaviour or could the rsynch process have messed up stuff?

Also, in the WebUI, what is the "Mark all DRUs as read-only" for?

Brahim wrote:Well, nothing is hard set.

Oh what the heck, I will work on FlexRAID Live! in parallel.

Wow, cool!!! Thank you!
I think that that would be a product you could easily sell to a lot of people. The manual rsynching is offputting for many. A live implementation would be serious competition for unRAID.