Incomplete rsynch with 1.1RC

[2009-03-23 01:49:47,437] INFO : Total process size = 277973042809
[2009-03-23 02:54:42,156] ERROR:
java.lang.NullPointerException
at java.lang.Void.<unknown>(Unknown Source)
at java.lang.Void.<unknown>(Unknown Source)
at java.lang.Void.<unknown>(Unknown Source)
at java.lang.Void.<unknown>(Unknown Source)
[2009-03-23 02:54:42,250] WARN : Incomplete - G:\MyStuff\Onlymine.min - size:4236701427 read:2306589294

Now I have .flxr files with a 23 March date and a flx.meta file still with an older date. Do I have to rebuild everything?
I stopped rsynching when I discovered that rsynching, validate and quick validate were not producing consistent results. Hoped the issue had been solved with RC1.1

Now I really think that, in case of a drive failure, I'd be defenseless.

This message was edited 1 time. Last update was at 23/03/2009 08:53:06

You are fine.

All this is saying is that a file got deleted before it was fully processed.
So, all you have to do is rsynch again.

The .flxr files don't have to have the same date.
Don't worry about it, no need to rebuild everything.

Also, what inconsistent results are your referring to?
They each have a different purpose.
Let me know if there is any confusion that needs clearing.

Also, starting with 1.1, there is an option to undo failed operation (rollback).

You should enable that feature if your system is displaying odd behaviors.

This message was edited 1 time. Last update was at 23/03/2009 16:57:29

But which file has been deleted? Certainly not the one indicated as Incomplete...

The inconsistent behaviour has been reported on the main forum before, under this account.

This message was edited 1 time. Last update was at 23/03/2009 17:23:11

I take back what I said.

Can I get more of the log?
I understand the intent to show the relevant portion, but it is possible that what's left out is more relevant than what's being shown.

That's the whole log, sorry.

gorman wrote:That's the whole log, sorry.

I think I found the issue.
A new build will be up tonight.

Alright, 1.1 RC1 build 003 is up: http://www.openegg.org/forums/posts/list/110.page

This bug was indeed present since 1.0 final and would in fact lead to an inconsistent behavior.
The bug was introduced when I was making some memory optimization prior to the 1.0 final release.

Thanks for the report.

Glad to help. One question. Considering what I described (basically rsynch being interrupted and my flx.meta file being with an older date than some of the .flxr files), do I have to rebuild from scratch or would a rsynch be enough, with the new version you uploaded?

Just an rsynch will do.

The date thing is normal.
Feel free to run validate or verify afterward.

Will do and report back. Thanks.

New log:

The file is a movie. It works without problems... shall I activate debug log or rebuild from scratch the RAID?

gorman wrote:New log:

[2009-03-23 01:49:47,437] INFO : Total process size = 277973042809 [2009-03-23 02:54:42,156] ERROR: java.lang.NullPointerException at java.lang.Void.<unknown>(Unknown Source) at java.lang.Void.<unknown>(Unknown Source) at java.lang.Void.<unknown>(Unknown Source) at java.lang.Void.<unknown>(Unknown Source) [2009-03-23 02:54:42,250] WARN : Incomplete - G:\MyStuff\Onlymine.min - size:4236701427 read:2306589294 [2009-03-26 00:49:51,234] INFO : Total process size = 322624208023 [2009-03-26 02:15:47,546] ERROR: java.lang.NullPointerException at java.lang.Void.<unknown>(Unknown Source) at java.lang.Void.<unknown>(Unknown Source) at java.lang.Void.<unknown>(Unknown Source) at java.lang.Void.<unknown>(Unknown Source) [2009-03-26 02:15:47,640] WARN : Incomplete - G:\MyStuff\Onlymine.min - size:3231290127 read:1301177994

The file is a movie. It works without problems... shall I activate debug log or rebuild from scratch the RAID?

Nope, download build 004: http://www.openegg.org/forums/posts/list/110.page

Follow these steps:
1. Run validate (not quick-validate)
2. Move all the files marked as "corrupted" to a different directory (it should really be one or just a few)
3. Rsynch
4. Move back the file(s) you moved in step 2
5. Rsynch

One of the early bug must have left some invalid entries in the metadata.

This message was edited 1 time. Last update was at 28/03/2009 08:14:22

I followed the procedure you described.

Ran validate. 16 files corrupted, 1 incomplete (the aforementioned one).
Moved all the files as suggested.
Rsynched.
Moved back the files.
Rsynched.
Ran validate once more. One of the above 16 files is still reported as corrupted. It's a video file, played back just fine, with no visible.

Log reported nothing for the second Rsynch and just this line for the second validate:

[2009-03-28 04:33:00,875] ERROR: Corrupted: M:\MyStuff\MyBusiness\MyVideo.min

What can I do? False positives undermine the whole validation sense...

Edit: I tried deleting the corrupted file and do a restore. Luckily I had the good sense of copying it somewhere else before.
The restore process restored the file, saying it was corrupted. The file had the exact size, byte for byte, but did not play at all. The original one I had copied before deleting played perfectly. I copied it back.

But at this point I doubt if FLEXRaid can offer me the safety I seek.

This message was edited 1 time. Last update was at 28/03/2009 09:02:52

Were you using build 005?
Which task gave you the incomplete file report?

It is possible that you might have to re-create the RAID.
1.1 RC1 was definitely buggy up to build 005.
I did make far too many code changes in 1.1 RC1.
This is the reason why it is important for you guys to test and report back.

However, I promise it is much stable now.

I will see if I can release and update to version 1.0 with just the fixes it needed for those that just want something stable without the additional features.

Can all those on 1.1 RC1 build 005 run verify and report back?
Thanks.

This message was edited 2 times. Last update was at 28/03/2009 10:11:38

Yup, 005.

The incomplete file report was the first one I opened this thread for. It was given to me by the validate task (not quick validate), on 005 (and previous version as well, the one I initially reported, which I think was 1.0).

I'll move out this single file, rsynch, move it back, rsynch and validate. If the error is still there I'll have to recreate the RAID, I'm afraid.

Edit: ok, did the "move out, rsynch, move back, rsynch" thing. The log for this shows:

[2009-03-28 14:18:25,593] INFO : Total process size = 1566168068
[2009-03-28 15:03:07,609] INFO : Total process size = 1566168068

With the process size exactly being, byte perfect, the same as the video that I'm having problems with.
I'll schedule a validate tonight, as now I need the machine and I don't want to have anything else running while validating.

I think it was given to you by the rsynch task, when the initial bugs were creeping up.

In all cases, I would clear the log, run validate, then track the exact number of reported corrupted files and move them.
Once I get confirmation of what works to fix the issues without recreating the RAID, I will try make its resolution automatic before 1.1 goes final.

Messages: Sun Mar 29 04:59:37 CEST 2009

* Task: validate
* Status: completed
* Success! The RAID is healthy and up to date!
* Started: 28 marzo 2009 20.00.00 CET
* Ended: 29 marzo 2009 4.59.32 CEST
* T-stamp: 29 marzo 2009 4.59.37 CEST

Hoooray!!!

The last "move out, rsynch, move back in, rsynch" appears to have been successful for the last very stubborn file.

Cool.

Can you now run verify on the RAID?

Brahim wrote:Cool.

Can you now run verify on the RAID?

Am I wrong in believing verify will take as much time as re-building the RAID from scratch?
It took 9 hours just for the validate...