PERFORMANCE THREAD

This message was edited 1 time. Last update was at 31/03/2010 13:03:36

Brahim wrote:Great.

Yeah, there is a performance hit if FlexRAID does not have a chance to self-optimize first.

What do you mean by this? Self-optimize?

This message was edited 2 times. Last update was at 31/03/2010 14:01:47

Brahim wrote:
The Simple-IO-Test package is generic, does not depend on a FlexRAID release, and only works on Windows.

Looks like I have a problem with the z:\ music drive..
N: is sata ][ like the z: drive
K: is a 400GB IDE drive so was expecting that result anyway

Will the performance of this and / or raid be improved by downloading the latest JRE 6 v 18 ?
I'm installing so will find out

Edit: Does not make any difference..

Sparks wrote:

Brahim wrote:Great.

Yeah, there is a performance hit if FlexRAID does not have a chance to self-optimize first.

What do you mean by this? Self-optimize?

It is internal to FlexRAID and not something you can control.
There are certain things FlexRAID does to optimize itself.
The optimization rules are lengthy and too complex to discuss here.

Smurf-IV wrote:

Brahim wrote:
The Simple-IO-Test package is generic, does not depend on a FlexRAID release, and only works on Windows.

I had to go and re-word my post as the package can work on Linux too.

The Simple-IO-Test package is generic and does not depend on a FlexRAID release.
Though the batch script only works on Windows, Linux users can modify it to get it to work on Linux.

Looks like I have a problem with the z:\ music drive..
N: is sata ][ like the z: drive
K: is a 400GB IDE drive so was expecting that result anyway

Indeed, drive Z:\ is your bottleneck.
Looks like a network drive to me. Isn't it?
If so, try to upgrade to a Gigabit network.

Will the performance of this and / or raid be improved by downloading the latest JRE 6 v 18 ?
I'm installing so will find out
Edit: Does not make any difference..

Again, FlexRAID is written in both Java and C/C++ and then compiled natively.
In other word, it does not run on JRE and you don't need Java to run it.

The Simple-IO-Test runs on Java and is only here to give you a hint.

Brahim wrote:

Sparks wrote:

Brahim wrote:Great.

Yeah, there is a performance hit if FlexRAID does not have a chance to self-optimize first.

What do you mean by this? Self-optimize?

It is internal to FlexRAID and not something you can control.
There are certain things FlexRAID does to optimize itself.
The optimization rules are lengthy and too complex to discuss here.

So is there any way to make sure it "self-optimizes" first or is that just thru trial and error to find the best combo of parameters, ie Processes, buffer size, etc.

Sparks wrote:

So is there any way to make sure it "self-optimizes" first or is that just thru trial and error to find the best combo of parameters, ie Processes, buffer size, etc.

Nothing you have control over.
Think of this as FlexRAID adapting to its environment as opposed to requesting that the environment adapts to it.

What users should focus on when it comes to performance are:
1. the buffer size (makes the biggest difference in performance)
2. the processes count

The difference between the optimal buffer size and the worst possible value can be as drastic as a 10x performance difference.

Brahim wrote:Update 07/15/2008:

All those having performance issues, please download the attached simple I/O test program and report your read figures.

Is the '4096' in these lines from the batch files the buffer size? Would changing it here be a simple way to test various buffer sizes for FlexRAID rsynch and verify operations?

This message was edited 2 times. Last update was at 01/04/2010 12:59:07

guy wrote:

Brahim wrote:Update 07/15/2008:

All those having performance issues, please download the attached simple I/O test program and report your read figures.

java -jar test.jar 4096 Drive1:\sample java -jar test.jar 4096 Drive1:\sample Drive2:\sample Drive3:\sample

Is the '4096' in these lines from the batch files the buffer size? Would changing it here be a simple way to test various buffer sizes for FlexRAID rsynch and verify operations?

Correct.
That is indeed the buffer size, and the test can be used to test the various buffer sizes.
Just make sure that the data set is large enough for the numbers to be meaningful.

I still prefer users to test directly against a sample RAID.
That is, create a test RAID in FlexRAID and record the start and end times (note: not the "T-stamp" time, use the "Started" and "Ended" values ).
After the RAID is created, you can delete the parity, change the buffer size, and re-create the RAID (the button to re-create is under the create button on the WebUI - under "advanced").
Command line users, can simply run the create task again to recreate the RAID.

This message was edited 4 times. Last update was at 02/04/2010 17:51:21

I made test runs on several buffer size settings.

Nforce 8200 mobo
amd Athlon x2 250 3ghz
some 1066Mhz ddr2 ram
3 * 1.5TB samsung EcoGreen 5400 ( 2xDRU, 1xPPU)
Win7 Home Basic 32bit
Flexraid 1.4 beta 4 T1+ engine

2 folders of big and small files, each dir is 15GB, 1 for each DRU

Buffersize and the time to create the raid
16K - 6m27s
32K - 6m27s
64K - 6m19s **** fastest
256K - 6m35s
1M - 6m30s
4M - 7m42s
10M - 7m17s
32M - 7m32s

Brahim wrote:What users should focus on when it comes to performance are:
1. the buffer size (makes the biggest difference in performance)
2. the processes count

The difference between the optimal buffer size and the worst possible value can be as drastic as a 10x performance difference.

I'm new to FlexRAID (haven't even installed yet; still reading as much as I can) and want to understand configuring the buffer size. Processess is self explanatory, but what drives the buffer size? Meaning is it number of drives and/or typical file sizes? As I expand my array is this something I may want to revisit? If so, would I have to delete my parity to make the change?

...BTW, I can't wait to start using this program!

Well, the buffer what you talk about is a simple read/write buffer when reading data from hdds and writing on them. You can set buffer size at raid creation time. YOU CAN change it each time before resync (and other operations, see wiki when does it matter).

So to answer your questions...
Q: "... but what drives the buffer size"
A: You can set it, for best read/write performance you may have to make tests ( ideal performance varies between system configurations), 1M buff size is default

And now i understand, maybe you think that buffer size is like stripe size or something, no... Flexraid does not "stripe" data.

Q: " Meaning is it number of drives and/or typical file sizes?"
A: No.

Q: " As I expand my array is this something I may want to revisit?"
A: No.

Q: " If so, would I have to delete my parity to make the change? "
A: No.

wtf?

greatus wrote:wtf?

Spammers...

Deleted those messages.

Brahim, is there a suggested buffer size? I have it at 1MB currently.