Wrong correlation coefficient

TeeChart VCL for Borland/CodeGear/Embarcadero RAD Studio, Delphi and C++ Builder.
Post Reply
provis
Newbie
Newbie
Posts: 6
Joined: Mon Oct 01, 2001 4:00 am
Location: France

Wrong correlation coefficient

Post by provis » Fri Apr 06, 2007 12:00 pm

We use TeeChartPro V7 in Borland c++ builder.
On a graph, we display the correlation function, which is a polynom
of degree 1 (f(x)=ax + b
The graph is correct.
The coefficient for degree 1 is correct.
The coefficient for degree 0 is INCORRECT.
We use AnswerMethod to get the coefficients.

Narcís
Site Admin
Site Admin
Posts: 14730
Joined: Mon Jun 09, 2003 4:00 am
Location: Banyoles, Catalonia
Contact:

Post by Narcís » Tue Apr 10, 2007 1:21 pm

Hi provis,

Could you please send us a simple example project we can run "as-is" to reproduce the problem here? You can either post your files at news://www.steema.net/steema.public.attachments newsgroup or at our upload page.

Thanks in advance.
Best Regards,
Narcís Calvet / Development & Support
Steema Software
Avinguda Montilivi 33, 17003 Girona, Catalonia
Tel: 34 972 218 797
http://www.steema.com
Image Image Image Image Image Image
Instructions - How to post in this forum

provis
Newbie
Newbie
Posts: 6
Joined: Mon Oct 01, 2001 4:00 am
Location: France

Post by provis » Fri Apr 13, 2007 2:16 pm

hi Narcis,
I will try to prepare a test case.
Until that, could you confirm you are not aware of known bugs
with correlation feature in your component.
thk

Narcís
Site Admin
Site Admin
Posts: 14730
Joined: Mon Jun 09, 2003 4:00 am
Location: Banyoles, Catalonia
Contact:

Post by Narcís » Fri Apr 13, 2007 3:01 pm

Hi provis,

Are you using TCustomTeeFunction for that?
Best Regards,
Narcís Calvet / Development & Support
Steema Software
Avinguda Montilivi 33, 17003 Girona, Catalonia
Tel: 34 972 218 797
http://www.steema.com
Image Image Image Image Image Image
Instructions - How to post in this forum

provis
Newbie
Newbie
Posts: 6
Joined: Mon Oct 01, 2001 4:00 am
Location: France

Post by provis » Tue Jul 10, 2007 7:39 am

Hi Narcis,
sorry for the late reply.
To get the coefficient factors, we use the TCurveFittingFunction
and AnswerVector.
I have made another simple test case, which unfortunately I cannot ship because it uses some internal classes.
Here is the description of my test, which you should be able to reproduce easily.
I have 2 series.
Serie 1 contains values 1,2,3,4,5,6
Serie 2 contains values 2000,20,30,40,50,60

We plot serie 2 against serie 1.
We draw the correlation polynom of degree 1.
The drawing is correct (polynom is Y=-274.29X+1326.7).
But when we ask for correlatrion factors using AnswerVector,
coefficient of degree 1 is correct, but coefficient of degree 0 is not.
It returns 1032 instead of 1326.
If I use a polynom of degree 2, only coefficient of degree 2 is correct.
Back to the polynom of degree 1, if I use function GetCurveYValue
with parameter X=0, I get 1326.7, which proves that the polynom
is drawn correctly.
It looks there is only a problem with AnswerVector.
Thanks for your help

Marjan
Site Admin
Site Admin
Posts: 745
Joined: Fri Nov 07, 2003 5:00 am
Location: Slovenia
Contact:

Post by Marjan » Tue Jul 10, 2007 2:17 pm

Hi.

I think the problem is the AnswerVector allways returns the coefficients of normalized equation.

Normalizing values prior to fitting avoids fp overflows when large x or y values are used. In short, fitting is performed on x=x-xmin and y=y-ymin values. If you take this into the account, AnswerVector will return correct polynomial coefficients.
Marjan Slatinek,
http://www.steema.com

provis
Newbie
Newbie
Posts: 6
Joined: Mon Oct 01, 2001 4:00 am
Location: France

Post by provis » Tue Jul 10, 2007 3:18 pm

thanks for the reply but sorry, I did not understand it.
What do you mean by normalized equation ?
What is fp overflow ?
What should I do to get the correct coefficient ?
How come the coefficient for the highest degree is correct and not the others ?
thanks

Marjan
Site Admin
Site Admin
Posts: 745
Joined: Fri Nov 07, 2003 5:00 am
Location: Slovenia
Contact:

Post by Marjan » Wed Jul 11, 2007 7:13 am

5886443 wrote:What do you mean by normalized equation ?
What should I do to get the correct coefficient ?
Normalized (in this case) means *all* that minimum x value is subtracted from all x values and minimum y value is subtracted from all y values. Then the actual fitting is performed on these normalized values. The actual fitting is therefore not done on:

Code: Select all

y=a(n)*x^n + a(n-1)*x^(n-1) +...  + a0
but

Code: Select all

y-ymin = b(n)*(x-xmin)^n + b(n-1)*(x-xmin)^(n-1) + ... + b0
where the a(n) and b(n) differ. The AnswerVector will allways return b(n) coefficients and if you want to get a(n), use the above relations to do transformation.
5886443 wrote:What is fp overflow ?
Floating point overflows - this happens if the numbers in fitting algorithm get too big.
5886443 wrote: How come the coefficient for the highest degree is correct and not the others ?
See my explanation above.
Marjan Slatinek,
http://www.steema.com

provis
Newbie
Newbie
Posts: 6
Joined: Mon Oct 01, 2001 4:00 am
Location: France

Post by provis » Wed Jul 11, 2007 9:16 am

thanks a lot, this is clear now;
If I understand you correctly, to get the expected factors,
I need to modify my series (add xmin to all x, add ymin to all y),
before using AnswerVector ?
Or is there another method/property I could use to get directly
the coefficients used to draw the polynom (the polynom
is not drawn using normalized equation)
thanks

Marjan
Site Admin
Site Admin
Posts: 745
Joined: Fri Nov 07, 2003 5:00 am
Location: Slovenia
Contact:

Post by Marjan » Mon Jul 16, 2007 6:36 am

Hi.

Not quite. Simply adding xmin and ymax won't do the trick. For example, "normalized" 1st degree equation is:

Code: Select all

y-ymin = b(1)*(x-xmin)+b(0)
By expanding this to:

Code: Select all

y = b(1)*x - b(1)*xmin+b(0)+ymin
you get the usuall a(1) and a(0) coefficients which define equation

Code: Select all

y=a(1)*x+a(0)
...
a(1) = b(1)
a(0) = -b(1)*xmin+b(0)+ymin
a(1) and a(0) are the coefficients you're after. For higher order polynomial you can use the same approach (although the formulas for a(i) will be a bit longer).
Marjan Slatinek,
http://www.steema.com

MikeC
Newbie
Newbie
Posts: 2
Joined: Mon Jan 07, 2002 5:00 am
Location: Isle of Wight

Post by MikeC » Tue Feb 05, 2008 5:31 pm

Marjan wrote:Hi.

Not quite. Simply adding xmin and ymax won't do the trick. For example, "normalized" 1st degree equation is:

Code: Select all

y-ymin = b(1)*(x-xmin)+b(0)
By expanding this to:

Code: Select all

y = b(1)*x - b(1)*xmin+b(0)+ymin
you get the usuall a(1) and a(0) coefficients which define equation

Code: Select all

y=a(1)*x+a(0)
...
a(1) = b(1)
a(0) = -b(1)*xmin+b(0)+ymin
a(1) and a(0) are the coefficients you're after. For higher order polynomial you can use the same approach (although the formulas for a(i) will be a bit longer).
I've been trying to expand your equations to 5th order but I am getting completely incorrect results. Could you please provide the formula to obtain coefficients a(2), a(3) and a(4). Thank you in advance.

Narcís
Site Admin
Site Admin
Posts: 14730
Joined: Mon Jun 09, 2003 4:00 am
Location: Banyoles, Catalonia
Contact:

Post by Narcís » Wed Feb 06, 2008 10:37 am

Hi MikeC,

The equation is different for each polynomial degree. You cannot apply 1st degree polynomial equation to 5th degree polynomial.

You shoud write out the 5th polynomial equation and use the same principle to extract coefficients a(0) to a(4). It's not complicated, but the equations are quite long. We suggest you to use Mathematica to extract a(0)..a(4) from system of equations.
Best Regards,
Narcís Calvet / Development & Support
Steema Software
Avinguda Montilivi 33, 17003 Girona, Catalonia
Tel: 34 972 218 797
http://www.steema.com
Image Image Image Image Image Image
Instructions - How to post in this forum

Anton
Newbie
Newbie
Posts: 5
Joined: Mon Apr 11, 2011 12:00 am

Re: Wrong correlation coefficient

Post by Anton » Mon May 02, 2011 9:36 am

And for whatever it is worth here are the conversions for second and third order polynominals.

Code: Select all

'y = a(2)*x^2 + a(1)*x + a(0)
'y-ymin = b(2)*(x-xmin)^2 + b(1)*(x-xmin) + b(0)

a(2) = b(2)
a(1) = b(1) - 2*b(2)*xmin
a(0) = b(2)*Math.Pow(xmin,2) - b(1)*xmin + b(0) + ymin

Code: Select all

'y = a(3)*x^3 + a(2)*x^2 + a(1)* x + a(0)
'y-ymin = b(3)*(x-xmin)^3 + b(2)*(x-xmin)^2 + b(1)*(x-xmin) + b(0)

a(3) = b(3)
a(2) = b(2) - 3*b(3)*xmin
a(1) = 3*b(3)*Math.Pow(xmin, 2) - 2*b(2)*xmin + b(1)
a(0) = -b(3)*Math.Pow(xmin, 3) + b(2)*Math.Pow(xmin, 2) - b(1)*xmin + b(0) + ymin


thierry
Newbie
Newbie
Posts: 2
Joined: Thu Sep 15, 2011 12:00 am

Re: Wrong correlation coefficient

Post by thierry » Sat Aug 04, 2012 10:07 am

I use teechartnet2010.

I need to use the polynomial coefficient with a polynomial degree 6.

I understand that your coefficient is for normalised value.

I draw a series point1.

I Apply polyfitting function on this curve. every things is OK.

I change polynomial coefficient by :

Code: Select all

            double[] tab_temp_poly = new double[polyFitting1.PolyDegree];

            xmin =  points1.MinXValue;
            ymin =  points1.MinYValue;

            tab_temp_poly[5] = polyFitting1.Coefficient(5);
            tab_temp_poly[4] = polyFitting1.Coefficient(4) - 5 * polyFitting1.Coefficient(5) * xmin;
            tab_temp_poly[3] = polyFitting1.Coefficient(3) + 10 * polyFitting1.Coefficient(5) * Math.Pow(xmin, 2)
                - 4 * polyFitting1.Coefficient(4) * xmin;
            tab_temp_poly[2] = polyFitting1.Coefficient(2) - 10 * polyFitting1.Coefficient(5) * Math.Pow(xmin, 3)
                + 4 * polyFitting1.Coefficient(4) * Math.Pow(xmin, 2) - 3 * polyFitting1.Coefficient(3) * xmin;
            tab_temp_poly[1] = polyFitting1.Coefficient(1) + 5 * polyFitting1.Coefficient(5) * Math.Pow(xmin,4)
                - 4 * polyFitting1.Coefficient(4) * Math.Pow(xmin,3) + 3 * polyFitting1.Coefficient(3) * Math.Pow(xmin,2)
                - 2 * polyFitting1.Coefficient(2) * xmin;
            tab_temp_poly[0] =  polyFitting1.Coefficient(0)
                - polyFitting1.Coefficient(5) * Math.Pow(xmin, 5)
                + polyFitting1.Coefficient(4) * Math.Pow(xmin, 4)
                - polyFitting1.Coefficient(3) * Math.Pow(xmin, 3)
                + polyFitting1.Coefficient(2) * Math.Pow(xmin, 2)
                - polyFitting1.Coefficient(1) * xmin
                + ymin;
I draw the curve with this coefficient and I get bad result.

what do you think of this?

can you help me?

thanks

thierry

thierry
Newbie
Newbie
Posts: 2
Joined: Thu Sep 15, 2011 12:00 am

Re: Wrong correlation coefficient

Post by thierry » Sat Aug 04, 2012 10:45 am

sorry it's mathematical error
now I do

Code: Select all

            double[] tab_temp_poly = new double[polyFitting1.PolyDegree];

            tab_temp_poly[5] = polyFitting1.Coefficient(5);
            tab_temp_poly[4] = polyFitting1.Coefficient(4) - 5 * polyFitting1.Coefficient(5) * xmin;
            tab_temp_poly[3] = polyFitting1.Coefficient(3) + 10 * polyFitting1.Coefficient(5) * Math.Pow(xmin, 2)
                - 4 * polyFitting1.Coefficient(4) * xmin;
            tab_temp_poly[2] = polyFitting1.Coefficient(2) - 10 * polyFitting1.Coefficient(5) * Math.Pow(xmin, 3)
                + 6 * polyFitting1.Coefficient(4) * Math.Pow(xmin, 2) - 3 * polyFitting1.Coefficient(3) * xmin;
            tab_temp_poly[1] = polyFitting1.Coefficient(1) + 5 * polyFitting1.Coefficient(5) * Math.Pow(xmin,4)
                - 4 * polyFitting1.Coefficient(4) * Math.Pow(xmin,3) + 3 * polyFitting1.Coefficient(3) * Math.Pow(xmin,2)
                - 2 * polyFitting1.Coefficient(2) * xmin;
            tab_temp_poly[0] =  polyFitting1.Coefficient(0)
                - polyFitting1.Coefficient(5) * Math.Pow(xmin, 5)
                + polyFitting1.Coefficient(4) * Math.Pow(xmin, 4)
                - polyFitting1.Coefficient(3) * Math.Pow(xmin, 3)
                + polyFitting1.Coefficient(2) * Math.Pow(xmin, 2)
                - polyFitting1.Coefficient(1) * xmin
                + ymin;
every things are ok.

Post Reply