Filtering normal distribution data

madah123

New member
Joined
Aug 2, 2017
Messages
1
Hello,

While this is not a school question, I felt it's appropriate to write here as it's mathematical in nature.

Let's say I have an increasing time series with noisy data; such that the noise is a white gaussian noise with zero mean and variance of sigma^2. Because the data is increasing, I wish to make a filter such that, if f(i+1)<f(i), then I will add a correction based on the expected increase, as I would know that the signal at i+1 was corrupted by negative value noise, so it would make sense to add to it.

Now the actual mathematical context of this question is that I tried to do a simple example in MATLAB

---------------------------------------------------------
clear all
clc
N=50; %Signal Length
M=25;%Number of samples
L=50; %Signal Step
xtrue=linspace(1,N,L)';% this would generate data {1,2,3,..50} a sample increasing data
x=zeros(N,M);
q=zeros(N,M);%number that will be used for correction


for i=1:M
signalnoise=randn(N,1);% zero mean with 1 standard deviation
x:),i)=xtrue+signalnoise;%
for k=1:N-1
if x(k+1,i)>x(k,i)
q(k,i)=x(k+1,i)-x(k,i);
end
end

end


xestimate1=mean(x,2);%without correction
correction =mean(q(q~=0));


for i=1:M
for k=1:N-1
if x(k+1,i)<x(k,i)
x(k+1,i)=x(k+1,i)+correction;
end
end

end


xestimate2=mean(x,2);%with correction
[r.m,b]=regression(1:50,xestimate1')
[r.m,b]=regression(1:50,xestimate2')


figure(1)
plot(xtrue)
hold on
plot(xestimate1)
figure(2)
plot(xtrue)
hold on
plot(xestimate2)
---------------------------------------------------------

Again the mathematical part is that what I do to set the correction is I calculate the correction value by averaging every instance in which, f(i+i)>f(i) I average f(i+1)-f(i). since the function I defined has constant increase.

I expect the average value to be increase which is 1 + the expected value of half normal distribution which is sqrt(2/pi), but that's not what I am getting. I am getting correction equal to pi/2 approximately. and if i subtract 1 from it, I get pi/2 -1.

Why is that?

Also is there a better approach to this. As I am not seeing improved RMSE for this method. I have a bachelors in electrical engineering, if my background is relevant to how you answer this.

Thank you for your support.
 
Top