Physical Meaning of Marcenko Pastur Theorem

 This post will describe the physical meaning of the Marcenko Pastur Theorem (MPT)

Physical meaning:
The Theorem allows us to predict the probability density function (PDF) of the eigen values of a covariance matrix. The covariance matrix is associated with an underlying process that has zero mean, has variance sigma^2, and comes from a process that is IID (Independent and Identically Distributed). 

Practical use:
The IID idea is crucial because it allows us to use the MPT as a template for the expected PDF when a process only has random noise, and not underlying signal. (PDF is for eigen value of the covariance matrix of an underlying process, adjusted to be with zero mean, and variance sigma^2). We can then determine the % of signal that is present in our observations, and can localise the signal to a certain eigen value and above, so we can then try to extract it. 

MPT works best when you have at least 10X observations for every independent variable. The higher the X the better the distribution

Results:
When you run the code below, you can see that:
1. The Eigen values will always be positive
2. The PDF will be centered around the value of the variance, and will almost be symetrically distributed
3. At higher variances, you will have peakier distributions, at lower variances, you will have almost a uniform distribution



Compute using Python:
import numpy as np, pandas as pd
import matplotlib.pyplot as plt
# ---------------

def mpPDF(var,q,pts):
    #q = T/N
    eMax = var*(1+((1./q)**.5))
    eMin = var*(1-((1./q)**.5))
    eVal = np.linspace(eMin,eMax,pts)
    pdf = (q/(2*np.pi*eVal*var))*(((eMax-eVal)*(eVal-eMin))**.5)
    pdf = pd.Series(pdf,index=eVal)
    return pdf

pdf0 = mpPDF(var=1.,q=100000/1000,pts=1000)
pdf1 = mpPDF(var=2.,q=100000/1000,pts=1000)
pdf2 = mpPDF(var=0.5,q=100000/1000,pts=1000)

fig, ax = plt.subplots()
plt.plot(pdf0,'r',label='Variance 1')
plt.plot(pdf1,'g',label='Variance 2')
plt.plot(pdf2,'b',label='Variance 0.5')
ax.set_xlabel('Eigen Value')
ax.set_ylabel('Probability of Eigen Value')
ax.legend()
plt.show()

Comments

Post a Comment

Popular posts from this blog

1.2 Structured Data: Information Driven Bars

Image
This post will describe how to process & store High Frequency financial data to support your Machine Learning Algorithm, based on how much information is contained in the data The post is directly based on content from the book "Advances in Financial Machine Learning" from Marcos Lopez de Prado Physical meaning: This approach is inspired by Claude Shannon's Information Theory, that is the basis of most compression algorithms. Shannon argued that only departure from the norm is new information. A famous example, is that the sun has risen in the morning. Yes, the event has happened, but because the probability of this event was 1.0, this event does not represent information. In information driven bars, we attempt to use the same insight, to compress the financial data. Here, we know that financial data is inherently noisy, and put guardrails around what level of variance we expect for a particular behavior. It is only when the variance exceeds these guardrails, that we ...
Read more

2.2 Labeling: Triple barrier method

Image
 This post will describe how to label a financial dataset to support supervised learning The post is directly based on content from the book "Advances in Financial Machine Learning" from Marcos Lopez de Prado Physical meaning: Algorithm description: Python code: # Import libraries import numpy as np import pandas as pd import matplotlib.pyplot as plt import math # Import data df = pd.read_csv( r'C:\Users\josde\OneDrive\Denny\Deep-learning\Data-sets\Trade-data\ES_Trades.csv' ) df = df.iloc[: , 0 : 5 ] df[ 'Dollar' ] = df[ 'Price' ] * df[ 'Volume' ] print (df.columns) # Generate thresholds d = pd.DataFrame(pd.pivot_table(df , values = 'Dollar' , aggfunc = 'sum' , index = 'Date' )) DOLLAR_THRESHOLD = ( 1 / 50 ) * np.average(d[ 'Dollar' ]) # Generate bars def bar_gen (df , DOLLAR_THRESHOLD): collector , dollarbar_tmp = [] , [] dollar_cusum = 0 for i , (price , dollar) in enumerate ( zip (df[ 'Price...
Read more

Denoise a Covariance Matrix using Constant Residual Eigen Value Method

Image
 This post will describe how to denoise a Covariance matrix using Constant Residual Eigen Value Method Practical use: The Eigen Values are naturally stored in decreasing order. Once we find the range of Eigen Values that correspond to signal (using fit to Marcenko Pastur Distribution), we can then reset all other eigen values below this threshold to a constant value. This basically vaccums all the eigen values containing random noise into a single Eigen value. These transformed Eigen values are used with the original Eigen Vectors, to recreate a denoised covariance matrix, that will be used for further processing. Compute using Python: # File: Denoise-Constant-Residual-Eigen.py import numpy as np,pandas as pd import matplotlib.pyplot as plt #--------------------------------------------------- def mpPDF(var,q,pts): # Marcenko-Pastur pdf # q=T/N eMin,eMax=var*( 1 -( 1. /q)** .5 )** 2 ,var*( 1 +( 1. /q)** .5 )** 2 eVal=np.linspace(eMin,eMax,pts) pdf=q/( 2 *np.pi*var*eVal)*...
Read more