This essay has been submitted by a student. This is not an example of the work written by professional essay writers.
# An Adaptive Online Learning Model for Flight Data Cluster Analysis

## Introduction:

### Method

### Data Transformation

### Cluster Analysis

### Gaussian Mixture Model

### Online EM algorithm:

### Projection function

### Algorithm:

### Conclusion

### Your time is important. Let us write you an essay from scratch

Pssst…
**we can write an original essay just for you.**

Any subject. Any type of essay.

We’ll even meet a 3-hour deadline.

121 writers online

In the past, the improvement of aviation safety was mostly based on the analysis of accidents, but the airline could not effectively detect and deal with the unknown safety hazard. In recent years, the aviation industry has adopted many data-based risk identification methods. The digital flight data recorder (FDR) data is used by many airlines for routine analysis to identify risks. In order to better identify factors in FDR data that cannot be artificially identified, some methods based on cluster analysis have been developed, such as ClusterAD (Li et al. 2015, Li et al. 2016). However, no matter which clustering method is used, these methods can only perform offline analysis. So, when the new data come in, the original model and the result cannot be adjusted accordingly. Cluster analysis need to be conducted over again by re-entering a large amount of old data and new data, which is computationally intensive and unable to track the changes in clusters as flight operations evolve. In practice, airlines consolidate new flight data and perform Flight operations quality assurance (FOQA) or Flight Data Monitoring (FDM) analysis every month. In order to make the cluster-based anomaly detection method compatible with airline’s practice, an online clustering method is needed to process new data every month.

In recent years, in the transportation direction, many methods have been developed to monitor the operation of transportation. (Chang et al., 2008; Shi and Abdel-Aty, 2015; Li et al., 2014; Shichrur et al., 2014; Toledo et al., 2008; Zhang et al., 2011). In the field of aviation, there have been many studies showing that clustering is a technology that can effectively identify various common patterns in operations. The Morning Reporting Package was one of the methods to detect anomalies in the aviation field at early stages (Amidan and Ferryman, 2005). The method uses statistical and mathematical based algorithms to identify abnormal flights, flight parameters and flight phases. By inputting discrete flight data, such as binary switches inside the cockpit, the Sequence Miner algorithm can detect abnormalities in pilot switch operations based on Longest Common Subsequence (LCS) metric (Budalakoti et al., 2006). Srivastava et al. developed a statistical method that discretizes continuous data to combine discrete data with continuous data (Srivastava, 2005). On top of this framework, Das et al. developed multi-core anomaly detection (MKAD). This method based on the theory of multiple kernel learning and adopted one-class Support Vector Machine (SVM) to detect anomalies from a large set of continuous and discrete data (Das et al., 2010). Cluster-AD developed by Li et al. is a technique which is directly applicable to solve the anomaly detection problem for flight operations (Li et al., 2015). This method transforms time series data into a high-dimensional vector, which represents one trajectory of a flight for take-off or approach phase. After dimensionality reduction, the method adopts DBSCAN to cluster all the flights to discovery the common operations. ClusterAD-DataSample uses the snapshot data at each time point as a data sample and adopts Gaussian Mixture Model to automatically recognize multiple typical patterns of flight operations (Li et al. 2016). Melnyk et al. adopt a semi-Markov switching vector autoregressive (SMS-VAR) model to represents each flight and detect anomalies based on measuring difference between the model’s prediction and data observation (Melnyk et al. 2016). A common challenge exists for all the above methods is that they all focus on the anomaly detection based on historical digital flight data. As new flight data come in, the methods cannot be update rapidly.

Sato et al. developed an online EM algorithm for normalized Gaussian network, which shows that the on-line EM algorithm can be seen as a stochastic approximation process to find the maximum likelihood estimator (Seto et al.). The idea of the online expectation-maximization (EM) algorithm for the method was derived by Xu, Jordan, & Hinton (Xu et al., 1995). Later, many studies adopt this idea to develop similar online EM algorithms for mixture models (Nowlan et al., 1991; Jordan et al., 1994; Liu et al., 2004). In this article, we adopt the idea of online EM algorithm (Xu et al., 1995) to update the parameters of Gaussian Mixture Model.

This study aims to develop an online cluster model to detect common patterns in the operations of aircrafts and update the identified patterns as new flight data come in. Compared with existing methods, the advantages of the new method lie in that it can (1) update the cluster model as new flight data are added into the model, and (2) track changes in clusters over time.

The main purpose of the method is to perform cluster analysis and update the parameters of the cluster model as new flight data come in. The workflow of the method in this paper is illustrated in Fig.1, which consists of two parts: offline parts and online parts. For offline part, the algorithm run only once to get the initial parameters of the cluster model and for online parts the algorithm run every time when new flight data come in and the clusters are updated.

Firstly, we need to normalize the fight parameters to have “zero mean and unit variance” for offline data. As for online data, we normalize them to the same standard with normalized offline data. After normalization, flight data are transformed into high-dimension vectors. Each vector represent a single flight, including some selected parameters.

v=[x_1^1,x_2^1,…,x_1^1,…,x_j^i,…,x_n^m] (1)

where x_j^iis the value of the jth flight parameter at time i. m is the number of flight parameters and n represent the number of samples for every flight parameter.

After the transformation step, we develop a clustering method based on a GMM to identify the different common patterns of the flights.

We use clustering to identify the same pattern in flight data. The advantage is that the clustering algorithm can automatically assign similar vectors to the same cluster without the need to manually add a label. The Gaussian mixture model is a typical clustering method and is often used in methods that need to know the statistical properties of each cluster. The advantage is that parameters describe the characteristics of each Gaussian component, making it easy to update the parameters in the online algorithm. The GMM with the K component is given by:

p(x│λ)=∑_(i=1)^K▒〖ω_i g(x|μ_i,∑_i)〗 (2)

Where x is a set of M-dimensional vectors, λ_i={ω_i,μ_i,∑_i} are the GMM parameters, K is the number of components of Gaussians and ω_i, i = 1,…, K are mixture weights, satisfying ∑_i^K▒〖ω_i=1〗, μ_i, i = 1,…, K are the mean vectors and ∑_i, i = 1,…, K are the covariance matrixes of Gaussians, and g(x|μ_i,∑_i) are the component Gaussian densities.

g(x|μ_i,∑_i)=1/√(〖(2π)〗^M |∑_i |) e^(-1/2 〖(x-μ_i)〗^’ ∑_i^(-1) (x-μ_i)) (3)

In order to determine the number of Gaussian mixture model components, K, we tried a number of Ks for sensitivity analysis. The final K is selected based on Bayesian Information Criterion (BIC), and the K with the smallest BIC is considered to be the best component.

The parameters of GMM (λ_0={ω_0,μ_0,∑_0}) in offline part are obtained by using expectation-maximization (EM) algorithm, which is a well-established method. And we use 1×〖10〗^(-6) as the termination tolerance (ε) for the objective function value.

For the online part, we use the parameters that we get for offline part as initial parameters and update the parameters with online EM algorithm which will be illustrated below.

In order to update the clusters as new flight data come in, we introduce a new algorithm to estimate the parameter of GMM based on new data and initial GMM parameters, named online (recursive) EM algorithm.

The online EM algorithm is composed of two parts. One is to get the updated parameters for new dataset from initial offline parameters and the other one is to combine the updated parameters with the offline parameters to get the final updated parameter. For the first part, we need to update the initial parameters when new data come in. This process can be viewed as a projection. So, we let

λ^((k+1))=Ξ(λ^((k) ),X ̅_(k+1)) (4)

Where λ^((k) ) are initial parameters and λ^((k+1)) are parameters after updating, X ̅_(k+1) are new dataset that come in and Ξ is the projection function that we will discuss below. To ensure the parameter converge, we let 〖{a〗_k,k≥0} be a sequence of positive numbers, satisfying

a_k>0,a_k→∞,∑_(n=1)^∞▒a_k^(1+δ) <∞, (5)

where δ∈(0,1) is a constant number. Then, at the kth iteration, the parameter λ is updated by

λ^((k+1))=(1-a_k ) λ^((k) )+a_k Ξ(λ^((k) ),X ̅_(k+1)) (6)

The second part of our online algorithm is to combine initial parameters and updated parameters through a certain proportion to get the final updated parameters. We let w be the weight of new data.

λ^new=(1-w) λ^initial+〖wλ〗^updated (7)

To get the projection function of the online EM algorithm, we utilize the fact that at each iteration, the parameter increments have a positive projection on the gradient of the likelihood function, which is established by Xu and Jordan (1996). In EM algorithm for GMM, the log likelihood is

L_N (X,λ)=∑_(t=1)^N▒ln〖(∑_(i=1)^K▒〖ω_i g(x_t |μ_i,∑_i)〗)〗 (8)

The online EM algorithm for GMM can be represent by

ω^(k+1)=ω^k+P_ω^((k)) ├ (∂L_N (X,λ))/∂ω┤|_(ω=ω^((k)) ) (9)

μ^(k+1)=μ^k+P_(μ_i)^((k)) ├ (∂L_N (X,λ))/(∂μ_i )┤|_(μ_i=〖μ_i〗^((k)) ) (10)

∑^(k+1)=∑^k+P_(∑_i)^((k)) ├ (∂L_N (X,λ))/(∂∑_i )┤|_(∑_i=〖∑_i〗^((k)) ) (11)

Where

P_ω^((k))=1/N [(■(ω_1^((k))&…&[email protected]⋮&⋱&⋮@0&…&ω_K^((k)) ))-ω^((k) ) 〖(ω^((k)))〗^T ]

P_(μ_i)^((k))=(∑_i^((k)))/(∑_(t=1)^N▒Pr^((k))(i│x_t,λ) )

P_(∑_i)^((k))=2/(∑_(t=1)^N▒Pr^((k))(i│x_t,λ) ) ∑_i^((k))⊗∑_i^((k))

The proof of (9)-(11) can be given by the following the same line as the derivation of Theorem 1 of Xu and Jordan (1996). Then we let

P_μ^((k))=(■(P_(μ_1)^((k))&⋯&[email protected]⋮&⋱&⋮@0&⋯&P_(μ_K)^((k)) ))

P_∑^((k))=(■(P_(∑_1)^((k))&⋯&[email protected]⋮&⋱&⋮@0&⋯&P_(∑_K)^((k)) ))

We can convert online EM algorithm to a gradient algorithm:

λ^((k+1))-λ^((k) )=P^((k)) L_N^’ (X ̅_(k+1),λ^((k) ) ), (12)

Table 1

Pseudo code of Online EM Algorithm

Input:

Normalized vectors of new digital flight data x_t

Initial parameters of offline GMM, λ^initial ,and the number of Gaussian components, K

Output:

New GMM parameters, λ^new

E step. For each data sample, determine the a posteriori probability for each Gaussian component i using the following equation.

Pr(i│x_t,λ)=(ω_i g(x_t│μ_i,∑_i ))/(∑_(j=1)^K▒〖ω_j g(x_t│μ_j,∑_i ) 〗)

where x_t is a normalized vector of digital flight data

M step. Update GMM parameters k. For each Gaussian component, update parameters using the following equation.

λ^((k+1))=(1-a_k ) λ^((k) )+a_k [λ^((k) )+P^((k)) L_N^’ (X ̅_(k+1),λ^((k) ) )]

Evaluate log likelihood

L_N (X,λ)=∑_(t=1)^N▒ln(∑_(i=1)^K▒〖ω_i g(x_t│μ_i,∑_i ) 〗)

If likelihood converge is smaller than the termination tolerance, ε,

Output λ^updated;

Else,

λ=λ^((k+1))

go to Step 2.

Combine initial parameters and updated parameters by the following scheme.

λ^new=(1-w) λ^initial+〖wλ〗^updated

Output λ^new

Where P^((k))=(■(P_ω^((k))&0&[email protected]&P_μ^((k))&[email protected]&0&P_∑^((k)) )) and L_N^’ (X ̅_(k+1),λ^((k) ) )=(∂L_N (X ̅_(k+1),λ))⁄(∂〖λ|〗_(λ=λ^((k) ) ) ). So, the proposed parameter update scheme (6) can be represented by

λ^((k+1) )=(1-a_k ) λ^((k) )+a_k Ξ(λ^((k) ),X ̅_(k+1) )

=(1-a_k ) λ^((k) )+a_k [λ^((k) )+P^((k)) L_N^’ (X ̅_(k+1),λ^((k) ) )] (13)

Digital flight data are collected by airlines every month even every day. It would be very time consuming to analyze all of the data every time new data is generated. Even in the data reading phase, it takes a lot of time, which will greatly affect the efficiency of the airline operation. We developed a method for online clustering of new data, which can cluster new data and update the original clusters when new data are added. The method was tested on real-world datasets provided by international airlines. Results show that this method is able to cluster new data and update the parameters of the clusters that are already exist. However, there are still some limitations in our method. On the one hand, the method can only put new data into the clusters that already exist and the number of clusters is fixed. When new clusters appear, our method cannot detect that changes and failed to add new clusters in. As airlines may adopt new technology or system on their aircraft, new clusters can be more valuable that the historical data. On the other hand, new digital flight data are not the only data generated by the airlines. Two types of new data related to anomaly detection are generated at airlines every day. One is the onboard flight data; the other one is the airline safety experts’ feedback on the anomaly detection results. Safety experts’ feedback is as important as new digital flight data and is a good way for us to identify the realistic characteristics of each particular flight state. It can also help the airline to recognize different risks in various states. Therefore, method that considers the real-time update of the two types of new data will be a better method to detect anomalies and give more information to the airline.

In future steps, we will continue to complete the part of anomaly detection in some statistical ways to find outliers in our clusters and conduct some analysis on the outlier and clusters to identify different common patterns in operations. We will also try to improve our method to enable the updating in the number of clusters. It will be very useful to identify new clusters in new data, which will tell more information to the airlines. Safety experts’ feedback is another part that we are interested in. We will try to use the experts’ feedback to identify the new outliers to avoid repeating work on the same type of outliers.

Remember: This is just a sample from a fellow student.

100% plagiarism free

Sources and citations are provided

To export a reference to this article please select a referencing style below:

GradesFixer. (2019). An Adaptive Online Learning Model for Flight Data Cluster Analysis. Retrived from https://gradesfixer.com/free-essay-examples/an-adaptive-online-learning-model-for-flight-data-cluster-analysis/

GradesFixer. "An Adaptive Online Learning Model for Flight Data Cluster Analysis." *GradesFixer*, 14 May. 2019, https://gradesfixer.com/free-essay-examples/an-adaptive-online-learning-model-for-flight-data-cluster-analysis/

GradesFixer, 2019. *An Adaptive Online Learning Model for Flight Data Cluster Analysis*. [online] Available at: <https://gradesfixer.com/free-essay-examples/an-adaptive-online-learning-model-for-flight-data-cluster-analysis/> [Accessed 12 July 2020].

copy to clipboard

Having trouble finding the perfect essay? We’ve got you covered.
Hire a writer

GradesFixer.com uses cookies. By continuing we’ll assume you board with our cookie policy.