Novelty Detection with One-Class Support
`Vector Machines
`
`John Shawe-Taylor and Blaž Žliˇcar
`
`Abstract In this paper we apply one-class support vector machine (OC-SVM)
`to identify potential anomalies in financial time series. We view anomalies as
`deviations from a prevalent distribution which is the main source behind the original
`signal. We are interested in detecting changes in the distribution and the timing of
`the occurrence of the anomalous behaviour in financial time series. The algorithm is
`applied to synthetic and empirical data. We find that our approach detects changes
`in anomalous behaviour in synthetic data sets and in several empirical data sets.
`However, it requires further work to ensure a satisfactory level of consistency and
`theoretical rigour.
`
`Keywords Financial time series • Novelty detection • One-class SVM
`
`1 Introduction
`
`We apply one-class support vector machine (OC-SVM) to synthetic and empirical
`data and test its ability to detect anomalous behaviour in a time series. Anomalous
`behaviour in this case is a combination of consecutive data points in a time series that
`do not belong to a distribution identified by the algorithm. We first briefly introduce
`the theory behind the OC-SVM. Then we present its application to novelty detection
`in a time series by using lagged returns as inputs. Results, main conclusions and
`recommendations for further research are outlined at the end.
`
`J. Shawe-Taylor • B. Žliˇcar ((cid:2))
`Department of CS, University College London, London, UK
`e-mail: j.shawe-taylor@cs.ucl.ac.uk; b.zlicar@cs.ucl.ac.uk
`
`© Springer International Publishing Switzerland 2015
`I. Morlini et al. (eds.), Advances in Statistical Models for Data Analysis,
`Studies in Classification, Data Analysis, and Knowledge Organization,
`DOI 10.1007/978-3-319-17377-1_24
`
`231
`
`Columbia Ex 2035-1
`Symantec v Columbia
`IPR2015-00375
`
`

`
`232
`
`J. Shawe-Taylor and B. Žliˇcar
`
`2 Background: Novelty Detection and One-Class SVM
`
`We begin by quoting a result from [1] that bounds the likelihood that data generated
`according to the same distribution used to train OC-SVM will generate a false alarm.
`Theorem 1 Fix (cid:2) > 0 and ı 2 .0; 1/. Let .c; r/ be the centre and radius of a
`hypersphere in a feature space determined by a kernel .x; x0/ D h(cid:4).x/; (cid:4).x0/i
`from a training sample S D fx1; : : : ; x`g drawn randomly according to a probability
`distribution D. Let g .x/ be the function defined by
`if kc (cid:2) (cid:2).x/k  rI
`=(cid:2); if r2  kc (cid:2) (cid:2).x/k2  r2 C (cid:2)I
`1;
`otherwise:
`Then with probability at least 1 (cid:2) ı over samples of size ` we have
`r
`
`0;
`
`(cid:2)kc (cid:2) (cid:2).x/k2 (cid:2) r2
`
`
`
`8ˆ< ˆ:
`
`g .x/ D
`
`ED Œg.x/  1
``
`
``X
`iD1
`
`g .xi/ C 6R2
`p
``
`
`(cid:2)
`
`C 3
`
`ln.2=ı/
`2`
`
`,
`
`where R is the radius of a ball in feature space centred at the origin containing the
`support of the distribution.
`Hence, the support of the distribution outside the sphere of radius r2 C (cid:2) centred
`at c is bounded by the same quantity, since g.x/ D 1 for such inputs and is less than
`1 elsewhere. Note moreover that the function g.x/ can be evaluated in kernel form
`if the optimisation is solved using its dual.
`The theorem provides the theoretical basis for the application of the OC-SVM
`and it is perhaps worth dwelling for a moment on some implications for time-series
`analysis.
`• Firstly, the assumption made by the theorem about the distribution D generating
`the training and test data has strong and weak elements:
`
`– It is strong in the sense that there are no assumptions made about the
`form of the input distribution D. It therefore applies equally to long-tailed
`distributions as it does to multivariate Gaussians. We will perform some
`experiments with real-world data in which any assumptions about the form
`of the generating distribution would be difficult to justify.
`– It is weak in the sense that it assumes the training data are generated
`independently and identically (i.i.d.), something that will not to be strictly true
`for time series. This assumption becomes more reasonable when the training
`data are drawn from separate parts of the time series.
`
`Columbia Ex 2035-2
`Symantec v Columbia
`IPR2015-00375
`
`

`
`Novelty Detection with One-Class Support Vector Machines
`
`233
`
`• The theorem is one sided in that it bounds the probability that data that arose from
`the training distribution are mistaken for novel outliers. It does not, however, say
`anything about the likelihood that novel data are not detected.
`
`In connection with the final point, Vert and Vert [3] provide an interesting analysis
`showing that if we generate negative data from an artificial background measure (cid:6)
`and train as a 2-class SVM, in the limit of large data the SVM will identify the level
`sets of the pdf of the training distribution relative to (cid:6). This suggests that it finds the
`minimal density with respect to (cid:6) consistent with capturing a given fraction of the
`input distribution. Hence, in this case, we can make assertions about the efficiency
`with which outliers are detected.
`
`3 Problem: Novelty Detection in Financial Time Series
`
`In order to apply OC-SVM to a single time series we follow the approach proposed
`by Ma and Perkins [2]. We extend this approach by adding an exponential decay
`parameter so that the more recent lags carry more weight than the older lags,
`since we are interested in detecting anomalies in the very short window before
`the occurrence of the extreme volatility, the underlying hypothesis being that the
`behaviour of the market changes before the occurrence of the spike in the time
`series.
`
`3.1 Data Preprocessing
`
`A data matrix is constructed in such a way so that the first column represents the
`original time series and every next column is a lag of the previous column. More
`specifically, for a time series variable x composed of observations x.t/ where t D
`1; : : : ; T (T being the number of time points, observations) we perform a vector-
`to-matrix transformation so that the dimensionality of the original column vector
`x changes from Tx1 to (T-d-1) x d forming a data matrix X. Here d represents our
`choice of the dimensionality expansion, i.e. the number of all columns in the newly
`formed matrix X in effect reflecting the number of lags we chose to include in the
`analysis. Alternatively, we can think of this in terms of extending the dimensionality
`of a data point x(t) to a row vector x(t) so that
`x.t/ D Œx.t/ : : : x.t (cid:2) d C 2/ x.t (cid:2) d C 1/
`
`(1)
`
`Then the newly formed data matrix in terms of row vectors becomes
`X D Œx.d/; : : : ; x.T/|
`
`(2)
`
`Columbia Ex 2035-3
`Symantec v Columbia
`IPR2015-00375
`
`

`
`234
`
`J. Shawe-Taylor and B. Žliˇcar
`
`with dimensions (T-d-1) x d (as suggested by Ma and Perkins [2]). We then take
`a step further and add a decay parameter c so that the weight of each next column
`falls exponentially with each lag. If we denote a row vector d D Œ1; : : : ; d then we
`define cd to be the row vector
`
`c D cd D Œc1; : : : ; cd
`
`(3)
`
`where c is an arbitrarily chosen decay parameter 0 < c < 1. The new data matrix
`taking into account the exponential time decay is then
`Xc D X ˇ D
`(4)
`where D is a matrix of decay factors D D 1| (cid:4) c and multiplication between X and
`D is element by element multiplication. Alternatively, if we denote the number of
`columns in X as j D 1; : : : ; d, then we can simply define the matrix D as having
`entries Dij D cj. Xc is then centred row-wise in a standard manner using a centering
`matrix C
`
`C D Id (cid:2) 1
`d
`
`Od
`
`so that the final centred data matrix with a time decay is:
`c D Xc C
`Xc
`
`(5)
`
`(6)
`
`3.2 Novelty Detection Algorithm
`
`In this section we present a step-by-step pseudo-algorithm of OC-SVM based
`novelty detection in time-series analysis.
`
`Input: a time series x(t) of length T. Output: points in time identified as novelties.
`
`(1) Vector-matrix transformation: Calculate Xc
`c using a range of different lags
`d D Œ2 W 20 to obtain 19 matrices of different dimensions XE D ŒXc
`
`2 : : : Xc20.
`The value of the decay parameter is set arbitrarily at c D 0:97.
`E is split in a train set (X-train) of length 2
`(2) Data sets: Each Xc
`3 T and the remaining
`third of observations comprise a test set (X-test). Further split X-train in half,
`that is into a sub-X-train and a val-X-train set.
`(3) Train OC-SVM: Apply OC-SVM to sub-X-train so as to obtain ˛i for each
`c in the array of matrices XE D ŒXc
`Xc
`
`2 : : : Xc20 individually and for all values of
`
`Columbia Ex 2035-4
`Symantec v Columbia
`IPR2015-00375
`
`

`
`Novelty Detection with One-Class Support Vector Machines
`
`235
`
`(cid:2) D 2i (where i D Œ(cid:2)10 W 10)1 and (cid:7) D 2j where (j D Œ(cid:2)15 W (cid:2)1). Then find
`pseudo-optimal do, (cid:2)o and sensitivity parameter (cid:7)o by locating the OC-SVM
`that achieved the highest accuracy on val-X-train.2
`(4) Test optimal OC-SVM(do,(cid:2)o,(cid:7)o): Use pseudo-optimal values determined in (3)
`to train OC-SVM(do,(cid:2)o,(cid:7)o) on X-train. Test the model on X-test and obtain the
`novelty signal for the test set.
`
`4 Experiments
`
`Firstly, we describe the construction of synthetic time series and present the
`empirical data sets (three stock market indices). Next, we comment on the results
`and outline the challenges.
`
`4.1 Data
`
`Both synthetic and empirical data sets are of about the same length (T Ñ
`5800). Synthetic time series are comprised of an original signal in the train-
`ing set while in the test sets we add anomalies (i.e. time intervals where the
`original time series is corrupted by an anomalous signal) on the intervals T D
`Œ5000W5050; 5300W5350; 5600W5650. We train the OC-SVM algorithm on a data set
`comprised solely of original (non-anomalous) time series and then test the optimal
`specification of the model on a test set that includes pre-defined intervals with
`anomalies. Synthetic time series are constructed as follows:
`(
`xa.t/ for t 2 Œ5000; 5050 ^ Œ5300; 5350 ^ Œ5600; 5650
`xo.t/ for t … Œ5000; 5050 ^ Œ5300; 5350 ^ Œ5600; 5650
`Here xo.t/ denotes the original time series and xa.t/ denotes the anomalous time
`series. Synthetic data sets are then the following three time series types:
`
`x.t/ D
`
`(7)
`
`1. Synthetic 1 time series is a sinusoid with a small standardised random noise in
`the training set, but with increased standard deviation of the error process on
`
`1We use radial basis kernel (RBF) so that k.x; y/ D exp.(cid:2)(cid:2)jjx (cid:2) yjj2/.
`2We use a term pseudo-optimal since we simply choose a specification that is able to contain all
`training data and consequently label sub-X-train data sample as novelty-free. Clearly, this is not
`necessarily the optimal solution nor is it the only solution and presents one of the main challenges
`related to novelty detection with OC-SVM.
`
`Columbia Ex 2035-5
`Symantec v Columbia
`IPR2015-00375
`
`

`
`236
`
`J. Shawe-Taylor and B. Žliˇcar
`
`specific intervals within the test set. The synthetic 1 signal follows
`/ C (cid:9).t/
`x1.t/ D sin.
`
`40(cid:8)
`N
`
`(8)
`
`where
`
`(cid:9).t/ D (cid:10) z.t/
`(9)
`and z.t/ (cid:5) N.0; 1/ with sigma term in the error process equal to (cid:10)o D 0:1 for the
`original signal xo and a slightly higher sigma term (cid:10)a D 0:2 for the anomalous
`signal xa.
`2. Synthetic 2 time series is constructed as a random process taking into account the
`empirical (sample) mean and standard deviation of an empirical time series (in
`our case the S&P500 stock market index). The original time series is constructed
`by adding an error term sampled from a standardised normal distribution and the
`anomalous signal is obtained by adding an error noise sampled from a student-t
`distribution. We write the original time series as
`x2.t/ D (cid:6) C (cid:9).t/
`
`(10)
`
`where
`
`(cid:9)o.t/ D (cid:10)SP500 z.t/
`
`(11)
`
`and z.t/ (cid:5) N.0; 1/.
`The anomalous signal follows the same process only with its noise term
`sampled from student-t distribution with six degrees of freedom
`(cid:9)a.t/ D (cid:10)SP500 z.t/
`
`(12)
`
`and z.t/ (cid:5) t6.
`3. Synthetic 3 time series is obtained by subtracting the mean from the synthetic
`2 signal and then taking the absolute value of the obtained time series. Such
`absolute returns are often used as a proxy for a volatility process in financial
`research. In other words, the synthetic 3 signal is equal to absolute error term in
`Eq. (10).
`
`x3.t/ D j(cid:9).t/j D j(cid:10)SP500 z.t/j
`(13)
`with z.t/ (cid:5) N.0; 1/ for the original time series and z.t/ (cid:5) t6 for the anomalous
`time series.
`
`Empirical data sets are time series of three stock market indices: S&P500,
`DAX30 and NIKKEI225. We work with adjusted daily closing prices obtained
`
`Columbia Ex 2035-6
`Symantec v Columbia
`IPR2015-00375
`
`

`
`Novelty Detection with One-Class Support Vector Machines
`
`237
`
`from Yahoo.Finance for a period of approximately 13 years, where we perform the
`following two transformations of the original series:
`
`1. Log returns are calculated by obtaining the difference between natural loga-
`rithms of a price at time t and price at time t (cid:2) 1:
`r.t/ D ln.
`pt
`pt(cid:2)1
`
`(14)
`
`/
`
`Then we subtract the mean of the return time series:
`rdm.t/ D r.t/ (cid:2) (cid:6)
`
`(15)
`
`2. Absolute returns are obtained by simply taking the absolute value of the log
`returns. Absolute returns are a frequently used proxy for a volatility measure in
`financial research
`
`rabs.t/ D jr.t/j
`
`(16)
`
`4.2 Results
`
`In this section we describe the results for OC-SVM algorithm applied to synthetic
`and empirical data sets without and with the exponential decay parameter in the
`preprocessing phase. The two algorithms are denoted with OC-SVM-ND (no decay)
`and OC-SVM-ED (exponential decay), respectively. Optimal model specification is
`indicated by adding optimal parameter values in brackets so that OC-SVM(do,(cid:2)o,(cid:7)o)
`denotes the specification of OC-SVM with optimal index values for the dimension
`(indicating number of lags), (cid:2) in RBF kernel and (cid:7) parameter in OC-SVM.3 Please
`note that this is a naive pseudo-optimisation simply assuming that the best OC-
`SVM is the one that is able to put a bound around the data in a training set. We use
`LIBSVM support vector machine toolbox. Finally, figures are moved to Appendix to
`prevent cluttering.
`
`3Where the numbers refer to the index not the value itself. For example, OC-SVM-ND (1,2,3)
`would denote the optimal specification of OC-SVM without the decay parameter, where the
`optimal lag dimension is the first dimension in the dimension array d D Œ2 W 20, i.e. do D 2,
`the optimal (cid:7) refers to the second position in the j array j D Œ(cid:2)15 W (cid:2)1, i.e.(cid:7) o D 2(cid:2)14, and the
`optimal (cid:2) refers to the third position in the i array i D Œ(cid:2)10 W 10, i.e. (cid:2)o D 2(cid:2)8.
`
`Columbia Ex 2035-7
`Symantec v Columbia
`IPR2015-00375
`
`

`
`238
`
`4.2.1 Synthetic Data
`
`J. Shawe-Taylor and B. Žliˇcar
`
`Synthetic time series are constructed so that we can investigate the performance
`and the ability of OC-SVM to detect novelties that were artificially inserted in the
`testing part of the various types of data sets. Ideally, no novelties would be detected
`in the valuation part of the training set. In the test set the best performance is the
`one detecting novelties in the previously determined anomalous intervals.
`Figure 1 displays the results for valuation part of the training set on the left side
`(with synthetic 1 at the top and OC-SVM-ND novelty signal at the bottom) and the
`test set on the right side. It shows that the model has the ability to learn the original
`signal in the training set since it correctly detects no novelties (novelty signal is
`equal to 1 at all times). When the same model is applied to the test set we see that
`it correctly identifies areas where anomalous data have been added to the original
`signal (grey areas). However, it also falsely returns novelty signal where no novelties
`have been added to the original synthetic signal, indicating that this particular OC-
`SVM specification is perhaps still too sensitive to outliers. Figure 2 shows results
`of the OC-SVM-ED (with exponential decay) applied to the same synthetic 1 time
`series. This model is also successful in identifying the anomalous areas in the test
`set with slightly lower number of false positives. The results for other two synthetic
`signals are displayed in Figs. 3, 4, 5, and 6. Both algorithms, without and with decay
`parameter, are able to detect the anomalous areas with a small number of false
`positives. Only Fig. 3 stands out as it displays a poor performance of OC-SVM-ND
`in the test set of a synthetic 2 time series.
`
`4.2.2 Empirical Data
`
`Our empirical experiments focus on whether or not the algorithm detects anomalies
`slightly before volatility spikes. Figures 7 and 8 display results of OC-SVM applied
`to absolute returns of the empirical time series without and with decay, respectively.
`Figure 9 shows the valuation part of a training sample on the left and test sample on
`the right side for the S&P500 stock market index. Top row displays the time series
`of S&P500, middle row the return time series and bottom row the novelty signal
`for OC-SVM-ND. In this case our algorithm detects two intervals as anomalous
`(around time points 600 and 1300). Figure 9 displays identical figure twice with the
`only difference being that the two charts in fifth and sixth row on the right side are
`magnified around the novelty point so as to show the case of early novelty detection
`(around point 600). However, the volatility spike around time point 1300 is not
`detected in advance. The same results in both of these volatility cases are obtained
`when OC-SVM is applied using the exponential decay parameter (Fig. 10). Also, the
`results are very similar when both types of algorithms (with and without the decay
`parameter) are applied to S&P500 absolute returns time series. In case of the DAX30
`index none of the two algorithms detect sudden increase in volatility in advance
`when applied to time series of returns (Figs. 11 and 12). When applied to absolute
`returns (Figs. 13 and 14) both algorithms (OC-SVM-ND and OC-SVM-ED) detect
`
`Columbia Ex 2035-8
`Symantec v Columbia
`IPR2015-00375
`
`

`
`Novelty Detection with One-Class Support Vector Machines
`
`239
`
`the second volatility spike in advance (around the time point 550) but their first
`detections (around the time point 350) temporally coincide with the volatility spike.
`In case of the NIKKEI225 index both OC-SVM-ND and OC-SVM-ED detect the
`biggest volatility increase in advance (around the time point 600) when applied
`to returns and absolute returns. However, both specifications also fail to detect in
`advance the second spike around the time point 1200, again when applied to either
`returns or absolute returns (Figs. 15, 16, 17 and 18).
`
`5 Conclusions and Further Research
`
`In this paper we investigate the application of the OC-SVM to novelty detection in
`financial time series. We add an exponential decay parameter when preprocessing
`the data to account for the reduced dependency related to older data points. We
`test the OC-SVM on synthetic data sets and find that our algorithm manages to
`consistently identify anomalous areas inserted artificially in our test sets. Building
`on these results we then apply the algorithm to empirical data, namely financial
`time series of three stock market indices: S&P500, DAX30 and NIKKEI225. The
`idea is that the projection of the market data into the feature space effectively
`allows for an inspection of market patterns we would normally not detect in the
`input space. This means that in cases when anomalous behaviour in the markets
`(reflected in the change of the distribution of the time series) has preceded the
`spike, our algorithm might be able to detect these anomalies. However, when the
`spike in volatility is the result of an unexpected exogenous event, OC-SVM will
`not be able to alert the user in advance since the time series is not reflecting
`the impending risk.4 Our experiments to some extent support this reasoning as
`the results show instances where OC-SVM, with and without a decay parameter,
`detects novelties occurring before volatility spikes, but such results are by no
`means conclusive. The unsupervised nature of OC-SVM allows for the detection of
`previously unseen observation, however it simultaneously prevents us from targeting
`a type of process (event). This makes it useful for novelty detection in synthetic data
`sets (where novelty points are known in advance) while making it problematic for
`the application to empirical data sets.5 Optimisation of the hyperparameter (cid:7) in OC-
`SVM is a challenge in itself and when applied to financial time-series analysis this
`problem becomes even more difficult. Future research could perhaps investigate a
`possible connection between (cid:7) and the level of randomness of the underlying time
`series. Also, it would be interesting to investigate the usefulness of one-class SVM
`for novelty detection in multivariate data.
`
`4Note that for the synthetic data this does not arise since the volatility takes immediate effect.
`5Especially when applied to extremely noisy data such as financial time series.
`
`Columbia Ex 2035-9
`Symantec v Columbia
`IPR2015-00375
`
`

`
`240
`
`Appendix
`
`J. Shawe-Taylor and B. Žliˇcar
`
`Fig. 1 Synthetic 1: OC-SVM-ND (19,7,1)
`
`Columbia Ex 2035-10
`Symantec v Columbia
`IPR2015-00375
`
`

`
`Novelty Detection with One-Class Support Vector Machines
`
`241
`
`Fig. 2 Synthetic 1: OC-SVM-ED (12,7,1)
`
`Columbia Ex 2035-11
`Symantec v Columbia
`IPR2015-00375
`
`

`
`242
`
`J. Shawe-Taylor and B. Žliˇcar
`
`Fig. 3 Synthetic 2: OC-SVM-ND (15,9,5)
`
`Columbia Ex 2035-12
`Symantec v Columbia
`IPR2015-00375
`
`

`
`Novelty Detection with One-Class Support Vector Machines
`
`243
`
`Fig. 4 Synthetic 2: OC-SVM-ED (1,7,9)
`
`Columbia Ex 2035-13
`Symantec v Columbia
`IPR2015-00375
`
`

`
`244
`
`J. Shawe-Taylor and B. Žliˇcar
`
`Fig. 5 Synthetic 3: OC-SVM-ND (8,8,8)
`
`Columbia Ex 2035-14
`Symantec v Columbia
`IPR2015-00375
`
`

`
`Novelty Detection with One-Class Support Vector Machines
`
`245
`
`Fig. 6 Synthetic 3: OC-SVM-ED (13,9,7)
`
`Columbia Ex 2035-15
`Symantec v Columbia
`IPR2015-00375
`
`

`
`Train: Price
`
`Test: Price
`
`J. Shawe-Taylor and B. Žliˇcar
`
`1500
`1000
`500
`Test: Absolute return
`
`1500
`1000
`500
`Test: OC−SVM−ND
`
`500
`
`1000
`
`1500
`
`2000
`
`1500
`
`1000
`
`500
`
`0
`
`0.1
`
`0.05
`
`0
`
`0
`
`01
`
`−1
`
`0
`
`1500
`1000
`500
`Train: Absolute return
`
`1500
`1000
`500
`Train: OC−SVM−ND
`
`500
`
`1000
`
`1500
`
`246
`
`2000
`
`1500
`
`1000
`
`500
`
`0
`
`0.1
`
`0.05
`
`0
`
`0
`
`01
`
`−1
`
`0
`
`Fig. 7 S&P500 absolute returns: OC-SVM-ND (17,10,4)
`
`Columbia Ex 2035-16
`Symantec v Columbia
`IPR2015-00375
`
`

`
`Novelty Detection with One-Class Support Vector Machines
`
`247
`
`Train: Price
`
`Test: Price
`
`1500
`1000
`500
`Test: Absolute return
`
`1500
`1000
`500
`Test: OC−SVM−ED
`
`500
`
`1000
`
`1500
`
`2000
`
`1500
`
`1000
`
`500
`
`0
`
`0.1
`
`0.05
`
`0
`
`0
`
`01
`
`−1
`
`0
`
`1500
`1000
`500
`Train: Absolute return
`
`1500
`1000
`500
`Train: OC−SVM−ED
`
`500
`
`1000
`
`1500
`
`2000
`
`1500
`
`1000
`
`500
`
`0
`
`0.1
`
`0.05
`
`0
`
`0
`
`01
`
`−1
`
`0
`
`Fig. 8 S&P500 absolute returns: OC-SVM-ED (17,10,5)
`
`Columbia Ex 2035-17
`Symantec v Columbia
`IPR2015-00375
`
`

`
`248
`
`J. Shawe-Taylor and B. Žliˇcar
`
`Train: Price
`
`Test: Price
`
`2000
`
`1500
`
`1000
`
`500
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Train: Daily return
`
`Test: Daily return
`
`0.1
`
`0.05
`
`0
`
`−0.05
`
`−0.1
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Train: OC−SVM−ND
`
`Test: OC−SVM−ND
`
`01
`
`−1
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Train: Price
`
`Test: Price
`
`2000
`
`1500
`
`1000
`
`500
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Train: Daily return
`
`Test: Daily return
`
`0.1
`0.05
`0
`−0.05
`−0.1
`550
`
`600
`
`650
`
`0
`
`500
`
`1000
`
`1500
`
`Train: OC−SVM−ND
`
`Test: OC−SVM−ND
`
`01
`
`−1
`
`2000
`
`1500
`
`1000
`
`500
`
`0.1
`
`0.05
`
`0
`
`−0.05
`
`−0.1
`
`01
`
`−1
`
`2000
`
`1500
`
`1000
`
`500
`
`0.1
`
`0.05
`
`0
`
`−0.05
`
`−0.1
`
`01
`
`−1
`
`0
`
`500
`
`1000
`
`1500
`
`550
`
`600
`
`650
`
`Fig. 9 S&P500 returns: OC-SVM-ND (6,8,5); bottom figure is identical to the top one with
`magnified bottom charts (Test: Daily return and Test: OC-SVM-ND) to demonstrate the early
`novelty detection by the algorithm
`
`Columbia Ex 2035-18
`Symantec v Columbia
`IPR2015-00375
`
`

`
`Novelty Detection with One-Class Support Vector Machines
`
`249
`
`Train: Price
`
`Test: Price
`
`2000
`
`1500
`
`1000
`
`500
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Train: Daily return
`
`Test: Daily return
`
`0.1
`
`0.05
`
`0
`
`−0.05
`
`−0.1
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Train: OC−SVM−ED
`
`Test: OC−SVM−ED
`
`01
`
`−1
`
`2000
`
`1500
`
`1000
`
`500
`
`0.1
`
`0.05
`
`0
`
`−0.05
`
`−0.1
`
`01
`
`−1
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Fig. 10 S&P500 returns: OC-SVM-ED (7,7,7)
`
`Columbia Ex 2035-19
`Symantec v Columbia
`IPR2015-00375
`
`

`
`Train: Price
`
`Test: Price
`
`J. Shawe-Taylor and B. Žliˇcar
`
`10000
`8000
`6000
`4000
`2000
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Train: Daily return
`
`Test: Daily return
`
`0.1
`
`0.05
`
`0
`
`−0.05
`
`−0.1
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Train: OC−SVM signal
`
`Test: OC−SVM signal
`
`01
`
`−1
`
`250
`
`10000
`8000
`6000
`4000
`2000
`
`0.1
`
`0.05
`
`0
`
`−0.05
`
`−0.1
`
`01
`
`−1
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Fig. 11 DAX returns: OC-SVM-ND (1,6,8)
`
`Columbia Ex 2035-20
`Symantec v Columbia
`IPR2015-00375
`
`

`
`Novelty Detection with One-Class Support Vector Machines
`
`251
`
`Train: Price
`
`Test: Price
`
`10000
`8000
`6000
`4000
`2000
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Train: Daily return
`
`Test: Daily return
`
`0.1
`
`0.05
`
`0
`
`−0.05
`
`−0.1
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Train: OC−SVM signal
`
`Test: OC−SVM signal
`
`01
`
`−1
`
`10000
`8000
`6000
`4000
`2000
`
`0.1
`
`0.05
`
`0
`
`−0.05
`
`−0.1
`
`01
`
`−1
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Fig. 12 DAX returns: OC-SVM-ED (1,6,8)
`
`Columbia Ex 2035-21
`Symantec v Columbia
`IPR2015-00375
`
`

`
`Train: Price
`
`Test: Price
`
`J. Shawe-Taylor and B. Žliˇcar
`
`10000
`8000
`6000
`4000
`2000
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Train: Daily return
`
`Test: Daily return
`
`252
`
`10000
`8000
`6000
`4000
`2000
`
`0.1
`
`0.05
`
`500
`
`1000
`
`1500
`
`Test: OC−SVM signal
`
`0.1
`
`0.05
`
`0
`
`0
`
`01
`
`−1
`
`500
`
`1000
`
`1500
`
`Train: OC−SVM signal
`
`0
`
`0
`
`01
`
`−1
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Fig. 13 DAX absolute returns: OC-SVM-ND (15,9,4)
`
`Columbia Ex 2035-22
`Symantec v Columbia
`IPR2015-00375
`
`

`
`Novelty Detection with One-Class Support Vector Machines
`
`253
`
`Train: Price
`
`Test: Price
`
`10000
`8000
`6000
`4000
`2000
`
`10000
`8000
`6000
`4000
`2000
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Train: Daily return
`
`Test: Daily return
`
`500
`
`1000
`
`1500
`
`Test: OC−SVM signal
`
`0.1
`
`0.05
`
`0
`
`0
`
`01
`
`−1
`
`500
`
`1000
`
`1500
`
`Train: OC−SVM signal
`
`0.1
`
`0.05
`
`0
`
`0
`
`01
`
`−1
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Fig. 14 DAX absolute returns: OC-SVM-ED (7,8,6)
`
`Columbia Ex 2035-23
`Symantec v Columbia
`IPR2015-00375
`
`

`
`x 104
`
`Train: Price
`
`J. Shawe-Taylor and B. Žliˇcar
`
`x 104
`
`Test: Price
`
`3
`2.5
`2
`1.5
`1
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Train: Daily return
`
`Test: Daily return
`
`0.1
`0.05
`0
`−0.05
`−0.1
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Train: OC−SVM−ND
`
`Test: OC−SVM−ND
`
`01
`
`−1
`
`254
`
`3
`2.5
`2
`1.5
`1
`
`0.1
`0.05
`0
`−0.05
`−0.1
`
`01
`
`−1
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Fig. 15 NIKKEI225 returns: OC-SVM-ND (16,9,1)
`
`Columbia Ex 2035-24
`Symantec v Columbia
`IPR2015-00375
`
`

`
`Novelty Detection with One-Class Support Vector Machines
`
`255
`
`x 104
`
`Train: Price
`
`x 104
`
`Test: Price
`
`3
`2.5
`2
`1.5
`1
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Train: Daily return
`
`Test: Daily return
`
`0.1
`0.05
`0
`−0.05
`−0.1
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Train: OC−SVM−ED
`
`Test: OC−SVM−ED
`
`01
`
`−1
`
`3
`2.5
`2
`1.5
`1
`
`0.1
`0.05
`0
`−0.05
`−0.1
`
`01
`
`−1
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Fig. 16 NIKKEI225 returns: OC-SVM-ED (16,9,1)
`
`Columbia Ex 2035-25
`Symantec v Columbia
`IPR2015-00375
`
`

`
`J. Shawe-Taylor and B. Žliˇcar
`
`x 104
`
`Test: Price
`
`x 104
`
`Train: Price
`
`256
`
`3
`2.5
`2
`1.5
`1
`
`3
`2.5
`2
`1.5
`1
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Train: Absolute return
`
`Test: Absolute return
`
`500
`
`1000
`
`1500
`
`Test: OC−SVM−ND
`
`0.1
`
`0.05
`
`0
`
`0
`
`01
`
`−1
`
`500
`
`1000
`
`1500
`
`Train: OC−SVM−ND
`
`0.1
`
`0.05
`
`0
`
`0
`
`01
`
`−1
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Fig. 17 NIKKEI225 absolute returns: OC-SVM-ND (16,9,1)
`
`Columbia Ex 2035-26
`Symantec v Columbia
`IPR2015-00375
`
`

`
`Novelty Detection with One-Class Support Vector Machines
`
`257
`
`x 104
`
`Train: Price
`
`x 104
`
`Test: Price
`
`3
`2.5
`2
`1.5
`1
`
`0
`
`500
`
`1000
`
`1500
`
`0
`
`500
`
`1000
`
`1500
`
`Train: Absolute return
`
`Test: Absolute return
`
`3
`2.5
`2
`1.5
`1
`
`0.1
`
`0.05
`
`500
`
`1000
`
`1500
`
`Test: OC−SVM−ED
`
`500
`
`1000
`
`1500
`
`0.1
`
`0.05
`
`0
`
`0
`
`01
`
`−1
`
`0
`
`500
`
`1000
`
`1500
`
`Train: OC−SVM−ED
`
`500
`
`1000
`
`1500
`
`—
`
`0
`
`0
`
`01
`
`−1
`
`0
`
`Fig. 18 NIKKEI225 absolute returns: OC-SVM-ED (16,9,1)
`
`References
`
`1. Cristianini, N., Shawe-Taylor, J.: Kernel Methods for Pattern Analysis. Cambridge University
`Press, Cambridge (2004)
`2. Ma, J., Perkins, S.: Time-series novelty detection using one-class support vector machines. In:
`Proceedings of the International Joint Conference on Neural Networks (2003)
`3. Vert, R., Vert, J-P.: Consistency and convergence rates of one-class SVMs and related algo-
`rithms. J. Mach. Learn. Res. 7, 815–854 (2006)
`
`Columbia Ex 2035-27
`Symantec v Columbia
`IPR2015-00375