PROJECT TITLE :

LTC: a Fast Algorithm to Accurately Find Significant Items in Data Streams

ABSTRACT:

Finding the top k most frequent items in databases has been a contentious issue recently. The problem of locating the top-k persistent items is a new one that has received an increasing amount of attention over the past few years. In actual use, users frequently want to know which items are significant, meaning that they are not only frequent but also persistent over time. There is no prior art that can simultaneously address both of the problems that have been outlined above. Additionally, for high-speed data streams, the prior art is incapable of achieving high accuracy when memory is constrained. In this paper, we define a brand new problem, which we call the "finding significant items" problem, and we suggest a brand new algorithm, which we call LTC, to address this problem. Long-tail Restoring and CLOCK are two of the most important techniques that are included, along with three different optimizations. In addition to this, the LTC has been expanded to support the finding of significant items with thresholds. In order to evaluate the efficacy of LTC, we first theoretically derive the appropriate rate and error bound, and then we run extensive experiments on three different real-world datasets. According to the findings of our experiments, LTC is capable of achieving an accuracy that is 105 times superior to that of other related algorithms in terms of the average relative error. In conclusion, LTC is applied to a DDoS detection task, and the results demonstrate that finding significant items is more effective than finding frequent items in the data set.