Karl Taht's Research Paper Blog

Authors: J.R.R. Uijlings, K.E.A. van de Sande, T. Gevers, and A.W.M. Smeulders Venue: Tech Report / IJCV 2012 Note that I did not read this paper in it's entirety, but mainly tried to focus on the ideas presented in this paper. Selective Search image segmentation uses a hierarchical clustering approach to segment the image into different related objects, motivated by the intuitive hierarchical relationship images have. When clustering, the algorithm takes into account the color similiarity of regions as well as texture similarity. For color similarity, they make use of multiple color scales such as RGB, grayscale, and HSV which each provide different properties. For texture measurements, they use SIFT (Scale-invarient feature transform). To analyze the hierarchical component, they compare there approach to flat clustering approach, Efficient Graph-based Image Segmentation. To analyze their overall performance, they compare to many other approaches, and do so rather exhaustivel...

Authors: Abigail See, Minh-Thang Luong, Christopher D. Manning (Stanford) Venue:    Arxiv This paper applies pruning techniques to encoder-decoder deep multi-layer recurrent architecture with LSTM as the hidden unit type. The paper tries various pruning types, but finds the most effective to be simply pruning weights of least magnitude overall. While overall the techniques are mostly brought over from pruning techniques used in CNN's and other networks, the papers does make note of interesting artifacts of pruning. Firstly, deeper neurons are more sensitive to pruning that early neurons. In other words, the deeper units are actually of more importance and more sensitive to even low-magnitude weights. Additionally, they find that the sparse models can even out perform the originals, and claim that this is most likely due to the "generalizing" effect that pruning has. They say that while training set accuracy decreases, validation set accuracy actually increases! Addition...

Authors: Yu Ji, Youhui Zhang, Wenguang Chen, Yuan Xie Venue: ASPLOS 2018 With the machine learning community trying to push the limits of neural networks on one hand, and the architecture community proposing their own constraints and data-flows to accelerate neural networks on the other hand, this paper tries to bridge the gap between the two communities by proposing a neural network compiler. The main aim of this paper is to run a given neural network on a given hardware, no matter what the constraints are. They achieve this by modelling the target NN as a computational graph, restructuring it based on the constraints of the target architecture, and fine-tuning the graph to minimise accuracy loss. One of the main conflicts between the NN and the hardware is the precision of inputs. This paper solves this issue by using an autoencoder network that produces the low-precision representation of the inputs. The accuracy loss incurred in translating  to low-precision values can be...

Authors: Min Du, Feifei Li, Guineng Zheng, Vivek Srikumar Venue:    CCS  2017 (Conference on Computer and Communications Security) DeepLog presents a method for parsing system logs automatically using natural language processing techniques, and an LSTM. The paper makes a few assumptions, such as a fixed set of log entry types, called keys. DeepLog constructs workflows from the underlying system log so that once an anomaly is detected, root cause analysis can be performed. Need to do a deeper dive on this. Full Text

Authors: Patrick Judd, Jorge Albericio, Tayler Hetherington, Tor M. Aamodt, Andreas Moshovos Venue:    MICRO 2016 Bit-Stripes presents are architecture which is able to scale almost linearly with the bit-precision width for neural networks. They do a design space exploration to find the minimum number of bits required for different networks to maintain within 1% accuracy of the original network. The paper takes a DaDianNao-like approach in terms of hardware, but thanks to the per-layer precision optimizations consumes less energy, and operates faster. Full Text