返回到首页 | 返回到 Paper Summary


✏️ A Paper A Day

Felt like I wasn’t reading enough – and what I was reading wasn’t sinking in enough. I also wanted to keep track of my sources in a more controlled manner. As a part of adding everything to my JabRef (maybe…), I figured I would write up my comments on papers.

The goal is to read and comment once a day and this post will be updated day by day according to the reading process.

✏️ A Paper A Day🔁 Generative Models👊 Adversarial Examples/Attacks🎵 Sound & Signal Processing📍 Anomaly DetectionUnsupervised / Domain Adaptation📉 Optimization & Generalization👍 Model Evaluation & Performance & Interpretion & Visualization🎛 Model Configuration〽️ ODE & PDE⚛️ Physics Related📚 Review, Survey & Lecture Notes🖼 Figure Design & Dimension Reduction

🔁 Generative Models

Image Generation from Small Datasets via Batch Statistics Adaptation. A Noguchi, T Harada [The University of Tokyo] (2019) arXiv:1904.01774

A Three-Player GAN: Generating Hard Samples To Improve Classification Networks. S Vandenhende, B D Brabandere, D Neven, L V Gool [KU Leuven] (2019) arXiv:1903.03496

O-GAN: Extremely Concise Approach for Auto-Encoding Generative Adversarial Networks. Jianlin Su [Sun Yat-sen University] (2019) arXiv:1903.01931 Reddit PaperWeekly

AVP: Physics-informed Data Generation for Small-data Learning. J Chen, Y Xie, K Wang, C Zhang, M A. Vannan, B Wang, Z Qian [Georgia Institute of Technology] (2019) arXiv:1902.01522

A Layer-Based Sequential Framework for Scene Generation with GANs. M O Turkoglu, W Thong, L Spreeuwers, B Kicanaoglu [University of Twente & University of Amsterdam] (2019) arXiv:1902.00671 Github

Perturbative GAN: GAN with Perturbation Layers. Y Kishi, T Ikegami, S O'uchi, R Takano, W Nogami, T Kudoh [National Institute of Advanced Industrial Science and Technology, Tsukuba, Japan & The University of Tokyo] (2019) arXiv:1902.01514

HyperGAN: A Generative Model for Diverse, Performant Neural Networks. N Ratzlaff, L Fuxin [Oregon State University] (2019) arXiv:1901.11058

Lie Group Auto-Encoder. L Gong, Q Cheng [University of Kentucky] (2019) arXiv:1901.09970

Maximum Entropy Generators for Energy-Based Models. Rithesh Kumar, Anirudh Goyal, Aaron Courville, Yoshua Bengio [Mila & CIFAR & IVADO] (2019) arXiv:1901.08508 Reddit PaperWeekly PaperWeekly

MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks. D Li, D Chen, L Shi, B Jin, J Goh, S Ng [National University of Singapore & UC Berkeley & ST Electronics (Info Security) Pte Ltd] (2019) arXiv:1901.04997

On Finding Local Nash Equilibria (and Only Local Nash Equilibria) in Zero-Sum Games. E V. Mazumdar, M I. Jordan, S. S Sastry [UC Berkeley] (2019) arXiv:1901.00838

Evaluating Generative Adversarial Networks on Explicitly Parameterized Distributions. S O'Brien, M Groh, A Dubey [MIT] (2018) arXiv:1812.10782 Github

InstaGAN: Instance-aware Image-to-Image Translation. S Mo, M Cho, J Shin [Korea Advanced Institute of Science and Technology (KAIST) & Pohang University of Science and Technology (POSTECH)] (2018) arXiv:1812.10889 OpenReview.net Github 机器之心

Improving Generalization and Stability of Generative Adversarial Networks. H Thanh-Tung, T Tran, S Venkatesh [Deakin University] (ICLR 2018) OpenReview.net Github

Finger-GAN: Generating Realistic Fingerprint Images Using Connectivity Imposed GAN. S Minaee, A Abdolrashidi [New York University & University of California, Riverside] (2018) arXiv:1812.10482

Generative Models from the perspective of Continual Learning. T Lesort, H Caselles-Dupré, M Garcia-Ortiz, A Stoian, D Filliat [Flowers Team (ENSTA ParisTech & INRIA)] (2018) arXiv:1812.09111 Github OpenReview.net

A Probe into Understanding GAN and VAE models. J Zhang, L Mi, M Shen [MIT] (2018) arXiv:1812.05676

A Style-Based Generator Architecture for Generative Adversarial Networks. T Karras, S Laine, T Aila [NVIDIA] (2018) arXiv:1812.04948 Code YouTube 机器之心

Intra-class Variation Isolation in Conditional GANs. R T. Marriott, S Romdhani, L Chen [Ecole Centrale de Lyon & IDEMIA] (2018) arXiv:1811.11296

Metropolis-Hastings Generative Adversarial Networks. R Turner, J Hung, Y Saatci, J Yosinski [Uber AI Labs] (2018) arXiv:1811.11357 Github Blog

Label-Noise Robust Generative Adversarial Networks. Takuhiro Kaneko, Yoshitaka Ushiku, Tatsuya Harada [The University of Tokyo & RIKEN] (2018) arXiv:1811.11165

Do GAN Loss Functions Really Matter?. Y Qin, N Mitra, P Wonka [KAUST & UCL] (2018) arXiv:1811.09567 Reddit

Copy the Old or Paint Anew? An Adversarial Framework for (non-) Parametric Image Stylization. N Jetchev, U Bergmann, G Yildirim [Zalando Research] (2018) arXiv:1811.09236 GitHub

Guiding the One-to-one Mapping in CycleGAN via Optimal Transport. G Lu, Z Zhou, Y Song, K Ren, Y Yu [Shanghai Jiao Tong University] (2018) arXiv:1811.06284

NEMGAN: Noise Engineered Mode-matching GAN. D Mishra, P AP, A J, P Pandey, S Chaudhury [Indian Institute of Technology Delhi] (2018) arXiv:1811.03692 GitHub

Bias and Generalization in Deep Generative Models: An Empirical Study. S Zhao, H Ren, A Yuan, J Song, N Goodman, S Ermon [Stanford University] (ICML2018) arXiv:1811.03259 GitHub

Language GANs Falling Short. Massimo Caccia, Lucas Caccia, William Fedus, Hugo Larochelle, Joelle Pineau, Laurent Charlin [MILA, Université de Montréal & MILA, McGill University & MILA, HEC Montréal & Google Brain & Facebook AI Research] (2018) arXiv:1811.02549

CariGANs: Unpaired Photo-to-Caricature Translation. K Cao, J Liao, L Yuan [Tsinghua University & Microsoft Research] (2018) arXiv:1811.00222 Blog App YouTube

Large Scale GAN Training for High Fidelity Natural Image Synthesis | OpenReview. (2018) More examples BigGAN Demo(Colab Notebook)

Generative adversarial networks and adversarial methods in biomedical image analysis. J M. Wolterink, K Kamnitsas, C Ledig, I Išgum [University Medical Center Utrecht & Imperial College London & Imagen Technologies] (2018) arXiv:1810.10352

Do Deep Generative Models Know What They Don't Know?. E Nalisnick, A Matsukawa, Y W Teh, D Gorur, B Lakshminarayanan [DeepMind] (2018) arXiv:1810.09136

Discriminator Rejection Sampling. Samaneh Azadi, Catherine Olsson, Trevor Darrell, Ian Goodfellow, Augustus Odena [UC Berkeley & Google Brain]. arXiv:1810.06758

Refacing: reconstructing anonymized facial features using GANs. D Abramian, A Eklund [Linkoping University] (2018) arXiv:1810.06455

ClusterGAN : Latent Space Clustering in Generative Adversarial Networks. S Mukherjee, H Asnani, E Lin, S Kannan [University of Washington] (2018) arXiv:1809.03627

Whispered-to-voiced Alaryngeal Speech Conversion with Generative Adversarial Networks. Santiago Pascual, Antonio Bonafonte, Joan Serrà, Jose A. Gonzalez [Universitat Polite`cnica de Catalunya & Telefo ́nica Research & Universidad de Ma ́laga, Spain] (2018) arXiv:1808.10687

The relativistic discriminator: a key element missing from standard GAN. Alexia Jolicoeur-Martineau [Lady Davis Institute Montreal, Canada] (2018) arXiv:1807.00734

Exploring Disentangled Feature Representation Beyond Face Identification. Yu Liu, Fangyin Wei, Jing Shao, Lu Sheng, Junjie Yan, Xiaogang Wang [The Chinese University of Hong Kong, SenseTime Group Limited, Peking University] (2018) arXiv:1804.03487

Evolutionary Generative Adversarial Networks. Chaoyue Wang, Chang Xu, Xin Yao, Dacheng Tao [2018] arXiv:1803.00657 PaperWeekly

Do GANs learn the distribution? Some Theory and Empirics. Sanjeev Arora, Andrej Risteski, Yi Zhang [Princeton University & MIT] (ICLR 2018) OpenReview.net

Do GANs actually learn the distribution? An empirical study. Sanjeev Arora, Yi Zhang [] (2017) arXiv:1706.08224 Reddit Blog

Generalization and Equilibrium in Generative Adversarial Nets (GANs). S Arora, R Ge, Y Liang, T Ma, Y Zhang [Princeton University & Duke University] (2017) arXiv:1703.00573 Blog Reddit

InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets. X Chen, Y Duan, R Houthooft, J Schulman, I Sutskever, P Abbeel [UC Berkeley & OpenAI] (2016) arXiv:1606.03657 Reddit

Improved Techniques for Training GANs. Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen [OpenAI] (2016) arXiv:1606.03498 Github Reddit Github PyTorch


👊 Adversarial Examples/Attacks

GanDef: A GAN based Adversarial Training Defense for Neural Network Classifier. G Liu, I Khalil, A Khreishah [New Jersey Institute of Technology & Qatar Computing Research Institute] (2019) https://arxiv.org/abs/1903.02585

Adversarial Attacks on Time Series. Fazle Karim, Somshubra Majumdar, Houshang Darabi […] (2019) arXiv:1902.10755

On Evaluating Adversarial Robustness. N Carlini, A Athalye, N Papernot, W Brendel, J Rauber, D Tsipras, I Goodfellow, A Madry [Google Brain & MIT & University of Tübingen] (2019) arXiv:1902.06705 Github Reddit

Adversarial Examples Are a Natural Consequence of Test Error in Noise. Nic Ford, Justin Gilmer, Nicolas Carlini, Dogus Cubuk [Google AI Residency] (2019) arXiv:1901.10513 Reddit

Towards a Deeper Understanding of Adversarial Losses. H Dong, Y Yang [Academia Sinica] (2019) arXiv:1901.08753 Github Blog

Image Transformation can make Neural Networks more robust against Adversarial Examples. D D Thang, T Matsui [Institute of Information Security] (2019) arXiv:1901.03037

Multi-Label Adversarial Perturbations. Q Song, H Jin, X Huang, X Hu [Texas A&M University] (2019) arXiv:1901.00546

Adversarial Transfer Learning. G Wilson, D J. Cook [Washington State University] (2018) arXiv:1812.02849

Defensive Dropout for Hardening Deep Neural Networks under Adversarial Attacks. S Wang, X Wang, P Zhao, W Wen, D Kaeli, P Chin, X Lin [Northeastern University & Boston university & Florida International University] (2018) arXiv:1809.05165

Are adversarial examples inevitable?. A Shafahi, W. R Huang, C Studer, S Feizi, T Goldstein (2018) arXiv:1809.02104

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples. Anish Athalye, Nicholas Carlini, David Wagner [MIT & Berkeley] (2018) arXiv:1802.00420 Github

Generating Natural Adversarial Examples. Z Zhao, D Dua, S Singh [University of California, Irvine] (2017) arXiv:1710.11342 Github [comment]



🎵 Sound & Signal Processing

VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Masking. Q Wang, H Muckenhirn, K Wilson, P Sridhar, Z Wu, J Hershey, R A. Saurous, R J. Weiss, Y Jia, I L Moreno [Google & Idiap Research Institute] (2018) arXiv:2018:04826 Home VentureBeat Tproger 机器之心 新智元

Phase-aware Speech Enhancement with Deep Complex U-Net. H Choi, J Kim, J Huh, A Kim, J Ha, K Lee [Seoul National University & NAVER Corp] (2019) arXiv:1903.03107 Home OpenReview.net

A Deep Generative Model of Speech Complex Spectrograms. A A Nugraha, K Sekiguchi, K Yoshii [RIKEN Center for Advanced Intelligence Project (AIP) & Kyoto University] (2019) arXiv:1903.03269

Utterance-level Aggregation For Speaker Recognition In The Wild. W Xie, A Nagrani, J S Chung, A Zisserman [University of Oxford] (2019) arXiv:1902.10107 Blog

catch22: CAnonical Time-series CHaracteristics. C H Lubba, S S Sethi, P Knaute, S R Schultz, B D Fulcher, N S Jones [Imperial College London] (2019) arXiv:1901.10200 Github

End-to-End Probabilistic Inference for Nonstationary Audio Analysis. W J. Wilkinson, M R Andersen, J D. Reiss, D Stowell, A Solin [Queen Mary University of London & Aalto University] (2019) arXiv:1901.11436

Unsupervised speech representation learning using WaveNet autoencoders. J Chorowski, R J. Weiss, S Bengio, A v d Oord [University of Wrocław & Google Research & DeepMind] (2019) arXiv:1901.08810

Kymatio: Scattering Transforms in Python. M Andreux, T Angles, G Exarchakis... [PSL Research University & Universite de Montreal & New York University] (2018) arXiv:1812.11214 Home

Deep Neural Networks for Automatic Classification of Anesthetic-Induced Unconsciousness. Konstantinos Patlatzoglou, etc. [etc.] (2018) PDF

Using Convolutional Neural Networks to Classify Audio Signal in Noisy Sound Scenes. M.V. Gubin [South Ural State University] (2018 GloSIC) PDF Github

Interpretable Convolutional Filters with SincNet. M Ravanelli, Y Bengio [Université de Montréal] (2018) arXiv:1811.09725 GitHub

A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data. C Zhang, D Song, Y Chen, X Feng, C Lumezanu, W Cheng, J Ni, B Zong, H Chen, N V. Chawla [University of Notre Dame & NEC Laboratories America & Columbia University] (2018) arXiv:1811.08055

Stochastic Adaptive Neural Architecture Search for Keyword Spotting. T Véniat, O Schwander, L Denoyer [Sorbonne Université & Facebook AI Research] (2018) arXiv:1811.06753 GitHub

Unifying Probabilistic Models for Time-Frequency Analysis. W J. Wilkinson, M R Andersen, J D. Reiss, D Stowell, A Solin [Queen Mary University of London & Aalto University] (2018) arXiv:1811.02489 GitHub

WaveGlow: A Flow-based Generative Network for Speech Synthesis. Ryan Prenger, Rafael Valle, Bryan Catanzaro [NVIDIA Corporation] (2018) arXiv:1811.00002 Github

Training neural audio classifiers with few data. J Pons, J Serrà, X Serra [Telefonica Research & Universitat Pompeu Fabra] (2018) arXiv:1810.10274 Github

End-to-end music source separation: is it possible in the waveform domain?. F Lluís, J Pons, X Serra [Universitat Pompeu Fabra] (2018) arXiv:1810.12187

Multilevel Wavelet Decomposition Network for Interpretable Time Series Analysis. Jingyuan Wang, Ze Wang, Jianfeng Li, Junjie Wu.[Beihang University] (2018) arXiv:1806.08946

Deep Convolutional Neural Networks On Multichannel Time Series For Human Activity Recognition. Jian Bo Yang, Minh Nhut Nguyen, Phyo Phyo San, Xiao Li Li, Shonali Krishnaswamy (2015) IJCAI2015

Towards a universal neural network encoder for time series. J Serrà, S Pascual, A Karatzoglou [Telefonica Research & Universitat Politecnica de Catalunya] (2018)

Sound Event Detection Using Spatial Features and Convolutional Recurrent Neural Network. Sharath Adavanne, Pasi Pertilä, Tuomas Virtanen [Tampere University of Technology] (DCASE 2017) arXiv:1706.02291 Github

Time Series Classification Using Multi-Channels Deep Convolutional Neural Networks. Yi Zheng, Qi Liu, Enhong Chen, Yong Ge, and J. Leon Zhao [USTC, et al.] (2014) WAIM2014


📍 Anomaly Detection

Deep CNN-based Multi-task Learning for Open-Set Recognition. P Oza, V M. Patel [Johns Hopkins University] (2019) arXiv:1903.03161 Github

Learning Confidence Sets using Support Vector Machines. W Wang, X Qiao [Binghamton University] (2018) arXiv:1809.10818

Novelty Detection with GAN, Mark Kliger, Shachar Fleishman [Amazon] arXiv:1802.10560

Learning Confidence for Out-of-Distribution Detection in Neural Networks. T DeVries, G W. Taylor [University of Guelph & Vector Institute] (2018) arXiv:1802.04865

Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples, Kimin Lee, Honglak Lee, Kibok Lee, Jinwoo Shin [Korea Advanced Institute of Science and Technology, University of Michigan, Google Brain] (ICLR 2018) arXiv:1711.09325


Unsupervised / Domain Adaptation

Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation. C Lee, T Batra, M H Baig, D Ulbricht [Apple Inc] (2019) arXiv:1903.04064




📉 Optimization & Generalization

Transfusion: Understanding Transfer Learning with Applications to Medical Imaging. M Raghu, C Zhang, J Kleinberg, S Bengio [Cornell University & Google Brain] (2019) arXiv:1902.07208

Decoupled Greedy Learning of CNNs. Eugene Belilovsky, Michael Eickenberg, Edouard Oyallon [University of Montreal , University of California Berkeley, CentraleSupelec and INRIA] (2019) arXiv:1901.08164 Github

Eliminating all bad Local Minima from Loss Landscapes without even adding an Extra Unit. J Sohl-Dickstein, K Kawaguchi [Google & MIT] (2019) arXiv:1901.03909

The Benefits of Over-parameterization at Initialization in Deep ReLU Networks. D Arpit, Y Bengio [Montreal Institute for Learning Algorithms] (2019) arXiv:1901.03611

Generalization in Deep Networks: The Role of Distance from Initialization. Vaishnavh Nagarajan, J. Zico Kolter [Carnegie-Mellon University] (NeurlPS 2017, 2019) arXiv:1901.01672

Elimination of All Bad Local Minima in Deep Learning. K Kawaguchi, L P Kaelbling [MIT] (2019) arXiv:1901.00279

Towards Understanding the Role of Over-Parametrization in Generalization of Neural Networks. B Neyshabur, Z Li... [Princeton & Toyota Technological Institute at Chicago & Facebook AI Research] (2018) arXiv:1805.12076 Github Reddit

Visualizing the Loss Landscape of Neural Nets. H Li, Z Xu, G Taylor, T Goldstein [University of Maryland & United States Naval Academy] (NIPS 2018) arXiv:1712.09913 Github Reddit

Gradient Descent Happens in a Tiny Subspace. G Gur-Ari, D A. Roberts, E Dyer [Institute for Advanced Study & Facebook AI Research & Johns Hopkins University] (2018) arXiv:1812.04754

A Sufficient Condition for Convergences of Adam and RMSProp. F Zou, L Shen, Z Jie, W Zhang, W Liu [Stony Brook University & Tencent AI Lab] (2018) arXiv:1811.09358

Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks. D Zou, Y Cao, D Zhou, Q Gu [University of California, Los Angeles] (2018) arXiv:1811.08888

Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers. Z Allen-Zhu, Y Li, Y Liang [Microsoft Research AI & Stanford University & University of Wisconsin-Madison] (2018) arXiv:1811.04918

A Convergence Theory for Deep Learning via Over-Parameterization. Z Allen-Zhu, Y Li, Z Song [Microsoft Research AI & Stanford University & UT-Austin] (2018) arXiv:1811.03962

Gradient Descent Finds Global Minima of Deep Neural Networks. S S. Du, J D. Lee, H Li, L Wang, X Zhai [CMU & University of Southern California & Peking University & MIT] (2018) arXiv:1811.03804

Identifying Generalization Properties in Neural Networks. H Wang, N S Keskar, C Xiong, R Socher [Salesforce Research] (2018) arXiv:1809.07402

Accelerating Natural Gradient with Higher-Order Invariance. by Yang Song Post paper


👍 Model Evaluation & Performance & Interpretion & Visualization

Unmasking Clever Hans Predictors and Assessing What Machines Really Learn. S Lapuschkin, S Wäldchen, A Binder, G Montavon, W Samek, K Müller [Fraunhofer Heinrich Hertz Institute & Technische Universitat Berlin & Singapore University of Technology and Design] (2019) arXiv:1902.10178

Seven Myths in Machine Learning Research. O Chang, H Lipson [Columbia University] (2019) arXiv:1902.06789 Blog

Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent. J Lee, L Xiao, S S. Schoenholz, Y Bahri, J Sohl-Dickstein, J Pennington [Google Brain] (2019) arXiv:1902.06720 Reddit

Impact of Fully Connected Layers on Performance of Convolutional Neural Networks for Image Classification. S H S Basha, S R Dubey, V Pulabaigari, S Mukherjee [Indian Institute of Information Technology Sri City] arXiv:1902.02771 Github

CHIP: Channel-wise Disentangled Interpretation of Deep Convolutional Neural Networks. X Cui, D Wang, Z. J Wang [University of British Columbia] (2019) arXiv:1902.02497

Are All Layers Created Equal?. C Zhang, S Bengio, Y Singer [Google] (2019) arXiv:1902.01996 新智元

Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet. W Brendel, M Bethge [Eberhard Karls University of Tübingen] (2019) OpenReview.net Reddit Blogcomment Blogcomment 过往Net,皆为调参?一篇BagNet论文引发学界震动

See Better Before Looking Closer: Weakly Supervised Data Augmentation Network for Fine-Grained Visual Classification. T Hu, H Qi [University of Chinese Academy of Sciences] (2019) arXiv:1901.09891

Deep Learning on Small Datasets without Pre-Training using Cosine Loss. B Barz, J Denzler [Friedrich Schiller University Jena] (2019) arXiv:1901.09054 Github

Using Pre-Training Can Improve Model Robustness and Uncertainty. D Hendrycks, K Lee, M Mazeika (2019) arXiv:1901.09960

Understanding Geometry of Encoder-Decoder CNNs. J C Ye, W K Sung [KAIST] (2019) arXiv:1901.07647

Attribute-Aware Attention Model for Fine-grained Representation Learning. K Han, J Guo, C Zhang, M Zhu [Peking University] (2019) arXiv:1901.00392

Multi-class Classification without Multi-class Labels. Y Hsu, Z Lv, J Schlosser, P Odom, Z Kira [Georgia Institute of Technology & Georgia Tech Research Institute] (ICLR 2019) arXiv:1901.00544

Are All Training Examples Created Equal? An Empirical Study. K Vodrahalli, K Li, J Malik [UC Berkeley] (2018) arXiv:1811.12569 知乎

Rethinking ImageNet Pre-training. K He, R Girshick, P Dollár [Facebook AI Research (FAIR)] (2018) arXiv:1811.08883

Efficient Identification of Approximate Best Configuration of Training in Large Datasets. S Huang, C Wang, B Ding, S Chaudhuri [University of Illinois & Microsoft Research & Alibaba Group] (2018) arXiv:1811.03250

Explaining Deep Learning Models - A Bayesian Non-parametric Approach. W Guo, S Huang, Y Tao, X Xing, L Lin [The Pennsylvania State University & Netflix Inc & Columbia University] (2018) arXiv:1811.03422

How deep is deep enough? - Optimizing deep neural network architecture. A Schilling, J Rietsch, R Gerum, H Schulze, C Metzner, P Krauss [University Hospital Erlangen & Friedrich-Alexander University Erlangen-N¨urnberg (FAU)] (2018) arXiv:1811.01753

Approximate Fisher Information Matrix to Characterise the Training of Deep Neural Networks. Z Liao, T Drummond, I Reid, G Carneiro [University of Adelaide & Monash University] (2018) arXiv:1810.06767 GitHub

A Performance Evaluation of Convolutional Neural Networks for Face Anti Spoofing. Chaitanya Nagpal, Shiv Ram Dubey (2018) arXiv:1805.04176

An Information-Theoretic View for Deep Learning. J Zhang, T Liu, D Tao [UBTECH Sydney AI Centre] (2018) arXiv:1804.09060

Understanding Individual Neuron Importance Using Information Theory. K Liu, R A Amjad, B C. Geiger [Technical University of Munich & Graz University of Technology] (2018) arXiv:1804.06679

Understanding Convolutional Neural Network Training with Information Theory. S Yu, R Jenssen, J C. Principe [University of Florida & University of Tromsø] (2018) arXiv:1804.06537

A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay. Leslie N. Smith. arXiv:1803.09820

Focal Loss for Dense Object Detection. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár [Facebook AI Research] (2017) arXiv:1708.02002 Github

How transferable are features in deep neural networks?. J Yosinski, J Clune, Y Bengio, H Lipson [Cornell University, University of Wyoming, University of Montreal] (2014) (NIPS 2014)



🎛 Model Configuration

Ising-Dropout: A Regularization Method for Training and Compression of Deep Neural Networks. H Salehinejad, S Valaee [University of Toronto] (2019) arXiv:1902.08673

LocalNorm: Robust Image Classification through Dynamically Regularized Normalization. B Yin, S Schaafsma, H Corporaal, H. S Scholte, S M. Bohte [Centrum Wiskunde & Informatica (CWI) & Holst Centre / IMEC] (2019) arXiv:1902.06550

On the Impact of the Activation Function on Deep Neural Networks Training. S Hayou, A Doucet, J Rousseau [Universiy of Oxford] (2019) arXiv:1902.06853 机器之心

TUNet: Incorporating segmentation maps to improve classification. Y Tian [New York University] (2019) arXiv:1901.11379

Weighted Channel Dropout for Regularization of Deep Convolutional Neural Network. Saihui Hou, Zilei Wang [USTC] (AAAI 2019) Paper 机器之心

Fixup Initialization: Residual Learning Without Normalization. H Zhang, Y N. Dauphin, T Ma [MIT & Google Brain & Stanford University] (2019) arXiv:1901.09321 Reddit Github

Training Neural Networks with Local Error Signals. A Nøkland, L H Eidnes [Trondheim] (2019) arXiv:1901.06656 GitHub

Flow Based Self-supervised Pixel Embedding for Image Segmentation. B Ma, S Liu, Y Zhi, Q Song [CuraCloud] (2019) arXiv:1901.00520

Bag of Tricks for Image Classification with Convolutional Neural Networks. Tong He, Zhi Zhang, Hang Zhang, Zhongyue Zhang, Junyuan Xie, Mu Li [AWS] (2018) arXiv:1812.01187 Reddit Slides[Reddit]

Linear Backprop in non-linear networks. Mehrdad Yazdani [University of California San Diego] (NIPS 2018) OpenReview 机器之心

Seeing in the dark with recurrent convolutional neural networks. T S. Hartmann [Harvard Medical School] (2018) arXiv:1811.08537

Dataset Distillation. T Wang, J Zhu, A Torralba, A A. Efros [Facebook AI Research & MIT & UC Berkeley] (2018) arXiv:1811.10959

Retina U-Net: Embarrassingly Simple Exploitation of Segmentation Supervision for Medical Object Detection. P F. Jaeger, S A. A. Kohl, S Bickelhaupt, F Isensee, T A Kuder, H Schlemmer, K H. Maier-Hein [German Cancer Research Center] (2018) arXiv:1811.08661 GitHub

Rethinking floating point for deep learning. Jeff Johnson [Facebook AI Research] (2018) arXiv:1811.01721 Github Blog

Quaternion Convolutional Neural Networks. Xuanyu Zhu / Yi Xu / Hongteng Xu / Changjian Chen. [Shanghai Jiao Tong University & Duke University] (ECCV2018) (pdf)

Why scatter plots suggest causality, and what we can do about it. C T. Bergstrom, J D. West [University of Washington] (2018) arXiv:1809.09328

Human activity recognition based on time series analysis using U-Net. Y Zhang, Y Zhang, Z Zhang, J Bao, Y Song [Beijing University of Posts and Telecommunications & AIdong Super AI] (2018). arXiv:1809.08113

Backprop Evolution. M Alber, I Bello, B Zoph, P Kindermans, P Ramachandran, Q Le [TU Berlin & Google Brain] (2018) arXiv:1808.01974

Smooth Loss Functions for Deep Top-k Classification. L Berrada, A Zisserman, M. P Kumar [University of Oxford] (2018) arXiv:1802.07595 Github


〽️ ODE & PDE

Augmented Neural ODEs. E Dupont, A Doucet, Y W Teh [University of Oxford] (2019) arXiv:1904.01681

Data Driven Governing Equations Approximation Using Deep Neural Networks. T Qin, K Wu, D Xiu [The Ohio State University] (2018) arXiv:1811.05537

Neural Ordinary Differential Equations. Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, David Duvenaud [University of Toronto, Canada] (NeurIPS 2018 | best paper) arXiv:1806.07366 Github Blog 机器之心(【硬核NeruIPS 2018最佳论文,一个神经了的常微分方程】这是一篇神奇的论文,以前一层一层叠加的神经网络似乎突然变得连续了,反向传播也似乎不再需要一点一点往前传、一层一层更新参数了。 )

【《Neural Ordinary Differential Equations》论文解读】《Paper Summary: Neural Ordinary Differential Equations》by Branislav Holländer http://t.cn/EGPEh8y

summary by Adrian Colyer http://t.cn/EqANCZ0

【神经常微分方程的PyTorch实现与分析(Jupyter Notebooks)】’Neural ODEs - Jupyter notebook with Pytorch implementation and investigation of Neural Ordinary Differential Equations' by Mikhail Surtsukov GitHub: http://t.cn/Ef3Qkw4

【神经常微分方程与对抗攻击】《Neural Ordinary Differential Equations and Adversarial Attacks》by Rajat Vadiraj Dwaraknath http://t.cn/EqW9Anb

【神经微分方程】《Neural Differential Equations - YouTube》by Siraj Raval http://t.cn/EqWCSBN

Forward-Backward Stochastic Neural Networks: Deep Learning of High-dimensional Partial Differential Equations. M Raissi [Brown University] (2018) arXiv:1804.07010




⚛️ Physics Related

Deep learning approach based on dimensionality reduction for designing electromagnetic nanostructures. Yashar Kiarashinejad, Sajjad Abdollahramezani, Ali Adibi [Georgia Institute of Technology] (2019) arXiv:1902.03865

The Calabi-Yau Landscape: from Geometry, to Physics, to Machine-Learning. Y He (2018) arXiv:1812.02893

DeepSphere: Efficient spherical Convolutional Neural Network with HEALPix sampling for cosmological applications. N Perraudin, M Defferrard, T Kacprzak, R Sgier [aSwiss Data Science Center (SDSC) & EPFL & ETH Zurich] (2018) arXiv:1810.12186

Toward an AI Physicist for Unsupervised Learning. T Wu, M Tegmark [MIT] (2018) arXiv:1810.10525

Using Machine Learning to Predict the Evolution of Physics Research. W Liu, S Saganowski, P Kazienko, S A Cheong [Nanyang Technological University & Wrocław University of Science and Technology] (2018) arXiv:1810.12116

hep-th. Y He, V Jejjala, B D. Nelson [University of London & Nankai University & Northeastern University] (2018) arXiv:1807.00735 [comment]

Deep Fluids: A Generative Network for Parameterized Fluid Simulations. B Kim, VC Azevedo, N Thuerey, T Kim, M Gross… [ETH Zurich & Technical University of Munich & Pixar Animation Studios] (2019) arXiv:1806.02071 Blog

Physics-guided Neural Networks (PGNNs). Anuj Karpatne, William Watkins, Jordan Read, Vipin Kumar [University of Minnesota] (2017) arXiv:1710.11431


📚 Review, Survey & Lecture Notes

A State-of-the-Art Survey on Deep Learning Theory and Architectures. PDF

Gradient Descent based Optimization Algorithms for Deep Learning Models Training. J Zhang [Information Fusion and Mining Laboratory] (2019) arXiv:1903.03614

Deep Learning for Image Super-resolution: A Survey. Z Wang, J Chen, S C.H. Hoi [Singapore Management University & South China University of Technology] (2019) arXiv:1902.06068 专知

Word Embeddings: A Survey. F Almeida, G Xexéo [Federal University of Rio de Janeiro] (2019) arXiv:1901.09069

Fitting A Mixture Distribution to Data: Tutorial. B Ghojogh, A Ghojogh, M Crowley, F Karray [University of Waterloo & Shiraz University of Technology] (2019) arXiv:1901.06708

A Survey of the Recent Architectures of Deep Convolutional Neural Networks. A Khan, A Sohail, U Zahoora, A S Qureshi [PIEAS] (2019) arXiv:1901.06032 机器之心

Artificial Neural Networks. B. Mehlig [University of Gothenburg, Sweden] (2019) arXiv:1901.05639 Reddit

Optimization Models for Machine Learning: A Survey. C Gambella, B Ghaddar, J Naoum-Sawaya [IBM Research Ireland & University of Western Ontario] (2019) arXiv:1901.05331

Deep Learning for Anomaly Detection: A Survey. R Chalapathy, S Chawla [University of Sydney & Qatar Computing Research Institute (QCRI)] (2019) arXiv:1901.03407

Revisiting Self-Supervised Visual Representation Learning. Alexander Kolesnikov, Xiaohua Zhai, Lucas Beyer [Google Brain] (2019) arXiv:1901.09005 Github Reddit

How Generative Adversarial Networks and Their Variants Work: An Overview. Yongjun Hong, Uiwon Hwang, Jaeyoon Yoo, Sungroh Yoon [Seoul National University] (2017, 2018v9) arXiv:1711.05914

A Survey on Multi-output Learning. D Xu, Y Shi, I W. Tsang, Y Ong, C Gong, X Shen [ University of Technology Sydney & Nanyang Technological University] (2019) arXiv:1901.00248

Analysis Methods in Neural Language Processing: A Survey. Y Belinkov, J Glass [MIT] (2018) arXiv:1812.08951 Github Blog

Neural Approaches to Conversational AI. J Gao, M Galley, L Li [Microsoft Research & Google Brain] (2018) arXiv:1809.08267

Recent Advances in Autoencoder-Based Representation Learning. M Tschannen, O Bachem, M Lucic [ETH Zurich & Google AI] (2018) arXiv:1812.05069

Learning From Positive and Unlabeled Data: A Survey. J Bekker, J Davis [KU Leuven] (2018) arXiv:1811.04820

Analyzing biological and artificial neural networks: challenges with opportunities for synergy?. David G.T. Barrett, Ari S. Morcos, Jakob H. Macke [DeepMind; Technical University of Munich, Germany] (2018) arXiv:1810.13373

Model Selection Techniques -- An Overview. J Ding, V Tarokh, Y Yang [University of Minnesota & Duke University] (2018) arXiv:1810.09583

Deep Learning with the Random Neural Network and its Applications. Y Yin [Imperial College] (2018) arXiv:1810.08653

The Frontiers of Fairness in Machine Learning. A Chouldechova, A Roth [CMU & University of Pennsylvania] (2018) arXiv:1810.08810

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey. N C Luong, D T Hoang, S Gong, D Niyato, P Wang, Y Liang, D I Kim [Nanyang Technological University & University of Technology Sydney & Chinese Academy of Sciences] (2018) arXiv:1810.07862

A Survey on Deep Learning: Algorithms, Techniques, and Applications. Pouyanfar S, Sadiq S, Yan Y, et al [ACM Computing Surveys (CSUR)] (2018) (pdf) (专知)

A Tale of Three Probabilistic Families: Discriminative, Descriptive and Generative Models. Y N Wu, R Gao, T Han, S Zhu [UCLA] (2018) arXiv:1810.04261

Deep learning for time series classification: a review. H I Fawaz, G Forestier, J Weber, L Idoumghar, P Muller [Université Haute Alsace] (2018) arXiv:1809.04356

A Survey on Deep Transfer Learning. C Tan, F Sun, T Kong, W Zhang, C Yang, C Liu [Tsinghua University] (2018) arXiv:1808.01974

Generalization Error in Deep Learning. D Jakubovitz, R Giryes, M R. D. Rodrigues [Tel Aviv University & University College London] (2018) arXiv:1808.01174

How convolutional neural network see the world - A survey of convolutional neural network visualization methods. Z Qin, F Yu, C Liu, X Chen [George Mason University & Clarkson University] (2018) arXiv:1804.11191

An Introduction to Image Synthesis with Generative Adversarial Nets. He Huang, Philip S. Yu, Changhu Wang [University of Illinois at Chicago & ByteDance AI Lab] (2019) arXiv:1803.04469

Visual Interpretability for Deep Learning: a Survey. Quanshi Zhang, Song-Chun Zhu [] (2018) arXiv:1802.00614

Deep Learning for Time-Series Analysis. John Gamboa [University of Kaiserslautern, Germany] (2017) arXiv:1701.01887

Deep Learning in Neural Networks: An Overview. Ju ̈rgen Schmidhuber [University of Lugano & SUPSI, Switzerland] (2014) arXiv:1404.7828

Curriculum learning. Y Bengio, J Louradour, R Collobert, J Weston [Montreal, Fabert, Princeton] (2009) PDF


🖼 Figure Design & Dimension Reduction

Deep Learning Multidimensional Projections. Mateus Espadoto, Nina S. T. Hirata and Alexandru C. Telea (2019) arXiv:1902.07958

CFUN: Combining Faster R-CNN and U-net Network for Efficient Whole Heart Segmentation. Z Xu, Z Wu, J Feng [Tsinghua University] (2018) arXiv:1812.04914 GitHub

Deep Paper Gestalt. J Huang [Virginia Tech] (2018) arXiv:1812.08775 GitHub YouTube 机器之心

A Tutorial on Distance Metric Learning: Mathematical Foundations, Algorithms and Software. J L Suárez, S García, F Herrera [University of Granada] (2018) arXiv:1812.05944

Improving Generalization for Abstract Reasoning Tasks Using Disentangled Feature Representations. X Steenbrugge, S Leroux, T Verbelen, B Dhoedt [Ghent University] (2018) arXiv:1811.04784

UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. Leland McInnes and John Healy [Tutte Institute for Mathematics and Computing] (2018) arXiv:1802.03426 Github




Need to be reviewed.....

Entropic GANs meet VAEs: A Statistical Approach to Compute Sample Likelihoods in GANs. Y Balaji, H Hassani, R Chellappa, S Feizi [University of Maryland & University of Pennsylvania] (2018) arXiv:1810.04147 [comment]

Analyzing the Noise Robustness of Deep Neural Networks. M Liu, S Liu, H Su, K Cao, J Zhu [Tsinghua University] (2018) arXiv:1810.03913 [comment]

Deep convolutional Gaussian processes. K Blomqvist, S Kaski, M Heinonen [Aalto university] (2018) arXiv:1810.03052 GitHub [comment]

Learning with Random Learning Rates. L Blier, P Wolinski, Y Ollivier [Facebook AI Research & Universite Paris Sud] (2018) arXiv:1810.01322 Github Blog [comment]

Interpreting Adversarial Robustness: A View from Decision Surface in Input Space. F Yu, C Liu, Y Wang, X Chen [George Mason University & Clarkson University & Northeastern University] (2018) arXiv:1810.00144 [comment]

Spurious samples in deep generative models: bug or feature?. B Kégl, M Cherti, A Kazakçı [CNRS/Universite Paris-Saclay & PSL Research University] (2018) arXiv:1810.01876 [comment]

Inhibited Softmax for Uncertainty Estimation in Neural Networks. M Możejko, M Susik, R Karczewski [Sigmoidal] (2018) arXiv:1810.01861 GitHub [comment]

Deep processing of structured data. Ł Maziarka, M Śmieja, A Nowak, J Tabor, Ł Struski, P Spurek [Jagiellonian University] (2018) arXiv:1810.01989 [comment]

Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow. Xue Bin Peng, Angjoo Kanazawa, Sam Toyer, Pieter Abbeel, Sergey Levine [University of California, Berkeley] (2018) arXiv:1810.00821

Taming VAEs. D J Rezende, F Viola [DeepMind] (2018) arXiv:1810.00597 [comment]

Adversarial Attacks and Defences: A Survey. A Chakraborty, M Alam, V Dey, A Chattopadhyay, D Mukhopadhyay [Indian Institute of Technology & The Ohio State University & Nanyang Technological University] (2018) arXiv:1810.00069 [comment]

Over-Optimization of Academic Publishing Metrics: Observing Goodhart's Law in Action. M Fire, C Guestrin [University of Washington] (2018) arXiv:1809.07841 [comment]

On the loss landscape of a class of deep neural networks with no bad local valleys. Q Nguyen, M C Mukkamala, M Hein [Saarland University & University of Tübingen] (2018) arXiv:1809.10749 [comment]

Conditional WaveGAN. Chae Young Lee, Anoop Toffy, Gue Jun Jung, Woo-Jin Han (2018) GitHub arXiv:1809.10636

An analytic theory of generalization dynamics and transfer learning in deep linear networks. A K. Lampinen, S Ganguli [Stanford University] (2018) arXiv:1809.10374 [comment]

Dropout is a special case of the stochastic delta rule: faster and more accurate deep learning. N Frazier-Logue, S J Hanson [Rutgers University] (2018) arXiv:1808.03578 [comment]

Grassmannian Learning: Embedding Geometry Awareness in Shallow and Deep Learning. J Zhang, G Zhu, R W. H Jr., a K Huang [The University of Hong Kong] (2018) arXiv:1808.02229 [comment]

Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Models. D Su, H Zhang... [IBM Research & University of California, Davis & MIT] (2018) arXiv:1808.01688 GitHub [comment]

Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining. Xia Li, Jianlong Wu, Zhouchen Lin, Hong Liu, Hongbin Zha. [Peking University] (2018) arXiv:1807.05698 GitHub [comment]

Toward Convolutional Blind Denoising of Real Photographs. S Guo, Z Yan, K Zhang, W Zuo, L Zhang [Harbin Institute of Technology & The Hong Kong Polytechnic University] (2018) arXiv:1807.04686 Github [comment]

Seamless Nudity Censorship: an Image-to-Image Translation Approach based on Adversarial Training. MD More, DM Souza, J Wehrmann, RC Barros (2018) ResearchGate [comment]

Classification and Geometry of General Perceptual Manifolds. SY Chung, DD Lee, H Sompolinsky [Harvard University] (2018) PRX [comment]

The GAN Landscape: Losses, Architectures, Regularization, and Normalization. K Kurach, M Lucic, X Zhai, M Michalski, S Gelly [Google Brain] (2018) arXiv:1807.04720 Github [comment]

Troubling Trends in Machine Learning Scholarship. Z C. Lipton, J Steinhardt [Stanford University] (2018) arXiv:1807.03341 [comment]

On the Spectral Bias of Deep Neural Networks. N Rahaman, D Arpit, A Baratin, F Draxler, M Lin, F A. Hamprecht, Y Bengio, A Courville [Heidelberg University & MILA] (2018) arXiv:1806.08734 [comment]

Opening the black box of deep learning. D Lei, X Chen, J Zhao [Shanghai University] (2018) arXiv:1805.08355 [comment]

Foundations of Sequence-to-Sequence Modeling for Time Series. V Kuznetsov, Z Mariet [Google Research & MIT] (2018) arXiv:1805.03714




返回到首页 | 返回到顶部

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.