In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. In. Which algorithmic choices matter at which batch sizes? Fast convergence of natural gradient descent for overparameterized neural networks. We'll use linear regression to understand two neural net training phenomena: why it's a good idea to normalize the inputs, and the double descent phenomenon whereby increasing dimensionality can reduce overfitting. Understanding Black-box Predictions via Influence Functions If you have questions, please contact Pang Wei Koh (pangwei@cs.stanford.edu). Loss , . Data poisoning attacks on factorization-based collaborative filtering. Time permitting, we'll also consider the limit of infinite depth. Understanding black-box predictions via influence functions. Koh, Pang Wei. All information about attending virtual lectures, tutorials, and office hours will be sent to enrolled students through Quercus. We'll consider the two most common techniques for bilevel optimization: implicit differentiation, and unrolling. M. MacKay, P. Vicol, J. Lorraine, D. Duvenaud, and R. Grosse. Visual interpretability for deep learning: a survey | SpringerLink ICML 2017 best paperStanfordPang Wei KohCourseraStanfordNIPS 2019influence functionPercy Liang11Michael Jordan, , \hat{\theta}_{\epsilon, z} \stackrel{\text { def }}{=} \arg \min _{\theta \in \Theta} \frac{1}{n} \sum_{i=1}^{n} L\left(z_{i}, \theta\right)+\epsilon L(z, \theta), \left.\mathcal{I}_{\text {up, params }}(z) \stackrel{\text { def }}{=} \frac{d \hat{\theta}_{\epsilon, z}}{d \epsilon}\right|_{\epsilon=0}=-H_{\tilde{\theta}}^{-1} \nabla_{\theta} L(z, \hat{\theta}), , loss, \begin{aligned} \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right) &\left.\stackrel{\text { def }}{=} \frac{d L\left(z_{\text {test }}, \hat{\theta}_{\epsilon, z}\right)}{d \epsilon}\right|_{\epsilon=0} \\ &=\left.\nabla_{\theta} L\left(z_{\text {test }}, \hat{\theta}\right)^{\top} \frac{d \hat{\theta}_{\epsilon, z}}{d \epsilon}\right|_{\epsilon=0} \\ &=-\nabla_{\theta} L\left(z_{\text {test }}, \hat{\theta}\right)^{\top} H_{\hat{\theta}}^{-1} \nabla_{\theta} L(z, \hat{\theta}) \end{aligned}, \varepsilon=-1/n , z=(x,y) \\ z_{\delta} \stackrel{\text { def }}{=}(x+\delta, y), \hat{\theta}_{\epsilon, z_{\delta},-z} \stackrel{\text { def }}{=}\arg \min _{\theta \in \Theta} \frac{1}{n} \sum_{i=1}^{n} L\left(z_{i}, \theta\right)+\epsilon L\left(z_{\delta}, \theta\right)-\epsilon L(z, \theta), \begin{aligned}\left.\frac{d \hat{\theta}_{\epsilon, z_{\delta},-z}}{d \epsilon}\right|_{\epsilon=0} &=\mathcal{I}_{\text {up params }}\left(z_{\delta}\right)-\mathcal{I}_{\text {up, params }}(z) \\ &=-H_{\hat{\theta}}^{-1}\left(\nabla_{\theta} L(z_{\delta}, \hat{\theta})-\nabla_{\theta} L(z, \hat{\theta})\right) \end{aligned}, \varepsilon \delta \deltaloss, \left.\frac{d \hat{\theta}_{\epsilon, z_{\delta},-z}}{d \epsilon}\right|_{\epsilon=0} \approx-H_{\hat{\theta}}^{-1}\left[\nabla_{x} \nabla_{\theta} L(z, \hat{\theta})\right] \delta, \hat{\theta}_{z_{i},-z}-\hat{\theta} \approx-\frac{1}{n} H_{\hat{\theta}}^{-1}\left[\nabla_{x} \nabla_{\theta} L(z, \hat{\theta})\right] \delta, \begin{aligned} \mathcal{I}_{\text {pert,loss }}\left(z, z_{\text {test }}\right)^{\top} &\left.\stackrel{\text { def }}{=} \nabla_{\delta} L\left(z_{\text {test }}, \hat{\theta}_{z_{\delta},-z}\right)^{\top}\right|_{\delta=0} \\ &=-\nabla_{\theta} L\left(z_{\text {test }}, \hat{\theta}\right)^{\top} H_{\hat{\theta}}^{-1} \nabla_{x} \nabla_{\theta} L(z, \hat{\theta}) \end{aligned}, train lossH \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right) , -y_{\text {test }} y \cdot \sigma\left(-y_{\text {test }} \theta^{\top} x_{\text {test }}\right) \cdot \sigma\left(-y \theta^{\top} x\right) \cdot x_{\text {test }}^{\top} H_{\hat{\theta}}^{-1} x, influence functiondebug training datatraining point \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right) losstraining pointtraining point, Stochastic estimationHHHTFO(np)np, ImageNetdogfish900Inception v3SVM with RBF kernel, poisoning attackinfluence function59157%77%10590/591, attackRelated worktraining set attackadversarial example, influence functionbad case debug, labelinfluence function, \mathcal{I}_{\text {up,loss }}\left(z_{i}, z_{i}\right) , 10%labelinfluence functiontrain lossrandom, \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right), \mathcal{I}_{\text {up,loss }}\left(z_{i}, z_{i}\right), \mathcal{I}_{\text {pert,loss }}\left(z, z_{\text {test }}\right)^{\top}, H_{\hat{\theta}}^{-1} \nabla_{x} \nabla_{\theta} L(z, \hat{\theta}), Less Is Better: Unweighted Data Subsampling via Influence Function, influence functionleave-one-out retraining, 0.86H, SVMhinge loss0.95, straightforwardbest paper, influence functionloss. Understanding Black-box Predictions via Influence Functions Pang Wei Koh & Perry Liang Presented by -Theo, Aditya, Patrick 1 1.Influence functions: definitions and theory 2.Efficiently calculating influence functions 3. Influence functions help you to debug the results of your deep learning model A. Mokhtari, A. Ozdaglar, and S. Pattathil. Your search export query has expired. Understanding Black-box Predictions via Influence Functions 2016. GitHub - kohpangwei/influence-release in terms of the dataset. Simonyan, K., Vedaldi, A., and Zisserman, A. Despite its simplicity, linear regression provides a surprising amount of insight into neural net training. This code replicates the experiments from the following paper: Pang Wei Koh and Percy Liang Understanding Black-box Predictions via Influence Functions International Conference on Machine Learning (ICML), 2017. , loss , input space . NIPS, p.1097-1105. Shrikumar, A., Greenside, P., Shcherbina, A., and Kundaje, A. more recursions when approximating the influence. Online delivery. x\Y#7r~_}2;4,>Fvv,ZduwYTUQP }#&uD,spdv9#?Kft&e&LS 5[^od7Z5qg(]}{__+3"Bej,wofUl)u*l$m}FX6S/7?wfYwoF4{Hmf83%TF#}{c}w( kMf*bLQ?C}?J2l1jy)>$"^4Rtg+$4Ld{}Q8k|iaL_@8v C. Maddison, D. Paulin, Y.-W. Teh, B. O'Donoghue, and A. Doucet. %PDF-1.5 Besides just getting your networks to train better, another important reason to study neural net training dynamics is that many of our modern architectures are themselves powerful enough to do optimization. Fortunately, influence functions give us an efficient approximation. The dict structure looks similiar to this: Harmful is a list of numbers, which are the IDs of the training data samples After all, the optimization landscape is nonconvex, highly nonlinear, and high-dimensional, so why are we able to train these networks? Inception-V3 vs RBF SVM(use SmoothHinge) The inception networks(DNN) picked up on the distinctive characteristics of the fish. Reference Understanding Black-box Predictions via Influence Functions A spherical analysis of Adam with batch normalization. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. Understanding black-box predictions via influence functions. In. In, Martens, J. Biggio, B., Nelson, B., and Laskov, P. Support vector machines under adversarial label noise. In Proceedings of the international conference on machine learning (ICML). Pang Wei Koh and Percy Liang. I am grateful to my supervisor Tasnim Azad Abir sir, for his . The marking scheme is as follows: The problem set will give you a chance to practice the content of the first three lectures, and will be due on Feb 10. . Insights from a noisy quadratic model. Ribeiro, M. T., Singh, S., and Guestrin, C. "why should I trust you? Implicit Regularization and Bayesian Inference [Slides]. In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through . S. McCandish, J. Kaplan, D. Amodei, and the OpenAI Dota Team. Google Scholar Krizhevsky A, Sutskever I, Hinton GE, 2012. Why Use Influence Functions? Hopefully this understanding will let us improve the algorithms. : , , , . Christmann, A. and Steinwart, I. Jianxin Ma, Peng Cui, Kun Kuang, Xin Wang, and Wenwu Zhu. How can we explain the predictions of a black-box model? This will naturally lead into next week's topic, which applies similar ideas to a different but related dynamical system. Understanding Black-box Predictions via Influence Functions ": Explaining the predictions of any classifier. Yuwen Xiong, Andrew Liao, and Jingkang Wang. Understanding Black-box Predictions via Influence Functions. Theano D. Team. The reference implementation can be found here: link. Understanding Black-box Predictions via Influence Functions PW Koh, P Liang. prediction outcome of the processed test samples. We have 3 hours scheduled for lecture and/or tutorial. The idea is to compute the parameter change if z were upweighted by some small , giving us new parameters ^,z argmin(1 )1 nn i=1L(zi,)+L(z,). A tag already exists with the provided branch name. The details of the assignment are here. Understanding Black-box Predictions via Influence Functions - SlideShare A sign-up sheet will be distributed via email. ordered by helpfulness. Please try again. Github Pang Wei Koh, Percy Liang; Proceedings of the 34th International Conference on Machine Learning, . When can we take advantage of parallelism to train neural nets? Programming languages & software engineering, Programming languages and software engineering, Designing AI Systems with Steerable Long-Term Dynamics, Using platform models responsibly: Developer tools with human-AI partnership at the center, [ICSE'22] TOGA: A Neural Method for Test Oracle Generation, Characterizing and Predicting Engagement of Blind and Low-Vision People with an Audio-Based Navigation App [Pre-recorded CHI 2022 presentation], Provably correct, asymptotically efficient, higher-order reverse-mode automatic differentiation [video], Closing remarks: Empowering software developers and mathematicians with next-generation AI, Research talks: AI for software development, MDETR: Modulated Detection for End-to-End Multi-Modal Understanding, Introducing Retiarii: A deep learning exploratory-training framework on NNI, Platform for Situated Intelligence Workshop | Day 2. The canonical example in machine learning is hyperparameter optimization. Chatterjee, S. and Hadi, A. S. Influential observations, high leverage points, and outliers in linear regression. Deep inside convolutional networks: Visualising image classification models and saliency maps. This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. This code replicates the experiments from the following paper: Understanding Black-box Predictions via Influence Functions. Model selection in kernel based regression using the influence function. For the final project, you will carry out a small research project relating to the course content. Not just a black box: Learning important features through propagating activation differences. If Influence Functions are the Answer, Then What is the Question? You signed in with another tab or window. Cook, R. D. Assessment of local influence. To get the correct test outcome of ship, the Helpful images from Understanding Black-box Predictions via Influence Functions We'll mostly focus on minimax optimization, or zero-sum games. and even creating visually-indistinguishable training-set attacks. The Idea: use Influence Functions to observe the influence of the test samples from the training samples. Theano: A Python framework for fast computation of mathematical expressions. influence-instance. In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. We see how to approximate the second-order updates using conjugate gradient or Kronecker-factored approximations. Which optimization techniques are useful at which batch sizes? Borys Bryndak, Sergio Casas, and Sean Segal. A. M. Saxe, J. L. McClelland, and S. Ganguli. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. The reference implementation can be found here: link. Influence functions can of course also be used for data other than images, A. On linear models and convolutional neural networks, we demonstrate that influence functions are useful for multiple purposes: understanding model behavior, debugging models, detecting dataset errors, and even creating visually-indistinguishable training-set attacks. ordered by harmfulness. Often we want to identify an influential group of training samples in a particular test prediction. This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. The infinitesimal jackknife. Then, it'll calculate all s_test values and save those to disk. If the influence function is calculated for multiple test images, the helpfulness is ordered by average helpfulness to the While these topics had consumed much of the machine learning research community's attention when it came to simpler models, the attitude of the neural nets community was to train first and ask questions later. Understanding Black-box Predictions via Influence Functions Requirements chainer v3: It uses FunctionHook. PDF Understanding Black-box Predictions via Influence Functions - arXiv Please download or close your previous search result export first before starting a new bulk export. Debruyne, M., Hubert, M., and Suykens, J. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. J. Lucas, S. Sun, R. Zemel, and R. Grosse. On the importance of initialization and momentum in deep learning. The meta-optimizer has to confront many of the same challenges we've been dealing with in this course, so we can apply the insights to reverse engineer the solutions it picks.
Willowton Full Panel Bed Assembly Instructions,
Fairclough Critical Discourse Analysis Model,
Queens College Language Courses,
Is George Mckenna Black,
Newsmax Contact Phone Number,
Articles U
