Ankit Pensia

Postdoctoral Fellow

IBM Research

Hi! I am a Herman Goldstine Postdoctoral Fellow at the MIT-IBM Watson AI Lab at Cambridge, MA.

I received my PhD in the Computer Sciences department at UW-Madison in 2023, where I was advised by Po-Ling Loh, Varun Jog, and Ilias Diakonikolas. During the Summer of 2022, I was a research intern at Google Research NYC with Pranjal Awasthi and Satyen Kale. Prior to Madison, I spent five memorable years at IIT Kanpur.

Please feel free to contact me at ankitpensia94@gmail.com.

## Research

My research interests include:

Thesis: link (Graduate Student Research Award from the CS Department)

Please see below for more details about my research.

## Selected Publications

A comprehensive list of my published works can be found here. Relevant information can also be found on my profiles at Google Scholar, dblp, and Semantic Scholar.

### Robust and Heavy-Tailed Statistics

Outliers and heavy-tailed distributions can pose significant challenges to standard inference procedures.
These challenges are addressed in the field of *robust statistics*, which focuses on developing algorithms that are resistant to the effects of these extreme and atypical data points.
I am interested in exploring the statistical and computational landscapes of such algorithms.

- Outlier Robust Mean Estimation with Subgaussian Rates via Stability [link] [abstract]

**NeurIPS**, 2020

We show that recent outlier-robust algorithms also achieve subgaussian confidence intervals for heavy-tailed distributions, and vice-versa. In particular, we identify the "stability" condition as the bridge between these two contamination models. In a recent paper at NeurIPS 2022, we extended these results to the*structured*setting when the mean is further constrained to be sparse, i.e., robust*sparse*mean estimation.

- Robust regression with covariate filtering: Heavy tails and adversarial contamination [link] [abstract]

Manuscript, 2020

We propose a simple (computationally-efficient) filtering step to remove potentially harmful covariates for robust regression. Combined with Huber Regression or Least Trimmed Squares, we obtain simple, fast, and provable algorithms for robust regression.

### Inference under Communication, Memory, and Privacy Constraints

The proliferation of big data has led to the development of distributed inference paradigms such as federated learning and the use of edge devices. These distributed setups impose constraints on communication bandwidth, memory usage, and privacy. My research focuses on understanding the impact of these constraints on sample complexity and developing computationally-efficient algorithms.

- Simple Binary Hypothesis Testing under Local Differential Privacy and Communication Constraints [link] [abstract]

**COLT**, 2023

We characterizes the minmax optimal sample complexity of hypothesis testing under local differential privacy and communication constraints, develops instance-optimal algorithms, and shows separation between the sample complexity for binary and ternary distributions. In this paper, we build on our recent prior work, where we had shown that the cost of only the communication constraints is mild (at most logarithmic) for binary hypothesis testing but severe (exponential) for $M$-ary hypothesis testing.

- Streaming Algorithms for High-Dimensional Robust Statistics [link] [abstract]

**ICML**, 2022

We develop the first (computationally-efficient and sample-efficient) streaming algorithm with $o(d^2)$ memory usage for a variety of robust estimation tasks; in fact, our algorithm uses only $\tilde{O}(d)$ space. All the prior robust algorithms needed to store the entire dataset in memory, which leads to super-quadratic memory. We also develop near linear-time robust mean estimation algorithm under a unified notion of "stability".

### Machine Learning and Statistics

I have a broad interest in the fields of machine learning and statistics. In particular, I have explored the representation power of neural networks and the generalization error of learning algorithms in my research.

- Optimal Lottery Tickets via SubsetSum: Logarithmic Over-Parameterization is Sufficient [link] [abstract]

**NeurIPS**, 2020

We prove the strong lottery ticket hypothesis with optimal bounds on over-parameterization. We prove these results by connecting pruning of neural networks to random subset sum, a well-studied topic in TCS. This leads to an exponential improvement in the required over-parameterization in the width.

- Generalization Error Bounds for Noisy, Iterative Algorithms [link] [abstract]

**ISIT**, 2018

We provide a general recipe to bound generalization error of noisy, iterative algorithms using information-theoretic tools. One prominent example of such algorithms is Stochastic Gradient Langevin Dynamics (SGLD).

## Teaching Experience

Graduate Teaching Assistant

Mathematical Foundations of Machine Learning, UW-Madison, Fall 2018

Introduction to Bioinformatics, UW-Madison, Fall 2017

Probabilistic Mobile Robotics, IIT Kanpur, Fall 2016, Spring 2016

Communication Systems, IIT Kanpur, Summer 2016

Academic Mentor

Representation and Analysis of Random Signals, IIT Kanpur, Fall 201