Machine Learning Reading Group (MLRG)

The UBC Machine Learning Reading Group (MLRG) meets regularly (usually weekly) to discuss research topics on a particular sub-field of Machine Learning.

Subscription

You can receive announcements about the reading group by joining our mailing list. To join the mailing list, please use an academic email address and send an email to majordomo@cs.ubc.ca with an empty subject line and with the following message body: “subscribe mlrg-l YOUR-EMAIL-ADDRESS”. Use the message body “unsubscribe mlrg-l YOUR-EMAIL-ADDRESS” to unsubscribe from the list. If you use a non-academic email address, we would have to verify it which could delay your subscription process.

Upcoming Talks

Fall 2022 – Transformers

Every Wednesday at 1:00PM in ICCS 146. A Zoom link will be provided in the weekly reminder email for those wishing to attend remotely.

Past Sessions

DatePresenterTopic and PaperSlides
Sept. 28DylanIntroductionSlides
Oct. 5AlanAttention is All You Need
https://arxiv.org/abs/1706.03762
Slides
Oct. 12HelenLanguage Models are Few Shot Learners (GPT-3)
https://arxiv.org/abs/2005.14165
Slides
Oct. 19~Postponed
Oct. 26CurtisAn Image is Worth 16×16 Words: Transformers for Image Recognition at Scale
https://arxiv.org/abs/2010.11929
Slides
Nov. 2MeharChain of Thought Prompting Elicits Reasoning in Large Language Models
https://arxiv.org/abs/2201.11903
Slides
Nov. 9~READING WEEK
Nov. 16~Rescheduled
Nov. 23~Cancelled
Nov. 30 IssamLearning Transferable Visual Models From Natural Language Supervision (CLIP)
https://arxiv.org/abs/2103.00020
Slides
Dec. 7AdamHierarchical Text-Conditional Image Generation with CLIP Latents
https://arxiv.org/abs/2204.06125
Slides
Dec. 14BettyNeural Scaling Laws + Exploring the Limits of Large Scale Pre-Training
https://arxiv.org/abs/2001.08361, https://arxiv.org/abs/2110.02095
Slides

Summer 2022 – Random Topics

Every Wednesday at 1:00 PM Online or In-Person (ICCS 238)

DatePresenterTopic and PaperSlidesIn-Person or Online
June 1IssamBuild End-to-End Machine Learning Projects for Large-Scale Optimization ExperimentsSlides
Colab
In-Person
June 8WuThe Riemannian Geometry of Deep Generative Models
https://arxiv.org/pdf/1711.08014.pdf
SlidesIn-Person
June 15BettyAccelerated Primal-Dual Gradient Method for Smooth and
Convex-Concave Saddle-Point Problems with Bilinear Coupling
https://arxiv.org/abs/2112.15199
SlidesIn-Person
June 22HelenActive Learning in Semantic Image Segmentation
Paper Paper2
SlidesIn-Person
June 29AlanA-NICE-MC: Adversarial Training for MCMC
https://arxiv.org/abs/1706.07561
SlidesIn-Person
July 6CurtisAdaptive Gradient Descent without Descent
https://arxiv.org/abs/1910.09529
In-Person
July 13NickNot All Samples Are Created Equal: Deep Learning with Importance Sampling
https://arxiv.org/pdf/1803.00942.pdf
In-Person
July 20AmruthaAssessing Generalization via Disagreement
https://arxiv.org/pdf/2106.13799.pdf
In-Person
July 27BaharehRelational Multi-Task Learning: Modeling Relations between Data and Tasks
https://openreview.net/pdf?id=8Py-W8lSUgy
In-Person
August 10DylanDiffusion and Score-Based Generative Models
Paper1 Paper2 Paper3
In-Person

Winter 2021 term 2 – Random Topics

DatePresenterTopic and PaperSlides
February 16WilderGradients are Not All You Need
https://arxiv.org/abs/2111.05803
March 2HelenOops I Took a Gradient
https://arxiv.org/abs/2102.04509
Slides
March 9AminNeural Tangent Kernels
https://arxiv.org/abs/1806.07572
Slides
March 16FredGradient Descent on Neural Networks Typically Occurs at the Edge of Stability
https://arxiv.org/abs/2103.00065
Slides
March 23BettyQuasi-Newton Methods for Machine Learning: Forget the Past, Just Sample
https://arxiv.org/abs/1901.09997
Slides
March 30AmruthaGradient Starvation: A Learning Proclivity in Neural Networks
https://arxiv.org/abs/2011.09468
Slides
April 6DylanDo Deep Generative Models Know What They Don’t Know?
https://arxiv.org/abs/1810.09136
Slides
April 13WuMonte Carlo Gradient Estimation in Machine Learning
https://arxiv.org/abs/1906.10652
Slides
April 20 BaharehInductive Representation Learning on Temporal Graphs
https://arxiv.org/abs/2011.09468
Slides
April 27CurtisSuper-Acceleration with Cyclical Step-sizes
https://arxiv.org/abs/2106.09687
Slides
Video

Winter 2021 term 1 – Responsible ML

Every Wednesday at 1:00 PM Online

DatePresenterTopic and PaperSlides
Oct 13LironneIntroductionSlides
Oct 20AmruthaDeepFake Detection Slides
Oct 27NamrataFairness without Demographics Slides
Nov 3WilderEthics and Governance of Artificial Intelligence Slides
Nov 10YashaImage Counterfactual Sensitivity Analysis for Detecting Unintended Bias Slides
Nov 17FredLessons from Archives Strategies for Collecting Sociocoltural Data Slides
Nov 24BettyOptimizing Long-term Social Welfare in Recommender Systems Slides
Dec 1DylanOn the Dangers of Stochastic Parrots: Can Language Models Be Too Big? Slides
Dec 8 Helen Understanding The Origins of Bias in Word Embeddings Slides

Summer 2021 – Geometric structures in machine learning

DatePresenterTopic and PaperSlides
Jul 7WuIntroductionSlides
Jul 14NickBasics of Geometric Deep LearningSlides
Jul 21WuBasics of manifoldsSlides
Jul 28FredRiemannian optimizationSlides
Aug 4ChristianNatural Wake-Sleep Algorithm
Aug 11VictorRiemannian optimization
Aug 18WilderNatural Gradient Boosting for Probabilistic Prediction
Aug 25BettyPredicting anticancer hyperfoods with graph convolutional networksSlides
Sep 1Wu(bonus) Basics of information geometry and matrix groups

Winter term 2 2020 – Modern Deep Learning Architectures

Every Wednesday at 1:00 PM Online

DatePresenterTopic and PaperSlides
Jan 27FredIntroductionSlides
Feb 3VictorComputer Vision: Deep models for segmentation
He et al. (2018), Mask R-CNN,
Slides
Feb 10AdamComputer Vision: 3D point clouds
Zhou Tuzel (2018), VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
Slides
Feb 17MarkNatural Language Processing: Attention models
Vaswani et al. (2017, Attention is All you Need
Slides
Feb 24JacquesSound and audio data: Text-to-speach
Oord et al. (2016), WaveNet: A Generative Model for Raw Audio
Slides
Mar 3BettyBatch Normalization
Santurkar et al. (2018), How Does Batch Normalization Help Optimization?
Slides
Mar 10DanicaGenerative Models: GANs and optimal transport
Arjovsky et al. (2017), Wasserstein Generative Adversarial Networks
Slides
Mar 17DylanGenerative Models: Autoregressive models and VAEs
Chen et al. (2017), Variational Lossy Autoencoder
Slides
Mar 24WilderNeural ODE
Chen et al. (2018), Neural Ordinary Differential Equations
Slides
Mar 31LironneGraph Neural Networks
Kipf and Welling (2016), Semi-Supervised Classification with Graph Convolutional Networks
Slides

Winter term 1 2020 – Tensor basics and applications

Every Wednesday at 1:00 PM Online

DatePresenterTopic and PaperSlides
Sep 30BettyMotivationSlides
Oct 7MarkTensor basics: Notation, operations, etc. (Kolda and Bader 2009)
http://www.kolda.net/publication/TensorReview.pdf
Slides
Oct 14BettyTensor basics: Complexity (Hillar and Lim 2013)
https://arxiv.org/abs/0911.1393
Slides
Oct 21BahareTensor factorization: Knowledge graphs (Kazemi and Poole 2018)
https://papers.nips.cc/paper/7682-simple-embedding-for-link-prediction-in-knowledge-graphs.pdf
Slides
Oct 28WilderLatent models: Gaussian mixture models (Hsu and Kakade 2013)
https://arxiv.org/pdf/1206.5766.pdf
Slides
Nov 4EmmanuelLatent models: Topic models (Anandkumar et al. 2014)
https://papers.nips.cc/paper/4637-a-spectral-algorithm-for-latent-dirichlet-allocation
Slides
Nov 11Remembrance Day
Nov 18FredHigher order methods: Estimate sequence methods (Baes 2009)
http://www.optimization-online.org/DB_FILE/2009/08/2372.pdf
Slides
Nov 25WuHigher order methods: Bi-level unconstrained minimization (Nesterov 2020)
https://dial.uclouvain.be/pr/boreal/object/boreal:227219
Slides
Dec 2SimarOther appl: Tensors in deep neural networks (Novikov et al. 2015)
https://papers.nips.cc/paper/5787-tensorizing-neural-networks
Slides
Dec 9AdamOther appl: Tensor completion in visual data (Liu et al. 2013)
https://www.cs.rochester.edu/u/jliu/paper/Ji-ICCV09.pdf
Slides

Summer term 2020 – Optimal Control

Every Wednesday at 1:00 PM Online

DatePresenterTopic
Jul 8WilderIntroduction and Overview – [pdf slides]
Jul 15CathyOptimal Control and Dynamical Systems – [pdf slides]
Jul 22BenClassic Control and Dynamic Programming – [pdf slides]
Jul 29BettyIterative LQR and Guided Policy Search – [pdf slides]
Aug 5JoeySample Complexity of the Linear Quadratic Regulator – [pdf slides]
Aug 12FredModel predictive control and Safe RL – [pdf slides]
Aug 19AaronEnd-to-End Training of Deep Visuomotor Policies – N/A
Aug 26DylanCombining Optimal control and Learning for Visual Navigation in Novel Environments – [pdf slides]

Winter term 2 2020 – Causality

Every Wednesday in room ICICS X836 at 1:00 PM

DatePresenterTopic
Jan 29JasonMotivation – [pdf slides]
Feb 5CathyClassical approaches – [pdf slides]
Feb 12BenCounterfactual Inference with Deep Learning – [pdf slides]
Feb 26AaronInstrumental Variables – [pdf slides]
Mar 4WuCausal Inference with VAEs – [pdf slides]
Mar 11BettyCausal Discovery – [pdf slides]
Mar 18FredOut-of-distribution Generalization
Mar 25JoeyConfounded Bandits
Apr 1AdamCausality in Computer Vision
Apr 8WilderCausality in Reinforcement Learning
Apr 15DevonCounterfactuals
Apr 22AlirezaMeta-learning Causal Structures

Winter term 1 2019 – Why Deep Learning Works

Every Wednesday in room ICICS X836 at 1:00 PM

DatePresenterTopic
Sep 25AaronMotivation – [pdf slides]
Oct 2JasonGeneralization of Neural Networks (arXiv ID: 1611.03530) – [pdf slides]
Oct 9AmirSharp Minima Generalize Poorly (arXiv ID: 1609.04836) – [pdf slides]
Oct 16AdamSharp Minima Can Generalize Well (arXiv ID: 1703.04933) – [pdf slides]
Oct 23WillCapacity Measures (arXiv ID: 1706.08947) – [pdf slides]
Oct 30CathyImplicit Regularization of Optimizers (arXiv ID: 1705.03071) – [pdf slides]
Nov 6BettyImplicit Bias of SGD: Matrix Factorization (arXiv ID: 1705.09280) – [pdf slides]
Nov 13FredImplicit Bias of SGD: Logistic Regression (arXiv ID: 1710.10345) – [pdf slides]
Nov 20JoeyGeneralization of Over-Parameterized Kernels (arXiv ID: 1802.01396) – [pdf slides]
Nov 27WilderOver-Parameterization and Bias-Variance (arXiv ID: 1812.11118) – [pdf slides]
Dec 4AlirezaBig Architectures and Overfitting the Test Set (arXiv ID: 1902.10811) – [pdf slides]
Dec 11BenOver-Parameterization: Generalization Bounds (arXiv ID: 1805.12076) – [pdf slides]

Summer term 2019 – Online Learning

Every Wednesday in room ICICS 146 at 1:00 PM

DatePresenterTopic
Jun 19YihanIntroduction to Online Learning – [pdf slides]
Jun 26CathyMultiplicative Weight Update – [pdf slides]
Jul 3AmitFollow the Leader – [pdf slides]
Jul 10ChrisIntroduction to Bandits – [pdf slides]
Jul 17SikanderContextual Bandits – [pdf slides]
Jul 24JasonThompson Sampling – [pdf slides]
Jul 31ManyouMarkovian Bandits – [pdf slides]
Aug 14LironneDueling Bandits
Aug 21YihanLinear Bandits

Winter term 2 2019 – Representation Learning

Every Monday in room ICICS 146 at 5:00 PM

DatePresenterTopic
Feb 4YifanIntroduction to Representation Learning – [pdf slides]
Feb 11AmirArtistic Style Transfer
Feb 18(Holiday)
Feb 25AaronGANS
Mar 4WilderManifold Learning
Mar 11CathyConvolutional Graph Embeddings – [pdf slides]
Mar 18JasonVariational Autoencoders
Mar 25MarjanGraph and Point Cloud Embeddings
Apr 1MichaelDisentanglement
Apr 8Canceled
Apr 15YihanDictionary Learning – [pdf slides]

Winter term 1 2018 – Reinforcement Learning 2

Every Monday in room ICICS 146 at 5:00 PM

DatePresenterTopic
Oct 15MarkMotivation/Overview – [pdf slides]
Oct 22YifanBayesian RL – [pdf slides]
Oct 29ChristianUseful Uncertainties in Reinforcement Learning – [pdf slides]
Nov 5SharanIntroduction to Bandits – [pdf slides]
Nov 12Cancelled
Nov 19AaronPolicy Gradient Algorithms – [pdf slides]
Nov 26BoyanVerification of NN Properties: Examples from Supervised and Reinforcement Learning
Dec 3WilderMethods and Applications for Inverse Reinforcement Learning
Dec 10MehrdadIntroduction to Contextual Bandits
Dec 17VadenIntroduction to Bayesian Non-Parametrics

Summer 2018

Every Tuesday in room ICICS 146 at 3:00 PM

DatePresenterTopic
May 08Emtiyaz KhanFast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam
May 15Geoff RoederBetter Inference through Lower-Variance Stochastic Gradients
May 22Brendan JubaLearning Abduction Under Partial Observability

Winter term 2 2018 – Parallel and Distributed Machine Learning

Every Tuesday in room ICICS 146 at 5:00 PM

DatePresenterTopic
Jan 30Mark SchmidtMotivation – [pdf slides]
Feb 6YashaDistributed file systems
Feb 13MichaelAsynchronous stochastic gradient
Feb 27SharanSynchronous stochastic gradient – [pdf slides]
Mar 6JulieParallel coordinate optimization – [pdf slides]
Mar 13DevonDecentralized gradient
Mar 20WuDecomposition methods
Mar 27RezaAsynchronous/distributed SAG/SDCA/SVRG
Apr 3VadenRandomized Newton and least squares on the cloud
Apr 10NasimParallel tempering and distributed particle filtering
Apr 17AlirezaDistributed deep networks
Apr 24RaunakBlockchain-based distributed learning

Winter term 1 2017 – Deep Learning Meets Graphical Models

Every Tuesday in room ICICS 146 at 4:00 PM

DatePresenterTopic
Sep 26MarkMotivation/overview – [pdf slides]
Oct 3IssamFCNs and CRFs
Oct 10JulietaRNNs
Oct 17MichaelBayesian neural nets 1: sampling
Oct 24JasonBayesian neural nets 2: variational
Oct 31DevonVariational autoencoders 1: basics/ – [pdf slides]
Nov 7SharanVariational autoencoders 2: variations – [pdf slides]
Nov 14MohamedGenerative adversarial networks 1: basics
Nov 21AlirezaGenerative adversarial networks 2: variations
Nov 28RaunakBeyond generative adversarial networks/ – [pdf slides]

Summer 2017 – Online, Active, and Causal learning

Every Tuesday in room ICICS 146 at 4:00 PM

DatePresenterTopic,,,
Jun 6Mark SchmidtMotivation/overview, perceptron, follow the leader. – [pdf slides]
Jun 13JulieOnline convex optimization, mirror descent – [pdf slides]
Jun 20AlirezaMulti-armed bandits, contextual bandits – [pdf slides]
Jun 27MichaelHeavy hitters
Jul 4RaunakRegularized FTL, AdaGrad, Adam, online-to-batch – [pdf slides]
Jul 11GlenBest-arm identification, dueling bandits,,
Jul 18NasimUncertainty sampling, variance/error reduction, QBC – [pdf slides]
Jul 25MohamedPlanning, A/B testing, Optimal experimental design,
Aug 1SannaRandomized controlled trials, do-calculus – [pdf slides]
Aug 8IssamGranger causality, independent component analysis,,
Aug 22EricCounterfactuals – [pdf slides]
Aug 29JasonInstrumental variables

Winter term 2 2017 – Reinforcement Learning

Every Tuesday in room ICICS 146 at 5:00 PM

DatePresenterTopic
Jan 10Mark SchmidtMotivation/Overview – [pdf slides]
Jan 17NasimMDPs (policy iteration, value iteration)
Jan 24JulieMonte Carlo (estimators, on-policy/off-policy learning) – [pdf slides]
Jan 31RaunakTemporal Difference Learning
Feb 7JenniferMulti-Step Bootstrapping/ – [pdf slides]
Feb 14MichaelFunction Approximation, TD-Gammon
Feb 21Cancelled
Feb 28RickyPlanning, Control with Approximation, and Eligibility Traces
Mar 7IssamOptimal control, flying helicopters
Mar 14SharanPOMDPs – [pdf slides]
Mar 21JasonPolicy gradients, Monte Carlo tree search, and AlphaGo
Mar 28JulietaValue-Iteration Networks
Apr 4GlenRL in Practice
Apr 11MichielPerspectives on Reinforcement Learning for Locomotion Skills
Apr 25IssamConnection between Generative Adversarial Networks and Inverse Reinforcement Learning

Winter term 1 2016 – Deep Learning

Every Wednesday in room ICICS 146 at 5:00 PM

DatePresenterTopic
Sep 21Mark SchmidtIntroduction – [pdf slides]
Sep 28JulieFeedforward neural nets, backpropagation – [pdf slides]
Oct 5MohamedNetwork-independent tricks – [pdf slides]
Oct 12IssamImageNet tricks
Oct 19JasonGraphical models – [pdf slides]
Oct 26SaifArtistic style transfer – [pdf slides]
Nov 2NasimRecurrent neural nets – [pdf slides]
Nov 9Stephen/KevinRecurrent neural nets 2
Nov 16RickyVariational autoencoders and Bayesian dark knowledge
Nov 23RezaGenerative adversarial networks
Nov 30AlirezaMemory nets, neural Turing, stack-augmented RNNs

Summer term 2016 – Miscellaneous

Every Wednesday in room ICCS146 at 5:00 PM

DatePresenterTopic
May 25Mark SchmidtIntroduction to Summer topics – [pdf slides]
Jun 1No meetingUAI camera-ready deadline
Jun 8SharanSpectral Methods (1) – [pdf slides]
Jun 15GeoffSpectral Methods (2) – [pdf slides]
Jun 22ChrisRelational Models
Jun 29SaifSubmodularity – [pdf slides]
Jul 6NasimGrammars – [pdf slides]
Jul 13EviatarContinuous graphical models – [pdf slides]
Jul 20Steven and KevinGaussian Copulas – [pdf slides]
Jul 27IssamLarge-scale kernels methods (1)
Aug 3JuliettaLarge-scale kernels methods (2)
Aug 10AlirezaChangepoint detection (1)
Aug 17MohamedChangepoint detection (2)
Aug 24JulieIndependent component analysis (1)
Aug 31RickyIndependent component analysis (2)

Winter term 2 2016 – Crash course on Bayesian methods

Every Wednesday in room ICICS 146 at 5:00 PM

DatePresenterTopic
Jan 06Mark SchmidtIntroduction to Bayesian methods – [pdf slides]
Jan 13NasimConjugate Priors, Non-Informative Priors – [pdf slides]
Jan 20GeoffHierarchical Modeling and Bayesian Model Selection – [pdf slides]
Jan 27IssamGaussian Processes and Empirical Bayes – [pdf slides]
Feb 3RickyBasic Monte Carlo Methods – [pdf slides]
Feb 10JasonMCMC – [website link]
Feb 24MichaelBayesian Optimization – [pdf slides]
Mar 2SharanVariational Bayes – [pdf slides]
Mar 9RezaStochastic Variational Inference – [pdf slides]
Mar 16MarkNon-Parametric Bayes 1 – [pdf slides]
Mar 23RezaNon-Parametric Bayes 2
Apr 6JulietaSequential Monte Carlo and Population MCMC
Apr 13RudyReversible-Jump MCMC
Apr 20AlirezaApproximate Bayesian Computation – [pdf slides]

Winter term 1 2015 – Crash course on optimization

Every Tuesday in room X836 at 5:00 PM

DatePresenterTopic
Sep 22Mark SchmidtIntroduction to convex optimization – [pdf slides]
Sep 29Mark SchmidtFirst-Order Methods – [pdf slides]
Oct 06JulietaStochastic Subgradient – [pdf slides]
Oct 13MohamedMinimizing Finite Sums – [pdf slides]
Oct 20JasonProximal-Gradient – [pdf slides]
Oct 27IvesFrank-Wolfe, ADMM – [pdf slides]
Nov 03JulieCoordinate Descent – [pdf slides]
Nov 10SharanOnline Convex Optimization – [pdf slides]
Nov 17Mark SchmidtMulti-Level Methods – [pdf slides]
Nov 24IssamNon-Convex Rates – [pdf slides]
Dec 01IssamParallel/Distributed – [pdf slides]
Dec 08(NIPS)
Dec 15AlirezaDeep Learning Local Optima – [pdf slides]

Summer term 2 2015 – Crash course on graphical models

Room ICICS 238 at 11:00 AM

DatePresenterTopic
Aug 17Mark SchmidtWhy learn about graphical models? – [pdf slides]
Aug 18Mark SchmidtInference in Chains and Trees – [pdf slides]
Aug 19JulieConditional Inference and Cutset Conditioning – [pdf slides]
Aug 20MehranJunction Tree – [pdf slides]
Aug 21AlirezaSemi-Markov/Graph Cuts – [pdf slides]
Aug 24Mark SchmidtMRF/CRF – [pdf slides]
Aug 25JulietaICM/Block/Alpha – [pdf slides]
Aug 26JasonMCMC/Herding – [pdf slides]
Aug 27AnkurHidden/RBM/Younes – [pdf slides]
Aug 28SharanStructure Learning – [pdf slides]
Aug 31Mark SchmidtVariational/MF – [pdf slides]
Sep 1NasimBethe/Kikuchi – [pdf slides]
Sep 2RezaTRBP/Convex – [pdf slides]
Sep 3IssamLP/SDP – [pdf slides]
Sep 4Mark SchmidtSSVM/BCFW – [pdf slides]