ICML and my notes on day 1

Thirty-fourth International Conference on Machine Learning is held in Sydney this year. The accumulative brain power of this event is overwhelming. I’m taking  notes, so below are few things that I found interesting.


Amongst most popular areas of interest for this year at the conference are:

  • Neural Networks and Deep Learning
  • Optimization (Continuous)
  • Online Learning
  • Generative Models
  • Graphical Models

 Machine Learning for Autonomous Vehicles

There is quite a bit of interest for this topic at the conference: 2h tutorial on day1 and full day workshop later in the conference. The tutorial was presented by Uber Advanced Technologies Group. It’s quite impressive that within slightly more than a year team has gone from building  their 1st self driving vehicle  (Dec’15) to self-driving cars picking up Uber passenger in Pittsburg  (Feb’17).

They use the following sensor on a car to collect data: Camera, LIDAR, RADAR, GPS,  IMU.

Datasets mostly mentioned in this talk: KITTY and Cityscapes.

I found useful the chart they presented from paper Speed/accuracy trade-offs for modern convolutional object detectors


For self-driver cars’ needs object detection needs to be 3D, where Camera data and LIDAR data gets fused.

Main components are:  3D  Proposal network and  Region based  fusion network.

Tracking of the objects: RNN + LSTM (see paper).

For e2e learning Imitation Learning (General Adversarial Imitation Learning is option to consider)   and\or Deep Reinforcement Learning are used.

Nice TEDX talk (15 mins)  on how self driving cars will transform  our lives.

Deep Learning Models for Health Care – Challenges and Solutions

Slides for the talk are here.

RNNs are used in almost every scenarios they’ve presented.

Impressive results in Dermatologist-level classification of skin cancer  —  AI performs better than human expert (72% accuracy vs 65%).

To address the problem of big-small data (for example age-specific training) Variational Adversarial Deep Domain Adaptation approach (paper) has been presented.

Missing data is a common problem for medical problems as  data is often not missing at random (doctor decides that patient X does not need  lab test Y). See paper “Recurrent Neural Networks for Multivariate Time Series with Missing Values” on how to deal with this (yup, RNNs again).

Incorporating domain knowledge (ontology) into the model: see paper “Multi-layer Representation Learning for Medical Concepts” and paper “GRAM: Graph-based Attention Model for Healthcare Representation Learning” (to be presented at KDD’17). Also one of GRAM’s nice benefits that you can train model using much smaller dataset.

Interpretable ML is still a thing for health care. One of the options is to use Gradient Boosting Trees (GBTs) to mimic deep learning models (paper) is one of the option. Another one is “RETAIN: An Interpretable Predictive Model for Healthcare Using Reverse Time Attention Mechanism” (paper), where RNNs  produce weights for doctor’s visit and weights for medical codes used during visits.

For what’s coming next in Healthcare & ML the following areas were highlighted:

– using genomic data, using plentiful data from various sport sensors

– continue advances in medical imaging

– improving model interpretation

– producing more complex output (detail diagnosis)


Slides for the talk are here.

Great turnaround for this talk!

Helpful tips and tricks for training language model:


Interesting applications :

  1. Evaluation of short computer programs (paper)
  2. Meta learning: doing image classification by showing only few examples per class  (paper)
  3. Lip reading (paper). Model performed much better than professional lip reader with 10 years of experience. Professional lip reader was able to correctly decipher less than 25% of the spoken words and model (lips only) is  deciphered 50%.
  4. Pixel RNNs (paper) –generating images pixel by pixel. Results look not pretty, but still impressive taking into consideration pixel-by-pixel approach. Follow-up Pixel CNNs (paper) generates much better pictures.

It’s been an amazing start, looking forward for more!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s