individualized learning system; optimization; active learning ; machine learning ; bayesian optimization; statistical learning theory
Yeo Theresa, KamalarubanParameswaran, SinglaAdish, MerchantArpit, AsselbornThibault, FauconLouis, DillenbourgPierre, CevherVolkan (2019), Iterative Classroom Teaching, in 33rd AAAI Conference on Artificial Intelligence
, Honolulu, Hawaii, USAAssociation for the Advancement of Artificial Intelligence, AAAI conference.
Hsieh Ya-Ping, Kao Yu-Chun, Mahabadi Rabeeh Karimi, Yurtsever Alp, Kyrillidis Anastasios, Cevher Volkan (2018), A Non-Euclidean Gradient Descent Framework for Non-Convex Matrix Factorization, in IEEE Transactions on Signal Processing
, 66(22), 5917-5926.
Bogunovic Ilija, Mitrovic Slobodan, Scarlett Jonathan, Cevher Volkan (2017), Robust Submodular Maximization: A Non-Uniform Partitioning Approach, in The 34th International Conference on Machine Learning (ICML)
, SydneyPMLR, Sydney.
Mitrovic Slobodan, Bogunovic Ilija, Norouzi-Fard Ashkan, Tarnawsi Jakub, Cevher Volkan (2017), Streaming Robust Submodular Maximization: A Partitioned Thresholding Approach, in Conference on Neural Information Processing Systems (NIPS), Long Beach, 2017. These
, Long Beach, California, USANIPS, LOng Beach, California, USA.
Rolland Paul, Scarlett Jonathan, Bogunovic Ilija, Cevher Volkan, High Dimensional Bayesian Optimization via Additive Models with Overlapping Groups, in 21st International Conference on Artificial Intelligence and Statistics (AISTATS)
, Lanzarotte, Spain AISTATS, Spain.
Automated learning from massive datasets promises potentially huge societal and technological benefits. Efficient learning from massive datasets, however, is currently a grand challenge, since the data growth in modern applications is significantly outpacing the growth of computation. Surprisingly, while probabilistic graphical models (PGM) currently form the most successful “learning machines” in practice, the existing theory and methods date back to the previous century. When confronted with the basic computational and storage challenges of the data deluge, this important probabilistic learning framework severely lacks a modern optimization infrastructure. As a result, this project will develop new joint optimization and learning theoretic foundations of PGM’s to not only break the computational barriers for scalability, but also certify the quality of our learning results. Our basic plan of attack is supported by our theoretical preliminary results of trading-off the data-size and computation with guarantees, as well as our recent computational methods that already solve tera-scale, convex optimization problems as relaxations to non-convex problems. To this end, we will investigate three inter-related research thrusts: (Thrust I) An accurate and scalable prediction framework with PGMs: This thrust exploits the underlying geometry of non-convex PGM learning formulations in order to obtain massive speed-ups in learning. We illus- trate how streaming data models, stochastic approximation, primal-dual smoothing, and conver- gence analysis come together to design new, heuristic-free algorithms with theoretical guarantees. (Thrust II) A flexible decision framework via Bayesian optimization: This thrust builds an active learning framework based on Bayesian optimization that adaptively queries an unknown function in order to build an explicit approximation or to optimize the function with theoretical guarantees. For this purpose, we unify key combinatorial structures (e.g., submodularity) with smoothness models (e.g., Gaussian processes) for rigorous guarantees. (Thrust III) Scalable foundations of learning machines with PGMs and BO:This thrust uses neural networks to significantly speed up the task of Bayesian optimization, and Bayesian optimization to optimize the parameters of neural networks. We will explore these connections and interactions in detail, hence allowing the above research directions to complement each other to the full extent possible for scalable automation. One key application that we focus on implementing in the proposed work is ``individualized learning systems'' (ILS). While textbooks and homework assignments effectively addressed the key educational challenges of the 19th and the 20th centuries, they are often insufficient in today's fast-paced developments as they are static, time-consuming to develop, and quickly become out-of-date. Moreover, the classic, monolithic learning style for all students has limited effectiveness in serving to the skill set, background, or goals of individual students. Similarly, while Massive Online Open Courses (MOOCs) in various platforms, such as Coursera and edX, have gained significant popularity on an unprecedented scale, they generally mimic the classical lecture style and its ``non-adaptive'' structure. Recent advances in educational research hints at new ways of enhancing the learning experience given the student's learning style and abilities. Our contention in this proposal is that by leveraging machine learning research along with educational research, we can bring the potentially great benefits of these advances closer to reality. The application of neural networks, Bayesian optimization, and their fusion represents not only a substantial leap over the state-of-the-art in machine learning but has the potential to lead Switzerland on the forefront of ILS and educational research with clear benefits to Swiss society and its economy.