• Login
    View Item 
    •   SMARTech Home
    • Georgia Tech Interdisciplinary Research Institutes (IRIs)
    • Institute for Data Engineering and Science (IDEaS)
    • IDEaS Seminars
    • View Item
    •   SMARTech Home
    • Georgia Tech Interdisciplinary Research Institutes (IRIs)
    • Institute for Data Engineering and Science (IDEaS)
    • IDEaS Seminars
    • View Item
    JavaScript is disabled for your browser. Some features of this site may not work without it.

    Second Order Machine Learning

    Thumbnail
    View/Open
    mahoney.mp4 (413.8Mb)
    mahoney_videostream.html (985bytes)
    transcription.txt (54.73Kb)
    Date
    2017-09-22
    Author
    Mahoney, Michael
    Metadata
    Show full item record
    Abstract
    A major challenge for large-scale machine learning, and one that will only increase in importance as we develop models that are more and more domain-informed, involves going beyond high-variance first-order optimization methods to more robust second order methods. Here, we consider the problem of minimizing the sum of a large number of functions over a convex constraint set, a problem that arises in many data analysis, machine learning, and more traditional scientific computing applications, as well as non-convex variants of these basic methods. While this is of interest in many situations, it has received attention recently due to challenges associated with training so-called deep neural networks. We establish improved bounds for algorithms that incorporate sub-sampling as a way to improve computational efficiency, while maintaining the original convergence properties of these algorithms. These methods exploit recent results from Randomized Linear Algebra on approximate matrix multiplication. Within the context of second order optimization methods, they provide quantitative convergence results for variants of Newton's methods, where the Hessian and/or the gradient is uniformly or non-uniformly sub-sampled, under much weaker assumptions than prior work.
    URI
    http://hdl.handle.net/1853/58822
    Collections
    • IDEaS Seminars [9]

    Browse

    All of SMARTechCommunities & CollectionsDatesAuthorsTitlesSubjectsTypesThis CollectionDatesAuthorsTitlesSubjectsTypes

    My SMARTech

    Login

    Statistics

    View Usage StatisticsView Google Analytics Statistics
    facebook instagram twitter youtube
    • My Account
    • Contact us
    • Directory
    • Campus Map
    • Support/Give
    • Library Accessibility
      • About SMARTech
      • SMARTech Terms of Use
    Georgia Tech Library266 4th Street NW, Atlanta, GA 30332
    404.894.4500
    • Emergency Information
    • Legal and Privacy Information
    • Human Trafficking Notice
    • Accessibility
    • Accountability
    • Accreditation
    • Employment
    © 2020 Georgia Institute of Technology