• Login
    View Item 
    •   SMARTech Home
    • Georgia Tech Interdisciplinary Research Centers (IRCs)
    • Machine Learning (ML@GT)
    • Machine Learning@Georgia Tech Seminars
    • View Item
    •   SMARTech Home
    • Georgia Tech Interdisciplinary Research Centers (IRCs)
    • Machine Learning (ML@GT)
    • Machine Learning@Georgia Tech Seminars
    • View Item
    JavaScript is disabled for your browser. Some features of this site may not work without it.

    Pruning Deep Neural Networks with Net-Trim: Deep Learning and Compressed Sensing Meet

    Thumbnail
    View/Open
    aghasi.mp4 (467.0Mb)
    aghasi_videostream.html (985bytes)
    transcription.txt (41.92Kb)
    Date
    2018-03-14
    Author
    Aghasi, Alireza
    Metadata
    Show full item record
    Abstract
    We introduce and analyze a new technique for model reduction in deep neural networks. Our algorithm prunes (sparsifies) a trained network layer-wise, removing connections at each layer by addressing a convex problem. We present both parallel and cascade versions of the algorithm along with the mathematical analysis of the consistency between the initial network and the retrained model. We also discuss an ADMM implementation of Net-Trim, easily applicable to large scale problems. In terms of the sample complexity, we present a general result that holds for any layer within a network using rectified linear units as the activation. If a layer taking inputs of size N can be described using a maximum number of s non-zero weights per node, under some mild assumptions on the input covariance matrix, we show that these weights can be learned from O(slog N/s) samples.
    URI
    http://hdl.handle.net/1853/59442
    Collections
    • Machine Learning@Georgia Tech Seminars [52]

    Browse

    All of SMARTechCommunities & CollectionsDatesAuthorsTitlesSubjectsTypesThis CollectionDatesAuthorsTitlesSubjectsTypes

    My SMARTech

    Login

    Statistics

    View Usage StatisticsView Google Analytics Statistics
    facebook instagram twitter youtube
    • My Account
    • Contact us
    • Directory
    • Campus Map
    • Support/Give
    • Library Accessibility
      • About SMARTech
      • SMARTech Terms of Use
    Georgia Tech Library266 4th Street NW, Atlanta, GA 30332
    404.894.4500
    • Emergency Information
    • Legal and Privacy Information
    • Human Trafficking Notice
    • Accessibility
    • Accountability
    • Accreditation
    • Employment
    © 2020 Georgia Institute of Technology