- \n
- \n\u201cPruning Neural Networks Without any Data by Iteratively Conserving Synaptic Flow<\/a>,\u201d Hidenori Tanaka (NTT Research Physics & Informatics Lab (PHI) Lab<\/a>; Department of Applied Physics<\/a>, Stanford), Daniel Kunin (co-first author, Institute for Computational & Mathematical Engineering<\/a>, Stanford), Daniel L. Yamins (Departments of Psychology<\/a> and Computer Science<\/a>, Stanford), Surya Ganguli (Department of Applied Physics, Stanford). This paper, which builds upon another paper<\/a> delivered at last year\u2019s event, addresses a problem involving existing algorithms for network pruning, or the compression of neural networks by removing parameters (connections) between artificial neurons. Interest in pruning has grown due to its potential to save time, memory and energy during a neural network\u2019s training and test phases. One problem with this technique, however, is layer collapse, which occurs when an entire layer is pruned, making a network untrainable. The answer proposed here is a new algorithm, Iterative Synaptic Flow Pruning (SynFlow), in which successive iterations enable it to go farther and reach higher levels of compression without layer collapse. \u201cWe have characterized and provably solved a key failure mode of existing neural network pruning algorithms and have taken a step toward achieving more efficient deep learning models,\u201d said Hidenori Tanaka, a senior scientist in the NTT Research PHI Lab. A new work summarizing findings in the pruning paper by using theoretical tools from physics, titled \u201cNeural Mechanics: Symmetry and Conservation Laws in Learning Dynamics,\u201d will be presented at the NeurIPS 2020 Machine Learning and Physical Sciences workshop<\/a>.\n<\/li>\n<\/ul>\n
- \n
- \n\u201cMeta-learning from Tasks with Heterogeneous Attribute Spaces<\/a>,\u201d Tomoharu Iwata (NTT Communication Science Laboratories), and Atsutoshi Kumagai (NTT Software Innovation Center). Standard machine-learning methods train networks on a specific task. New tasks require another large amount of training data. One way around the related time and cost is \u201cfew-shot learning,\u201d a framework for learning from fewer examples. Existing models, however, assume that training and target tasks share the same attribute space, and tasks with heterogeneous attribute spaces to-date have assumed only two tasks and require target data sets for training. This paper introduces a new \u201cfew-shot\u201d learning method for tasks with heterogeneous attribute spaces. The model infers latent representations of each attribute and each response from a few labeled instances. Then responses of unlabeled instances are predicted with the inferred representations using a prediction network. \u201cBased on the knowledge learned from a wide-variety of training datasets, the proposed method can quickly adapt to and learn new tasks even when their attribute spaces are different from the training datasets,\u201d said Iwata, a distinguished researcher at NTT Communication Science Laboratories.\n<\/li>\n<\/ul>\n
- \n
- \n\u201cBaxter Permutation Process<\/a>,\u201d Masahiro Nakano, Akisato Kimura, Takeshi Yamada, Naonori Ueda (NTT Communication Science Laboratories). A Bayesian nonparametric (BNP) model for Baxter permutations (BPs), termed BP process (BPP), is proposed and applied to relational data analysis. \u201cWith our algorithm, we are able to find any combination of clusters with any sizes without any prior information, such as the number of clusters,\u201d said Masahiro Nakano, a researcher at NTT Communication Science Laboratories. \u201cOur achievement opens up new possibilities for applying Bayesian nonparametric machine learning, expected to be one of the next-gen machine learning technologies, to multi-dimensional \u2014 or even infinite-dimensional \u2014 data.\u201d This paper was selected as a Spotlight paper and will be presented during an online session on December 10.\n<\/li>\n<\/ul>\n
\n\u201cThere is no better place to explore the overlap between machine learning and computational neuroscience than the annual NeurIPS event,\u201d said Yoshihisa Yamamoto, PHI Lab Director. \u201cWe are excited to see the latest paper by Dr. Tanaka and his Stanford colleagues, as well as those by our colleagues at the NTT Software Innovation Center and NTT Communication Science Laboratories and expect the fields of neural networking and machine learning will benefit from the efficiencies and expanded capabilities that they are proposing.\u201d\n<\/p>\n
\nThis year\u2019s seven-day virtual NeurIPS event includes an expo, conference sessions, tutorials and workshops. The authors of these papers will participate in the event through poster and short recorded presentations. A follow-up to the \u201cPruning Neural Networks\u201d paper, as noted above, will be presented at one of the event\u2019s workshops. As an indication of the vitality of this sub-field of neuroscience, the event organizers noted a 40 percent year-over-year<\/a> increase in the number of submitted abstracts, similar to the growth from 2018 to 2019. Papers in the areas of algorithms, deep learning and applications comprised 66 percent of the papers that were reviewed. Among this year\u2019s keynote speakers<\/a> are Christopher Bishop, director of the Microsoft Research Lab in Cambridge, England; Shafi Goldwasser, Director of the Simons Institute for the Theory of Computing; and Marloes Maathuis, Professor of Statistics at ETH (the Swiss Federal Institute of Technology) in Zurich.\n<\/p>\n
About NTT Research<\/b><\/p>\n
\nNTT Research opened its Palo Alto offices in July 2019 as a new Silicon Valley startup to conduct basic research and advance technologies that promote positive change for humankind. Currently, three labs are housed at NTT Research: the Physics and Informatics (PHI) Lab, the Cryptography and Information Security (CIS) Lab, and the Medical and Health Informatics (MEI) Lab. The organization aims to upgrade reality in three areas: 1) quantum information, neuro-science and photonics; 2) cryptographic and information security; and 3) medical and health informatics. NTT Research is part of NTT, a global technology and business solutions provider with an annual R&D budget of $3.6 billion.\n<\/p>\n
\nNTT and the NTT logo are registered trademarks or trademarks of NIPPON TELEGRAPH AND TELEPHONE CORPORATION and\/or its affiliates. All other referenced product names are trademarks of their respective owners. \u00a9 2020 NIPPON TELEGRAPH AND TELEPHONE CORPORATION\n<\/p>\n
<\/p>\n
View source version on businesswire.com: <\/span>https:\/\/www.businesswire.com\/news\/home\/20201203005284\/en\/<\/a><\/span><\/p>\n
\nNTT Research:
\n
Chris Shaw
\n
Vice President, Global Marketing
\n
NTT Research
\n<\/a>
+1-312-888-5412
\n
chris.shaw@ntt-research.com
\n<\/a><\/p>\nMedia:
\n
Stephen Russell
\n
Wireside Communications<\/a>\u00ae
\n
For NTT Research
\n
+1-804-362-7484
\n
srussell@wireside.com<\/a><\/p>\nKEYWORDS:<\/b> United States North America California<\/p>\n
INDUSTRY KEYWORDS:<\/b> Software Research Networks Internet Hardware Data Management Technology Other Education University Education Science Other Technology Other Science<\/p>\n
- \n\u201cBaxter Permutation Process<\/a>,\u201d Masahiro Nakano, Akisato Kimura, Takeshi Yamada, Naonori Ueda (NTT Communication Science Laboratories). A Bayesian nonparametric (BNP) model for Baxter permutations (BPs), termed BP process (BPP), is proposed and applied to relational data analysis. \u201cWith our algorithm, we are able to find any combination of clusters with any sizes without any prior information, such as the number of clusters,\u201d said Masahiro Nakano, a researcher at NTT Communication Science Laboratories. \u201cOur achievement opens up new possibilities for applying Bayesian nonparametric machine learning, expected to be one of the next-gen machine learning technologies, to multi-dimensional \u2014 or even infinite-dimensional \u2014 data.\u201d This paper was selected as a Spotlight paper and will be presented during an online session on December 10.\n<\/li>\n<\/ul>\n
- \n\u201cMeta-learning from Tasks with Heterogeneous Attribute Spaces<\/a>,\u201d Tomoharu Iwata (NTT Communication Science Laboratories), and Atsutoshi Kumagai (NTT Software Innovation Center). Standard machine-learning methods train networks on a specific task. New tasks require another large amount of training data. One way around the related time and cost is \u201cfew-shot learning,\u201d a framework for learning from fewer examples. Existing models, however, assume that training and target tasks share the same attribute space, and tasks with heterogeneous attribute spaces to-date have assumed only two tasks and require target data sets for training. This paper introduces a new \u201cfew-shot\u201d learning method for tasks with heterogeneous attribute spaces. The model infers latent representations of each attribute and each response from a few labeled instances. Then responses of unlabeled instances are predicted with the inferred representations using a prediction network. \u201cBased on the knowledge learned from a wide-variety of training datasets, the proposed method can quickly adapt to and learn new tasks even when their attribute spaces are different from the training datasets,\u201d said Iwata, a distinguished researcher at NTT Communication Science Laboratories.\n<\/li>\n<\/ul>\n
![\"Logo\"](\"https:\/\/mms.businesswire.com\/media\/20201203005284\/en\/755976\/3\/NTTResearch_Symbol.jpg\")
<\/td>\n<\/tr>\n