基于第一性原理推导出的白盒子深度网络 ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reduction

SoftwareTeacher

《编程之美》作者

领域专家: 产品设计技术领域

2022-05-09 12:49:26

（从马毅老师的微博转载）
基于第一性原理推导出的白盒子深度网络《ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reduction》一百多页的论文，终于在Journal of Machine Learning Research（JMLR）上正式发表了：网页链接。这篇论文从它的题目看就备受关注。文章主体投稿到去年在ICML‘21，引起不小争议：它在四个reviewers都一致同意接受的前提下被area chair给无端拒稿了。媒体都有报道。随后，它又几乎以最快速度被JMLR审稿和接受。当时，大家可能对这个新的框架的实用效果如何，存有疑义。我们目的完全在推导严密的数学原理，根本没有不太管什么SOTA不SOTA，即使在这个正式版本里，我们也没有放最好的调参结果。不过随后一系列的发展（包括CTRL等等）证明，正是这项工作，让我们找到了深度网络的目的和来源。现在看来，应该可以把分类与生成模型学习，把监督学习、无监督学习、以及连续学习都作为特例严格统一在同一组基本原理、同一个计算框架下。再没有什么Heuristics 和 Engineering Tricks。NeurIPS后应该会透露更多新进展（包括应用和理论方面）。我近一两年来关于这个题目也应邀做了将近50场次学术报告，包括上周在FAIR Yann LeCun组以及这个月底在ICASSP的大会开场报告。喜欢玩黑盒子的同学，可以继续掷骰子碰运气；只是我们要在原理引导的正确方向上，越走越远、越做越快了。

...全文

1036 1 打赏收藏转发到动态举报

写回复

1 条回复

切换为时间正序

请发表友善的回复…

发表回复

CSDN-Ada助手 2023-01-13

打赏
举报

回复

您可以前往 CSDN问答-人工智能发布问题, 以便更快地解决您的疑问

OpenGL® ES™ is the industry’s leading software interface and graphics library for rendering sophisticated 3D graphics on handheld and embedded devices. The newest version, OpenGL ES 3.0, makes it possible to create stunning visuals for new games and apps, without compromising device performance or battery life. In the OpenGL ES 3.0 Programming Guide, 2nd Edition, the authors cover the entire API and Shading Language. They carefully introduce OpenGL ES 3.0 features such as shadow mapping, instancing, multiple render targets, uniform buffer objects, texture compression, program binaries, and transform feedback. Through detailed, downloadable C-based code examples, you’ll learn how to set up and program every aspect of the graphics pipeline. Step by step, you’ll move from introductory techniques all the way to advanced per-pixel lighting and particle systems. Throughout, you’ll find cutting-edge tips for optimizing performance, maximizing efficiency with both the API and hardware, and fully leveraging OpenGL ES 3.0 in a wide spectrum of applications. All code has been built and tested on iOS 7, Android 4.3, Windows (OpenGL ES 3.0 Emulation), and Ubuntu Linux, and the authors demonstrate how to build OpenGL ES code for each platform. Coverage includes EGL API: communicating with the native windowing system, choosing configurations, and creating rendering contexts and surfaces Shaders: creating and attaching shader objects; compiling shaders; checking for compile errors; creating, linking, and querying program objects; and using source shaders and program binaries OpenGL ES Shading Language: variables, types, constructors, structures, arrays, attributes, uniform blocks, I/O variables, precision qualifiers, and invariance Geometry, vertices, and primitives: inputting geometry into the pipeline, and assembling it into primitives 2D/3D, Cubemap, Array texturing: creation, loading, and rendering; texture wrap modes, filtering, and formats; compressed textures, sampler objects, immutable textures, pixel unpack buffer objects, and mipmapping Fragment shaders: multitexturing, fog, alpha test, and user clip planes Fragment operations: scissor, stencil, and depth tests; multisampling, blending, and dithering Framebuffer objects: rendering to offscreen surfaces for advanced effects Advanced rendering: per-pixel lighting, environment mapping, particle systems, image post-processing, procedural textures, shadow mapping, terrain, and projective texturing Sync objects and fences: synchronizing within host application and GPU execution This edition of the book includes a color insert of the OpenGL ES 3.0 API and OpenGL ES Shading Language 3.0 Reference Cards created by Khronos. The reference cards contain a complete list of all of the functions in OpenGL ES 3.0 along with all of the types, operators, qualifiers, built-ins, and functions in the OpenGL ES Shading Language. Table of Contents Chapter 1. Introduction to OpenGL ES 3.0 Chapter 2. Hello Triangle: An OpenGL ES 3.0 Example Chapter 3. An Introduction to EGL Chapter 4. Shaders and Programs Chapter 5. OpenGL ES Shading Language Chapter 6. Vertex Attributes, Vertex Arrays, and Buffer Objects Chapter 7. Primitive Assembly and Rasterization Chapter 8. Vertex Shaders Chapter 9. Texturing Chapter 10. Fragment Shaders Chapter 11. Fragment Operations Chapter 12. Framebuffer Objects Chapter 13. Sync Objects and Fences Chapter 14. Advanced Programming with OpenGL ES 3.0 Chapter 15. State Queries Chapter 16. OpenGL ES Platforms Appendix A. GL_HALF_FLOAT Appendix B. Built-In Functions Appendix C. ES Framework API Book Details Title: OpenGL ES 3.0 Programming Guide, 2nd Edition Author: Aaftab Munshi, Budirijanto Purnomo, Dan Ginsburg, Dave Shreiner Length: 560 pages Edition: 2 Language: English Publisher: Addison-Wesley Professional Publication Date: 2014-03-10 ISBN-10: 0321933885 ISBN-13: 9780321933881

最近在学习马毅老师的 NeurIPS 2020 中稿文章 MCR2，当时引起很多的报告，并且也上了热搜。抱着看一看的态度，认真学习了几篇核心的文章，并进行理解。作为 ReduNet: A White-box Deep Network from the Principle 的前置文章之一的，Deep Networks from Optimizing Rate Reduction，首先对其进行了学习。文章考虑的问题是针对分类问题，如何能够使得分类更加准确。使用的方法考虑到了数据的压缩与表示，非常类似对比学

鱼羊丰色发自凹非寺量子位报道 | 公众号 QbitAI4位评审一致接收，按理说这篇顶会论文就没跑了。可就在这两天，UC伯克利的马毅教授却遇上了一件“奇事”：课题组投稿ICML 2...

今天分享另一篇利用CLIP预训练模型进行聚类的论文，首先对于这个题目的THE PRINCIPLE OF RATE REDUCTION我是比较困惑的，我之前并没有接触过，但没关系，原文中会解释，文章中提到用the principle of Maximal Coding Rate Reduction来学习特征表示，这个原则叫做最大编码率衰减原则，详见出处。如果有对预训练模型感兴趣的，也可以看看我分享的上一篇来自CVPR的论文，指路。

Abstract Learning sophisticated feature interactions behind user behaviors is critical in maximizing CTR for recommender systems. Despite great progress, existing methods seem to have a strong bias to...

High-Dimensional DA

215

社区成员

37

社区内容

发帖

与我相关

我的任务

社区管理员

加入社区

近7日
近30日
至今

加载中

查看更多榜单

社区公告

暂无公告

试试用AI创作助手写篇文章吧

+ 用AI写文章