Tag: cs

Undefined Behavior

Recently we’ve heard a few people imply that problems stemming from undefined behaviors (UB) in C and C++ are largely solved due to ubiquitous availability of dynamic checking tools such as ASan, UBSan, MSan, and TSan. We are here to state the obvious — that, despite the many excellent advances in tooling over the last few years, UB-related problems are far from solved — and to look at the current situation in detail.

IEEE floats are broken

IEEE floats are broken. Posits beat floats at their own game:
• superior accuracy, dynamic range, closure
• Bitwise-reproducible answers (at last!)
• Proven better answers with same number of bits
• …or, equally good answers with fewer bits
• Simpler, more elegant design can reduce silicon cost, energy, and latency.

Enhancing color vision

We present an approach that can enhance human color vision by breaking the inherent redundancy in binocular vision, providing different spectral content to each eye. This technique represents a significant enhancement of the spectral perception of typical humans, and may have applications ranging from camouflage detection and anti-counterfeiting to art and data visualization.

Outrageously Large Neural Networks

We finally realize the promise of conditional computation, achieving greater than 1000x improvements in model capacity with only minor losses in computational efficiency on modern GPU clusters. We introduce a Sparsely-Gated Mixture-of-Experts layer (MoE), consisting of up to 1000s of feed-forward sub-networks. We present model architectures in which a MoE with up to 137B parameters is applied convolutionally between stacked LSTM layers. On large language modeling and machine translation benchmarks, these models achieve significantly better results than state-of-the-art at lower computational cost.

Leapfrogging Moore’s Law

fun idea: save compute power by selectively using lower precision math, then reinvest the savings:

We run the algorithm in double precision to a given error bound and measure energy consumption. This is our energy budget. We next run the algorithm in single precision for a number of iterations, followed by double precision for a number of iterations, consuming the same energy as before, and measure the error bound. The ratio between the first error bound and the second is the achieved improvement factor. For Laplace, we were able to achieve an improvement factor of 10^4, for Rosenbrock,10^8

Wycheproof

In cryptography, subtle mistakes can have catastrophic consequences, and mistakes in open source cryptographic software libraries repeat too often and remain undiscovered for too long. Good implementation guidelines, however, are hard to come by: understanding how to implement cryptography securely requires digesting decades’ worth of academic literature. We recognize that software engineers fix and prevent bugs with unit testing, and we found that many cryptographic issues can be resolved by the same means. These observations have prompted us to develop Project Wycheproof, a collection of unit tests that detect known weaknesses or check for expected behaviors of some cryptographic algorithm. Our cryptographers have surveyed the literature and implemented most known attacks. As a result, Project Wycheproof provides tests for most cryptographic algorithms, including RSA, elliptic curve crypto, and authenticated encryption.

100x smaller Static Checkers

Modern static bug finding tools are complex. They typically consist of 100s of 1000s of lines of code, and most of them are wedded to 1 language (or even 1 compiler). This complexity makes the systems hard to understand, hard to debug, and hard to retarget to new languages, thereby dramatically limiting their scope. This paper reduces the complexity of the checking system by addressing a fundamental assumption, the assumption that checkers must depend on a full-blown language specification and compiler front end. Instead, our program checkers are based on drastically incomplete language grammars (“micro-grammars”) that describe only portions of a language relevant to a checker. As a result, our implementation is tiny— 2500 lines of code, 100x smaller than a typical system. We hope that this dramatic increase in simplicity will allow developers to use more checkers on more systems in more languages. We implement our approach in µchex, a language-agnostic framework for writing static bug checkers. We use it to build micro-grammar based checkers for 6 languages (C, the C preprocessor, C++, Java, JavaScript, and Dart) and find over 700 errors in real-world projects.