Welcome to Disarray

A year ago, I was dabbling in model-based reinforcement learning, testing out PPO and SHAC. While PPO struggled with almost everything we threw at it, SHAC was unexpectedly strong and remarkably stable-though it came with one big catch: it needs a fully differentiable environment, which is rarely easy to build. Me, clearly operating at the absolute peak of my deep-learning-researcher powers, naturally wondered: what if I just tossed a neural network at the problem? And, against all expectations-including mine-it sort of worked.

In this post, I introduce a wild idea I came up with a few weeks ago— an outer-loop optimization scheme inspired by the K-Level Policy Gradients paper. In short, the concept is that each layer’s updates should consider how other layers are being updated. This approach yields noticeable performance improvements, though at a significant computational cost. Since this is an early exploration, there’s still plenty of room for refinement.

In this post, I will provide a simple tutorial on how to use PyTorch's distributed package. We will cover the basics of setting up a distributed environment, initializing the process group and up until a multi-node multi-GPU inference example. Bear in mind that I am no expert on this topic, so if you find any mistakes or have suggestions for improvement, please let me know!

In one of the previous posts, we have seen how tying embeddings can be destabilize the training if the the data do not satisfy certain assumptions (see here). In this post, we will explore a simple idea to get the best of both worlds: early training boost with tied embeddings and late training stability with untied one. This was a research idea that I had in mind however it did not work as well as expected so I decided to share it here.

This is the second part of our series on chaos theory. In this post, we will introduce the concept of differentiable manifolds and tangent spaces. Using these, we will explore the differentiation of maps between these objects. Finally, we will bring everything together with a simple example.

A brief introduction to chaos theory from a differential topolody perspecitive from a guy that is studying these topics for fun. I will share some of the things I learned from Alligood et al. (1998). This is also an attempt to understand the basics of differential topology. Here we discuss basics of differential topology and basic definitions of dynamical systems.

This summer, I was lucky enough to being accepted to ICML 2024 as spotlight poster. I told this story on reddit (you can see the original post here). However, now that I have a blog, I thought it would be nice to keep it here as well. Well, enough with the introduction, let's get to the story behind the paper titled By tying embedding you are assuming the distributional hypothesis.

For anyone who read through the toy models of superposition. This is mostly a rehash of the same ideas presented in the third section of Elhage et al. 's work. However, to keep things new, I will introduce a regularization term to the loss function. This will allow us to explore the phase diagram in a novel scenario. This analysis suggests that regularization inhibits superposition.

In this post, I will discuss T-Free. A recent paper from aleph-alpha that introduces a new method for learning embedding representations that promises to be both memory and computationally efficient. Of course this post is based from their original paper. However, I will share some of my insights and thoughts on their work.

In this post, I will explore how to find the population minimizer for the conditional risk associated with Categorical Cross Entropy. We will use only basic fundamental probability principles in order to make the derivation accessible to anyone with a basic understanding of probability. The only analytical tool needed are the Lagrange multipliers for constrained optimization problems.