Hello world! Last summer I wrote a short paper entitled "Entropy as a Topological Operad Derivation," which describes a small but interesting connection between information theory, abstract algebra, and topology. I blogged about it here in June 2021, and the paper was later published in an open-access journal called Entropy in September 2021. In short, it describes a correspondence between Shannon entropy and functions on topological simplices that obey a version of the Leibniz rule from calculus, which I call "derivations of the operad of topological simplices," hence the title.

By what do those words mean? And why is such a theorem interesting?

‍To help make the ideas more accessible, I've recently written a new article aimed at a wide audience to explain it all from the ground up. I'm very excited to share it with you! It's entitled "A New Perspective of Entropy," and a trailer video is below:

As mentioned in the video, the reader is not assumed to have prior familiarity with the words "information theory" or "abstract algebra" or "topology" or even "Shannon entropy." All these ideas are gently introduced from the ground up.

Today I am excited to share that the Math3ma platform has recently grown in a small yet personal way. This new endeavor is in its early stages, but it is one that is close to my heart and gives life to the reasons I started this blog six years ago. A more personal announcement can be found in a new article I wrote for the university, but I'd like to give an update here as well.

This semester I joined The Master's University (TMU), a small private Christian university in southern California, as a visiting research professor of mathematics. I am still a full-time research mathematician in the tech world, but I've also been collaborating part time with the math, science, and engineering faculty at TMU to launch a little research hub on the university's campus and online.

We are calling it The Math3ma Institute, and the website is now live: www.math3ma.institute.

What is The Math3ma Institute?

I've branded our little venture an "institute," though its function is notably different from that of other research institutions. Our vision is that The Math3ma Institute will grow into a place where TMU faculty and advanced undergraduates, along with external colleagues, can engage in research activities in various STEM fields—not just mathematics—and where those results will then be made accessible to a broad audience in easy-to-understand ways. This is summarized in the trifecta boldly displayed on our homepage: discover. share. repeat.

To this end, we are aiming to produce several publicly-available resources, including the launch of semi-annual journal described below.

I recently had the pleasure of chatting with Grant Sanderson on the 3Blue1Brown podcast about a variety of topics, including what first drew me to math and physics, my time in graduate school, thoughts on category theory, basketball, and lots more. We also chatted a bit about Math3ma and its origins, so I thought it'd be fun to share this "behind the scenes" peek with you all here on the blog. Enjoy!

Welcome to the final installment of our mini-series on the new preprint "An Enriched Category Theory of Language," joint work with John Terilla and Yiannis Vlassopoulos. In Part 2 of this series, we discussed a way to assign sets to expressions in language — words like "red" or "blue" – which served as a first approximation to the meanings of those expressions. Motivated by elementary logic, we then found ways to represent combinations of expressions — "red or blue" and "red and blue" and "red implies blue" — using basic constructions from category theory.

I like to think of Part 2 as a commercial advertising the benefits of a category theoretical approach to language, rather than a merely algebraic one. But as we observed in Part 1, algebraic structure is not all there is to language. There's also statistics! And far from being an afterthought, those statistics play an essential role as evidenced by today's large language models discussed in Part 0.

Happily, category theory already has an established set of tools that allow one to incorporate statistics in a way that's compatible with the considerations of logic discussed last time. In fact, the entire story outlined in Part 2 has a statistical analogue that can be repeated almost verbatim. In today's short post, I'll give lightning-quick summary.

It all begins with a small, yet crucial, twist.

Today I'd like to share a bit of math involving ideas from information theory, algebra, and topology. It's all in a new paper I've recently uploaded to the arXiv, whose abstract you can see on the right. The paper is short — just 11 pages! Even so, I thought it'd be nice to stroll through some of the surrounding mathematics here.

To introduce those ideas, let's start by thinking about the function $d\colon[0,1]\to\mathbb{R}$ defined by $d(x)=-x\log x$ when $x>0$ and $d(x)=0$ when $x=0$. Perhaps after getting out pencil and paper, it's easy to check that this function satisfies an equation that looks a lot like the product rule from Calculus:

Functions that satisfy an equation reminiscent of the "Leibniz rule," like this one, are called derivations, which invokes the familiar idea of a derivative. The nonzero term $-x\log x$ above may also look familiar to some of you. It's an expression that appears in the Shannon entropy of a probability distribution. A probability distribution on a finite set $\{1,\ldots,n\}$ for $n\geq 1$ is a sequence $p=(p_1,\ldots,p_n)$ of nonnegative real numbers satisfying $\sum_{i=1}^np_i=1$, and the Shannon entropy of $p$ is defined to be

Now it turns out that the function $d$ is nonlinear, which means we can't pull it out in front of the summation. In other words,  $H(p)\neq d(\sum_ip_i).$ Even so, curiosity might cause us to wonder about settings in which Shannon entropy is itself a derivation. One such setting is described in the paper above, which shows a correspondence between Shannon entropy and derivations of (wait for it...) topological simplices!