There is no vaccine for climate change

As I sat waiting the requisite 15 minutes to make sure I didn’t go into anaphylactic shock, I looked out over the arena and reflected on the historic nature of that moment. A building designed for basketball, concerts, and large-scale events sat empty for nearly a year because gatherings were a threat to public health. It had then been repurposed into a makeshift medical facility where vaccines were being administered on a mass scale. There’s something dark — and decidedly not normal — about a space designed for fun being used as a medical facility.

UIC arena sits empty as people get vaccines on the upper level

I felt grateful for the incredible work of the many brilliant and hard-working people both before and during the pandemic who allowed us to reach that moment. But fear of what this medical marvel might symbolize was also on my mind. We had all been waiting for medicine to end the pandemic, any too many people had been ignoring epidemiology’s “inconvenient” non-medical interventions like social distancing and mask wearing. We had been passively waiting for science to save us with vaccines, and this time we got lucky: science delivered.

This technological solutionism, waiting for a technological savior instead of making sacrifices, is at play in climate change, too. I am absolutely thrilled that mRNA vaccine technology was practically ready and waiting to be applied to SARS-CoV-2 in record time, but it scares me that it reinforces a solutionist attitude: “See? Science saved us from the pandemic, so it’ll also save us from climate change!”

There is no vaccine for climate change. We’ll need science to get us out of this, yes, but also political will. Political will to reign in corporations. Political will to fund science that can get us even partway there. Political will to do things that hurt in the short term before the status quo does even more damage to more people. Had we heeded epidemiologists’ advice on COVID, millions of lives around the world could have been saved. Let’s not make the same mistake with climate change, squandering the remaining time we have while waiting for a scientific miracle.

Encode Mighty Things

On February 22, 2021, NASA landed the Perseverance rover on Mars. Encoded in the parachute used for landing was the JPL motto and Teddy Roosevelt quote, “Dare mighty things”. Despite being a secret before landing, the internet quickly decoded the message.

Perseverance parachute

I thought that was pretty cool, so decided to make a little site that lets you make your own parachute using the same encoding. Other people have explained the encoding better than I could, so without further adieu, here is Encode Mighty Things.

Encode Mighty Things

Telephone colophon: Or, how I overengineered my call audio

Toward the beginning of the pandemic, a friend asked me how she could use an external vocal mic and a guitar with a pickup on Zoom calls. Sounds easy, right?

But to have the amount of control a musician really wants, it turned out to be a bit more involved. Plus, when working from home for a microphone company, it’s pretty common to use a decent mic in meetings.

This post explains the setup I’ve been using for my calls.

desk with mic, headphones, and speakers
(Don’t let the speakers fool you. Use headphones or it’ll feed back when echo cancellation is turned off!)

Ingredients:

The goals are:

  1. Mix the mic or other inputs going into the USB interface in the DAW
  2. Be able to hear/monitor the mix
  3. Route the output of the DAW to a Zoom call
  4. Be able to hear the far end of the call through the same headphones as monitoring the mix

Get Blackhole

The key ingredient here is BlackHole, a virtual audio driver that acts as a passthrough from each input to the corresponding output1. This actually needs two instances of BlackHole because Zoom can only send and receive from the first two channels of any audio interface. Fortunately, they offer direct downloads (email required) of each (and have nice instructions for building from source). I have one called BlackHole 16ch and one called BlackHole 2ch, which — surprise — have 16 channels and 2 channels, respectively.

The 16-channel BlackHole device will function as the Zoom speaker; the 2-channel BlackHole will be the Zoom “microphone”.

Set up an aggregate audio device

Reaper will handle all of the audio routing, but since it doesn’t support having different input and output devices, the first thing to do is create an aggregate device in Audio MIDI Setup. This allows the system to treat multiple devices as a single device with all of the channels from the individual devices. It doesn’t really matter what order you add them to the aggregate device, but it should include both BlackHole devices and the audio interface. I have the USB interface set to be the clock source, with the two BlackHole instances set for drift correction.

audio midi setup screenshot

Route and mix in the DAW

Once I got the audio devices set up, I had to route everything in Reaper. The general approach is:

  • The master out is going to Zoom, and my ears for monitoring
    In other words, (almost) every track in the DAW is “normal” in the sense that what I hear is what the far end of the call hears
  • The output of Zoom is not going to the master out, so it doesn’t feed back

But before doing that, make sure Reaper is set to use the aggregate device in the device preferences.

Reaper device preferences

For every input I want to mix, I created a track. Selecting the input for that track feels almost like just using the regular USB audio interface, but with a whole bunch of other channels thanks to being aggregated with BlackHole.

Reaper input selection

By default, Reaper sends each track to the master out, but in order to hear live input, you have to arm the track and turn on record monitoring.

A fun bonus of routing through a DAW is that you can use plugins! I use a simple NR plugin to deal with HVAC noise, and some compression.

The master out needs to go two places:

  • The USB interface so you can hear in your headphones
  • BlackHole, to get it into Zoom

So, from the master track’s routing window, add outputs to the USB interface and the two channels of the 2-channel BlackHole interface. The fader/mute button for the USB interface on the output routing of the master is how I adjust whether/how much of myself I want to monitor in my ears.

master output routing

That’s it for everything I want to send to Zoom, but I still want to be able to hear the far end of the call. I could just tell Zoom to send out to the hardware interface, but I want it in the DAW, too. This is useful for recording a tape sync, and so you don’t have to mess up your monitoring volume to change the volume of the far end.

For that, I created a special track and set its input to the 16–channel BlackHole instance. When you set up Zoom to use a particular output device, it sends the audio to the first two channels, so I had to use channels 1 and 2. Here’s where the track becomes special: you have to make sure it doesn’t send to the master out (it’ll feed back if you do). Instead, send it directly to the USB interface’s out.

zoom monitor routing

And that’s it for the DAW.

reaper's mixer

Set up Zoom

The basics of setting up Zoom are simple: it receives the master output of the DAW by setting its microphone to BlackHole 2ch, and setting the speaker to BlackHole 16ch sends the far end’s audio to the DAW on channels 1 and 2 of BlackHole 16ch. Since you can control the output level from the DAW, I maxed out Zoom’s output and input faders and turned off the automatic gain control.

zoom audio settings

That’s really all you need for the basics, but Zoom has a bunch of cool advanced audio settings. Under “Music and Professional Audio” you can tell Zoom to let you turn off all of its audio processing, sending “original sound”. This is great, because what’s the point of having a decent mic if Zoom is going to band-limit and compress it to death? You can also turn on stereo, but I only use that if I really need to, which is rare. (Keep in mind that in order to actually activate these settings, you have to press “Turn on original sound” in the upper left of a call.)

zoom advanced audio settings

Bonus! Sharing system sound

Zoom can share system sound, but when using a setup like this, I don’t recommend it. Turning it on activates some sort of additional virtual audio device on the system, which can mess with things. Remember that Zoom can only send audio out on the first two channels of a device. Thankfully, the system isn’t so limited. To share system sound, I went back to Audio MIDI Setup and under Configure Speakers told it that for stereo out, BlackHole 16ch uses channels 3 and 4.

Audio MIDI Setup speaker config

Now I can set my system output device to BlackHole 16ch, make a new track in the DAW, set its input to BlackHole 16ch, and system sound comes in there. So the far side of a Zoom call comes in on channels 1 and 2, and system sound on 3 and 4.

And that’s it. Happy calling!

  1. I used BlackHole because it’s free and did what I needed. You can achieve the same thing with a nice UI using Loopback from the excellent Rogue Amoeba. 

Entropic Timer: The Information Entropy of Crossing the Street

You know those countdown timers at crosswalks? Sometimes when crossing the street, I like to try to guess what number it’s on even when I can’t see the whole thing (like when approaching the intersection at an oblique angle).

Crosswalk signal countdown timer

This got me (over)thinking: if I want to know how much time is left, is it better to see the right side of the countdown timer (approaching from the left), or the left side (approaching from the right)? In other words, does the left or right side of the display carry more information?

These timers use seven-segment displays. Even if you didn’t know they were called seven-segment displays, you see them all over the place. They use seven separate segments, labeled A–G, to create each of the 10 digits from 0–9.

A
B
C
D
E
F
G

To form each of the ten digits, the seven segments are turned on (1) or off (0) in different combinations. Here are the standard representations of 0–9.

ABCDEFG
1111110
0110000
1101101
1111001
0110011
1011011
1011111
1110000
1111111
1111011

The seven segments aren’t on all turned on an equal number of times over the course of the ten digits. That means seeing some segments turned on is more probable than others.

.8
.8
.9
.7
.4
.6
.7
On for how many digits?
Segment A8/10
Segment B8/10
Segment C9/10
Segment D7/10
Segment E4/10
Segment F6/10
Segment G7/10

So how can we tell which of these seven segments communicates the most information?

Information entropy

The segments that are on or off for close to half the digits contain more information than those that are either on or off for most digits. This is intuitive for the same reason a fair coin toss contains more information than tossing a coin with heads on both sides: you’re less certain what you’re going to get, so learn more by observing the value.

Claude Shannon’s1 concept of entropy from information theory is a good way to quantify this problem. Entropy, \(H\), is defined as

\[H(X) = -\sum_{i = 1}^{n} P(x_i)\log_bP(x_i)\]

Oh no.

Here’s what’s that means in the case of a seven-segment display. \(X\) is a random variable representing whether a segment is on or off. Since a segment can only have two states, the random variable \(X\)’s actual values are either on or off. \(P\) is the probability operator, so \(P(x_i)\) really means the probability that a segment is on or off. (\(b\) is the base of the logarithm. We’re going to use 2 because we like bits.)

Let’s take segment A as an example. It’s on for 8 out of 10 digits, and off for 2 out of 10. That means the probability of seeing it on is 0.8, and the probability of seeing it off is 0.2. In other words (well, symbols), \(P(x_{\mathrm{on}}) = 0.8\) and \(P(x_{\mathrm{off}}) = 0.2\).

Plugging that in,

\[H(A) = -0.8 \log_2 0.8 - 0.2 \log_2 0.2 = 0.722\]

In Shannon’s terms, there are 0.722 bits of information communicated by segment A of a seven-segment display.

Doing this for all seven segments, we get these entropy values:

.72
.72
.47
.88
.97
.97
.88
Shannon entropy
Segment A0.721928
Segment B0.721928
Segment C0.468996
Segment D0.881291
Segment E0.970951
Segment F0.970951
Segment G0.881291

It sure looks like segments E and F carry the most information. That makes sense because they’re the closest to being on/off 50% of the time. Guess it’s better to approach an intersection from the right in order to see the left-hand segments.

But wait.

When approaching an intersection, you can see both right segments (B and C), or both left segments (E and F). A pair of segments from a single display are anything but independent because they’re both showing part of the same digit, so we can’t just add up their entropies.

Instead, treat each pair as if it holds a single value. Taken together, two segments can take on any of four values (off–off, off–on, on–off, on–on), which is binary for 0–3.

Segments B & CBinaryDecimal
On – On 113
On – On 113
On – Off102
On – On 113
On – On 113
Off – On011
Off – On011
On – On 113
On – On 113
On – On 113
Segments E & FBinaryDecimal
On – On 113
Off – Off 000
On – Off102
Off – Off 000
Off – On 011
Off – On011
On – On113
Off – Off 000
On – On 113
Off – On 011

In this case, our random variable \(X\) can take on four possible values rather than just two. Taking segments E and F as an example, the joint value is 0 for 3/10 digits, 1 for 3/10 digits, 2 for 1/10 digits, and 3 for 3/10 digits. Going back to the initial definition of entropy, we get

\[H(EF) = -\tfrac{3}{10}\log_2 \tfrac{3}{10} - \tfrac{3}{10}\log_2 \tfrac{3}{10} - .1\log_2.1 - \tfrac{3}{10}\log_2 \tfrac{3}{10} = 1.90\]

So we get 1.16 bits of information in joint segments B–C, and 1.90 bits in joint segments E–F. So there you have it: it’s still better to approach an intersection from the right.

But wait!

When was the last time you walked up to an intersection and only saw the timer on one number? If you look for at least half a second (on average), you’ll see it tick down.

A
B
C
D
E
F
G

Luckily, Wikipedia says that

For a first-order Markov source (one in which the probability of selecting a character is dependent only on the immediately preceding character), the entropy rate is:

\[H(\mathcal{S}) = -\sum_i p_i\sum_jp_i(j)\log p_i(j)\]

where \(i\) is a state (certain preceding characters) and \(p_{i}(j)\) is the probability of \(j\) given \(i\) as the previous character.

But actually, I don’t like this notation, so I’m going to rewrite it as

\[H(\mathcal{S}) = -\sum_i P(x_i)\sum_j P(x_j|x_i)\log_b P(x_j|x_i)\]

Alright, then. The probability of seeing a given state is the same as before. As for the conditional probabilities, let’s go back to the 0–3 binary values and assume 0 loops back to 92. If we see segments B and C in a 1 state (off–on), the next tick it will be in a 1 state half the time, and a 3 state half the time. Going through the rest of the states and transitions, we get these transition probabilities:

State transition probabilities

So for segments E and F, when \(i = 0\) and \(j = 2\), \(P(x_i) = \frac{3}{10}\) as with before, and \(P(x_j|x_i) = \frac{1}{3}\) because, as those circles show, a 0 transitions to a 2 a third of the time.

Now it’s just a matter of an inelegant nested for loop to determine that the first-order entropy rate of segments B–C is 1.00 bits, and 1.03 bits for segments E–F.

So, if you can manage to stare at either the left or right segments for a whole second, you’re still better off looking at the left segments, but not by much.

I’ll leave figuring out the entropy rates for looking at it longer as an exercise for the reader, because I’m done overthinking this (for now).


The 7-segment display CSS is on CodePen.

  1. Shannon and I both got undergrad degrees in EE from the University of Michigan, but he went on to create information theory, and I went on to write this stupid blog post. 

  2. This makes sense for the 1s place for segments B–C, but not for E–F. 

Is yontef early this year (dot com)

If two points (or posts) make a trend, interactive data visualizations of the Hebrew calendar are a thing I blog about now. In the long and storied tradition of single-use novelty sites, I’ve created isyontefearlythisyear.com (and its evil twin, isyonteflatethisyear.com). Now you can point to real data when the conversation inevitably comes up before every holiday.

screenshot showing an early Chanukah 2018

When is Passover (or Easter)?

A few weeks ago, @iamreddave tweeted a plot of Easter dates since 1600. I thought it was a very cool looking pattern, with very clear cyclicality.

Being Jewish, I immediately thought of the calendrical connection between Easter and Passover. Specifically, since Easter is usually around Passover, does the 19-year cycles of Hebrew leap years play a role in when Easter falls?

Very briefly (and approximately), a solar year is aligned with the seasons (because a year is one orbit of the earth around the sun), but the Hebrew calendar is based on a lunar calendar in which a month is determined by one cycle through the phases of the moon. The solar year is approximately 365 days, while 12 lunar months are approximately 354 days, or 11 days shorter. If the Hebrew calendar were a pure lunar calendar, over time the months would drift around the year. To make up for this shortfall, a 30-day leap month is added to the Hebrew calendar every two to three years, seven times in a 19-year cycle (years 3, 6, 8, 11, 14, 17, and 19). (30 days × 7 years ≈ 11 days × 19 years. Hey, I said this explanation is approximate.)

To see the effect of Hebrew leap years on Easter dates, I recreated iamreddave’s graph, but with larger points for leap years and points colored by position in the 19-year cycle.

Interact with these graphs at https://projects.noahliebman.net/pesach-easter/

Easter dates

What jumps out to me is that all of the late Easter dates are Hebrew leap years, which is what you’d expect when an additional month has recently been inserted, but all of the early Easter dates are also Hebrew leap years.

Passover, on the other hand, always occurs late in a leap year, as you’d expect:

Passover dates

Toggling between the two, it looks like it’s years with the latest Passovers that get leap-year–early Easters.

Animating between Easter and Passover

Zoom in a bit and you’ll find that the early Easter dates are always years 8, 11, and 19 of the 19-year cycle:

Easter, zoomed in on about 20 years

I thought maybe this happens because the Christian 19-year cycle is shifted by three years from the Jewish cycle (2014 was the first year of the Christian cycle, while 2017/5777 is the first year of the Jewish cycle), but this isn’t the case. Here’s what seems to be happening:

Easter is (by definition) the first Sunday after the full moon after the vernal (in the northern hemisphere) equinox. Typically, that’s the full moon of Nissan (the Hebrew month which contains Passover), but in those three years the leap month pushes Passover so late that it’s a full month later than the equinox. In other words, in those years the new moon that marks the start of Nissan is at least ~14 days after the equinox, which puts a full moon very shortly after the equinox, which is still in Adar II (the month before Nissan).

Shout out to the Hebcal team for their amazing tools!

Interact with these graphs at https://projects.noahliebman.net/pesach-easter/

On fear, nationalism, and oppression in Shmot

With parshat Shmot coinciding with the inauguration (err, Put-in) of Donald Trump, this image from Yossi Fendel has been making the rounds on social media. It quotes the eighth verse of the parsha (and book):

וַיָּ֥קָם מֶֽלֶךְ־חָדָ֖שׁ עַל־מִצְרָ֑יִם אֲשֶׁ֥ר לֹֽא־יָדַ֖ע [אֶת־יוֹסֵֽף]׃

A new king arose over Egypt who did not know [Joseph].

A new king arose who did not know. Image by Yossi Fendel

It’s an ominous image, and makes an important point, but it’s the next couple of sentences that have really stuck me for the last several years:

וַיֹּ֖אמֶר אֶל־עַמּ֑וֹ הִנֵּ֗ה עַ֚ם בְּנֵ֣י יִשְׂרָאֵ֔ל רַ֥ב וְעָצ֖וּם מִמֶּֽנּוּ׃

And he said to his people, “Look, the Israelite people are much too numerous for us.

הָ֥בָה נִֽתְחַכְּמָ֖ה ל֑וֹ פֶּן־יִרְבֶּ֗ה וְהָיָ֞ה כִּֽי־תִקְרֶ֤אנָה מִלְחָמָה֙ וְנוֹסַ֤ף גַּם־הוּא֙ עַל־שֹׂ֣נְאֵ֔ינוּ וְנִלְחַם־בָּ֖נוּ וְעָלָ֥ה מִן־הָאָֽרֶץ׃

Let us deal shrewdly with them, so that they may not increase; otherwise in the event of war they may join our enemies in fighting against us and rise from the ground.”

The liturgy talks a lot about the exodus from Egypt, but focuses far less on why the Israelites became enslaved in the first place. The answer, this parsha makes clear, is fear. Fear of shifting demographics. Fear of an ethnic group that looked different, spoke differently, and had different practices and customs — yet served an important economic function by doing the job no Egyptian was willing to do.

Faced with that fear from shifting demographics, the Pharaoh had at least a couple of courses of action. He could have pushed an agenda of multiculturalism, encouraging the Egyptians and Israelites to get to know one another, thereby mitigating their fear. Instead, he felt that it was more important to maintain what he considered the fundamentally Egyptian character of Egypt.

The United States — at least in theory — was founded not as “a place for a people”, but as a place for all people. Sadly, there are people who believe that America was a white country (back when it was great or something 🙄), and they are now feeling the same fear and oppressive urges the biblical Pharaoh felt.

This is precisely the danger that comes along with ethnic, racial, or religious nationalism. A nation founded as “a place for a people” cannot simultaneously offer full and equal rights/privileges to all, and continue to exist should that people become a minority. And the only ways to maintain the “desired” demographics are exclusion and oppression. Whether it’s in the context of Trump-emboldened white nationalism here in America, or Zionism, its moral equivalent, let’s learn from this week’s well-timed parsha: national ideals that depend on maintaining certain demographics are inherently oppressive.

In a place like America, although changing demographics can bring up a natural fear of the stranger, it also provides us with an opportunity to not be like Pharaoh and to strive for a multicultural ideal. The Torah reminds readers that, because the Israelites were strangers in Egypt, not only is one forbidden to oppress the stranger [1, 2], but it explains how: by loving that stranger [3]. But loving the stranger is abstract. Perhaps it’s better to take a cue from the JPS translation and befriend the stranger. Friends are way less scary than strangers.

Some thoughts on the Electoral College

I just read Madison’s Federalist Paper #10. Very interesting stuff. At a high level, the purpose of electors is to mitigate the effect of “factions”, which he defines (all emphasis in block quotes is mine):

By a faction, I understand a number of citizens, whether amounting to a majority or a minority of the whole, who are united and actuated by some common impulse of passion, or of interest, adverse to the rights of other citizens, or to the permanent and aggregate interests of the community.

A p’shat interpretation though a contemporary lens would seem to be a strong argument in favor of the electors being unfaithful and voting against Trump. After all, he explicitly threatened the rights of several groups of citizens, and his authoritarian tendencies pose a threat to the “aggregate interests of the community.”

Indeed, it is common to say that the purpose of the Electoral College is to protect the public good from the irresponsible or uneducated will of the people, and that’s also true:

The effect of [a Republic], on the one hand, to refine and enlarge the public views, by passing them through the medium of a chosen body of citizens, whose wisdom may best discern the true interest of their country, and whose patriotism and love of justice will be least likely to sacrifice it to temporary or partial considerations. Under such a regulation, it may well happen, that the public voice, pronounced by the representatives of the People, will be more consonant to the public good, than if pronounced by the People themselves, convened for the purpose.

However, Madison’s actual concern, it seems, is that non–land-owning voters1 would overwhelm the landed class. He even explicitly calls out “an equal division of property” as exactly the type of “wicked project” a representative republic can protect against.

But the most common and durable source of factions has been the various and unequal distribution of property. Those who hold, and those who are without property, have ever formed distinct interests in society. Those who are creditors, and those who are debtors, fall under a like discrimination. A landed interest, a manufacturing interest, a mercantile interest, a moneyed interest, with many lesser interests, grow up of necessity in civilized nations, and divide them into different classes, actuated by different sentiments and views. The regulation of these various and interfering interests forms the principal task of modern Legislation, and involves the spirit of party and faction in the necessary and ordinary operations of the Government.

It’s also impossible to ignore the effects of media and technology. We are hardly a united country, but the divisions depend on sociological environment (racial, religious, and ethnic diversity, wealth, rural–urban, etc.), not proximity.

The influence of factious leaders may kindle a flame within their particular States, but will be unable to spread a general conflagration through the other States: A religious sect may degenerate into a political faction in a part of the Confederacy; but the variety of sects dispersed over the entire face of it, must secure the National Councils against any danger from that source.

A conflagration can now easily spread across the continent.

Here in 2016 we have a situation where the Electoral College is about to vote for a candidate who is “adverse to the rights of other citizens, or to the permanent and aggregate interests of the community” when they are supposed to be the ones “whose wisdom may best discern the true interest of their country.” Therefore, it is easy to argue that they should vote counter to the will of the voters in their states. On the other hand, had Bernie Sanders been elected (if only!), someone reading the very same document could argue that the citizens whose rights are being infringed upon are the wealthy 1% whose property would be at risk of “[more] equal division”.

My take is that the threats to the Republic in the face of a Trump presidency are sufficient enough, and the adverse effects on the rights of citizens substantial enough, that the electors should vote for Hillary Clinton. The argument that a more left-leaning economic policy would infringe on the right of the 1% to hold their wealth breaks down because the effect would not be sufficiently “adverse”, and a better-functioning, more equitable economy is in “the true interest of their country.”

I’m sure there are other historical arguing for and against the Electoral College, but based on this one, I believe the electors should elect Hillary Clinton.

  1. Franchise was being slowly extended to non–land-owning white men in various states at the time. Wikipedia 

Vote your conscience

What’s happening at the Republican National Convention doesn’t feel real, but it’s real. The self-aggrandizing nominee for president claimed, “I alone can fix it.” Later, chants of “Yes you will, yes you will.” This is not about policies; it’s fear and cult of personality.

Fascism is the following (copied and pasted from here):

  • Glorification of the past (before the debasement of the nation); past seen as glorious, source of inspiration for the present.
  • Exaltation of force, strength, violence: slogans, symbols, costumes, insignias, military. Promotes discipline, sacrifice, blind obedience to the leader.
  • A reaction (defines itself through reaction to something else): against those that have debased the nation, those that disunite it, that cannot defend it against its enemies.
  • In fascism, the enemies of the nation are old corrupt politicians, foreigners, especially Jews, communism (promoted by Jews).

And let’s not forget the calls of “America First”, which is a reference to the political party of Nazi sympathizer Charles Lindburgh.

By all means, vote your conscience. I just hope that your conscience tells you that, above all, this man and his party must be defeated.

Quantified cantillation III: sequences

First post
Second post

Earlier this year I published a couple of blog posts with some descriptive statistics of trop in the Torah. One of the biggest shortcomings of those posts was that they didn’t deal with the order of trop at all. This is a pretty big shortcoming when you consider that many trop come in pairs/groups, or that certain trop frequently or necessarily follow certain other trop. So, this time around I created an interactive tool I’m calling (for lack of creativity) the Trop Sequence Explorer. If you haven’t checked it out yet, I’d suggest playing around with it a bit; it’ll give you context for the rest of this post.

Basically, it shows each trop listed in order from most to least common. When you click one, it shows you all trop that can follow it and how often each one occurs in that sequence. In other words, it shows transition probabilities to each trop conditional on all trop that come before it in a sequence. There’s also a graph at the bottom that shows how often the selected sequence occurs in each perek of the Torah. Clicking a bar in the graph shows the text of the p’sukim in that perek that contain the current sequence.

Trop Explorer screenshot

What follows is a bit of the thought process that went into its creation, some issues I ran into, and some interesting observations. Feel free to jump to the section that’s most interesting to you.

The Jewish Nerd section

Back in the fall, I was gabbaiing and noticed two tevirs in a row. “How often does that happen?”, I wondered. Seven times, it turns out. It’s pretty well known that a zarka has to be followed by a segol or a munakh segol, but it turns out that the latter is actually more common (by a 13-point margin).

Beyond the factoids, there are other fun things to come across. Parallel sentence structures often have parallel trop, even when the trop itself is not that common. In B’midbar 26, gadol is used at a much higher rate than normal, mostly on names in a genealogy; it really pops out in the bar graph.

Gadol in B'midbar 26

One of the most surprising things for me, though, is how relatively unique each pasuk is. Once you get more than three or four levels deep in the tree, there are surprisingly few p’sukim that match that sequence. This is even true for seemingly common sequences. A pasuk that is merkha tipkha etnakhta merkha tipkha sof pasuk only happens 43 times in the entire Torah.

As I was creating the Sequence Explorer, I encountered some challenges and needed to make some decisions about how it used trop data. One question several people have raised is: Why are there ever trop following a sof pasuk? Shouldn’t a sof pasuk, by definition, be the end of a pasuk? The answer is that there are two sets of trop used for the 10 Commandments, the takhtonim, which are used for private study, and the elyonim, which are used for public readings. I chose to use the elyonim because I wanted to examine how trop are read out loud. The problem is that the two sets of trop also have different pasuk divisions. Even though I used the elyon trop, I had to use the takhton pasuk divisions, because the takhton divisions seem to be more standard, and are the ones returned by the Sefaria API, which is what I used to pull the in actual pasuk text when you click on a perek’s bar in the bar graph. Perhaps at some point I’ll add a setting so people can explore both versions.

Many authorities consider munakh legarmeh a separate trop. I decided not to count it separately for two reasons. The simple technical reason is that there is not a different Unicode character for it (distinct from munakh), so I would have to detect it based on context. The other is that, by definition, the munakh legarmeh is a munakh that precedes another munakh. Since that’s exactly the type of data this app shows, it felt both redundant and somewhat circular to distinguish a trop by what follows it. If you click the munakh, the number of munakhs that follow it should be equal to the number of munakh legarmehs.

Seeing sequences also helped me find issues in the data that I couldn’t see otherwise. For example, I found a couple instances where the data showed four pashtas in a row, but this wasn’t really the case. Trop typically indicate where the stress should fall in a word, but some trop must be placed at either the beginning or the end of a word regardless of stress. To help readers, many sources, including — I found out — the Tanach.us data source I used, put such trop on a word twice: once in the required position, and once where the stress falls. I cleaned out those doublings by searching for any word with two trop on it, and if the two trop were the same, I deleted one of them. Hopefully there was no collateral damage from that.

Another oddity was that there were ten tsinnorits and one geresh mukdam. This was odd because those trop aren’t used in the Torah, even if their lookalikes, zarka and geresh are. It seems like they were used for typesetting reasons — their placement on a word is slightly different — so I just lumped them in with their respective lookalikes.

There were also a number of p’sukim with no sof pasuk. I’m not sure exactly why, but I fixed them. Being able to see the bar graph across the bottom was hugely helpful in seeing that this was an issue.

Speaking of the bar graph at the bottom, aggregating by perek is somewhat arbitrary. At some point I would like to try aggregating in other ways, such as by parshah.

The Design Nerd section

I knew pretty early on that I wanted to do some sort of Markov chain–like visualization of transition probabilities, but I set the idea aside to do real work, which, fortunately, happened to involve learning D3. When I turned my attention back to this, I realized two things:

  1. Pairwise transition probabilities aren’t that interesting in isolation; sequences are much more interesting. (In other words, you need memory in your Markov chain.)

  2. As in the previous posts, we have the complete dataset. Descriptively exploring that is very different from wanting to make predictions or generate new sequences, which is a more typical use of Markov chains.

So, I settled on the basics of a design, but without a few key features. The original idea was a tree, where each level would show the conditional probability of going to a particular trop given all those that had come before it. The plan was just to show simple squares with a trop symbol, its name, conditional probability, and conditional count. And, there was no bar graph at the bottom to show where a given sequence occurred.

Original whiteboard sketch (or what's left of it)

It wasn’t until I was sketching out the visual design for the squares — well after I had it actually working — that I came up with the idea of shading them in, making them into a histogram of sorts. Since they seem to follow something not entirely unlike a Poisson distribution, I thought about log-weighting them, but decided it would be more straightforward not to since I’m also showing raw counts.

Once I could play with building sequence trees, I pretty quickly wanted to know where in the Torah those sequences were. And so, the bar graph at the bottom was born. For most of the time I was building it, clicking a bar would just open that perek on Sefaria. Using the Sefaria API to pull in the text of the actual p’sukim was one of the last features to go in.

The Programming Nerd section

When I first started thinking about how to implement this, my intuition was to have the data structure match the tree structure of the interface. It felt elegant, and it seemed like a good idea at the time. I wrote a recursive function (after fighting with mutable container objects in Python) to go through the trop strings and build a giant JSON file shaped like this:

[{
  "name": "munakh",
  "count": 5456,
  "children": [
    {
     "name": "revii",
     "count": 1410,
     "children": […]
     },
     {
     "name": "katan",
     "count": 4350,
     "children": […]
     }
     …
  ]}
  …
]

Well, that turned out to be 8.6 MB — way too big to download as part of a web app. A similar file that listed which prakim had which sequences was over two gigabytes. I wrote most of the UI (locally) with these two files. Thankfully, I finally realized that I could just download a 760 kB list of raw trop strings and search for sequences on demand in the browser. And that, folks, is why I’m in HCI, not real computer science. Derp.

Finally, D3 was great to work with. Being able to define a simple linear scale like this

var x = d3.scale.linear()
    .domain([0, width])
    .range([width, 0]);

even made it easy to work right-to-left when SVG objects have their origins in the upper left-hand corner.

Future work

I’m a grad student, so how can I resist a Future Work section? There are a number of features I’d like to add at some point. As I hinted at earlier in this post, it would be nice to be able to aggregate the bar graph by parshah instead of just perek. Combining other aggregations, like sefer, with the ability to limit sequence queries to certain parts of the text would open the door to adding the rest of the Tanakh. (The Emet books would be outta control!) And color coding disjunctive and conjunctive trop would be a nice way to see more structure in sequences. If you want to take a stab at any of these things, have a look at the issues list for this project on GitHub.

And if you’ve made it this far without actually using the app, go play with it now!