Fly Dodge

A small browser game where a simulated fly dodges falling obstacles, steered by a network wired from real synapse counts in the male fruit fly central nervous system connectome.

The project asks one question: what happens when you drive a virtual fly with the wiring of a real fly's brain and nervous system instead of hand-written dodge rules? To get there it needed four things, and each one is documented here:

Provenance tags

Throughout these docs, three small tags show where a statement comes from:

from the data read from the dataset files or the real outputs pasted during the project.   assumption a choice made for the model, not established by the data.   literature a claim from a published paper, cited in Sources.

A note on voice: "I" in these docs is Claude, who wrote the code and this documentation with you. "The author" is you, the project owner, whose laptop and terminal produced the real outputs quoted here.

Status: what has and has not been checked

How it works at a glance

1. Choose the eyes. Four looming-detector cell types (LPLC2, LC4, LC6, LPLC1), 569 neurons in total, are the fly's only sensors.
2. Find what they connect to. The export script follows their synapses to descending neurons, both directly and through one interneuron, and groups neurons by cell type and body side.
3. Write the circuit. 8 sensor nodes, 40 interneuron nodes, 40 descending-neuron nodes and 482 weighted connections go into one JSON file.
4. Run it. Each falling obstacle produces a looming signal for the left or right eye. The network turns it into activity in left-side and right-side descending neurons. A simple rule turns that activity into movement.

Quick start

Path A: just see the game

1. Open game/fly_dodge.html in Chrome, Firefox or Edge. It needs no internet.
2. Click Choose file and pick tests/STANDIN_not_real_circuit.json, or your own flybrain_circuit.json if you have already made one.
3. Click in the arena to drop obstacles. Hold the button before releasing to make a bigger one.

Path B: the full pipeline on the real dataset

1. Make a Python environment (needed on Arch Linux, see Setting up Python):

   bash scripts/00_setup_venv.sh
   source ~/flybrain-venv/bin/activate

2. Download the three tables (about 1.1 GB) into a working folder:

   mkdir -p ~/malecns && cd ~/malecns
   bash /path/to/flybrain-dodge/scripts/01_download.sh

3. Inspect them (optional but recommended, prints schemas and cell-type counts):

   python /path/to/flybrain-dodge/scripts/02_inspect_schema.py | tee inspect_output.txt

4. Build the circuit (a few minutes; needs roughly 4 to 5 GB of free RAM):

   python /path/to/flybrain-dodge/scripts/03_export_circuit.py | tee report.txt

   This writes flybrain_circuit.json into the current folder.
5. Play. Either open the game and choose the JSON file, or serve both from one folder so it loads automatically:

   cp flybrain_circuit.json /path/to/flybrain-dodge/game/
   cd /path/to/flybrain-dodge/game && python -m http.server 8000
   # then open http://localhost:8000/fly_dodge.html

The male CNS dataset

The data comes from the MaleCNS connectome, an electron-microscopy reconstruction of the central nervous system (brain and ventral nerve cord) of a male Drosophila melanogaster. Every neuron was traced and every synapse located, so the result is a wiring diagram: which neuron connects to which, and through how many synapses. This project uses release male-cns:v1.0.

What is offered, and how big it is

Beyond the tables, the page offers skeletons in several formats and coordinate spaces, a Neo4j database with its input CSVs, and the image and segmentation volumes. Those are not sized on the page.

The file names contain minconf-0.5. I read this as a minimum synapse-confidence cut-off of 0.5, but the page does not spell it out, so check the dataset documentation before relying on that reading.

What to download

The circuit needs only three of the seven tables, about 1.2 GB (13 + 42 + 1,100 MB = 1,155 MB).

On the author's laptop the three files came down without trouble: the transfers reported 13.81 MB, 41.27 MB and 0.97 GB, at roughly 4 MB per second, so the big file took about four minutes. curl reports 0.97 G for the weights file where the page says 1.1 GB; I did not investigate the small gap (rounding or binary versus decimal units are the likely explanations).

Memory

The game itself needs very little. The only memory-hungry step is the export script, which reads a 152-million-row table in chunks of 10 million rows and keeps only the rows it needs. Budget roughly 4 to 5 GB of free RAM for it (my estimate; on a 16 GB laptop with the desktop running this should fit). If it gets killed, see Troubleshooting.

An alternative: query instead of download

neuPrint's API returns connectivity for chosen cell types without any bulk download. It needs a free account and an API token. It is slower to iterate with than local files, but it uses almost no disk.

pip install neuprint-python
python -c "
from neuprint import Client
client = Client('https://neuprint.janelia.org', dataset='male-cns:v1.0', token='YOUR_TOKEN')
"

Setting up Python

The author works on Arch Linux with Python 3.14.7. The first attempt to install the two libraries failed with this message:

error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try 'pacman -S python-xyz' ...

This is PEP 668: the distribution refuses system-wide pip install so that pip cannot break system packages. The fix is a virtual environment, which is what scripts/00_setup_venv.sh creates:

python -m venv ~/flybrain-venv
source ~/flybrain-venv/bin/activate      # fish shell: source ~/flybrain-venv/bin/activate.fish
pip install pandas pyarrow

After that both packages installed, and the prompt showed (flybrain-venv). You need to run the source line in every new terminal before using the scripts. If pip ever cannot find a pyarrow build for a very new Python, the fallback is sudo pacman -S python-pandas python-pyarrow and then running the scripts with the system python.

Reading the tables

Running 02_inspect_schema.py prints each table's columns and first rows. All values below are the real output; wide tables are shown with selected columns. The complete text is in examples/real_inspect_output.txt.

Annotations from the data

211,577 rows, one per neuron, with 36 columns. The ones this project uses are bodyId, type, superclass, somaSide, subclass and exitNerve.

The first row is worth a look: the dataset itself labels body 10001 as the right-hand giant fiber (DNp01(GF)_R, hemibrain name "Giant Fiber", with a synonym note pointing to Kennedy and Broadie 2018). That is how the project knows which descending neuron is the giant fiber, without relying on memory.

Neurotransmitters from the data

1,835,518 rows. That is far more than the annotation table because it also covers segments that were never annotated as neurons; the export script joins on bodyId and simply ignores the rest. The script uses consensus_nt.

Notice the confidence column. The giant fiber's per-neuron prediction is only 0.50, while the other two are 0.88 to 0.96, and VCH also has a ground_truth value. Predicted transmitters are predictions, and the model inherits their uncertainty.

Weights from the data

151,856,684 rows and just three columns: body_pre, body_post and weight (a synapse count).

Read the first row as "neuron 10352 makes 2,591 synapses onto neuron 10351". The three rows shown happen to run from strongest downwards, but I have not checked whether the whole table is ordered that way.

Gotchas found while exploring

• Many neurons have no superclass. 44,877 of 211,577 have a blank superclass, and some descending neurons have a blank type (4 of them in the type list). The export script labels blank types unnamed and never lets them into the circuit.
• Some types are compound. One descending-neuron type appears as DNp51,DNpe019 (4 neurons). It is treated as its own type.
• Side is a column, not a naming convention. The script builds node names as type_side (for example LPLC2_L) from somaSide, and uses ? when the side is blank.
• Newer pandas hides text columns from a dtype check. See Mistakes found along the way.

Choosing the cells

The tables classify every neuron into a superclass. These counts are real and add up to exactly 211,577:

The sensors: four looming detectors

Looming is the visual signature of something growing on the retina, which usually means it is getting closer. The visual projection neurons carry visual features out of the optic lobe. The four types chosen are all in that group, and the counts below are real:

The counts sum to 569 sensor neurons. The left and right numbers match the annotations' somaSide column; the export script's first lines print exactly these figures.

Other looming-related types were available but left out. LC16 has 182 neurons and is described alongside LC4, LPLC2 and LC6 as looming-responsive with avoidance behaviour. Adding it is one edit to INPUT_TYPES in the export script. Another type, LPLC4 (97 neurons), also sits among the visual projection neurons and was not used as a sensor; the real report shows it as the highest-ranked interneuron, so it is in the circuit, just not at the input.

The outputs: descending neurons

Descending neurons are the cells whose axons run from the brain down the nerve cord to control the body. The dataset has 1,314 of them in 481 types; the largest types are small (for example DNg08 with 23 neurons and DNpe008 with 19). The named ones this project cares about come in left/right pairs of one neuron each: DNp01 (the giant fiber), DNp02 to DNp07, DNp10, DNp11, DNa02 and DNb01 all have exactly 2 neurons. The export script does not pick descending neurons by name. It picks the 40 with the most synaptic drive from the chosen sensors.

Building the circuit file

scripts/03_export_circuit.py reduces 152 million neuron-to-neuron connections to a circuit of 88 nodes. This section walks through what it does, in order. The settings at the top of the script are the only things you normally edit.

INPUT_TYPES = ['LPLC2', 'LC4', 'LC6', 'LPLC1']   # the sensors
N_INTER = 40     # interneuron nodes kept
N_DN    = 40     # descending-neuron nodes kept
MIN_EDGE = 10    # drop type-to-type edges with fewer total synapses
SIGN = {'acetylcholine': 1, 'gaba': -1, 'glutamate': -1}   # anything else -> 0

Nodes are cell type plus side

A single neuron is too fine-grained for a small game, and a whole cell type ignores left and right. The script's compromise is a node = type + soma side, for example LPLC2_L (94 neurons) or DNp01_R (1 neuron). The edge between two nodes is the sum of synapses over every neuron pair. Node size is kept (n) because it matters when weights are normalised later.

1. Load and label. Read annotations, drop duplicate IDs, fill blanks (unnamed type, ? side), join consensus_nt from the neurotransmitter table, and build the type_side name for every neuron.
2. Pick the endpoints. Sensors are all neurons whose type is in INPUT_TYPES (569 neurons). Outputs are all neurons with superclass == "descending_neuron" (1,314).
3. Scan the weights once. Read the 152-million-row table in 10-million-row chunks. Keep rows that leave a sensor, and rows that arrive at a descending neuron. On the real data this left 505,699 sensor-outgoing edges and 818,956 descending-neuron-incoming edges. Everything else is discarded as it is read, which is why memory stays bounded.
4. Find the paths. Direct: a sensor synapses straight onto a descending neuron. Two-step: a sensor synapses onto some neuron X, and X synapses onto a descending neuron. X can be any neuron that is not itself a sensor or a descending neuron.
5. Keep the 40 most important interneuron nodes. Each node X is scored by the geometric mean of what it receives from sensors and what it sends to descending neurons: score = sqrt(in × out). Unnamed nodes are skipped.
6. Keep the 40 most driven descending-neuron nodes. A node's drive is its direct synapses from sensors plus its synapses from the kept interneurons. Unnamed nodes are skipped.
7. Write the outputs. Nodes and edges go to flybrain_circuit.json. Edges with fewer than MIN_EDGE = 10 synapses are dropped. A report is printed to the terminal (see the next section).

Worked example: why LPLC4 ranks first real numbers

Step 5 with the real totals from the report. "in" is synapses received from the sensors, "out" is synapses sent to descending neurons, and the transmitter is the node's majority consensus_nt.

LPLC4_R wins even though PVLP151_R receives more from the sensors (5,621 versus 2,194), because the geometric mean rewards nodes that are strong on both sides of the relay. PVLP011_R receives the most of all (10,334) but sends almost nothing on (238), so it ranks 13th. The scores computed here descend in exactly the order the script printed, which confirms this is the rule the script applies. The score column is my arithmetic from the reported in/out columns.

The sign rule assumption

Each node takes the majority consensus_nt of its neurons. Acetylcholine counts as excitatory (+1), GABA and glutamate as inhibitory (−1), and any other or missing transmitter as 0, meaning the connection has no effect in the game. Glutamate is a simplification: in flies it can act either way depending on the receptor. Of the top 15 interneuron nodes above, 10 are cholinergic, 3 glutamatergic and 2 GABAergic.

What the reduction leaves out

• Paths longer than two steps, and any interneuron-to-interneuron connection. Only sensor→descending and sensor→X→descending paths exist in the circuit.
• All but the top 40 interneuron nodes and top 40 descending-neuron nodes. The choice is by synaptic drive, not by what the neurons do.
• Edges under 10 synapses.
• Everything inside a node: individual neurons, their positions, and their timing.

Reading the export report

Besides the JSON, the script prints a report. Here is how to read the descending-neuron table, using the real output (full text in examples/real_export_report.txt).

What the table shows counted from the real 40-row table

• Direct wiring is same-side. 32 of the 40 nodes receive direct input, and 31 of those receive it from one side only (left-side nodes from left sensors, right-side from right). The single exception is DNp70_R: 6 synapses from the left and 207 from the right.
• Two-step wiring mixes sides. For 28 of the 40 nodes, at least 70% of the interneuron-relayed drive comes from the same side. For 10 nodes it is between 30% and 70%, and for 2 nodes (both DNg97) it comes mostly from the opposite side.
• Some nodes get almost all their input from the opposite eye. DNp04_L has 8,785 direct synapses from left sensors and none from the right, yet only 35% of its two-step drive is left-derived. Its right-eye influence arrives through interneurons.
• The giant fiber is a moderate case. About 63% of DNp01_L's two-step drive is left-derived and about 66% of DNp01_R's is right-derived, so roughly a third comes from the opposite eye on both sides.
• 8 nodes have no direct sensor input at all (for example DNp26 on both sides, DNp57_L, DNp07_L and both DNbe001 and DNg97 nodes) and are reachable only through interneurons.

I treat somaSide as the eye side, i.e. a left-soma looming detector is taken to see the left visual field. That is the usual arrangement for these cell types but I did not verify it for each one in this dataset.

The circuit JSON

flybrain_circuit.json is what the game reads: one object with a meta block, a list of nodes and a list of edges. It is on the order of 100 KB (my estimate for 88 nodes and 482 edges; ls -l will tell you). The layout below is schematic: the field names are exact, values marked … or in angle brackets are placeholders.

{
  "meta":  { "dataset": "male-cns v1.0",
             "input_types": ["LPLC2", "LC4", "LC6", "LPLC1"],
             "sign_rule": { "acetylcholine": 1, "gaba": -1, "glutamate": -1 },
             "min_edge": 10 },
  "nodes": [
    { "id": "LPLC2_L", "type": "LPLC2", "side": "L", "layer": "input",
      "n": 94, "nt": "<majority transmitter>", "sign": <+1, -1 or 0> },
    { "id": "LPLC4_R", "type": "LPLC4", "side": "R", "layer": "inter",
      "n": <neurons>, "nt": "acetylcholine", "sign": 1 },
    { "id": "DNp01_R", "type": "DNp01", "side": "R", "layer": "dn",
      "n": 1, "nt": "<…>", "sign": <…>, "subclass": "lt", "exit_nerve": "" }
  ],
  "edges": [
    { "pre": "<node id>", "post": "<node id>",
      "total": <synapses summed over all neuron pairs>,
      "per_post_neuron": <total / n_post>,
      "n_pre": <…>, "n_post": <…> }
  ]
}

Only the sensor→descending, sensor→interneuron and interneuron→descending kinds of edge exist. The game ignores any edge that points at a sensor, and any edge whose endpoints are missing from nodes.

Checking a claim against the file

The JSON is small enough to interrogate directly. scripts/04_input_contributions.py prints, for the strongest descending-neuron nodes, how many direct synapses come from each sensor type:

python scripts/04_input_contributions.py flybrain_circuit.json 12

Use it to check, for example, whether DNp01 receives direct input from LC6 in this dataset, or how large LPLC2's share is compared with LC4's. Remember that edges under 10 synapses were already dropped.

From obstacle to eye signal

The fly has no camera. Each frame, every falling obstacle is turned into a single number per eye that stands for "how strongly is something looming at me on this side". Everything in this section is assumption: the connectome says nothing about how the sensor neurons are driven.

Per obstacle

Let p be the obstacle's centre relative to the fly, v its velocity relative to the fly, r its radius, d = |p| and R = 7 px the fly's radius.

closing speed     c   = −(p·v) / d                      (skip the obstacle if c ≤ 0: it is receding)
time of closest   tca = −(p·v) / |v|²
miss distance     b   = | p + v·tca |
on-course gate    g   = exp( −( b / (1.3·(r+R) + 12) )² )
expansion rate    e   = r·c / (d² + 0.5·r²)             (per second)
bearing           φ   = atan2(px, py)  in degrees, + = right of heading
half-width        α   = asin( min(1, r/d) )

• Expansion rate e is how quickly the obstacle's angular size grows. The exact figure for a circle is r·c / (d·√(d²−r²)); I soften the denominator so it stays finite when the obstacle is very close.
• The gate g is 1 for an obstacle heading straight for the fly and falls off for one that will pass wide. It stands in for the collision-course selectivity of looming detectors such as LPLC2, so a ball that will miss does not trigger an escape.

Per eye

s_R(φ) = smoothstep(−25°, +5°, φ) × ( 1 − smoothstep(130°, 170°, φ) )       s_L(φ) = s_R(−φ)
ℓ_eye  = Σ over obstacles  [ mean of s_eye over 5 bearings in φ±α ] × g × e
a_eye  = ℓ² / (ℓ² + ℓ0²)                                                 ℓ0 = looming threshold

Averaging the eye sensitivity across the obstacle's angular width matters for big, close obstacles that fill both eyes. a_eye is a Naka-Rushton saturating curve between 0 and 1; the Looming threshold slider sets ℓ0 (default 0.25 per second). Lower values make the fly react earlier.

How the eye signal reaches the circuit

Every sensor node on the left receives a_left and every sensor node on the right receives a_right, passed through a 20 ms first-order lag. All four sensor types get the same number, because the data does not tell me how their tuning differs. Sensor nodes with unknown side (?) get the average. Obstacles that have already burst are ignored.

The rate model

Each node has an activity a between 0 and 1, standing for the average firing of its neurons. Sensor nodes are set from the eye signals. Every other node follows the equations below.

weight             w_ij = sign(node i) × per_post_neuron(i → j)                      from the data
drive              x_j  = gain × ( Σ_i w_ij · a_i ) / scale(layer of j)  + noise      noise: Gaussian, σ = 0.04
response           f(x) = x / (1 + x)  if x > 0,  else 0
dynamics           τ · da_j/dt = −a_j + f(x_j)                                        τ = 30 ms, Euler step 1/120 s

• Why per_post_neuron. If every presynaptic neuron fires at rate a, an average postsynaptic neuron receives per_post_neuron × a synapse-weighted drive. This normalises for how many neurons a node contains, so a big population is not automatically stronger.
• Why x/(1+x). A rectified saturating curve keeps every node between 0 and 1 no matter how large the synapse counts are.
• Why noise. It breaks ties. Without it, a perfectly symmetric situation (an obstacle straight overhead) would leave the fly frozen or always choosing the same side. Idle noise is small: in a 10-second test with no obstacles, the fly drifted at most 17 px from home.

Calibration: turning raw synapse counts into sensible activity

Raw counts span orders of magnitude (from the report: tens to thousands). Feeding them straight into f would either saturate everything or silence everything, so at load time the game calibrates three constants from one reference stimulus: all left-side sensors at activity 0.8 and all right-side sensors at 0.

1. scale(inter) = the median of the positive interneuron drives. A typical driven interneuron then sits at x = 1, half saturation (a = 0.5).
2. With those interneuron activities, scale(dn) = the median of the positive descending-neuron drives, in the same way.
3. motor_scale is chosen so that the summed left-side descending activity for this stimulus equals 0.8.

The right-side summed activity under the same stimulus is then a direct measure of how well the circuit separates sides. The game prints it when a file loads: "a strong looming object on the left drives left-side descending neurons to 0.80 and right-side ones to …". A value near 0 means a strongly lateralised circuit; a value near 0.80 means the circuit barely distinguishes left from right. On my made-up stand-in it was 0.53; for the real file, read your own status line.

The three constants are computed once from the unmodified weights and then held fixed, so the controls in the game (silencing, rewiring) change the behaviour rather than silently re-tuning it.

From neurons to movement

The circuit ends in descending-neuron activity. The dataset cannot tell me what each of these neurons does to a body, so the game uses a single rule for all of them. assumption

S_L = motor_scale × Σ activity of left-side descending nodes       S_R = the same for the right side
T      = S_L + S_R                                                   overall threat
drive  = T / (T + T0)                                                T0 = 0.35
asym   = tanh( κ · (S_L − S_R) / (S_L + S_R + 0.05) )                κ = 10 ("Left/right decisiveness")
sym    = 1 − |asym|
v_x    = vmax · drive · asym                                         (+ = move right)
v_y    = − vmax · retreat · drive · sym                              (− = move down)     retreat = 0.5,  vmax = 560 px/s

In words: activity on the left pushes the fly right, and activity on the right pushes it left; if both sides are active about equally, the fly backs away downwards; if nothing is active, it drifts back home.

Try it

Homing, the only way up

When the threat drive is low the fly drifts back to its home point (450, 135) with a speed of min(200, 3 × distance) px/s, scaled by (1 − drive). That is the only source of upward motion. The tables carry no information about where in the visual field a neuron looks, so nothing in the wiring says "an obstacle is below, go up".

Body dynamics

v ← v + ( v_command + v_home·(1 − drive) − v ) · dt / 0.05          (τ_v = 50 ms)
position ← position + v · dt         clamped to the arena;  altitude limited to 11–430 px

The rule also has an obvious blind spot: vertical avoidance is only "back away when both sides fire", so a fly cannot choose to climb over an obstacle. Section Where to go next discusses what would be needed to change that.

The arena and the fly

From the default drop height to the fly's home altitude, an obstacle falls 448 units, which takes about 1.0 s and ends at about 900 units/s (my arithmetic from the gravity constant). The fly can move at up to 560 units/s, so it has real time to react only if it notices the obstacle early. Lowering the drop height makes the game harder; the looming threshold slider makes the fly react sooner or later.

Hits and the "dodged" score

• Hit: centre distance below r + 7. Counted once per obstacle; the fly is knocked away at 260 units/s and flashes red.
• Threat: an obstacle counts as a threat if, when it spawns, |x − fly.x| < r + 7 + 25. Only threats can be "dodged". Drops far to the side would otherwise inflate the score.
• Dodged: a threat that passes entirely below the fly, or reaches the floor, without hitting it.

Using the game

The game is a single file, game/fly_dodge.html, with no libraries and no network use. It has two panels: the arena on the left and the fly's circuit on the right.

Loading the circuit

• Choose file, or drag flybrain_circuit.json onto the box. The file is read in the browser and goes nowhere else.
• Paste the JSON into the text box if a file picker is unavailable (some embedded viewers block it).
• Automatic: if the page is served over HTTP from the same folder as flybrain_circuit.json (for example with python -m http.server), it loads by itself. Opening the HTML straight from disk cannot do this, because browsers block reading a neighbouring file from a file:// page.

After loading, the status line under the arena shows the size of the circuit and the calibration result described in The rate model. A malformed file gives a plain-language error instead of a blank screen.

Dropping obstacles

• Click in the arena: an obstacle appears at the click's horizontal position, at the drop height (dashed line), and falls.
• Hold the button: a ghost circle grows at the top for up to 1.2 seconds; releasing drops it. Sliding the pointer sideways while holding moves it. Sizes run from radius 8 to 60.
• Touch screens work the same way. The Drop one button places a random obstacle, and is the keyboard-friendly way to play.

What you see in the arena

What you see in the circuit panel

• Three bands from top to bottom: looming detectors (the eyes), interneurons, descending neurons. Left-side nodes are in the left half of each band and right-side nodes in the right half.
• Each circle fills in with its activity. Colour shows the node's kind: amber excitatory, violet inhibitory, teal for descending neurons.
• Lines are the connections; they brighten when the presynaptic node is active, in proportion to the connection's strength. Amber lines excite and violet lines inhibit.
• The most active nodes get a label. Hover any node for its cell type, neuron count, neurotransmitter, sign and current activity.
• Two bars at the bottom show S_L and S_R, and a sentence says what the motor rule is doing, for example "Threat 0.61 → sidestep right 84%".

Controls

Experiments to try

The controls are there so you can test whether the wiring matters, instead of taking it on faith. Hit and dodge counts are noisy, so run each condition for at least a minute of Rain, press Reset score between conditions, and treat differences of a few counts as noise.

1. Intact against cut brain. Turn Rain on for 60 s with the defaults, note dodged and hit, reset, tick cut the brain and repeat. The brain-cut fly should be hit far more often. On the stand-in circuit over 90 simulated seconds: intact 12 dodged and 1 hit; brain cut 4 dodged and 8 hits.
2. Intact against rewired. Repeat with rewire ticked. See the caveat in Results: this gap depends on the decisiveness setting.
3. Silence one sensor type at a time. Because all sensor types get the same signal, differences between them reveal how much each one's wiring contributes to escape-related descending neurons in your file.
4. Watch a left drop and a right drop. Drop an obstacle well to the left of the fly, then well to the right, while watching the circuit panel. Which descending nodes light up on which side? Compare S_L and S_R with the calibration line printed at load.
5. Head-on drop. Drop directly above the fly. Both eyes see it, S_L and S_R are close, and noise decides which way it goes; try a few times and see the split.
6. Turn the knobs. Lower the drop height and raise the looming threshold to make the game hard; raise decisiveness to see how much of the fly's success comes from that one setting.

How it was tested

Everything under tests/ can be re-run. The simulation core is loaded straight from the game file, so the tests always match the shipped code.

The stand-in circuit

not real data All the simulation tests run on a circuit I built by hand, saved as tests/STANDIN_not_real_circuit.json. Its interneuron in/out totals, its direct sensor-to-descending totals and its left-share percentages are copied from the real report. Everything else, including which interneuron connects to which descending neuron, the 75/25 same-side/other-side split and the 40/20/20/20 division between sensor types, is invented. It also has left/right structure built in. That is enough to test that the code works and that the controls behave; it cannot say whether the real circuit dodges well.

Pipeline test on synthetic tables

The export script the author ran on real data was reproduced from the project record, so I checked the copy in scripts/ by running scripts 02, 03 and 04 on synthetic Feather tables (255 neurons, 23,450 edges). They ran cleanly, produced a 24-node, 192-edge circuit, and that circuit passed the same 17 page checks as the stand-in.

What is not tested

• The game with the real flybrain_circuit.json.
• Real rendering: pixels, layout on phones, frame rate. The canvas was mocked.
• Browsers other than the headless one, and touch input on a real device.
• Statistical significance: results below come from fixed seeds and one circuit.

Results

All numbers are from running tests/ on the stand-in circuit (not real data), default settings (κ = 10, retreat 0.5, looming threshold 0.25), 112 single drops per condition.

Single drops: percentage of drops that hit the fly

• Silencing all sensors gives about the same result as cutting the brain, as it should.
• Silencing LPLC2 alone barely matters here (3 versus 4 hits). In the stand-in every sensor type feeds the same descending neurons (LPLC2 with a 40% share, the others 20% each), so silencing one leaves plenty of drive; this says nothing about the real LPLC2.

Direction check

An obstacle of radius 30 dropped at different horizontal offsets from the fly (negative = to the left), showing how far the fly's x position moved at most (positive = right):

With intact wiring the fly moves away from the obstacle every time (right when it is on the left, left when it is on the right). The offset-0 case is decided by noise. With rewired wiring the fly moves left whatever the side of the obstacle, and the two hits are exactly the two cases where the obstacle was on the left and the fly moved toward it.

90 seconds of random rain

How much does the result depend on the knobs?

Percentage of the 112 drops that hit the fly, at looming threshold 0.25:

What to conclude, and what not to

• The mechanics work. The fly moves away from the side that is looming, is idle when nothing threatens, and does much worse without its eyes or its brain.
• The intact-versus-rewired gap is modest and knob-dependent. At the default it is 3% versus 12%, but at κ = 8 with strong retreat they are equal (8% each). Random rewiring often produces a constant sidestep bias, which dodges reasonably well on its own. A higher decisiveness setting makes both look good.
• None of this is evidence about the real fly. The stand-in has its left/right structure built in, so an intact advantage is close to guaranteed. The real test is running your own file with the same controls.

Mistakes found along the way

These are real problems that came up during the project, kept here because they explain several design decisions.

The 3D version

Everything above describes the original game: a hovering dot dodging obstacles in a flat, side-on arena. A second version puts the same brain in a room the fly actually flies through in three dimensions, with a real digitized fly model in place of the dot. This part of the docs covers only what's different. Anything not mentioned here — the connectome, the export script, the rate model's equations, the sign rule — is identical to the 2D version; the 3D game imports the exact same Brain class unchanged.

The reason for building this is explained in Where to go next above: memory was never the obstacle to a 3D room, and 3D doesn't fix the main limitation of this project (the tables give left/right, not elevation), but it does let the connectome's real left/right wiring do a more natural job — steering a moving body instead of pushing a hovering dot sideways.

Free flight and 3D sensing

The fly no longer hovers. It cruises forward at all times, and the brain's output steers it rather than moving it directly.

Body and room

Orientation is yaw (turn) and pitch (climb/dive) only, no roll in the physics — the visual bank you see is cosmetic, added only in rendering (see The 3D fly model). The local frame is right-handed: forward is local +Z, up is local +Y, right is local +X, computed each step as right = (cos yaw, 0, −sin yaw), fwd = (sin yaw cos pitch, sin pitch, cos yaw cos pitch), up = right × fwd.

Sensing: the same eye math, now in full 3D

The 2D game measured a bearing in degrees around a fixed frame. The 3D game measures it relative to the fly's actual facing, using vector dot products instead of a fixed angle, so the same left/right split now depends on which way the fly is pointed:

p = obstacle.pos − fly.pos            (as before: closing speed, time-to-closest-approach, miss distance, gate g, expansion rate e — unchanged formulas, see From obstacle to eye signal)
azimuth  φ = atan2( p·right, p·fwd )        + = right of the fly's nose
elevation ε = atan2( p·up, horizontal distance )    + = above the fly

The same eyeSens(φ) ramp from the 2D game turns azimuth into a left/right split. Elevation feeds a new, separate quantity:

vBias = (Σ sign(ε) · strength) / (Σ strength)      strength = gate × expansion rate, summed over obstacles

vBias runs from −1 (threats mostly below) to +1 (mostly above). I checked its sign in isolation before wiring it in: a closing obstacle placed above the fly gives vBias = 1, level gives 0, below gives −1.

Motor: yaw from the data, pitch from geometry

T = S_L + S_R                                    drive = T / (T + T0)             T0 = 0.4
asym = tanh( κ (S_L−S_R) / (S_L+S_R+0.05) )    κ = 9 default (was 10 in 2D)
yawCmd   = k_yaw · drive · asym                  k_yaw = 1.0
pitchCmd = −k_pitch · drive · vBias              k_pitch = 0.6 default, slider-adjustable

Yaw is the direct 3D equivalent of the 2D sidestep: it comes from S_L and S_R, the same real synapse-weighted descending-neuron activity as everywhere else in this project. Pitch has no connectome basis at all — the tables carry no information about elevation in the visual field, so vBias is pure scene geometry, not neural activity. It's gated by drive (computed from the real brain output) so the fly only dives or climbs when something real is actually threatening it, but the direction of that dive/climb is my invention, not a finding. The game's own on-page notes say this plainly, not just these docs.

When nothing threatens the fly, a gentle, also-assumed autopilot steers it back toward the room's centre and mid-height so it doesn't just fly into a wall — the free-flight equivalent of the 2D game's homing behaviour, needed for playability, not derived from data.

Obstacles

Spawn near the ceiling (Y=152 of a 165-unit-tall room) at a random or clicked X/Z, each with its own small constant sideways drift assigned at spawn (26 / (1 + 0.12r) units/s, so smaller obstacles drift more), and gravity pulling them down. They bounce off the four walls (reversing the relevant drift component) but burst on touching the floor, fading over 0.35 s, exactly like landed obstacles in the 2D game.

The 3D fly model

The fly is no longer a placeholder cone. It's a real digitized model — the user supplied a fly.rar archive, which turned out to be a 1999 Viewpoint Datalabs insect model shipped in several formats (OBJ, 3DS, DXF, LWO) with two textures. None of this came from the connectome; it's a stand-in visual, not derived from any biological data in this project.

Extraction

The archive was a solid RAR (all files compressed as one stream). The freely available unrar-free couldn't extract it ("RAR solid archive support unavailable", and it silently stopped after the first file); unar handled it correctly and extracted all eight files.

What was inside

The mesh itself: 1,635 vertices, 3,258 triangles (no quads), split into two named groups — fly_body (2,962 triangles) and fly_wing (296 triangles) — each with its own material and texture.

Finding the orientation

An OBJ file states no convention for which axis is forward. The raw vertex bounding box gave a first clue: X (wingspan) ranges ±2.75mm, Y (height) 0 to 2.62mm, and Z (length) −2.95mm to +1.64mm — asymmetric, meaning the model isn't centred at its own origin in Z. To find out which end was which, I rendered the raw triangles as orthographic wireframes (top, side, front views) directly from the vertex data, colour-coded by group. That made the shape unambiguous: a round head with legs splayed near the front, tapering to a larger abdomen at the back, wings attached near the shoulder. Head is at +Z, up is +Y, and X is left/right (the model is bilaterally symmetric, so handedness there doesn't visually matter).

This mattered because the game's fly orientation math (above) already treats local +Z as forward and +Y as up — so once the model's own axes were confirmed to mean the same thing, it could be dropped straight into that frame with no extra rotation.

Getting it into the browser

A published page can't fetch outside files — no loading a separate .obj, .mtl or .tif at runtime. Everything had to be preprocessed into something that could be embedded as text in the HTML itself:

1. Parse the OBJ myself (a few dozen lines of Python) rather than shipping an OBJ-parsing library. For each triangle corner, look up its position and UV coordinate and "unroll" them into flat, non-indexed arrays — simpler than reconstructing an indexed mesh, and small enough at this triangle count (9,774 corners total) not to matter.
2. Centre and scale. Vertices were in metres (a real fly's actual size, ~5.5 × 2.6 × 4.6mm). Centred on the bounding-box midpoint, then scaled ×2000 so the model's total length comes out to about 9.2 game units — comparable to the old placeholder cone's length of 8, and the reason the fly's collision radius went from 3 to 5.
3. Skip normals. Rather than compute or ship vertex normals, the browser calls BufferGeometry.computeVertexNormals() once at load time — a standard three.js method, one line, no extra data to embed.
4. Convert and shrink the textures. TIFF isn't a web format. Both were converted to PNG and downsized to a maximum of 256px on the long edge (554×420 → 256×194 for the body; 652×420 → 256×164 for the wing).
5. Embed everything as base64. Two float32 position arrays, two UV arrays, two PNGs — all inlined directly in the page's JavaScript as base64 strings, decoded in the browser with atob into Float32Arrays and data-URI textures.

The finished fly_dodge_3d.html is 459 KB in total — still small enough to open instantly, nowhere near the 16 MB a published page allows.

Checks made, and what's still unverified

• Checked: decoding the embedded base64 back into floats reproduces the exact expected triangle counts and a bounding box matching the intended ~9.2-unit scale; both embedded PNGs have valid file signatures; the whole page still passes its headless test suite with the real mesh wired in.
• Not checked: how it actually renders. There's no GPU in the environment these docs and the game were built in, so I have never seen the textures mapped onto the geometry, the shading, or the proportions against the room. If it looks flipped, inside-out, or oddly scaled, the fix is almost certainly in the centring/scale step or the material's side setting above — not a deep problem.

Camera and room

Two camera modes

A Camera card in the controls switches between them; only one is active at a time.

Switching to Free starts exactly from where the chase camera was looking at that instant (computed from the chase camera's position and look-at target, converted to yaw/pitch), so there's no jump. Arrow keys only act when Free mode is on and the browser's focus isn't inside a text field, slider or other form control — so they don't hijack a slider you're adjusting with the keyboard.

Design choice, not a connectome result: free flight plus a chase-or-manual camera was chosen over the original hover-in-place 2D design specifically because it gives the connectome's real left/right wiring something more natural to do — steering a moving body's turn, rather than pushing a stationary dot sideways.

A paler room

The room's background was originally a dark navy space, unrelated to the rest of this project's colour scheme. It now uses the same background colour as the surrounding page — literally the same CSS custom property, --bg — in both light and dark themes, so the 3D view sits inside the page rather than looking like a separate dark window cut into it. Grid lines and room-edge wireframes were recoloured to stay visible against a pale background (previously they only had to work against near-black); ambient lighting was brightened to match. This is a styling change only — it touches no simulation logic.

Testing the 3D version

The same approach as the 2D game: test the simulation math directly in Node first, then test the page's wiring in a headless browser. Nothing here involves an actual GPU — none was available while building this, so rendering itself (whether the textures map correctly, whether proportions and lighting look right) has never been seen, only reasoned about.

Simulation checks (Node, no browser)

Page wiring checks (headless browser, three.js's API stubbed)

There's no WebGL in this environment, so three.js itself was replaced with a small stand-in implementing only the calls the page actually makes (geometry, materials, vectors, the camera's getWorldDirection), enough to run the real page code and confirm it doesn't throw and does what it should — not to confirm anything renders correctly. All 25 checks pass:

• The original 16 (loading, bad-file errors, click/hold-to-grow, every slider, silence/rewire/cut-brain controls, hover on every circuit node, rain, pause, reset) — unchanged, still pass with the real fly model wired in.
• 9 new: starts in Chase mode; switching to Free updates both buttons and the on-screen hint text; Arrow-Up actually moves the camera; the camera stays still with no keys held; dragging in Free mode does not drop an obstacle; a plain click in Free mode still does; switching back to Chase restores its hint text; Chase mode's hold-to-grow still works after switching modes; no JavaScript errors anywhere in the sequence.

What was checked and what wasn't, specific to 3D

Real, assumed, unverified

One place that lists every step of the pipeline by how well supported it is. If you extend the project, this is the list to challenge first.

Troubleshooting

File reference

flybrain-dodge/
├── README.md
├── docs/index.html                      this documentation (offline, single file)
├── game/
│   ├── fly_dodge.html                   the 2D game (single file, no dependencies)
│   └── fly_dodge_3d.html                the 3D game (single file; fly model + textures embedded, 459 KB)
├── scripts/
│   ├── 00_setup_venv.sh                 create the Python environment
│   ├── 01_download.sh                   fetch the 3 tables (~1.1 GB)
│   ├── 02_inspect_schema.py             schemas, first rows, cell-type counts
│   ├── 03_export_circuit.py             tables → flybrain_circuit.json + report
│   └── 04_input_contributions.py        which sensor types feed which descending neuron
├── tests/
│   ├── harness.js, sweep.js, tune.js, rain.js           2D simulation tests
│   ├── ui_smoke.js, package.json                        2D headless page checks (needs npm install)
│   ├── make_synthetic_feather.py                        tiny fake tables for a quick pipeline run
│   ├── STANDIN_not_real_circuit.json                    made-up circuit, used by both games' tests
│   └── 3d/
│       ├── core3d.js                    the 3D simulation core (imported by the tests below)
│       ├── test3d.js, test3d_rain.js    3D simulation tests: idle, direction, hit rate, rain
│       ├── smoke3d.js, three_stub.js    3D headless page checks with three.js's API stubbed
│       └── fly_model_data.js            the embedded fly model, as its own file (for reference)
└── examples/
    ├── real_export_report.txt           output of script 03 on the real data
    └── real_inspect_output.txt          output of script 02 on the real data

flybrain_circuit.json is deliberately not included: it is produced on your machine from the real tables by script 03, and I never had a copy. The 3D game's fly model is included pre-built inside fly_dodge_3d.html itself; the original fly.rar the user supplied is not redistributed here.

Parameter reference

Export script (03_export_circuit.py)

World constants (CFG in the game)

Tunable defaults (DEFAULTS, exposed as sliders where marked)

Other constants inside the code: sensor input lag 20 ms; network noise σ 0.04; homing blends with weight 1 − drive; hit knock-back 260 units/s; eye fields as in From obstacle to eye signal.

Glossary

Where to go next

A 3D room — built

This was discussed, then built; see The 3D version for the full writeup. The predictions from the original discussion held up:

• Memory was not the obstacle. The finished fly_dodge_3d.html is 459 KB, and the 1.1 GB weights file was only ever needed once, offline, to build the small circuit JSON. No measurement of in-browser memory use was possible in this environment (no GPU), but nothing in the design should come close to being a problem on a 16 GB laptop.
• 3D did not fix the main limit, as predicted. Left/right steering is data-driven, exactly as expected. Climb and dive are a geometric stand-in with no connectome support, exactly as predicted — and with a third axis actually in use, that gap is more visible in the finished game than it was in the 2D one, not less.
• The retinotopy route was not attempted. Elevation still doesn't come from the data; it comes from where obstacles happen to sit around the fly right now. The syn-points route described below remains open.
• The design choice was settled: free flight, not hovering-in-place. The fly turns and dives away from threats rather than sidestepping them, which is a better fit for what the real wiring's left/right signal can actually justify.

What wasn't part of the original discussion, and got added while building: a real fly model in place of the placeholder cone (from a user-supplied file, see The 3D fly model), and two camera modes including a free-flying manual camera (see Camera and room).

Still open: elevation from real data

The 12.7 GB syn-points table holds synapse positions. As I understand it, the looming detectors' dendrites are arranged retinotopically in the optic lobe, so synapse positions could in principle give each neuron an approximate look direction. I have not checked that the table supports this, and the mapping would be approximate. It would also mean modelling individual neurons: 569 sensors plus 1,314 descending neurons, and the neuron-level tables the export script filters (505,699 sensor-outgoing rows and 818,956 descending-incoming rows, with direct connections counted in both). That is feasible in a browser only with a compact binary file, and reading syn-points would need batch reading rather than loading it into pandas. This is the one piece of the original 3D discussion that remains exactly where it was.

Smaller steps that need no new data

• Add LC16 to INPUT_TYPES and compare the resulting circuit.
• Use script 04 to check the literature claims about LC4, LPLC2 and LC6 against the real synapse counts.
• Let each sensor type have its own tuning (for example size-sensitive versus speed-sensitive) once you have a reason from the literature.
• Include three-step paths, or interneuron-to-interneuron edges, and see whether the left/right structure changes.
• Replace the single motor rule with roles for individual descending neurons taken from published function.

Project log

How the project unfolded, in order.

1. Sizing the download. The dataset page lists seven tables totalling about 24.1 GB, plus skeletons, a database and image volumes that are not sized. The EM volume shape implies about a petabyte uncompressed (my arithmetic).
2. The idea. A game in which a blue dot at the bottom dodges obstacles that fall when the user clicks, steered by the male connectome. I explained that the full connectome cannot run in a web page, so the plan became a small subset: sensors, relays and descending neurons, with the real weights.
3. Choosing the files. Only 1.2 GB of the 24 GB was needed: annotations, neurotransmitters and weights. The three files downloaded fine.
4. Setting up Python. pip refused on Arch; a virtual environment fixed it.
5. First look at the tables. Schemas and sample rows appeared. The weights table has 151,856,684 rows. The cell-type search returned nothing because of the pandas string-dtype issue; fixed by selecting columns by name.
6. Confirming the cell types. Counts for LPLC2 (185), LC4 (126), LPLC1 (134), LC6 (124) and the descending-neuron classes confirmed the plan was feasible.
7. Building the circuit. The export script ran on the real data, producing 88 nodes and 482 edges and the report reproduced in this documentation. The report showed same-side direct wiring, mixed two-step wiring, and LPLC4 as the top relay.
8. Designing the game around the data. Because only left/right is in the data, the earlier idea of four movement groups was dropped in favour of the left/right, back-away and homing rule.
9. Testing and repairing. A stand-in circuit built from the report numbers exposed a broken control, floor clutter and a head-on weakness (all in Mistakes found along the way). The page was tested headlessly and published as a hosted page.
10. Discussing 3D. Concluded that memory is not the limit and that elevation from data would need the synapse-position tables.
11. This package. Documentation, scripts, tests, examples and the game bundled for offline use. While preparing it I checked the literature, which led to the LC6 correction and the caveat on the direction rule; and I reproduced the export script from the project record and re-ran it on synthetic data.
12. Building the 3D version. Free-flight physics and full-3D sensing, reusing the same Brain class unchanged. A stand-in circuit test run again exposed real problems before anything was published: a naive test scenario that reported 0% hits in every condition (a flawed test, not a game bug — fixed by testing against continuous rain instead of staged single drops).
13. Adding the real fly model. The user supplied fly.rar, a 1999 digitized insect model. Extracting it needed a proper unrar (the free one couldn't handle its solid-archive compression). The model's orientation wasn't stated anywhere in the file, so I rendered orthographic wireframes of the raw geometry to work it out before placing it in the scene. Parsed, centred, scaled and embedded as base64 so the published page needs no external files.
14. Camera and room styling. Added a Chase/Free camera toggle (arrow keys plus click-drag for the free camera) and repainted the room to match the surrounding page's background instead of a dark, unrelated navy. A stale-object-reference bug in the camera tests themselves (see Testing the 3D version) was caught and fixed before publishing.
15. This update. Extended this documentation with the 3D version's own sections, and updated the file reference, assumptions table and roadmap to match.

Sources and credits

Dataset

MaleCNS connectome, release v1.0. male-cns.janelia.org/download. A collaboration between FlyEM (HHMI Janelia), the University of Cambridge (Dept. of Zoology), the MRC Laboratory of Molecular Biology, and Google Research. Licensed CC-BY: credit the dataset if you share results, and use the citation the project site gives (the download page mentions a manuscript but I did not identify its reference here).

Literature

• Ache JM, Polsky J, Alghailani S, Parekh R, Breads P, Peek MY, Bock DD, von Reyn CR, Card GM. Neural basis for looming size and velocity encoding in the Drosophila giant fiber escape pathway. Current Biology 29(6):1073–1081, 2019. doi:10.1016/j.cub.2019.01.079. Used for: LC4 (velocity) and LPLC2 (size) inputs to the giant fiber.
• Jang H, Goodman DP, Ausborn J, von Reyn CR. Azimuthal invariance to looming stimuli in the Drosophila giant fiber escape circuit. Journal of Experimental Biology 226(8):jeb244790, 2023. doi:10.1242/jeb.244790. Used for: bilateral integration and azimuth invariance in looming-responsive descending neurons.
• Klapoetke NC, Nern A, Peek MY, Rogers EM, Breads P, Rubin GM, Reiser MB, Card GM. Ultra-selective looming detection from radial motion opponency. Nature 551, 2017. Used for: LPLC2 as a looming detector.
• "Spatial readout of visual looming in the central brain of Drosophila", eLife, doi:10.7554/eLife.57685. Used for: LC6 responses and its relation to the giant fiber pathway (I have cited it by title and DOI because its author list was not in the material I consulted).

3D fly model

A digitized fly model (OBJ/3DS/DXF/LWO with two textures), dated August 1999 and marked "Created with Viewpoint Interchange, www.viewpoint.com" in its own file header — Viewpoint Datalabs was a commercial 3D model library of that era. Supplied by the user as fly.rar; its original licence terms were not available to me, so treat its reuse as the user's own to determine. Not connectome data, and not covered by the dataset's CC-BY licence above.

Tools

Python with pandas and pyarrow; the game and tests in plain JavaScript; jsdom for the headless page tests; three.js (r128) for 3D rendering; Pillow for the 3D fly model's texture conversion; unar for extracting the model's solid RAR archive.