Python API¶
Stochastic simulation of loop-extrusion factors (LEFs) on a 1D lattice.
Describe a model, then run it:
lattice = lefsim.Lattice(3000, {"site": [600, 1300], "captures_from": ["right", "left"], "strength": 0.9})
world = lefsim.populate(lattice, 20, seed=1) # or lefsim.World(lattice, lefs_table)
model = lefsim.Model(world, lefsim.Rates(step=1.0, respawn=2e-3, respawn_blocked=2e-2, respawn_captured=2e-3))
sim = lefsim.Sim(model, seed=1)
sim.run_until(1_000.0) # advance
rec = sim.record_until(2_000.0, dt=1.0, events=True) # advance and record
Recorded data comes back as Batch objects: Arrow record
batches exposed through the Arrow PyCapsule Interface, so they can be passed
straight to pyarrow.table, polars.DataFrame or queried from duckdb, without
this package depending on any of them. Sim.stream runs
lazily as an Arrow stream, and Replayer rebuilds states
from a recorded event log.
Building a model¶
Lattice
¶
Lattice(n_sites: int, ctcfs: Union[Mapping[str, Any], Any, None] = None)
A 1D lattice of n_sites sites, some of them CTCFs.
ctcfs is a table with one row per CTCF: site, captures_from
('left' or 'right': the side that captures) and optionally strength
(the probability, in [0, 1], that a landing leg interacts; 1 by
default). Pass any Arrow table (pyarrow, polars, Lattice.to_arrow())
or a dict of columns, where captures_from and strength may be single
values for all CTCFs.
to_arrow
¶
to_arrow() -> Batch
The CTCF table (site, captures_from, strength), with
lefsim.n_sites in the schema metadata.
to_dict
¶
to_dict() -> Any
As plain Python data (dicts, lists, numbers and strings), in the same shape as the JSON.
World
¶
World(lattice: Lattice, lefs: Union[Mapping[str, Sequence[Any]], Any, None] = None)
The state of a simulation at one moment: a lattice and the LEFs on it.
lefs is a table with one row per LEF (row i is LEF i): columns
left_pos and right_pos, and optionally left_mobility and
right_mobility ('moving' by default). Pass any Arrow table (pyarrow,
polars, World.to_arrow(), Sim.state()) or a dict of sequences.
Every LEF is checked: legs on the lattice and not crossed, no two LEFs on one site, and a 'captured' leg only on a CTCF that could have captured it (a left-facing leg by a CTCF that captures from the right, a right-facing leg by one that captures from the left).
to_arrow
¶
to_arrow() -> Batch
The LEF table: lef_id, left_pos, right_pos, left_mobility,
right_mobility.
effective_distance
¶
effective_distance(a: int, b: int, *, step_length: int, hop_length: int) -> int
The effective distance between sites a and b: the shortest path
when a path walks the lattice (step_length per step) and may cross
LEFs (hop_length per crossing; across a chain of LEFs side by side,
a rosette, a hop per LEF and the junctions between them free). Both
lengths are integers, so distances are exact; divide by step_length
for lattice steps.
effective_distances
¶
effective_distances(sites: Sequence[int], targets: Optional[Sequence[int]] = None, *, step_length: int, hop_length: int) -> Any
Effective distances (see effective_distance). Without targets,
every pair of sites, condensed as scipy.spatial.distance.pdist
(squareform makes it a matrix). With targets, each of sites to
each of targets, row-major. Returns an array.array("q");
numpy.asarray wraps it without copying.
to_dict
¶
to_dict() -> Any
As plain Python data (dicts, lists, numbers and strings), in the same shape as the JSON.
populate
¶
populate(lattice: Lattice, n_lefs: int, *, seed: int = 0, spawn_rule: SpawnRuleName = 'uniform_non_ctcf') -> World
A world with n_lefs LEFs placed one at a time by a spawn rule, exactly
as respawns place them. seed drives only this placement.
Rates
¶
Rates(step: Optional[float] = None, respawn: Optional[float] = None, respawn_blocked: Optional[float] = None, respawn_captured: Optional[float] = None)
Event rates (per unit time): step for a moving, unblocked leg (a leg
facing a CTCF from behind splits it between passing and eviction);
respawn, the basal rate, for a LEF with both legs free to move;
respawn_blocked for a LEF with either leg blocked (by a leg ahead or the
lattice's end); respawn_captured for one with either leg captured,
which wins over blocked. Eviction unloads a LEF at once; a leg whose
partner is captured is immune to it. Checked (positive and finite) by
Model.
respawn_blocked
property
¶
respawn_blocked: float
Respawn rate of a LEF with either leg blocked (by a leg ahead or the lattice's end), unless the other is captured.
respawn_captured
property
¶
respawn_captured: float
Respawn rate of a LEF with a captured leg, whatever the other leg does.
to_dict
¶
to_dict() -> Any
As plain Python data (dicts, lists, numbers and strings), in the same shape as the JSON.
Model
¶
Model(world: World, rates: Optional[Rates] = None, *, spawn_rule: SpawnRuleName = 'uniform_non_ctcf')
A complete model: a world, its rates, and a spawn rule. Everything a run needs except a seed.
Checks that the rates are positive and finite, and that the spawn rule
can always place the world's LEFs. to_json() and Model.from_json()
save and load it as a versioned document.
to_dict
¶
to_dict() -> Any
As plain Python data (dicts, lists, numbers and strings), in the same shape as the JSON.
from_dict
classmethod
¶
from_dict(d: Any) -> Model
Load from plain Python data shaped as to_dict returns it, with the same checks as
construction.
to_json
¶
to_json(*, indent: Optional[int] = None) -> str
As the versioned JSON model document (lefsim.model, version 2); indent pretty-prints.
Running¶
Sim
¶
Sim(model: Model, *, seed: int = 0, start_time: float = 0.0)
A running simulation of a Model.
seed drives the whole run: the same model and seed give the same run,
bit for bit, on every platform. start_time sets the clock, e.g. to
continue from a state recorded at that time. The model is copied in;
sim.model() returns the current state as a new Model.
run_* methods advance the run; record_* methods advance it and
return what happened as Arrow data. copy.copy(sim) forks the run: the
copy continues exactly as the original would.
next_event_time
property
¶
next_event_time: Optional[float]
Time of the next event, or None if none remain.
model
¶
model() -> Model
The current state as a Model: the world now, with this run's rates
and spawn rule. Sim(sim.model(), seed=..., start_time=sim.time)
continues from here with fresh randomness.
initial_model
¶
initial_model() -> Model
The model this run started from. With seed and start_time, it
says where a recording came from: Sim(sim.initial_model(),
seed=sim.seed, start_time=sim.start_time) replays the run from its
start, event for event.
state
¶
state(*, layout: Layout = 'lef') -> Batch
The current state as a one-snapshot Batch, without advancing.
step
¶
step() -> Optional[tuple[float, Literal['Step', 'Respawn', 'Evict'], int]]
Fire one event. Returns (time, action, id) — id is a leg_id
for "Step" and "Evict", a lef_id for "Respawn" — or None if no
events remain.
run_until
¶
run_until(t_end: float) -> RunReport
Fire every event at or before t_end, then set the clock to t_end.
Releases the GIL while running; Ctrl-C interrupts.
record_until
¶
record_until(t_end: float, *, dt: Optional[float] = None, t0: float = 0.0, events: bool = False, keyframes: bool = False, layout: Layout = 'lef') -> Recording
Run until t_end, recording grid snapshots (with dt, on the grid
t0 + k * dt, inclusive of t_end) and/or event deltas (with
events=True; add keyframes=True to open the event table with one
Keyframe row per leg holding the full state). Consecutive calls on
the same grid continue it seamlessly. Releases the GIL while running;
Ctrl-C interrupts.
record_events
¶
record_events(n: int, *, dt: Optional[float] = None, t0: float = 0.0, events: bool = False, keyframes: bool = False, layout: Layout = 'lef') -> Recording
Fire up to n events, recording as in record_until; snapshots
cover grid points up to the current time.
record_snapshot_chunk
¶
record_snapshot_chunk(t_end: float, *, dt: float, t0: float = 0.0, max_snapshots: int, layout: Layout = 'lef') -> Recording
Run toward t_end, taking at most max_snapshots grid snapshots.
stopped is "recorder_full" when the chunk filled first: call again
for the next one. Consecutive chunks tile the grid exactly.
record_event_chunk
¶
record_event_chunk(t_end: float, *, max_rows: int, keyframes: bool = True) -> Recording
Run toward t_end, logging event deltas into a chunk of about
max_rows rows. With keyframes (the default) the chunk opens with
the full state, so it can be replayed on its own, and may exceed
max_rows by the rows of a zero-delay chain in progress (chunks only
end at settled times). stopped is "recorder_full" when the chunk
filled first: call again for the next one. time is the time through
which the chunk describes the run.
stream
¶
stream(t_end: float, *, dt: Optional[float] = None, t0: float = 0.0, events: bool = False, keyframes: bool = False, layout: Layout = 'lef', max_snapshots: Optional[int] = None, max_rows: Optional[int] = None) -> Stream
Hand this simulation to a lazy Arrow stream that runs it to t_end,
one bounded chunk per batch, as the record_*_chunk methods would.
Pass exactly one of dt (grid snapshots, at most max_snapshots per
batch) or events=True (deltas, about max_rows rows per batch).
With events=True, keyframes=True, every batch opens with one
Keyframe row per leg, so each batch can be replayed on its own; a
batch may then exceed max_rows by the rows of a zero-delay chain
in progress (typically two), since batches only end at settled times.
The stream can be iterated for Batches, or passed to any Arrow
PyCapsule consumer (e.g. pyarrow.RecordBatchReader.from_stream,
polars.DataFrame, duckdb). It runs without the GIL, so it can be
consumed from any thread.
The stream's schema metadata says where it came from, once for the
whole stream: lefsim.model (the model the run started from, as
JSON) and lefsim.run (JSON: seed as a string, start_time,
record_start, the requested record_end or null, lefsim_version).
The Sim is moved into the stream and can't be used afterwards; copy
it first (copy.copy(sim)) to keep going from here.
RunReport
¶
What a run did: events fired, the time reached, and why it stopped ("reached_time", "reached_event_count", "exhausted", or for the chunk methods "recorder_full").
stopped
instance-attribute
¶
stopped: Stopped
Why the run stopped: "reached_time", "reached_event_count", "exhausted", or, for the chunk methods, "recorder_full".
Recording
¶
What a record_* call recorded, plus its run report fields.
Data¶
Batch
¶
An immutable Arrow record batch.
Implements __arrow_c_array__, __arrow_c_stream__ and
__arrow_c_schema__, so it can be passed directly to e.g.
pyarrow.table(...), polars.DataFrame(...) or queried by name from
duckdb.
metadata
property
¶
metadata: dict[str, str]
Schema metadata, e.g. {"lefsim.kind": "snapshots", ...}.
Stream
¶
A simulation that runs lazily as it is read, one Arrow batch at a time.
Iterate it for lefsim.Batches, or pass it to an Arrow PyCapsule stream
consumer. Either way it can be read only once.
time
property
¶
time: float
The time the batches read so far (by iterating) describe the run through: where the next batch starts. At the end of the stream it's the time the stream was asked to end at.
to_parquet
¶
to_parquet(path: Any, *, compression: str = 'zstd') -> None
Write this stream to a Parquet file as it runs, one row group per
batch, so the file indexes itself for random access
(Replayer.from_parquet). Only for keyframed event streams
(events=True, keyframes=True). The footer carries lefsim.model
and lefsim.run, with record_end set to where the data actually
ends (if writing stops early, e.g. on Ctrl-C, the file is still
valid up to there), and lefsim.row_groups = "keyframed".
compression is any codec pyarrow writes; the default, zstd, and
snappy are also readable by the TS package's lefsim/parquet.
Requires pyarrow. Consumes the stream.
Replaying¶
Replayer
¶
Replayer(lattice: Lattice, n_lefs: int)
Rebuilds the state of a run at any recorded time from a keyframed event
log (events=True, keyframes=True), with no simulation.
Use Replayer.from_arrow(...) to load a whole log, or push() chunks as
they arrive. Then seek(t) (any direction) or advance_to(t) (forward,
cheaper), and read state(), world() or snapshots(dt). Rows before
the first keyframe are skipped, so a log may start partway into a run.
The state at time t is the state after every event at or before t,
exactly what Sim.run_until(t) leaves. world_at(t) returns it as a
World, to start a new run from.
row_groups_read
property
¶
row_groups_read: int
How many Parquet row groups from_parquet has read so far (0 for a
replayer not backed by a file).
start
property
¶
start: Optional[float]
Earliest time that can be sought to (the first keyframe), or None.
from_arrow
classmethod
¶
from_arrow(source: Union[Any, Iterable[Any]], lattice: Lattice, *, n_lefs: Optional[int] = None, end: Optional[float] = None) -> Replayer
Load a whole keyframed event log: any Arrow table, batch or stream
(pyarrow, polars, a Stream, pq.ParquetFile(...).iter_batches())
or an iterable of them. Batch boundaries don't matter: the log is
re-split at its keyframes. n_lefs defaults to the size of the
first keyframe. end is the time through which the log describes
the run (e.g. the t_end it was recorded to); it defaults to the
last row, since event rows alone can't say how long the state after
the last event lasted.
from_parquet
classmethod
¶
Open a Parquet file written by Stream.to_parquet for random access:
only the footer is read up front, and each seek, advance_to or
snapshots reads just the row groups it needs (one per keyframed
chunk, found from the footer's time statistics). The lattice and the
end come from the file's lefsim.model and lefsim.run; pass
lattice for a file without them. A file not written that way
(e.g. by duckdb COPY) is read whole, as from_arrow would.
Requires pyarrow.
push
¶
push(events: Any, end: Optional[float] = None) -> None
Append a chunk of event rows (whole keyframes only). end is the
time through which the chunk describes the run (a Recording's
time); it defaults to the chunk's last row.
advance_to
¶
advance_to(t: float) -> float
Move forward to t (or back, by seeking), clamped to the data: it
never fails. Returns the time reached.
state
¶
state(*, layout: Layout = 'lef') -> Batch
The state at time as a one-snapshot Batch, like Sim.state().
snapshots
¶
snapshots(dt: float, *, t0: float = 0.0, t_start: Optional[float] = None, t_end: Optional[float] = None, layout: Layout = 'lef') -> Batch
Snapshots on the grid t0 + k * dt from t_start through t_end
(default: the end of the log): the same table
Sim.record_until(dt=...) records. Without t_start, starts from
the current time (or the first keyframe), so consecutive calls on the
same grid continue it.
effective_distances
¶
effective_distances(sites: Sequence[int], *, dt: float, t0: float = 0.0, t_start: Optional[float] = None, t_end: Optional[float] = None, step_length: int, hop_length: int) -> tuple[Any, Any]
For each point of the grid snapshots(dt, ...) would take, the
effective distance of every pair of sites (condensed, as
World.effective_distances). Returns (times, distances):
array.array("d") of grid times, and array.array("q") of distances,
row-major, one row of k(k-1)/2 per time. A replayer opened with
from_parquet reads row groups as it walks the grid.