Skip to content

Python API

Stochastic simulation of loop-extrusion factors (LEFs) on a 1D lattice.

Describe a model, then run it:

lattice = lefsim.Lattice(3000, {"site": [600, 1300], "captures_from": ["right", "left"], "strength": 0.9})
world = lefsim.populate(lattice, 20, seed=1)  # or lefsim.World(lattice, lefs_table)
model = lefsim.Model(world, lefsim.Rates(step=1.0, respawn=2e-3, respawn_blocked=2e-2, respawn_captured=2e-3))
sim = lefsim.Sim(model, seed=1)

sim.run_until(1_000.0)  # advance
rec = sim.record_until(2_000.0, dt=1.0, events=True)  # advance and record

Recorded data comes back as Batch objects: Arrow record batches exposed through the Arrow PyCapsule Interface, so they can be passed straight to pyarrow.table, polars.DataFrame or queried from duckdb, without this package depending on any of them. Sim.stream runs lazily as an Arrow stream, and Replayer rebuilds states from a recorded event log.

Building a model

Lattice

Lattice(n_sites: int, ctcfs: Union[Mapping[str, Any], Any, None] = None)

A 1D lattice of n_sites sites, some of them CTCFs.

ctcfs is a table with one row per CTCF: site, captures_from ('left' or 'right': the side that captures) and optionally strength (the probability, in [0, 1], that a landing leg interacts; 1 by default). Pass any Arrow table (pyarrow, polars, Lattice.to_arrow()) or a dict of columns, where captures_from and strength may be single values for all CTCFs.

n_sites property

n_sites: int

Number of sites.

n_ctcfs property

n_ctcfs: int

Number of CTCF sites.

n_plain property

n_plain: int

Number of non-CTCF sites.

to_arrow

to_arrow() -> Batch

The CTCF table (site, captures_from, strength), with lefsim.n_sites in the schema metadata.

to_dict

to_dict() -> Any

As plain Python data (dicts, lists, numbers and strings), in the same shape as the JSON.

from_dict classmethod

from_dict(d: Any) -> Lattice

Load from plain Python data shaped as to_dict returns it, with the same checks as construction.

to_json

to_json(*, indent: Optional[int] = None) -> str

As a JSON string; indent pretty-prints.

from_json classmethod

from_json(s: str) -> Lattice

Load from a JSON string as to_json writes it, with the same checks as construction.

World

World(lattice: Lattice, lefs: Union[Mapping[str, Sequence[Any]], Any, None] = None)

The state of a simulation at one moment: a lattice and the LEFs on it.

lefs is a table with one row per LEF (row i is LEF i): columns left_pos and right_pos, and optionally left_mobility and right_mobility ('moving' by default). Pass any Arrow table (pyarrow, polars, World.to_arrow(), Sim.state()) or a dict of sequences.

Every LEF is checked: legs on the lattice and not crossed, no two LEFs on one site, and a 'captured' leg only on a CTCF that could have captured it (a left-facing leg by a CTCF that captures from the right, a right-facing leg by one that captures from the left).

lattice property

lattice: Lattice

The lattice.

n_lefs property

n_lefs: int

Number of LEFs.

to_arrow

to_arrow() -> Batch

The LEF table: lef_id, left_pos, right_pos, left_mobility, right_mobility.

effective_distance

effective_distance(a: int, b: int, *, step_length: int, hop_length: int) -> int

The effective distance between sites a and b: the shortest path when a path walks the lattice (step_length per step) and may cross LEFs (hop_length per crossing; across a chain of LEFs side by side, a rosette, a hop per LEF and the junctions between them free). Both lengths are integers, so distances are exact; divide by step_length for lattice steps.

effective_distances

effective_distances(sites: Sequence[int], targets: Optional[Sequence[int]] = None, *, step_length: int, hop_length: int) -> Any

Effective distances (see effective_distance). Without targets, every pair of sites, condensed as scipy.spatial.distance.pdist (squareform makes it a matrix). With targets, each of sites to each of targets, row-major. Returns an array.array("q"); numpy.asarray wraps it without copying.

to_dict

to_dict() -> Any

As plain Python data (dicts, lists, numbers and strings), in the same shape as the JSON.

from_dict classmethod

from_dict(d: Any) -> World

Load from plain Python data shaped as to_dict returns it, with the same checks as construction.

to_json

to_json(*, indent: Optional[int] = None) -> str

As a JSON string; indent pretty-prints.

from_json classmethod

from_json(s: str) -> World

Load from a JSON string as to_json writes it, with the same checks as construction.

populate

populate(lattice: Lattice, n_lefs: int, *, seed: int = 0, spawn_rule: SpawnRuleName = 'uniform_non_ctcf') -> World

A world with n_lefs LEFs placed one at a time by a spawn rule, exactly as respawns place them. seed drives only this placement.

Rates

Rates(step: Optional[float] = None, respawn: Optional[float] = None, respawn_blocked: Optional[float] = None, respawn_captured: Optional[float] = None)

Event rates (per unit time): step for a moving, unblocked leg (a leg facing a CTCF from behind splits it between passing and eviction); respawn, the basal rate, for a LEF with both legs free to move; respawn_blocked for a LEF with either leg blocked (by a leg ahead or the lattice's end); respawn_captured for one with either leg captured, which wins over blocked. Eviction unloads a LEF at once; a leg whose partner is captured is immune to it. Checked (positive and finite) by Model.

step property

step: float

Step rate of a moving, unblocked leg.

respawn property

respawn: float

Basal respawn rate: a LEF with both legs free to move.

respawn_blocked property

respawn_blocked: float

Respawn rate of a LEF with either leg blocked (by a leg ahead or the lattice's end), unless the other is captured.

respawn_captured property

respawn_captured: float

Respawn rate of a LEF with a captured leg, whatever the other leg does.

to_dict

to_dict() -> Any

As plain Python data (dicts, lists, numbers and strings), in the same shape as the JSON.

from_dict classmethod

from_dict(d: Any) -> Rates

Load from plain Python data shaped as to_dict returns it, with the same checks as construction.

to_json

to_json(*, indent: Optional[int] = None) -> str

As a JSON string; indent pretty-prints.

from_json classmethod

from_json(s: str) -> Rates

Load from a JSON string as to_json writes it, with the same checks as construction.

Model

Model(world: World, rates: Optional[Rates] = None, *, spawn_rule: SpawnRuleName = 'uniform_non_ctcf')

A complete model: a world, its rates, and a spawn rule. Everything a run needs except a seed.

Checks that the rates are positive and finite, and that the spawn rule can always place the world's LEFs. to_json() and Model.from_json() save and load it as a versioned document.

world property

world: World

The world.

lattice property

lattice: Lattice

The world's lattice.

rates property

rates: Rates

The rates.

spawn_rule property

spawn_rule: SpawnRuleName

The spawn rule's name.

to_dict

to_dict() -> Any

As plain Python data (dicts, lists, numbers and strings), in the same shape as the JSON.

from_dict classmethod

from_dict(d: Any) -> Model

Load from plain Python data shaped as to_dict returns it, with the same checks as construction.

to_json

to_json(*, indent: Optional[int] = None) -> str

As the versioned JSON model document (lefsim.model, version 2); indent pretty-prints.

from_json classmethod

from_json(s: str) -> Model

Load a JSON model document as to_json writes it, with the same checks as construction.

Running

Sim

Sim(model: Model, *, seed: int = 0, start_time: float = 0.0)

A running simulation of a Model.

seed drives the whole run: the same model and seed give the same run, bit for bit, on every platform. start_time sets the clock, e.g. to continue from a state recorded at that time. The model is copied in; sim.model() returns the current state as a new Model.

run_* methods advance the run; record_* methods advance it and return what happened as Arrow data. copy.copy(sim) forks the run: the copy continues exactly as the original would.

time property

time: float

Current simulation time.

n_events property

n_events: int

Total events fired so far.

next_event_time property

next_event_time: Optional[float]

Time of the next event, or None if none remain.

n_sites property

n_sites: int

Number of lattice sites.

n_lefs property

n_lefs: int

Number of LEFs.

n_legs property

n_legs: int

Number of legs (two per LEF).

rates property

rates: Rates

The rates this run uses.

seed property

seed: int

The seed this run was started with.

start_time property

start_time: float

The clock at construction.

model

model() -> Model

The current state as a Model: the world now, with this run's rates and spawn rule. Sim(sim.model(), seed=..., start_time=sim.time) continues from here with fresh randomness.

initial_model

initial_model() -> Model

The model this run started from. With seed and start_time, it says where a recording came from: Sim(sim.initial_model(), seed=sim.seed, start_time=sim.start_time) replays the run from its start, event for event.

world

world() -> World

The current state as a World.

state

state(*, layout: Layout = 'lef') -> Batch

The current state as a one-snapshot Batch, without advancing.

step

step() -> Optional[tuple[float, Literal['Step', 'Respawn', 'Evict'], int]]

Fire one event. Returns (time, action, id) — id is a leg_id for "Step" and "Evict", a lef_id for "Respawn" — or None if no events remain.

run_until

run_until(t_end: float) -> RunReport

Fire every event at or before t_end, then set the clock to t_end. Releases the GIL while running; Ctrl-C interrupts.

run_events

run_events(n: int) -> RunReport

Fire up to n events.

record_until

record_until(t_end: float, *, dt: Optional[float] = None, t0: float = 0.0, events: bool = False, keyframes: bool = False, layout: Layout = 'lef') -> Recording

Run until t_end, recording grid snapshots (with dt, on the grid t0 + k * dt, inclusive of t_end) and/or event deltas (with events=True; add keyframes=True to open the event table with one Keyframe row per leg holding the full state). Consecutive calls on the same grid continue it seamlessly. Releases the GIL while running; Ctrl-C interrupts.

record_events

record_events(n: int, *, dt: Optional[float] = None, t0: float = 0.0, events: bool = False, keyframes: bool = False, layout: Layout = 'lef') -> Recording

Fire up to n events, recording as in record_until; snapshots cover grid points up to the current time.

record_snapshot_chunk

record_snapshot_chunk(t_end: float, *, dt: float, t0: float = 0.0, max_snapshots: int, layout: Layout = 'lef') -> Recording

Run toward t_end, taking at most max_snapshots grid snapshots. stopped is "recorder_full" when the chunk filled first: call again for the next one. Consecutive chunks tile the grid exactly.

record_event_chunk

record_event_chunk(t_end: float, *, max_rows: int, keyframes: bool = True) -> Recording

Run toward t_end, logging event deltas into a chunk of about max_rows rows. With keyframes (the default) the chunk opens with the full state, so it can be replayed on its own, and may exceed max_rows by the rows of a zero-delay chain in progress (chunks only end at settled times). stopped is "recorder_full" when the chunk filled first: call again for the next one. time is the time through which the chunk describes the run.

stream

stream(t_end: float, *, dt: Optional[float] = None, t0: float = 0.0, events: bool = False, keyframes: bool = False, layout: Layout = 'lef', max_snapshots: Optional[int] = None, max_rows: Optional[int] = None) -> Stream

Hand this simulation to a lazy Arrow stream that runs it to t_end, one bounded chunk per batch, as the record_*_chunk methods would.

Pass exactly one of dt (grid snapshots, at most max_snapshots per batch) or events=True (deltas, about max_rows rows per batch). With events=True, keyframes=True, every batch opens with one Keyframe row per leg, so each batch can be replayed on its own; a batch may then exceed max_rows by the rows of a zero-delay chain in progress (typically two), since batches only end at settled times. The stream can be iterated for Batches, or passed to any Arrow PyCapsule consumer (e.g. pyarrow.RecordBatchReader.from_stream, polars.DataFrame, duckdb). It runs without the GIL, so it can be consumed from any thread.

The stream's schema metadata says where it came from, once for the whole stream: lefsim.model (the model the run started from, as JSON) and lefsim.run (JSON: seed as a string, start_time, record_start, the requested record_end or null, lefsim_version).

The Sim is moved into the stream and can't be used afterwards; copy it first (copy.copy(sim)) to keep going from here.

RunReport

What a run did: events fired, the time reached, and why it stopped ("reached_time", "reached_event_count", "exhausted", or for the chunk methods "recorder_full").

n_events instance-attribute

n_events: int

Events fired by this call.

time instance-attribute

time: float

Simulation time after the call.

stopped instance-attribute

stopped: Stopped

Why the run stopped: "reached_time", "reached_event_count", "exhausted", or, for the chunk methods, "recorder_full".

Recording

What a record_* call recorded, plus its run report fields.

snapshots instance-attribute

snapshots: Optional[Batch]

Grid snapshots, if dt was given.

events instance-attribute

events: Optional[Batch]

Event deltas, if events=True.

n_events instance-attribute

n_events: int

Events fired by this call.

time instance-attribute

time: float

Simulation time after the call: the recording describes the run through this time.

stopped instance-attribute

stopped: Stopped

Why the run stopped: "reached_time", "reached_event_count", "exhausted", or, for the chunk methods, "recorder_full".

Data

Batch

An immutable Arrow record batch.

Implements __arrow_c_array__, __arrow_c_stream__ and __arrow_c_schema__, so it can be passed directly to e.g. pyarrow.table(...), polars.DataFrame(...) or queried by name from duckdb.

num_rows property

num_rows: int

Number of rows.

num_columns property

num_columns: int

Number of columns.

column_names property

column_names: list[str]

The column names, in order.

metadata property

metadata: dict[str, str]

Schema metadata, e.g. {"lefsim.kind": "snapshots", ...}.

to_pyarrow

to_pyarrow() -> Any

As a pyarrow.Table (requires pyarrow).

to_polars

to_polars() -> Any

As a polars.DataFrame (requires polars).

to_pandas

to_pandas() -> Any

As a pandas.DataFrame (requires pyarrow and pandas).

Stream

A simulation that runs lazily as it is read, one Arrow batch at a time.

Iterate it for lefsim.Batches, or pass it to an Arrow PyCapsule stream consumer. Either way it can be read only once.

column_names property

column_names: list[str]

The column names of every batch, in order.

closed property

closed: bool

Whether the stream has been fully read or handed off.

time property

time: float

The time the batches read so far (by iterating) describe the run through: where the next batch starts. At the end of the stream it's the time the stream was asked to end at.

to_parquet

to_parquet(path: Any, *, compression: str = 'zstd') -> None

Write this stream to a Parquet file as it runs, one row group per batch, so the file indexes itself for random access (Replayer.from_parquet). Only for keyframed event streams (events=True, keyframes=True). The footer carries lefsim.model and lefsim.run, with record_end set to where the data actually ends (if writing stops early, e.g. on Ctrl-C, the file is still valid up to there), and lefsim.row_groups = "keyframed". compression is any codec pyarrow writes; the default, zstd, and snappy are also readable by the TS package's lefsim/parquet. Requires pyarrow. Consumes the stream.

Replaying

Replayer

Replayer(lattice: Lattice, n_lefs: int)

Rebuilds the state of a run at any recorded time from a keyframed event log (events=True, keyframes=True), with no simulation.

Use Replayer.from_arrow(...) to load a whole log, or push() chunks as they arrive. Then seek(t) (any direction) or advance_to(t) (forward, cheaper), and read state(), world() or snapshots(dt). Rows before the first keyframe are skipped, so a log may start partway into a run.

The state at time t is the state after every event at or before t, exactly what Sim.run_until(t) leaves. world_at(t) returns it as a World, to start a new run from.

row_groups_read property

row_groups_read: int

How many Parquet row groups from_parquet has read so far (0 for a replayer not backed by a file).

lattice property

lattice: Lattice

The lattice of the recorded run.

n_lefs property

n_lefs: int

Number of LEFs in the recorded run.

ready property

ready: bool

Whether a keyframe has been applied, i.e. whether there's a state.

time property

time: float

Current replay time (-inf until ready).

start property

start: Optional[float]

Earliest time that can be sought to (the first keyframe), or None.

end property

end: float

Latest time the log describes.

from_arrow classmethod

from_arrow(source: Union[Any, Iterable[Any]], lattice: Lattice, *, n_lefs: Optional[int] = None, end: Optional[float] = None) -> Replayer

Load a whole keyframed event log: any Arrow table, batch or stream (pyarrow, polars, a Stream, pq.ParquetFile(...).iter_batches()) or an iterable of them. Batch boundaries don't matter: the log is re-split at its keyframes. n_lefs defaults to the size of the first keyframe. end is the time through which the log describes the run (e.g. the t_end it was recorded to); it defaults to the last row, since event rows alone can't say how long the state after the last event lasted.

from_parquet classmethod

from_parquet(path: Any, lattice: Optional[Lattice] = None) -> Replayer

Open a Parquet file written by Stream.to_parquet for random access: only the footer is read up front, and each seek, advance_to or snapshots reads just the row groups it needs (one per keyframed chunk, found from the footer's time statistics). The lattice and the end come from the file's lefsim.model and lefsim.run; pass lattice for a file without them. A file not written that way (e.g. by duckdb COPY) is read whole, as from_arrow would. Requires pyarrow.

push

push(events: Any, end: Optional[float] = None) -> None

Append a chunk of event rows (whole keyframes only). end is the time through which the chunk describes the run (a Recording's time); it defaults to the chunk's last row.

advance_to

advance_to(t: float) -> float

Move forward to t (or back, by seeking), clamped to the data: it never fails. Returns the time reached.

seek

seek(t: float) -> float

Jump to t in either direction (clamped to end).

state

state(*, layout: Layout = 'lef') -> Batch

The state at time as a one-snapshot Batch, like Sim.state().

world

world() -> World

The state at time as a World.

world_at

world_at(t: float) -> World

Seek to t and return the state there as a World.

snapshots

snapshots(dt: float, *, t0: float = 0.0, t_start: Optional[float] = None, t_end: Optional[float] = None, layout: Layout = 'lef') -> Batch

Snapshots on the grid t0 + k * dt from t_start through t_end (default: the end of the log): the same table Sim.record_until(dt=...) records. Without t_start, starts from the current time (or the first keyframe), so consecutive calls on the same grid continue it.

trim_before

trim_before(t: float) -> None

Drop data no longer needed to seek to t or later.

effective_distances

effective_distances(sites: Sequence[int], *, dt: float, t0: float = 0.0, t_start: Optional[float] = None, t_end: Optional[float] = None, step_length: int, hop_length: int) -> tuple[Any, Any]

For each point of the grid snapshots(dt, ...) would take, the effective distance of every pair of sites (condensed, as World.effective_distances). Returns (times, distances): array.array("d") of grid times, and array.array("q") of distances, row-major, one row of k(k-1)/2 per time. A replayer opened with from_parquet reads row groups as it walks the grid.