heapprof.reader: The high-level API

class heapprof.reader.Reader(filebase: str)

Bases: object

Reader is the basic API for reading a heap profile.

hasDigest() → bool

Test if this Reader already has a digest (i.e. an ._hpc file). If not, you can create one using makeDigest; this is a slow operation (O(minutes), usually) so isn’t done on its own.

makeDigest(timeInterval: float = 60, precision: float = 0.01, verbose: bool = False) → None

Parse the ._hpm and ._hpd files to form a digest. You need to do this before most of the methods will work.

NB that this method deliberately does not check self.hasDigest(); you can use this to stomp an existing digest and create a new one.

NB also that if this function is interrupted (say, with a ctrl-C) it should still yield a valid ._hpc file; the file will just stop at whatever time makeDigest() has gotten up to by the time it was stopped. This doesn’t apply if the function is interrupted in a way that kills the interpreter midway through, like kill -9’ing the process, though.

Parameters

• timeInterval – The time interval between successive snapshots to store in the digest, in seconds.

• precision – At each snapshot, stack traces totalling up to this fraction of total memory used at that frame may be dropped into the “other stack trace” bucket. This can greatly shrink the size of the digest at no real cost in usefulness. Must be in [0, 1); a value of zero means nothing is dropped.

• verbose – If true, prints status information to stderr as it runs.

close() → None

Close the reader. After doing this, the reader is no longer usable.

initialTime() → float

Return the time, in seconds since the epoch, when the program run started. This is useful if you want to compare this to logs data.

finalTime() → float

Return the time, in seconds since the epoch, of the last snapshot stored.

elapsedTime() → float

Return the relative time between program start and the last snapshot.

samplingRate() → Dict[int, float]

Return the sampling rate parameters passed to the profiler.

snapshotInterval() → float

Return the time interval, in seconds, between successive time snapshots in the digest.

trace(traceindex: int) → Optional[List[heapprof.types.TraceLine]]

Given a trace index (of a sort which you can get from various other functions), return a proper stack trace. A value of None means that we have no trace stored for this index.

rawTrace(traceindex: int) → Optional[List[heapprof.types.RawTraceLine]]

Like trace(), but the raw trace doesn’t include the actual lines of code, so is cheaper to fetch.

snapshots() → Sequence[heapprof.types.Snapshot]

Return a sequence of all the time snapshots in the digest.

snapshotAt(relativeTime: float) → heapprof.types.Snapshot

Return the snapshot closest in time (rounding down) to the indicated relative time.

property hpm

Access to the low-level HPM interface.

property hpd

Access to the low-level HPD interface.

property hpc

Access to the low-level HPC interface.

fastGetUsage(snapshot: heapprof.types.Snapshot, lines: Tuple[heapprof.types.RawTraceLine, ...], cumulative: bool = True) → Tuple[Tuple[int, ...], int]

This is a function to quickly fetch usage numbers without computing full usage graphs. Given a snapshot and a list of N raw trace lines, this will return an array with N+1 elements. The first N elements are the cumulative (or local) usage at the indicated trace lines; the last element is the total usage for all trace lines.

class TimePlot(times, totalUsage, lines, labels)

Bases: tuple

property times

Alias for field number 0

property totalUsage

Alias for field number 1

property lines

Alias for field number 2

property labels

Alias for field number 3

pyplot(scale: int = 1048576) → None

Show this plot using pyplot. Note that this method will only work if you have separately pip installed matplotlib; we deliberately don’t add this, as it would create a lot of dependency bloat!

The scale is a scaling applied to byte quantities.

timePlot(lines: Optional[Dict[str, Union[str, heapprof.types.RawTraceLine]]] = None) → heapprof.reader.TimePlot

Sometimes, after you’ve looked at usage graphs and so on, you want to see how memory usage in certain parts of the program is varying over time. This function helps you with that.

Parameters

lines –

If given, this is a map from display label to lines of code whose usage you want to monitor. In this case, the output data will show memory usage at those lines, in addition to overall usage by the program.

The lines may be specified either as RawTraceLine, or as “filename:lineno”. This latter form is provided for convenience while debugging.

flowGraph(snapshot: heapprof.types.Snapshot) → heapprof.flow_graph.FlowGraph

Compute a FlowGraph given a snapshot. See flow_graph.py for the details of what these are and how they can be used for analysis; these are one of your best ways to both visualize a snapshot or analyze time-dependence.

Use this method if you want a graph visualization of your data.

NB: The first call to flowGraph on a Reader may be a bit slow, because it has to load up all the stack traces from the ._hpm file; once that cache is warm, future reads will be much faster.

flowGraphAt(relativeTime: float) → heapprof.flow_graph.FlowGraph

compareFlowGraphs(dotFile: str, *relativeTimes, **kwargs) → None

Generate a graph view of the comparison of a bunch of different time slices, and save the result as a .dot file to the given name. See FlowGraph.compare() for the kwargs available.

flameGraph(snapshot: heapprof.types.Snapshot, output: TextIO) → None

Write a snapshot in Brendan Gregg’s “collapsed stack” format. This format can be visualized as a Flame graph with tools like speedscope.app. (NB that if you’re using speedscope, only the “left heavy” and “sandwich” views will make any sense; the “time order” view is intended to show CPU profiles over time, which would be nonsensical for this type of data)

For speedscope, see https://github.com/jlfwong/speedscope, or use the hosted version at https://www.speedscope.app.

flameGraphAt(relativeTime: float, output: TextIO) → None

writeFlameGraph(filename: str, when: Union[float, heapprof.types.Snapshot]) → None

Convenience helper: Grab a snapshot at a particular relative time, and write it in collapsed stack format to filename.