The Design of VisiData: Display Engine
Every displayed screen is an instance of the
Sheet class, which has several members:
topRowIndexindex of row just below header row
cursorColIndex(computed, O(ncols)); index into .columns
cursorCol: columns[cursorColIndex] (equivalent to visibleCols[cursorVisibleColIndex])
nScreenRows: simply based on terminal height, the exact number of rows that can be displayed.
visibleRows: slice of onscreen rows; 'visible' has different meaning than visibleCols
visibleCols: (O(ncols), cached between frames): list of all non-hidden columns
leftVisibleColIndexis the first non-key column on the left.
Sheet.calcColLayout(), assignments ignored)
- populated by
- must be a randomly addressable sequence
- can only be reordered if rows is mutable
- rows is usually a list. [future] could be other types, would go terabyte easily then
- populated by
- populated by init or copied from subclass
- can be populated by reload if data dependent
*VisibleCol* property is an index into
Originally, cursorColIndex was the canonical column position, but this made many things more complicated than necessary. In particular, this allowed the cursor to be on a hidden column, which had to be considered for many column commands.
The canonical column position is now cursorVisibleColIndex, so the cursor is always on a non-hidden column by design. cursorColIndex (and others) must now be computed, which can be expensive if there are many columns, but these computed properties can be cached, and the resulting code is cleaner.
The fundamental abstraction of VisiData, is that every Column is essentially a function of the row.
A VisiData column, in essence, is simply a function which takes a row and returns a value.
The Column object has other associated niceties (any of which may be passed as kwargs to init) - name set with
^ - getter(r): the main function called by Column.calcValue() - setter(sheet,col,row,val): the function called by Column.setValue() set with 'e' 'zd' or 'Del' sets to None - width: '0' if hidden;
None if should be auto-set on first visibility (default) adjusted by '-' and '_' - type: int, str, float, date, currency, anytype (default) set with ~@#$% (cannot be reset to anytype from interface) - fmtstr strftime style format for date columns:
%Y-%m-%d by default new style python format:
name, type, width, and fmtstr can all be edited on the Columns metasheet.
[will probably become getter(col, row) and setter(col, row, val). Use col.sheet if needed][previously was %s C-style format, which I might still prefer]
Though I resisted this for a long time, Columns are now associated with their sheet; the same exact Column object cannot be used on multiple sheets. In practice this seems okay; just use
Sheet.addColumn(copy(Column)) and addColumn will set the .sheet member of the new Column properly.
The Column's sheet is in the .sheet member, but that is not set until
Sheet.recalc() (first reload). This can be a problem when a getter or setter needs to know the sheet; it can't pass it into the constructor, and the getter is not given the column. The solution is to assign columns in the reload() and bind the sheet in the lambda.
[same recursive lookup for colors as with commands][drop color precedence altogether; cell overrides row overrides col overrides hdr] [attributes (bold, underline, reverse) always used for header/column/row and not configurable]
Here is an extremely simple sheet that shows a list of all global variables with their values:
from vdtui import *
Sheet.__init__(*names, **kwargs) joins the names with
options.name_joiner, and other kwargs besides
source may be provided for convenience. - columns are set in reload, because they require the sheet's context, for the source dict, to be bound to the lambda. [Future: setter=lambda col,row,val: col.sheet.source[row] ] - The structure of the row objects is so important that I have taken to including a 'rowdef' comment above every sheet, describing what kind of object each row is. - The getters are passed into the Column init kwargs directly. The default getter is the identity function, so the 'key' getter is actually unnecessary. - There are other possible designs of this sheet: - rows could be list(self.source.items()); but then when an item changes, this sheet would not change until reload (^R) - could take generic dict; use source
class VisiData and vd
The VisiData singleton (accessible via
scrFull: the curses screen object
sheets: a list;
sheetsis the actively displayed sheet
vd.screenHeight are the dimensions of the current terminal screen. (
sheet.windowHeight are the dimensions of that sheet's specific window, including status line).
visidata.run() and vd.mainloop()
VisiData.mainloop(scr) is the main display loop. It calls
draw() on the top sheet, left and right statuses, and handles commands, until there are no more sheets.
VisiData exits when this function returns.
This function must be called with the curses screen object. Applications should call the module-level
run(*sheets) with the sheets they want pushed initially, and VisiData will initialize curses to its liking.
Handles drawing everything on the screen but the status bars.
Sheet.calcColLayout() uses: -
Sheet.leftVisibleColIndex: leftmost visible non-key column
and computes: -
Sheet._visibleColLayout (dict of vcolidx to (onscreen x, w)); only for onscreen columns -
Sheet.rightVisibleColIndex (the rightmost visible column) - any None Column.width (sets to max width of values in onscreen rows)
For symmetry, there is also
Sheet._rowLayout (dict of rowidx to onscreen y), computed during draw(). This allows for multi-line rows.
Sheet.calcColLayout should be called at least whenever a Column is added, deleted, or its width is changed. In practice, it is called at the beginning of every draw cycle anyway. It is also used to get a good 'feel' during pageLeft() and checkCursor().
For each cell,
Column.getCell(row) produces a
DisplayWrapper, which has the whole deal: Cell returns DisplayWrapper, which is the whole deal; the original value, the fully typed and formatted display string, a note character and note color, associated error
valuethe raw result returned from the getter
displayfull string in the cell
errorif any error occurred (use getattr(error))
noteon far right of cell, in notecolor
notecolor, text string ('green')
[DisplayWrapper will allow per-character colors; should colorizers be called before or after, or be subsumed by DW?]
getCell does the most amount of work per cell, and there are several other internal getters which do less work for a purpose other than display. From the least amount of work to the most:
Column.getValue(row)The main function to call to get the raw value. Will cache the result if
Column._cachedValuesis a mapping (which it is, if
cache=Trueis in the Column init kwargs. (In VisiData proper,
z'will add or reset the cache for the current column].
Column.calcValue(row)Computes the raw value every time. This is the function to override if subclassing Column.
Column.getTypedValue(row)Produces a guaranteed result, coerced to Column.type. When every cell in this column simply must have a value (like sort).
Column.getCell(row)This returns a DisplayWrapper no matter what. It does type conversion, formatting, decoding, exception handling, and annotating. The only thing it doesn't do is colorizing.
Column.getDisplayValue(row)Returns a guaranteed string value equivalent to what would be displayed in wide-enough cell. Deprecated: use
cursorValue is equivalent to cursorCol.getValue(cursorRow)
, and so on withcursorTypedValue