A lot of features have been added to VisiData. So many, in fact, that we are calling the next big release v2.0. But with so many changes, we need your help to test and find bugs.
If you can handle the sharp cutting edge of VisiData, then go ahead and install the latest pre-release version:
pip3 install git+https://github.com/saulpw/visidata@v2.-3.0
Then, play with the new features and file any issues you come across.
And we are steadily approaching the finish line! With v2.-3 all of the major features that we wanted to be part of v2.0 are complete. Since v2.-2, 16 issues filed by users had code written for them and 6 PRs written by non-maintainers have been merged in. As always, the full summary of changes has been listed in the CHANGELOG. The large bulk of the changes were improvements in VisiData internals, but a few features definitely stood out.
It is here! VisiData can now show the two sheets on the Sheets Stack simultaneously.
Shift+Zsplits the screen in half so that the second-most-recently accessed sheet is visible in a second pane.
z Shift+Zsplits the screen, and queries for the height of the second pane
g Shift+Z sets the current pane to the whole screen (effectively closing an already split screen)
Ctrl+^swaps which sheet is in the current pane
TAB jumps to the other pane
For more information, check out the blog post on the feature.
There are some formats (csv, tsv, json, etc) for which VisiData contains both a loader and a saver. A roundtrip is a trait where if a file is loaded, and then immediately saved to the same format, there should be a minimal or no diff. It is important that diffs be informative and only show the actual changes that occurred to the contents of data.
vd -b input.json -o output.json
Until now, this was not the case with jsons. The columns would get shuffled upon save.
Now, the column order is preserved upon save. This was a big blocker towards incorporating the the json/jsonl format for internal data sources.
This leads to....
If you save the CommandLog as .vdj, it will get saved in the .jsonl format. This .vdj can be replayed and loaded, just like a .vd.
Since the jsonl format permits the inclusion of tabs and newlines within cells, this will provide space to these formats to further develop.
Thanks so much to everyone that is helping us prepare for v2.0. This release will feature a bucketload of bugfixes and additions along with the following new features. Between v2.-1 and v2.-2, 26 issues filed by users had code written for them.
This endeavor was hinted in this interview:
There is one thing that I do wish that I could solve, and I'm sure that it is solvable, but I haven't managed it yet. The command log reports every action that you take, including all the dead ends that you wind up finding yourself in. Those are sometimes handy to have on there, so you can see what you did and didn't want to keep around. But when you're getting to the end, and you want to do it again, you want to cull all those dead ends out, and just get to the place that you currently are. And so I want to have some kind of graph or tree that would get you from your current state and only show the commands that you took to get there.
and now it is here!
Shift+D is now mapped to
cmdlog-sheet, and takes you to the current sheet's commandlog, with all the other loose ends removed. If missed,
gShift+D goes to the standard, classic commandlog that contains everything.
It has become apparent that the write-mode of the DirSheet was a conceptually separate tool, even if the interface fits in seamlessly in VisiData. Data sheets and filesystem editing are two radically different use cases.
So from 2.-2 and on, the filesystem manipulation part of VisiData was moved into the entry point vls.
To play with, type
pip3 install git+git://github.com/saulpw/visidata/vsh@develop
and then type
vls. Please share any feedback here.
(To learn about the write-mode of the DirSheet, check out the following case study video.)
The package is called
vsh because it is going to be a compilation of vd interfaces for Linux utilities.
On top of
vls, it also includes:
|Please note, that there are two open bugs for defermods at the time of this release. Subscribe to those issues to get updates on when they have been fixed.|
Activating this plugin adds two modes to VisiData: Defermods and Trackmods.
Defermods enables modification for sheets that are deferred (e.g. sqlite and vls). Saving to source is deferred until
Ctrl+S). Deleted rows and modified cells are colored as pending changes until committed. To save to a different file (not source), use
Trackmods tracks changes in sheets whose source has a path. Modifications are colored yellow until changes are committed with
Ctrl+S) or saved with
These modes are not on by default,
from plugins.defermods import * must be added to visidatarc (it can also be installed through the PluginsSheet. Sheets must set .defermods and/or .trackmods. By default BaseSheet has .defermods=False and .trackmods set to True when defermods is imported.
defermods is not on by default because: 1) deletes with defermods enabled take longer; especially delete-selected 2) it is easier for users to consciously opt-in to a novel VisiData experience than to opt-out
As requested by aborruso!
+:subsheet:col:row as a command line argument now sets
subsheet to be the topsheet upon load, with cursor located in cell at
col. All of the components are optional. For example,
+:subsheet:: sets the topsheet to be
subsheet with the cursor located at the top left.
+::col:row sets the cursor in the cell at
col in the topsheet. The toplevel source index can be explicitly named, if there is more than one with
This currently works for the html, xls, xlsx, xlsb, hdf5, and sqlite loaders.
Previously, if a user launched
vd without a source, the DirSheet for the current directory would pop up.
vd without a source file, now opens the vdmenu, a menu of core sheets. Press
Enter to open the sheet referenced in the current row.
Edit: this feature is currently under revision.
Alt+1 will go to the first loaded sheet,
Alt+2 goes to the second etc. A sheet's number is in the lower left of the status line, and in the new
shortcut column on the SheetsSheet.
The Alt (or Esc- prefix) layer is otherwise reserved for your personal bindings of builtin and/or custom commands. Use
^[x as the keybinding for
By popular request, VisiData now has undo/redo for certain operations. Press
Shift+U to undo the last modification on the current sheet, and then
Shift+R to redo what you just undid.
[In general undo will only be available for commands that could change which data is saved, which means that hiding a column is undoable but changing its width is not.]
Please let us know if any command is missing an undo, or if there is anything about an undo or redo operation that feels "weird".
EDIT: This feature has been moved out of VisiData core and into the plugin: defermods.py. See the list of features in 2.-2 for details.
When I made the DirSheet and gave it the power to delete files, and modify metadata, I quickly realized it needed to be a "deferred sheet". Modifications and deletions were staged instead of committed immediately. Then this concept transferred well to vgit, and now it seems like it might be generally useful.
Now, most sheets in VisiData will keep track of their additions/modifications/deletions, and color them accordingly. You can save the changes back to the source with the usual
Ctrl+S and revert all changes back to the most recent save with
Please let us know how the modify/save/confirm process feels on any sheet, with particular attention to derived sheets and metasheets.
Previously, VisiData only had single-line rows; to expand long text you had to increase the width of the column.
Now, VisiData v2.x supports optional multi-line rows! Toggle it by pressing
v on any sheet with truncated values, and cells in all columns will expand their height and wordwrap to show the full value.
Please let us know how multi-line rows looks and feels for you; in particular how scrolling with multi-line rows feels.
Per user request, we have added a feature flag for range binning of numeric columns.
If you set the option
True, Freqency and Pivot tables with numeric columns will bin the values into ranges.
If you play with it, let us know how it feels, and if you find any issues with edge cases.
Did you know that if you want VisiData to do something new or different, you can write a plugin and start using it right away? Some plugins have already been written for 1.5.2, like jsvine's vdnormcol and vddedupe.
2.0 will add a PluginsSheet, which fetches the list of known plugins from visidata.org/plugins.tsv, and allows you to easily install and activate (
a), and/or deactivate (
d) them. For instructions on how to manually incorporate plugin code, see our docs and dev checklist.
This plugin framework is what makes VisiData a platform, and not just a tool.
The API reworking that we're doing is substantial, so existing plugins and .visidatarc files may need to be changed in a bunch of small ways. With 2.0, we are going to refine and document a stable and reasonable API for plugin authors to rely on.
And we're doing this work, because the plugin and app ecosystem for VisiData very well might become more important than the VisiData core utility itself. Like some people use emacs mostly for magit or org-mode, or jupyter and pandas with python.
There are also lots more changes! Even if you do not end up playing with any of the new features, just using VisiData as you usually do and filing issues as they come would be an enormous help.