Version 1.0 - Updated November 23rd, 2014

The User Interface of Pipecut

This section is part tutorial, part documentation.

If you prefer manpages, you'll find that kind of layout here.


Pipecut is designed to be highly intuitive for the experienced Unix user, and to let you work in a highly fluid workflow. It is expected that users want to be able to start in their shell of choice, take advantage of pipecut's functionality when it's wanted, and carry back their results for use outside of pipecut.

The main User Interface for pipecut is an ncurses full-screen interface. It is made to be similar in ways to the vi editor, and many letters of the alphabet drive commands that will insert an entire Unix command with one keystroke.

Nothing is finalized. To date, a few dozen people have heard the description of pipecut, or seen me demonstrate it to them. That's simply not a wide enough set of feedback to declare that I've made the right decisions in all respects of the UI. There are many features yet to come, and as they are implemented I'll need to assign keys to them - and they may make better use of certain keys than the commands I've got there now. In the end, pipecut will have a remappable key configuration, so you can customize it the way that works for you - but that's lower on the task list than other features I want done for the first release.

There are some simplifying assumptions in place right now. Right now, every pipeline starts with 'cat filename' - I know how I want to remove that limitation, but it's useful for the time being to be able to assume that the input is in a file that can be opened, and read at will, and that it's a simple command that the filename can be parsed out of. Supporting awk -F: -f awkprog file1 file2 file3 will happen later.

Use Case #1

pipecut filename
            


Pipecut takes a filename argument. Pipecut opens this file read-only, and won't modify it. Run in this mode, pipecut will enter the ncurses UI and display one screenful of the input file, minus two reserved lines in the status area at the bottom of the screen. The first status line shows 'cat filename', and will be highlighted (it's the current Blade). The second status line shows the current mode/flag settings.

Press '?' for help. Don't press 'h' - that will transform the data with 'hexdump -C'. If you do press a key other than '?', you can press backspace to delete the blades you've added.

Suppose you're still at the first Blade. type the letters 'UHhcw'. Your toolset (the command line on the first status line) will now read 'cat filename | sort | uniq -c | sort -nr | hexdump -C | cat -n | wc', and the wc blade will be highlighted - while the screen will display the typical three number output of the wc Unix utility. In fact, the top portion of the screen updated with the transformation of the content as you typed each letter. Use the cursor keys to move left and right and browse what the data looks like at each blade. Seeing the data and your command line together at the same time, and being able to interact with both is the key to how pipecut provides a speedup vs manual command line recall, editing, execution through more/less, quitting out to the shell, and repeating. In pipecut you can accomplish all of that in one keystroke.

While in the middle of the toolset, you can press the 'Delete' key to remove the current blade. I don't yet provide a UI option to insert new blades mid-toolset.

Pipecut is taking another shortcut - and depending on the source file you chose, you may have noticed that the wordcount numbers were on the small side. That's because pipecut 'cuts' the top of the file, just one screenful, and applies your transformations on just that portion of the data. If it didn't do that, then adding a blade such as 'sort' could result in a long delay, if your input file is large. It does mean that the sample output you see is not exactly the same as running the command line in your shell. To execute the toolset as native Unix commands, on the full input file, and view the result in your PAGER, hit the pipe symbol '|'. For whatever blade you are on, the toolset as far as the current blade will be executed, along with your PAGER.

Modes

Pipecut offers modes, including 'command' and 'literal', and sub-modes, including 'Regular Expression', and 'Look Ahead'.

When you can press 's' and add a 'sort' blade, you're in command mode. If one of the blades is highlighted as your current blade, you are in command mode.

If you press the single-quote " ' " character, your cursor will drop in after a | following the last blade, and you can type a literal Unix command. Until you hit Enter, you are in literal mode.

If you are in command mode and hit 'x', you'll see an " | egrep -v ' " blade appended to the toolset, and your cursor will appear to let you type in the desired regular expression to eXclude from the data.

Press Enter after you type the Regex (no need to close the single-quote " ' " - pipecut will do that). Now you'll be back in Command mode, with the highlight on your egrep -v blade.

Press 'L' to turn on the Look-Ahead flag (sub-mode). Now if you cursor to the left once, you'll see highlights on screen wherever your next blade's expression (the Regex you just typed) appears. These are the lines that will be excluded by the following blade.

Press cursor right and you'll see them disappear. Cursor left and right a few times for the fun of it (you know you want to, if you haven't already done it).

Turn off the Look-Ahead flag by pressing 'L' while in command mode.

RE (Regular Expresion) mode

Turn on the Regex flag by pressing 'R'. Type a 'g' to add an egrep blade, and now you will see that as you type your regular expression, on-screen matches highlight as you type.

More Use Cases

Use case #2

 

Quit out of pipecut by pressing 'q'. In your shell, type ' cat filename | sort | uniq -c | sort -nr | hexdump -C | cat -n | wc ' and press Enter. This is a command line you've been working in the shell to build up. Now, at the next shell prompt, type 'history | pipecut'.

Pipecut will read your shell history. It will find the last pipeline you executed, and create a toolset from it. It re-opens /dev/tty, and puts you into the curses interface right where you would be if you had started with 'pipecut filename' and added all of these commands inside pipecut.

 

Use case #3

 

Running 'pipecut -t toolsetname' allows you to use pipecut as a filter on the Unix command line. This is the one use case where you won't end up in the pipecut ncurses user interface. Pipecut will simply retreive the named toolset from your sqlite toolbox, and act as a traditional Unix filter - stdin -> processing -> stdout.

Pipecut was first displayed in a presentation Nov 1, 2014 at MeetBSD California. Pipecut was created by David W. Maxwell