martanne

> Long-waited NetBSD support in CI would be awesome to be finished too.

As the maintainer of the linked vis project, I can confirm that the provided SSH access to the CI environment is very convenient and saves a lot of time.

I would also like to see the NetBSD support completed. It turns out that up until last week, we missed a #define to compile out of the box[1].

[1] https://git.sr.ht/~martanne/vis/commit/01eef9f87b72b0c147367...

The Plan 9 manual pages and papers are probably a good starting point:

http://man.cat-v.org/plan_9/

http://doc.cat-v.org/plan_9/4th_edition/papers/

> A terminal app that has terminal features doesn't violate any principles of simplicity.

It still depends how these features are implemented.

For example in dvtm scroll back history is made searchable by piping it to $PAGER. Similarly, copy mode is implemented by using $EDITOR as an interactive filter.

> Adding tmux just to get scrollback goes against Unix philosophy.

There are also simpler tools which can provide scrollback support.

The whole Unix tty subsystem goes against the "Unix philosophy", that is why it was completely scraped in Plan 9.

You can be sure that the Kakoune authors are familiar with vi(m).

Having said that, if you like both vi(m) and the structural regular expression support of sam/acme you might be interested in vis which combines the two:

https://github.com/martanne/vis

These operations are already supported by using structural regular expressions. As an example

    x g/foo

will select all lines containing foo. Similarly

    x v/foo

will select all lines not containing foo. Sorting etc. is taken care of by piping text through external tools.

I was more interested in common editing tasks for huge files which according to this thread a lot of people perform using sublime text.

Thanks for the feedback!

Yes syntax highlighting for large files is a hard issue. I'm not really aware of an accurate an high speed solution supporting editing operations in huge files.

In principle the underlying data structure used by vis supports all modifications with linear complexity in the number of editing operations since file load. This is independent of the file structure (i.e. single line files should be well supported). However the frontend code hasn't yet been optimized so in practice there might be some problems.

Unless one specifies the blackhole register when deleting large parts of a file this will create an in memory copy (to enable later pasting at a different location). Better would be to keep a reference to the existing immutable text region.

A number of people expressed the need to edit large files. For the development of my own editor[0] I would be interested to know what kind of usage patterns most often occur. What are the most important operations? Do you search for some (regex) pattern? Do you go to some specific line n? Do you copy/paste large portions of the file around? Do you often edit binary files? If so what types and what kind of changes do you perform?

[0] https://github.com/martanne/vis

Does somebody know how it compares to radare2?

As the main developer of an editor (vis) using a similar segmented data structure (a piece chain, similar to a rope, but storing the text junks in a double linked list, thus asymptotically worse) I can attest that the performance is generally very good. Some work will be required to adapt/write a regexp library around some kind of iterator interface, but in general I found that persistent data structures are very well suited for editors.

One problem with the gap buffer is that as soon as you have multiple cursors/selections or in my case structural regular expression support, then you have to move the gap around all the time. Also undo/redo support is simply not as elegant as with a persistent data structure.

For syntax highlighting copying the relevant text region into a contiguous memory block is simple and works well. For now I'm trying a completely stateless approach to syntax highlighting i.e. the text coloring is always completely recalculated. This trades highlighting accuracy for simplicity, but in practice the results aren't too bad. Efficient syntax highlighting for huge files is a hard problem.

How do you plan to encode binary data in your JSON-based protocol? Base64? This will only increase the overhead.

While the human readability of JSON is nice, it has some serious flaws when the goal is to handle arbitrary data.

I have yet to decide what kind of protocol to use in my own vis editor[1], hence why I'm asking.

[1] https://github.com/martanne/vis

Thanks for the hacking file, it is a good read for people like myself who are interested in text editor implementations. If time permits I will try delve a bit into the joe code base.

As for the syntax highlighting in vis, yes it currently is stateless and only considers a fixed range of text prior to the start of the window. Having no state at all simplifies some things and allows vis to highlight even extremely large files at the expense of the occasional glitch. In practice it seems to work reasonably well.

This approach also requires that the highlighting is fast, because nothing is cached, it is completely redone after every cursor motion. I'm quite happy with LPeg so far, it is convenient to express grammars in a high level language like Lua. It also supports nested grammars nicely, but due to the completely stateless approach taken by vis this is currently not fully exploited. As an added bonus there were already ~100 lexers (of admittedly varying quality) available for use.

Kakoune is a fine editor.

Regarding vis, Lua is optional (you lose syntax highlighting though). libtermkey is a 3 file library and despite what the author claims it works, is useful and could be maintained in the vis repository if necessary.

If you think anything C++ related is lighter than something C based, then we just have to agree to disagree.

The nice thing about the pointer as mark thingy is that while the offset from the start of the file might change when something is inserted before it, the pointer will remain the same.

Anyway this was just an example. I agree that higher level languages (including functional ones) have their own merit. If I would start again today I might consider Rust. But again the LLVM dependency seems kind of scary.

Again for me an ideal base system is built upon a kernel, libc (musl), C compiler (cparser/libfirm), coreutils (sbase/ubase, toybox, busybox), editor (vis) etc. Writing a C compiler is non-trivial, but doable. Creating a C++ compiler on the other hand ...

A self contained (including terminfo entries etc), statically linked vis binary weights in at around ~800K. This allows usage in resource constrained sytems, I don't think the same would be possible using e.g. Haskell.

Some things are just more convenient/efficient to do in C. As an example the mark handling[1] used to represent cursors/selection relies on pointer arithmetic.

Other things like the syntax highlighting are implemented in Lua which is high level but still has low resource usage.

And yes part of the choice is also philosophical. I consider an editor a core system tool which should have minimal dependencies.

[1] https://github.com/martanne/vis/blob/02c6df7cd4bca89506cf1d0...

Yes the lack of binary packages is a known problem.

The whole project is still somewhat in flux, feel free to contribute by filling a packaging request to your favorite distribution ...

Historically it originates from a community which is comfortable building from source.

I agree there are a lot of interesting ideas in 9term/sam/acme and Oberon (which was the initial inspiration for the project). In the future I would like to experiment with integrating a few more, but unfortunately time is a limiting factor.

> Does it build with LuaJIT?

I'm not sure. I think a few Lua >= 5.2 dependencies have crept in, but it shouldn't be difficult to fix this if desired.

LuaJIT's FFI is indeed very nice, but it is unfortunate that Lua has essentially been forked after the 5.1 release. The current C->Lua interface has been manually written.

Yes from a design philosophy standpoint vis is closer to kakoune than {neo,}vim. The implementation is quite different though (no C++, no boost, no home grown scripting/extension/syntax highlighting language).

Reposting an answer from a related thread.

The main difference is that vis is written from scratch while neovim inherited an old and hard to maintain code base. This allows vis to experiment with various ideas:

- native multiple cursors/selection support

- structural regular expression support

- different core data structure resulting in efficient large file support

- LPeg based syntax highlighting

- Lua as an in process extension language

- client/server design (not yet implemented)

without having to care about backward compatibility and legacy systems. Of course this has also some drawbacks for example there is no existing plugin ecosystem etc

Everything involving searching is currently probably very slow for large files, this is especially true for backward searches. The reason being that we currently rely on the regexp engine of libc and thus have to copy the underlying text to a contiguous memory area.

Navigating large files with motions like 50%, 70% should however be instantaneous. Navigating by line number is again somewhat slower, because the file actually needs to be read in this case.

Aside from that :v/ has a different meaning in vis.

As the producer of the "video" (it is actually ASCII, you can copy paste stuff out of it) I agree, here is an annotated version:

    - x/pattern/  extracts stuff from the text, creates a selection for every match
    - some normal vi motions are used to modify the selections, "o" to move the
      cursor from one end of the selection to the other, "w" to move the cursor
      to the start of the next word
    - y/,/ similar to x/ but create a selection for everything which does not match,
      hence splitting the existing selection  into multiple ones
    - \ trim selections i.e. remove leading and trailing white space
    - v/Vis keep all selection not matching Vis
    - g/F|C keep all selection matching F|C
    - + rotate selection contents on the same line
    - v/Com keep all selection not matching Com
    - + rotate some more
    - ESC clear selections, ESC remove all cursors
    - x/pattern select different regions
    - .,/^\} extend selection by searching in forward direction until the first match
      of } at the start of a line
    - { i/#if 0\n/ a/\n#endif/ } insert some text before every selection with i/text/
      and append some text after every selection with a/text/
   
    - change file, select similarily indented code block using a text object: vi<Tab>
    - x/\w+ extracts all words from selection
    - Tab align selections
    - C-c remove first selection column (i.e. remove first selection on every line)
    - Esc clear selections, switch to normal mode, move to next { with f{
    - i<S-Tab> goto insert mode and align cursors

    - search a variable, select it with C-n
    - C-n select next match
    - C-x skip this match, that is clear it and select next match
    - C-n select next match
    - c vi change operator, delete selection, switch to insert mode
    - rename the variable, switch back to normal mode
    - undo the changes

I'm not really familiar with the vim code base. From the little I've seen I'm astonished that people are actually willing to voluntarily work on it.

Having said that, googling revealed a bug report[1] where one of the main neovim developers states he would rather avoid "messing with a very obscure part of the C code." So yes, it seems to be difficult to implement into the existing code base.

There exists a plugin which adds some features, but because it is not supported by the editor core (hence the above emphasis on "native") it isn't properly integrated with other plugins etc.

[1] https://github.com/neovim/neovim/issues/211

No the main difference is that vis is written from scratch while neovim inherited an old and hard to maintain code base.

This allows vis to experiment with various ideas:

- native multiple cursors/selection support

- structural regular expression support

- different core data structure resulting in efficient large file support

- LPeg based syntax highlighting

- Lua as an in process extension language

- client/server design (not yet implemented)

without having to care about backward compatibility and legacy systems. Of course this has also some drawbacks for example there is no existing plugin ecosystem etc.

Yes vipe is indeed useful for editors which do not support it by themselves. Personally I find a solution solely based on pipes without temporary files more elegant.

I should probably let dvtm fall back to vipe if it is installed. Also I will have to check whether there is an easy fix to make vipe+vis work.

Vim is actually a relatively bad filter (some reasons are mentioned in the caveats section of the article). I explicitly designed my vis editor[1] in such a way that it can be used as an interactive filter. The following works as expected, use :wq to write to stdout:

    printf "foo\nbar\nbaz\n"  | vis - | sort

dvtm uses this mechanism to implement its copy mode.

I also recently integrated sam's structural regular expression based command language into the editor. This might be useful for people who want the power of stream editors but with instant visual feedback.

However keep in mind that this feature is relatively new thus likely still contains some bugs ...

[1] https://github.com/martanne/vis

Well technically you could inline the Javascript+CSS stuff and you would get a single HTML file ;) No need to write anything, it already exists.

Anyway I didn't mean to discourage you from working on your project.

I just would like to use a decent terminal recording solution to showcase some of my own projects (e.g. abduco+dvtm, vis[1]). Ideally it would support:

* copy pasting from the screen cast

* pausing / adjusting playback speed

* simple self hosting mechanism

* conversion to gif as preview for inclusion in Markdown files or for environments where Javascript is disabled

* overlay to show pressed keys

AFAIK all but the last point is possible with asciinema. In the past I've used mkcast[2] which uses a fork of screenkey and is a pain to use. Apparently its developer now recommends xscast.

Anyway it is a mess to use and as a result the featured screen casts for my projects are rarely updated and by now completely out of date.

[1] https://github.com/martanne/vis

[2] https://github.com/KeyboardFire/mkcast

asciinema stores the raw program output including any terminal escape sequences in a JSON format[1] and then uses a javascript terminal emulation library to replay/recreate the terminal state in your browser. The player, made up of one Javascript and one CSS file, can be self hosted[2].

[1] https://github.com/asciinema/asciinema/blob/master/doc/ascii...

[2] http://blog.asciinema.org/post/self-hosting/

asciinema[1] does a similar thing but captures a whole terminal session. If somebody would add key input overlay support[2], it would actually be tremendously useful for demonstration purposes of interactive, mainly keyboard driven, terminal applications.

[1] https://asciinema.org/ [2] https://github.com/asciinema/asciinema/issues/105

As the primary author of these tools, please report core dumps/stack traces of the crashes (if you get hold of them). At least for my use cases, the current git versions seem to be stable ...

Having said that, if I ever find the time I would like to overhaul/replace dvtm's terminal emulation component.