Hacker News Reader

Comment by azinman2 | Hacker News Reader

This is a great example that shows how poor most non-web debugging and logging. The ability to mix and match data types and presentation format is extremely useful. Almost everywhere else we just have printf. And that’s printf going to stdout — there isn’t a universally deployed format to even allow it to travel easily over the network and be viewed on one of many useful and advanced viewers.

I think part of the problem is that programmers rarely want to get into these supportive details; they just printf and get on with it.

View on HN

Damn, glad to see this, but they're also kinda encroaching on my territory! :P

I've been working on something like this that's language agnostic and works with not only tables but also trees, graphs, lists, hashmaps, etc., and animates the visuals as the data structures are modified in real time (and you can pause, playback, step, etc.): http://symbolflux.com/projects/avd

The api I was conceiving earlier was more complex, but if you read the copy there, you'll notice it's essentially the same now: `MObject.monitor(...)` or MTable.monitor() MStateMachine.monitor() etc.

Edit: really am happy to see some other projects like this in total, partly because I want to use 'em now if possible—but also because I've had a surprisingly hard time communicating to other developers why it might be desirable to do something like this in the first place, so I'll be very happy to have general awareness raised.

I read the first line of their docs as rather dry humor, "Displays tabular data as a table." —like yeah, of course you would want to see tabular data as a table, and hierarchical data as a tree, etc.!

That video was impressive. Are you using OpenGL/gpu for the animations?

Yes, sadly I rolled my own engine directly on top of Java/OpenGL, rather than using something pre-existing. (There were some decent reasons for it considering how specialized the visuals I'm working with are—but it was a big mistake over all. Should've done it in a more prototype-y fashion).

To code the animations I use something in the engine which resembles CSS transitions and can be attached to arbitrary Java objects, which I call 'property modulators'. So most of the animation you see comes from setting up one of those on one property or another.

Would this possibly work on top of WebGL?

Most of the rendering code would be a pretty direct port—I don't think I'm using anything from OpenGL that isn't supported by WebGL. That said, it is a desktop Java application and it would be non-trivial to port it to javascript. And that said, it probably would've been best to do it in javascript in the first place, but I did basically no web dev when I started, so... shrug

Looks really interesting, which platforms languages etc. does it support?

There are two clients in progress at the moment. By far the more complete one is for Java; the other is for javascript.

Since I've had such a difficult time communicating the potential benefit of the fully general piece of software, I've been exploring applications that I can build on top of it which do visualizations for more specific things (rather than leaving it to folks' imaginations to roam the wide possibilities of general data structure visualization :)).

As an example, my sister and I started building a 'code tutor' app in Electron, where you would be presented with problem sequences—but when you tested the code for your solution, you could watch what your algorithm actually did to the data (imagine a failed attempt to sort a list). This was the motivation for writing the javascript client. I integrated Lucidity's visuals by running it headless and streaming frames over TCP to the Electron app.

I think the next thing I'm gonna do though is just finish general object monitoring for the Java client, so that you could get something a little like the Chrome dev tools object printer in Java—but of course this one will allow you to watch algorithms performed on your objects :)

Btw, source for the (rather incomplete) javascript client is here if anyone's curious: https://github.com/westoncb/JS-Watcher

This looks really cool and I would love to play with it but I can't seem to find any download link. Is it publicly available?

Unfortunately, no :/ I should get that together soon, but there are still some rough edges I've wanted to work on before issuing a release.

At a quick glance it looks like it's targeted towards mutable data structures. Can it support (and would it make sense to support) immutable data? E.g. it would be nice to have a Haskell state monad that also visualizes its state.

You are correct, it's very much targeted toward mutable data structures. It'd be interested in looking more into the state monad and considering the possibilities, but I haven't yet.

Actually part of how I see the role of systems like this is part of the necessary conditions for more safely working with mutable data. I think mutable data is more risky, but also super easy to work with, so if we had ways of making it more reliable we should explore them. I envision languages where this sort of visualization is a core feature designed in tandem with the runtime etc.

Neat! Very interesting ideas! The Vacuum package[1] seems like something you could look into if you want to explore Haskell. Good luck with your project!

[1]: https://hackage.haskell.org/package/vacuum

Thanks! Looks pretty cool. I also found this: http://felsin9.de/nnis/ghc-vis/

Cool.

A small suggestion: Allow to not show so much.

A problem I have with most log-like solutions is that them bombard you with so much info and you need parse them after the fact, yet none of the log visualization have a way for that (just ctrlf, with luck).

A simple improvenment? Put a search box in top of the displayer, and a way to make contextual search, (like :tree.at Code.py), and a way to save search patterns, so I can put the whole search syntaxt and just fill boxes (like :mySearch foo)

Hey mamcx—totally agree. Actually, I've got designs for something very much like what you suggest. I was actually thinking of designing a simple query language for it so that you could search for certain types of relationships etc. too.

Wow, super cool! How does this cope with cycles?

This is amazing. I love it.

Only for those developers that insist in not using IDEs.

Common Lisp and Smalltalk environments have something like console.table since its early days.

Xerox even went to the trouble of adding similar capabilities to the Mesa/Cedar enviroment, because they thought they should cater to their Interlisp-D and Smalltalk developers.

Java and .NET even allow to provide metadata so that debuggers know what is the best way to render the data in debug view.

Visual Studio allows similar capabilities for C++ via data visualizers.

Now if you insist in vim and Emacs, there is printf.

It has little to do with IDEs and everything to do with debuggers. When writing code in emacs or vim, as long as you compile with -g and run the binary with gen, you get the features you're talking about.

Unless you are speaking about something like DDD, not really.

Also it has to do with IDE, because in an IDE there is two way communication between both tools, like the ability to do edit-and-continue.

Look at this: https://news.ycombinator.com/item?id=17310418

It's not quite as fancy as an IDE, but it's a lot more than printf.

Could you show a screenshot?

Here's a recording of using GDB: https://s.mort.coffee/d/vid/gdb-demo.mp4

Here's a recording of using GDB's TUI mode: https://s.mort.coffee/d/vid/gdb-tui-demo.mp4

Note that I'm not a GDB pro, I just recently started trying to use it more instead of printf, so there are probably lots of stuff I didn't show off. (For example, I wish I'd shown the `bt` command to print stack traces).

Another thing that's worth noting is that with GDB, if your program crashed and generated a coredump, you can use GDB to look at how everything was when it crashed, just like if you were debugging a running program. This is extremely useful on for example embedded systems where you don't have GDB installed, or when the crash is hard to reproduce.

I know that GDB is powerful and scriptable and very useful in environment where you don't have anything else. But I'm sorry to say, what you showed wasn't very discoverable and the UI was faaar from amazing.

I don't know if you've tried it, but whenever you have a bit of time try Visual Studio's (or even Visual Studio Code's) debugger. They're among the best visual debuggers out there. Or IntelliJ's.

You see directly all the variables as you step through code, you can see array contents directly, they're super discoverable, etc.

It's such a shame that there's no glory (or money) to be had in designing CLI/TUIs. I also get the impression that it would be really hard to change things, OSS maintainers have a reputation for being very stubborn about workflow changes (sometimes for good reasons, sometimes not) :(

GDB's main power is that it's scriptable. You can track down some pretty gnarly bugs with that. You don't necessarily need that capability for languages like JS, because there's far less danger of getting into serious trouble (stack corruption, etc.), so it's hard to appreciate. But yeah, discoverable it is not haha.

All major Java and .NET IDEs have debugging APIs as well.

As for the data visualizers, as mentioned, with Java and .NET I can do it with a bit of annotations and the debugger takes it from there.

Or in C++'s case when using Visual Studio, a bit of XML describing the metadata.

Oh yeah, totally true. All I meant was GDB's power interface is scripting. I definitely think IDE debuggers are better.

Thats just a textual display of the structures, not a different presentation (i.e. a graphical tree or table, allbeit in text mode).

BTW: In GDB, all commands have shortcuts i.e. 'b'=='break'. (Saves a lot of time).

What's gen? It's not very googlable ('gen debugger' gives nothing relevant), and asking my package manager for a package providing 'gen' just gives some game.

Oh wow, I didn't notice that. I wrote gdb, but my phone autoincorrected it to gen.

I'm amazed sometimes at what people are able to accomplish banging rocks together and doing things the hard way.

Me, I'm lazy, so I lean on my tools to make my life easy. And there's some fantastic tooling out there; the biggest problem is usually discovering that a helpful feature exists and how to use it effectively.

Yes, that is why Visual Studio team started doing talks about how to use the debugger.

They started to realize that there were many developers doing feature requests for stuff Visual Studio already supports.

Do you know how to record all backtraces of all threads and child processes with the VS debugger? Just curious. I did it lately with gdb [0], but with a bit guerrilla method. It was my try at localizing the origins of dreadlocks. I would appreciate pointers for something better for dreadlocks, but the original question still stands.

I understand that what I did would be a bit close to a poor man's reversible debugger.

[0] https://gist.github.com/hadrianw/5b8d33a4b353c49e7dbd6eb55f8...

I usually make use of the tasks and thread viewers.

https://docs.microsoft.com/en-us/visualstudio/debugger/debug...

https://docs.microsoft.com/en-us/visualstudio/debugger/map-m...

Otherwise Intellitrace is the way to go.

https://docs.microsoft.com/en-us/visualstudio/debugger/intel...

I will start using IDEs again once they get instant, mouse-free, java-crawltime-env free and get easily extensible like most modern editors are.

Can you please provide a screenshot of something like Console.table() for a tree data structure on your extensible modern editor?

This would be fairly easy to build in Emacs for almost any language, but I can't see how this is so useful. Furthermore, if working with Org mode source blocks, it can render structured data as tables upon evaluation. Say you have this block:

  #+BEGIN_SRC elisp
  (list (list 'col1 'col2) (list 1 2))
  #+END_SRC

(I don't use quote syntax to not confuse non-lispers) Hitting C-c C-c on it produces the following block just below it:

  #+RESULTS:
  | col1 | col2 |
  |    1 |    2 |

With Org source blocks you can have a literate program even using multiple languages and still communicating easily between source code blocks and both generated and static tables. It can easily be intermixed with formatted text and exported to a suite of various formats, including plain text source code (so-called "tangled source").

Edit: just for the sake of it, another example, this one in Python:

  #+BEGIN_SRC python
  return [[x for x in range(1,5)],
          [chr(y) for y in range(ord('a'),ord('e'))]]
  #+END_SRC
  
  #+RESULTS:
  | 1 | 2 | 3 | 4 |
  | a | b | c | d |

I don't get why in this day and age we can't have proper text mode interfaces.

Freaking IBM had box drawing characters in freaking 1985, how hard is it to have them in 2018? https://www.ascii-codes.com/

See this:

  ┌─┬┐  
  │ ││  
  ├─┼┤  
  └─┴┘

vs the crappy table.

Do you believe this adds some kind of clarity to the data ?

Yes. There's a reason no one (except for hardcore geeks) uses ASCII tables.

Do you know what that reason is, and can you elucidate me ?

They don't look nice...

Beauty is in the eye of the beholder, true, but the majority of the beholders want a table to actually look like a table. If you give me pen and paper, I'm not going to draw weird half lines around the table, I'm going to draw the table first, line fully drawn everywhere and then add the content.

It's misaligned on my screen. Still, it would not be too hard to hack that in to Org.

Even GDB TUI comes short to what Turbo products with their Turbo Vision based UI were capable of on MS-DOS.

  | Hello | Planet | Earth |
  |-------+--------+-------|
  | foo   |      4 |    88 |
  | bar   |    265 |   123 |
  |-------+--------+-------|
  | sum   |    269 |   211 |

This org table looks clean enough to me.

The disadvantage of using box drawing characters is that they do not always align nicely - depending on the font, renderer, and couple other things. You can't go wrong with basic ASCII and a fixed-width display.

The point of Org mode is that you can type the table.

How do you type ┌─ ?

It's easier to use a font that renders "regular" keyboard characters as borders, akin to https://github.com/tonsky/FiraCode

   Now if you insist in vim and Emacs, there is printf.

That's just plain wrong. GDB for example has had pretty printers for STL data structures for many years. Those operate on the underlying structure of the data types and you can write printers for your own data types pretty easily (gdb uses python for these).

Apart from that, there is structured hierarchical logging, which seems to be on the way of becoming an industry standard and is considered best practice in some ecosystems (golang comes to mind)

> Visual Studio allows similar capabilities for C++ via data visualizers.

No reason this couldn't be possible for Emacs, I guess nobody bothered. In particular, Common Lisp with SLIME already makes use of "presentations" in REPL, which nicely combine CL's print-object output with inspector/debugger capabilities.

Ultimately, no one limits you to printing out simple text. In various Lisp software I wrote, I ended up displaying profiling info inline in REPL, like this:

          CHARTED-LET*                  
                              0    9.249
                              ˫--------˧
              SOME-VAR: 0.006 |
             OTHER-VAR: 0.004 |
      SOME-COMPUTATION: 2.036 |==.
             A-SUMMARY: 0.062 |
      DATA-FOR-A-CHART: 3.486 |====
  SOME-BIG-COMPUTATION: 9.249 |==========
       SOME-TREE-STUFF: 0.005 |

(EDIT: there were pretty Unicode characters in that output, but my browser / HN ate them. See https://github.com/tkych/cl-spark for how it should look like.)

(all it took was a simple macro on top of a sparkline lib). I dumped custom execution stats out-of-band into a HTML file that, with little bit of static JS, rendered explorable charts. I've sent data straight into new Emacs buffers for processing, with a help of this simple macro:

https://gist.github.com/TeMPOraL/8715c9dd9837e0b601d1cdce059...

(can be trivially modified to make the new buffer automatically assume a given major-mode).

Point is, you don't have to limit yourself to simple, uninterpreted printfs. Write your own debugging tools! And while I'm not up to date with C#, most other languages and IDEs I've seen don't move beyond dumb printfs. Java definitely doesn't (hint: Grep Console is a very useful plugin to IntelliJ and Eclipse). And this Console.table() is just a visual gimmick. I'm actually disappointed that with all the capabilities for interactive HTML rendering, that's the best the Web standardized on so far.

When I think about Common Lisp debugging capabilities usually what comes to my mind is Allegro, Lisp Works, MCL, CCL, not SLIME in text mode single window.

Sure, but a lot of CL development these days is done with SBCL and SLIME - Allegro and LispWorks are expensive, and whatever cool stuff CCL has would require me to buy a Mac.

So much this! I've long wanted something where by I can chain, into a sort of tree, to more easily follow or hide large subcomponents and subservices. For example, say I make two calls in my service A to two other services B and C, and all report logs to a centralized location; that could get rendered as,

  ─ info: GET /foo/endpoint
    ├─ info: user <user ID> valid auth
    ├─ info: request to service B
    │  ├─ debug: opening new connection
    │  ├─ debug: success, result = ...
    │  └─ info: request took 1 seconds
    ├─ info: request to service B
    │  ├─ debug: opening new connection
    │  ├─ debug: success, result = ...
    │  └─ info: request took 1 seconds
    ├─ info: preparing result took 1 seconds
    └─ info: http request took 3 seconds

with the ability to hide sections, perhaps grep for only certain messages (particularly if you keep the formatting and the message separate, this should be doable, I think), attach metadata to messages…

As it is, we have a fairly standard shove everything into syslog, and then pipe to a downstream logging system and a local file. But the downstream system is not very good at search (this is probably mostly our fault) and requires the message to be in JSON, so the stuff in the log file is _also_ JSON, b/c that's what syslog got. There are definitely better ways with our existing tools, but it sure makes one dream up what the perfect logging solution could look like.

Tracing calls across threads/servers is exactly the kind of stuff that should be easy and out of the box. If we had this built into a kind of universal logging protocol, you can imagine some viewers getting extremely sophisticated with network-level analysis, parsing & plotting times for distributions, anomaly detection, etc. Other viewers might be more geared towards mobile development, which might emphasize where network requests were generated in the code and its effect on the UI. Both should be possible from the same input data stream.

This is what I did my MSc thesis on. I should consider open sourcing the framework, even though it’s relatively “homework grade”

A couple of places to look though:

- opentracing.io - Fonseca et al’s X-Trace work

I don’t have a link handy, and the visualizations haven’t aged well, but you can probably find a copy of my thesis or conference paper under Anthony Arkles in google scholar. I extended the X-Trace protocol a bit to make it easier to reassemble function calls that potentially had parallelism.

Html5 timing API + chrome devtools performance tab "user timing" section is a great workflow in my opinion for frontend.

Take a look at lnav (https://lnav.org), you might be pleasantly surprised.

> As it is, we have a fairly standard shove everything

Assuming "everything" is "static log statements" which is the first issue I have with this, a developer must have thought before deployment that this could be useful, which usually results in a lot of useless garbage and a lack of actually useful information. And little by little you build up actually technically useful data collection, and rather than being spread throughout the codebase all of the probes are centralised and readable.

I've been thinking about using dynamic instrumentation tools (bcc/dtrace) for that purpose instead, you know you need something when you actually do need it, at that point you can add it to the probes/instrumentation (which is external to the program and deployable separately), and all information would be collected in a structured form in a database you can interact with (probably not something relational).

If you're seriously interested in building this and would be open to employment for that purpose – please drop me a line (contact info in my profile). This absolutely deserves to exist and it would be a natural extension of what we're building at Scalyr. I'd love to talk.

1. this is tracing not logging, it's not ordered by time (as opposed to Console.table)

2. there are lots of systems having done this in production for many years, take a look at opencensus.io for googles open source version

>Almost everywhere else we just have printf

Because you see the black console almost everywhere. We debug tabular or structured data in visual debugger, console or specialized snapshot-view that is not hard to create. All depends on programmer’s will to make their life easier. Console.table() is just one of the simplest things that you could write for yourself in dynamic language, but the fact that most web “devs” cared to read the docs only after it hit top of HN makes it look amazing.

>how poor most non-web debugging and logging

Praise the web.

> shows how poor most non-web debugging and logging

Nonsense - web programming is inherently tied to the browser, which severely limits debugging and output options.

> The ability to mix and match data types and presentation format is extremely useful.

That's a common feature of dynamic type languages.

> Almost everywhere else we just have printf.

Most languages have logging libraries, and many have them built into the stdlib or runtime. Popular languages usually have many logging libraries to choose from.

> that’s printf going to stdout

Of course - printf(3) by definition "shall place output on the standard output stream stdout"[1]. If you want stderr or any other output stream, use fprintf(3).

    #include <stdio.h>
    int main() {
        fprintf(stderr, "Error: %s\n", "LP0 on fire");
        return 0;
    }

Other languages usually handle that for you as part of the logging library.

> there isn’t a universally deployed format

Sure there is: "text"[2].

> to even allow it to travel easily over the network

Copying files to a remote host is easy with scp. There are numerous ways to send data over a network. Web software is often more work as you have to move your debugging data/logs out of the browser first or use a specialized tool that does that for you. Sending program output realtime to a remote host only requires appending "| ssh ${host} \"${remote_cmd}\"" to your command.

> advanced viewers

That's great if you have specific needs and a viewer for your complex data structures, but the only universal solution is text. Fortunately, there are many tools available to handle structured data as text.

> I think part of the problem is that programmers rarely want to get into these supportive details; they just printf and get on with it.

I think you might want to explore the work programmers have done in other languages.

[1] http://pubs.opengroup.org/onlinepubs/9699919799/functions/pr...

[2] http://www.catb.org/esr/writings/taoup/html/ch05s01.html

You're totally missing my point(s). Of course web programming is tied to the browser, but why does that mean debugging via logs has to be tied to whatever particular console is attached to where a program is running? Why can't I have a console.table() equivalent in Python, and Ruby, and C, and Go, and Java that all can render via a common protocol to any viewer that can handle it?

By printf I didn't _literally_ mean printf -- I meant any logging type equivalent whether that IS printf or a built-in logging system. The point is there's no inter-op between them with any rich detail, which prohibits universal log viewers.

When I say travel over the network, I don't mean scp. I mean allow my iOS/Raspberry Pi/local Erlang server device to stream in real-time it's logs to any viewer that I want, whether that's on the same device or somewhere else in the world. That's the kind of portability required when you can't guarantee that complex display can be part of the same system that generated it -- that's a unique quality of web browsers. IT'S ABOUT THE PROTOCOL, NOT THE MECHANISM.

Text is great, and most logs will be in text. But I also sometimes want to inspect data, or expand JSON, or view an image, etc etc. But let's even stick with text -- a good viewer would let me set up filters on a host of structured qualities -- filenames, lines, log levels, keywords, etc. It would output in a way that allows me to easily view such info in a UI that supports infinite scroll back, easy copy/paste, elegantly formats so different log instances are clearly separate from each other, adds color to enhance readability, allows me to hide/collapse sections or duplicate entries, provides columns for the structured logging fields like timestamps, allow interactive sorting on these columns like Excel, etc etc all without losing my data.

Apple's Console.app is an elementary example of what I'm suggesting I'd like to see. Except I don't want it tied to the syslogs of an Apple device. I want every language/platform to be able to output to it in real-time. And for it's capabilities to be as rich or richer than what we have in the browser. We already have universal text editors like VIM/Emacs -- why can't we have universal log viewers?

"printf" will always have it's place, because when things are hosed, there's a good chance you can't push stuff to network, but things have to be really hosed to stop you pushing some chars to a UART.

> I mean allow my iOS/Raspberry Pi/local Erlang server device to stream in real-time it's logs to any viewer that I want, whether that's on the same device or somewhere else in the world.

e.g. syslog (RFC5424)? -e- You can even add some extra fields of your choosing if you so desire: https://tools.ietf.org/html/rfc5424#section-6.3

> But let's even stick with text -- a good viewer would let me set up filters on a host of structured qualities -- filenames, lines, log levels, keywords, etc. It would output in a way that allows me to easily view such info in a UI that supports infinite scroll back, easy copy/paste, elegantly formats so different log instances are clearly separate from each other, adds color to enhance readability, allows me to hide/collapse sections or duplicate entries, provides columns for the structured logging fields like timestamps, allow interactive sorting on these columns like Excel, etc etc all without losing my data.

e.g. ELK, Graylog etc?

lnav + mitmdump!

lnav looks really neat. Thanks!

> Text is great, and most logs will be in text. But I also sometimes want to inspect data, or expand JSON, or view an image, etc etc. But let's even stick with text -- a good viewer would let me set up filters on a host of structured qualities -- filenames, lines, log levels, keywords, etc.

So, it sounds like you want a debugging library you can include in the language you are using to output structured debug lines and a client side that goes along with it to represent the usefully (or just use text+JSON and a JSON aware client), and to just stick it on top of syslog. I think that pretty much covers what you're asking for.

>So, it sounds like you want a debugging library you can include in the language you are using to output structured debug lines and a client side that goes along with it to represent the usefully (or just use text+JSON and a JSON aware client), and to just stick it on top of syslog. I think that pretty much covers what you're asking for

Except the most important thing: that ability to be included as standard, readily available, and well supported in languages.

Well, that's why I specified it as a library. A C compatible Lin is likely easiest for most languages to trivially support through an FFI, and it all it's doing is outputting structured text based on the input, that's trivial to hook up. It's not like it can be too language specific because if it's supposed to work in a bunch of languages, it has to be generic enough to handle them.

Although in the end you probably aren't getting much out of it, as a dynamic language is fairly easy to just pump stuff through a JSON converter, and static/strongly typed ones probably require enough boilerplate that the added benefit is slight.

"Why can't I have a console.table() equivalent in Python"

I get a console.table() equivalent in Python by using Jupyter Notebook, which is a web-based REPL shell, and pandas, which is a library for working with tabular data. Pandas displays HTML tables when used with Jupyter.

> Of course - printf(3) by definition "shall place output on the standard output stream stdout"[1]. If you want stderr or any other output stream, use fprintf(3).

I reject your C-centric world for one of my own making. ;)

  # perldoc -f printf | head -n3
       printf FILEHANDLE FORMAT, LIST
       printf FILEHANDLE
       printf FORMAT, LIST
  # perl -e 'printf STDERR "%0.2f\n", 2;' >/dev/null
  2.00

he was quite specific in saying printf(3), which is defined by POSIX

Yes, the reply was quite specific in rewriting printf to include the man page section, but the original comment did not.

This response managed to say very little and just give excuses for the current state of things.

>Nonsense - web programming is inherently tied to the browser, which severely limits debugging and output options.

Web programming includes cli and server side tools (Node for one, JS testing tools, etc), has several first class logging SaaS and locally installed options, and even has stuff like the developer tools protocol, which allows one to debug Chrome running code in VS Code for example...

>That's a common feature of dynamic type languages.

That's merely an "ability", it's by no means a "common feature" in dynamic languages to have different output options in the example of e.g. console.table() (which itself is limited anyway).

>Copying files to a remote host is easy with scp.

That manual busywork is not what the parent asks for.

>That's great if you have specific needs and a viewer for your complex data structures, but the only universal solution is text.

You've just restated the problem -- only with a positive spin.

A proper debugger like the one in VS and general structured logging would like to challenge that statement. I'll take those over what you have in the web space any day.

What are the criteria that define whether a debugger is 'proper' or not?

There are many, but for my line of work:

* Conditional Breakpoints

* Function breakpoints to add ad-hoc logging

* Show type information

* Ability to custom render state, a graph for instance.

* GUI to browse the state

* CLI to query the state

* Ability to alter state

* Multithread debugging support

* Record execution history and state so you can debug after the fact for hard non reproducible bugs.

* Remote debugging

* Support for network traffic debugging

Thanks for this list and I'm full agreement.

Chrome developer tools will do all of these except

> * Record execution history and state so you can debug after the fact for hard non reproducible bugs.

^ Which would frankly be awesome. Various libraries provide their own userland support for this but it's all fragmented with hugely varying levels of quality and flexibility, but out of the box support would make me so happy (unless this already exists in which case, someone please enlighten me!).

A couple years back at university, I created DoodleDebug[1,2], tackling some of your mentioned issues. It can be thought of as a graphical version of System.out.println(). There is a similar output format customization mechanism as toString(), but you can also customize the rendering of arbitrary types you don't control the source code of.

DoodleDebug's output is HTML-based and interactive with semantic zoom. That means you can click parts of printed objects to inspect them.

However, I didn't work on it in years due to lack of time, and even the linked description page is a bit outdated (it's not an Eclipse plugin anymore).

[1]: http://scg.unibe.ch/wiki/projects/DoodleDebug [2]: https://github.com/CedricReichenbach/DoodleDebug

Edit: Maybe this PDF of my Bachelor's thesis gives a better overview of what DD is and does: http://scg.unibe.ch/archive/projects/Reic13a.pdf

One of the under-shared features of Rust that I adore is #[derive(Debug)] -- having object output like a dynamic language on low-level, statically typed struct definitions is AWESOME.

Huge boon to productivity, especially when leveraged in tests.

PowerShell uses objects as output. They get automatically formatted and save you from custom parsing code.

I wish PowerShell and/or some of its ideas had a bit more mindshare outside of the MS ecosystem.

Other folks have talked about this at length (e.g. https://mkremins.github.io/blog/unix-not-acceptable-unix/), but the fact that there is no standard way to pass anything other than strings of plain text between UNIX shell commands is incredibly limiting.

Heck even if we could pass a flag (say, -j) to something like ls and have it spit out JSON instead of plain text would be an upgrade.

If you are debugging a program inserting printf or other debug prints in the code you are doing it wrong, for debugging you should use a debugger.

It's slow and inefficient, you add the debug print, then you need to recompile the program, maybe you need another debug print, recompile the program, and so on, and then you must remember to delete all the debug prints that you added! And let's hope that your debug print didn't add some strange side effect (they might have!).

With a modern debugger you could print everything you want, at every memory location, you could set watchpoints, isn't it better than printf ? Also where you printf on an embedded system...

Sometimes you just can't do that, without completely breaking everything while the program is halted in the debugger. Particularly real-time, multithreaded applications.

In those situations, even adding printf statements can be enough to change the behavior of the system.

Not necessarily. It's about timescales and other environmental factors. Pausing in a debugger is measured in seconds. Your network-using application won't notice extra half ms for a print statement, but might change its behaviour if you pause it for 5 seconds, as requests will timeout. A video game running in fullscreen won't mind printing to console/file in the background, but may not handle well being interrupted in the middle of rendering and having you switch focus to the debugger (less of a problem these days, but I remember this being a pain ~10 years ago).

Sometimes it's just easier. Especially if you have a nice incremental build system or an interpreter to which you can just send expressions and definitions and see the results.

Edit: also, you don't need to remember to weed out debug prints, you can, in almost all languages, define a function or a macro to do that which can be undefined or an ignore function if not built in debug mode. With cpp or Lisp, that'd be a compile time decision, and I suppose many languages could optimise out a function that is defined to just ignore it's arguments and do nothing.

Sometimes you may not want to pause execution. Adding a print statement is often faster than starting a debugging session - especially in languages like Common Lisp, where you can simply recompile a single function of a running program. Another benefit of print statements is that they're append-only, not self-overwriting - that is, I can stick one inside a loop and quickly collect a bunch of values to review, or even copy over to another tool for analysis.

A modern debugger is a powerful tool (especially if the language you're using isn't itself very powerful). It's worth knowing your way around it. But that doesn't mean you shouldn't be using simple print statements whenever that's faster or more useful.

If you use something like Symbolics Genera or CLIM, the printed output is actually still connected to the data. Thus you can use relatively primitive PRINT functions and still be able to drill into the objects later.

I use often in Lisp the INSPECT function with a GUI inspector. Each call to INSPECT puts the data into the inspector. For example (inspect (list :loop-i i :value n)) puts the list with the data into the inspector. The inspector has a history, which then allows me to see the various items. Thus this is similar to a print statement, but records the actual objects into a separate tool.

Didn't use INSPECT function like that, thanks!

For the sake of completeness, I'll add that the output-connected-to-data works to some extent with Emacs & SLIME as well. These are so-called "presentations" - if you print an object, SLIME will connect the printed text to actual Lisp object, which allows you to inspect it later, or even copy and paste within the REPL - as long as you copy and paste the whole "presentation text", the underlying association will remain.

Clojure's not often mentioned clojure.inspector namespace contains two functions: inspect-table and inspect-tree. They roughly approximate Console.table() and Console.dir(). I tend to get more mileage out of inspect-tree, but even so I find it limiting. I dream for a graphical object diffs and a version that when passed an atom or ref, auto-updates when it changes.

On the one hand, I like it. It's good this function is there.

On the other hand, I'm disappointed. With all the power the web browser tools have - basically, arbitrary HTML rendering in console - this is the best we get? A simple, dumb table? There are so many to be explored here!

> I think part of the problem is that programmers rarely want to get into these supportive details; they just printf and get on with it.

A lot of programmers aren't aware that yes, they can and should modify their environment to suit their needs. Yes, they are allowed and should write their own debugging tools. Those tools don't have to be complicated. You can start with an ASCII-based console.table() equivalent. Or a script that parses the output and feeds it to some half-baked D3-based visualization. Or even gnuplot. Or graphviz. Plenty of possibilities there, and yet most devs I know never even think of them.

I want a thing that can plot a numerical array as a line graph of data points. A histogram and a scatter plot would be nice, too.

And I don't mean for computational/data science exploring, I've got numpy/matplotlib for that. But really as a debugging tool in the browser's dev console. It's not uncommon that I have an array of numbers (or something that looks like one), a few hundred or more. Finding the bug or unexpected behaviour in my program is often about finding the numbers in such a list that "stand out" in some way. Now I could specify and filter those elements that "stand out", but I could also plot a graph and grasp its behaviour in a single glance. Just dumping the numbers to the console and hunting for aberrant behaviour likely takes me out of my flow, where a graph would not.

Yup. This is something I'd love to have too - in particular, point/line plots and scatterplots at resolution greater than possible in text-only terminals. I already render text-based histograms when I need.

Come to think of it, it shouldn't be hard to extend Emacs to eat specially prefixed output (e.g. #<scatterplot: (1 2) (3 4)...>), feed the data to external program and place a rendered chart inline. I might pick it up as a small project. Thanks!

> there isn’t a universally deployed format to even allow it to travel easily over the network and be viewed on one of many useful and advanced viewers.

printf("<tabe><tr><td>%s</td></tr></table>", htmlescape(blabla), ...);

Copy paste to the browser and you're done.

My absolute favorite web-console feature (in Chrome and Firefox at least) is the ability to go back in the REPL history to edit a previous function definition. A standard REPL just gives you back one line at a time, so if you want to make a change to line 3 of a 4 line function, then you have to type:

    UP UP UP UP ENTER UP UP UP ENTER UP UP

before you can edit that line. But the web consoles allow you to type UP just once, and then move your cursor around the entire function definition. I wish all REPLs had this feature.

You should try binding UP/DOWN to history-search-backward and history-search-forward[1]. Adding this to your ~/.inputrc

    "\e[A": history-search-backward
    "\e[B": history-search-forward

enables searching by the current line's prefix in all programs that use readline (a lot of languages use readline for their REPL). That is, if you type "cp<UP>" you immediately get the previous command that started with "cp", regardless of how far back it is in your history.

[1] https://codeinthehole.com/tips/the-most-important-command-li...

This is a useful trick, but I don't see how it addresses the issue they talked about. As I understand it, they were saying that Firefox and Chrome's Javascript REPL's scroll up through expressions rather than lines. If you say "const f = () => {" and press enter, it doesn't submit it until you close that brace, and when you use the up arrow to scroll back to it later, you get the whole thing (newlines preserved) instead of single lines of it.

> UP UP UP UP ENTER UP UP UP ENTER UP UP

If the REPL uses readline (or mimics it), then

    UP UP UP UP ^O ^O

will do the same thing as that sequence.

This is trivial in the right environments. Not shockingly, in emacs I get that.

It actually takes a little getting used to at first, that up literally just moves the cursor up into the output of the previous command. Nowdays, it is severely restricting to have the same commands move the cursor that change the current command. (That is, left/right move cursor, up/down do a ton more in most terminals. That is not the case in emacs shells.)

Really, though, the repl is a fun stepping stone to just sending function definitions from the file you are working on into an environment. https://github.com/skeeto/skewer-mode is a fun video showing the idea for javascript, amusingly.

That said, there was a fun video a while back of the guy that did Minecraft live coding a video game. Basically, the game would be running, and he could make a change and it would live update, just like the javascript video above. But it was in java.

This is harder in non-managed runtimes, I believe. But not impossible.

That, in turn, shows how far the web has come. I remember a time when we didn't even have console.log()!

Good point! The first time I used firebug, probably 3 years before I used Chrome, it blew my mind and immediately made clear how much faster it was going to make styling pages, debugging errors, and just testing JavaScript expressions.

AWS Cloud Watch Logs can take JSON format log events.

This displays better in the AWS console -- and also it enables better queries than raw text, e.g. `{$.response.code >= 400}`.

While I agree it's nice they can take it, I don't think their viewer is particularly readable or advanced. Apple's very basic Console.app is a much stronger example, and it's super simple in capabilities.

Eclipse, VisualStudio, and all other IDEs have had tabular expression viewers for decades.

We are too hell-bent on that unix philosophy that everything is a text, stdin, stderr and pipes are enough. It is not. We should have ways to log objects. We should have ways to pass those objects around and serialise, deserialise and inspect them.

It's not just text. Grep is not enough for investigation.

Is there a reversible AKA time travelling debugger for JavaScript?

I think the closest thing for this would be something like SpyJS, which is available for both NodeJS and browser JavaScript in JetBrains products such as Idea and Webstorm.

You can see previous executions of the code you're interested in, with a highlight of the paths followed through the logic and snapshots of the variables as they were at the time.

> how poor most non-web debugging and logging

Well, since that time when chrome was unable to find the javascript file of the page I was looking at, forcing me to go back to internet explorer, I disagree.

Also is it possible to edit the javascript code without reloading the page? I know how to do it with java+eclipse, but not in the browser.

Reply on news.ycombinator.com