justinsaccount

> make it possible to write better, safer, and more efficient C++

Right, but is it still possible to write bad, unsafe, inefficient C++ that looks indistinguishable from the safe kind?

What is being removed are the DRM drivers. Unless you're trying to play quake on your server you won't even notice.

If someone was trying to run an Active directory domain controller on a windows VM on a mac mini, what would you tell them?

If someone was trying to run FCP on an OS X VM on a linux box, what would you tell them?

It's not the fault of docker or linux containers that people insist on trying to use the technology on another unsupported operating system.

Apple is free to port the docker daemon to run natively on OS X.

Running the docker daemon in a linux VM on your OS X system and then complaining about performance is silly.

> I honestly don't know how Docker got so popular when it sucks so much on anything that is not Linux.

How do these sound to you?

I honestly don't know how Active Directory got so popular when it sucks so much on anything that is not Windows.

I honestly don't know how Final Cut Pro got so popular when it sucks so much on anything that is not OS X.

With things like docking stations/usb-c docks/other adapters, the actual mac address doesn't necessarily identify a machine.

Who said anything about the worldwide air traffic control system? Did you mean to reply to a different thread?

Yep, this is the exact point I tried to make above and got heavily downvoted.

If you can't move fast when things are working well, you can't move fast when things are broken. Acting like moving slow is going to prevent things from ever breaking is just wishful thinking.

So for the things that don't need a lot of change, what are some characteristics of the system?

Does CI/CD exist? Does CI even exist? Are deployments automated? Is data sanity checked before loading? Is there a development environment? Do things like hourly snapshots exist? Can you easily provision a replacement system from scratch and restore data from a known good snapshot?

Or, is every process manual, slow, and error prone because there's never been a need to move fast.

Look at this one sentence in the article:

> In the overnight hours of Tuesday into Wednesday, FAA officials decided to shut down and reboot the main NOTAM system -- a significant decision, because the reboot can take about 90 minutes, according to the source.

So they do a reboot, that takes 90 minutes for some reason, and then that didn't even fix the problem. Their system that needs a lot of stability is now broken.

True.. which is why you want simple systems that are easy to understand and iterate on.

And as I said, people think the alternative to "fast, full of bugs, 0 planning" is automatically "slow, no bugs, lots of planning" but it's often "slow, lots of planning, just as many bugs"

"Moving faster" doesn't mean "frantic" either. Who said frantic?

I'm not sure what point you are trying to make. Physics? super heroes? knobs?

People like to think that the alternative to "move fast and break things" is "move slowly and not break things" but it's not, it's "move slowly, break things anyway, then take days to resolve the problem because you never learned how to move fast".

We have that one and accidentally programmed it to run first thing in the morning. Turns out that works pretty well. Acts as a bit of an alarm.. runs from like 7:30 to 8:30 or so. Manages to completely fill itself up with pet hair every other day.

There's newer json support in clickhouse where you can load arbitrary json objects into a JSON column type:

https://clickhouse.com/blog/getting-data-into-clickhouse-par...

Behind the scenes it flattens things into real columns with auto detected types.

I suppose if you planned on running many queries that would be faster, but I bet octosql could also convert it to something like parquet for similar gains.

There's also some tricks you can do with json filtering if strings are involved, if you have a query like "select path from http_logs where hostname = 'news.ycom...'" you can read a whole chunk of data, do a `memmem` for the string and if no matches, not even bother decoding ANY of the json.

> If you had a million of them to process thats 4 hours

If you had 20 bank accounts sending you monthly statements for 100 years you'd have 24,000 of them. 50ms each would take you 20 minutes to process the 100 year backlog.

If you have a million of them to process then you're a bank or similar type of institution that can devote resources to this, either computational or developer time to optimize things. A 128 core c6a.32xlarge would turn the runtime from 4 hours into 1.9 minutes and cost $5 to run for an hour.

You can make python binary search super fast if you use mmap. here's a version of that I had lying around, it's probably correct.

  import os
  import mmap
  
  def do_mmap(f):
      fd = os.open(f, os.O_RDONLY)
      size = os.lseek(fd, 0, 2)
      os.lseek(fd, 0, 0)
      m = mmap.mmap(fd, size, prot=mmap.PROT_READ)
      return m, size, fd
  
  SEEK_SET = 0
  SEEK_CUR = 1
  
  class Searcher:
      def __init__(self, file):
          self.file = file
          self.map, self.size, self.fd = do_mmap(file)
  
      def close(self):
          self.map.close()
          os.close(self.fd)
  
      def find_newline(self):
          self.map.readline()
          return self.map.tell()
  
      def binary_search(self, q):
          pos = 0
          start = 0
          end = self.size
          found = False
          #this can get stuck with start = xxx and end = xxx+1, probably from the \r\n
          while start < end - 2:
              mid = start + (end-start)//2
              self.map.seek(mid)
              pos = self.find_newline()
              if pos > end:
                  break
              line = self.map.readline()
              if q < line:
                  end = mid
              elif q > line:
                  start = mid
  
          while True:
              line = self.map.readline()
              if not line.startswith(q): break
              yield line
  
  if __name__ == "__main__":
      import sys
      q = sys.argv[1]
      s = Searcher("pwned-passwords-sha1-ordered-by-hash-v6.txt")
      import time
      ss = time.perf_counter()
      res = s.binary_search(q.upper().encode())
      for x in res:
          print(x)
      ee = time.perf_counter()
      print(ee-ss)

Use look:

  look $(echo -n password | sha1sum | cut -d ' ' -f 1 | tr a-z A-Z) pwned.txt

from man page:

NAME

look - display lines beginning with a given string

DESCRIPTION

The look utility displays any lines in file which contain string. As look performs a binary search, the lines in file must be sorted (where sort(1) was given the same options -d and/or -f that look is invoked with).

example:

  justin@box:~/data$ time look $(echo -n secret123 | sha1sum | cut -d ' ' -f 1 | tr a-z A-Z) pwned-passwords-sha1-ordered-by-hash-v6.txt 
  F2B14F68EB995FACB3A1C35287B778D5BD785511:17384

  real 0m0.212s
  user 0m0.005s
  sys 0m0.001s

  justin@box:~/data$ time look $(echo -n secret123 | sha1sum | cut -d ' ' -f 1 | tr a-z A-Z) pwned-passwords-sha1-ordered-by-hash-v6.txt 
  F2B14F68EB995FACB3A1C35287B778D5BD785511:17384

  real 0m0.002s
  user 0m0.003s
  sys 0m0.001s

Sticking an official looking IBM label complete with (r)s onto a new battery made from scratch seems like a bad idea and a way to get sued.

> In both those companies, people die if the software doesn't work right

A non-zero number of people are dead due to issues with Tesla software.

Does network effect even apply if it's compatible with existing git repositories?

Cool, maybe they could get the rest of the government agencies to stop requiring bullshit FIPS certified crypto implementations that are all written in memory-unsafe programming languages.

Right, arr[index] is sugar for *(arr+index) which is then also *(index+arr) and index[arr]

edit: fixed escaped asterisks :-)

This might speed up the py impl a few %

  --- a/py/src/cpu.py
  +++ b/py/src/cpu.py
  @@ -260,7 +260,6 @@ class CPU:
               param = None
               cmd_str = cmd.name
               self.PC += 1
  -        self._debug_str = f"[{self.PC:04X}({ins:02X})]: {cmd_str}"

I remember trying to fix Katello once and trying to figure out why "candlepin" was broken because it couldn't talk to "gutterball". Everything about the situation was terrible.

From the linked bug..

> It takes about 50ms to parse an int string with 100,000 digits and about 5sec for 1,000,000 digits. The float type, decimal type, int.from_bytes(), and int() for binary bases 2, 4, 8, 16, and 32 are not affected.

Sure seems strange to set the limit to 4300. 50ms is not a DoS.

> if you use the VM disk for your workspace, which you should.

Yep, if you try to build something that is stored on /mnt/c you'll have the same horrible performance.

It's not the COW filesystem that is slow, it's bind mounting the host filesystem into the VM. This ends up using something like 9p or sshfs.

This is a cool project and yay nix, but:

> Inside a container the file system is very slow

Because you are not using "a container".

You are using a container that happens to be running in a linux VM on your OS X laptop. It's not the containers fault, it's the entire virtual machine that you are running because you're trying to run technology built on top of one operating system while running a completely different one.