HNHacker News
TopNewBestAskShowJobs

neopallium

136 karma · joined August 25, 2010

  Name: Robert G. Jakabosky
  Location: Guangzhou, China
  Remote: Yes
  Willing to relocate: Maybe
  Technologies: Rust, C/C++, Lua, Java, Node.js, Javascript/HTML/CSS, Substrate,
    MySQL/Postgres/MongoDB, JIT/Compilers, FIX protocol, ZeroMQ,
    FreeSwitch/Asterisk, Docker
  Résumé/CV: https://cn.linkedin.com/in/neopallium
  Other profiles: https://github.com/Neopallium,
    http://stackoverflow.com/cv/neopallium-34934
  Email: rjakabosky@sharedrealm.com
  Wechat: robert2051
Full-stack developer. Worked with stock market datafeeds (BridgeFeed,FIX,Activ Financial) for about 7 years. IVRS/VOIP using FreeSwitch/Asterisk. Implemented a JIT/static compiler for Lua [0][1].

I am a Canadian who moved to China about 3 years ago. Anyone else around Guangzhou/Shenzhen/Hong Kong? I am interested in meeting others around here.

0. https://github.com/Neopallium/llvm-lua

1. https://github.com/Neopallium/slua

[ my public key: https://keybase.io/neopallium; my proof: https://keybase.io/neopallium/sigs/jvP68uqQEiNiYP6NcRz8Ps8axbNXG62Pv94A3KIww04 ]

submissionscomments
neopallium··on Establishing a UART Root Connection to a TP-Link Wireless Router
I needed to get the PPPoE account off a router once (My boss didn't remember the router's admin password), so I connected the WAN port to my laptop and captured the PPPoE packets. I might have had to use a second ethernet port (USB hub with eth port) to allow the old router to talk to the telcom modem. Just bridge the two ports and capture the traffic.
neopallium··on StackOverflow user talks about GPT4
Create a StackOverflow + GPT site where questions can be asked in public with both human and AI answers. Make it clear to users that the content will be used in future training sets.
neopallium··on Emitting Safer Rust with C2Rust
I used c2rust to start rewriting OpenJpeg into Rust code [0].

It was easy to get the Rust code compiled and working as a drop-in-replacement for the C Library. This has been a big help with refactoring the unsafe Rust code into safe Rust (manual work). OpenJpeg has a great testsuite that has allowed testing that each refactor step doesn't add new bugs (has happened at least 3 times).

The original run of c2rust generated 96,842 lines of Rust code (about 1 year ago), now it is down to 46,873 lines code. A lot of the extra 50k lines of code were from C macros that got expanded and from constant lookup tables (C code had 10-30 values per line, Rust 1 value 1 line).

For anyone looking to use c2rust to port C code to Rust, I recommend the following:

  1. Setup some automated testing if it doesn't exist already.
  2. Do refactoring in small amounts, run the tests and commit the changes before doing more refactoring.
  3. Use "search/replace" tools (`sed`) to help with rewriting common patterns.  Make sure to follow #2 when doing this.
  4. Don't re-organize the code until after most of the unsafe code has been rewritten.  This will allow easier side-by-side comparison with the original C code.
  5. c2rust expands macros and constants from `#define`.  Being able to do side-by-side comparison of the C code will help with adding constants back in and removing expanded code with Rust macros or just normal Rust functions.
[0] https://github.com/Neopallium/openjpeg/tree/master/openjp2-r...
neopallium··on DALL-E 2 has a secret language
Would it be possible to build a rosetta stone for this secret language with prompts asking for labeled pictures of different categories of objects? Or prompts about teaching kids different words?
neopallium··on The overengineered solution to my pigeon problem
A motion activated sprinkler could work too. Or buy a motion detector and have it activate plant water sprayers (the kind used for automatic watering of potted plants).
neopallium··on James Webb: 'Fully focused' telescope beats expectations
If both Humble and Webb were pointed at the same spot of the sky at the same time, the images could be merged to get a wider range of the light spectrum.
neopallium··on C2rust: Transpile C to Rust
I just started using c2rust on openjpeg [0] (jpeg 2000 encoder/decoder) today and already have it working as a drop in replacement for the C libopenjp2.so on Linux. Still has a lot of unsafe code, but it does work. Which will be a big help with testing during refactoring to idiomatic safe Rust.

c2rust also has a refactor command that helps with refactoring the generated Rust code.

[0] https://github.com/Neopallium/openjpeg/tree/c2rust

neopallium··on Rust is now overall faster than C in benchmarks
The Rust compiler can auto-vectorize loop code.

This blog post shows how to write simple idiomatic Rust code that will allow the compiler to auto-vectorize:

http://cliffle.com/p/dangerust/6/

neopallium··on Rust is now overall faster than C in benchmarks
Someone wrote a 6 part of blog post [0] about porting that nbody benchmark from C to Rust. They went from a straight line by line port using unsafe rust using the same SSE based design to clean no-unsafe and no SSE rust code that was faster then the original C code with hand optimized SSE.

It is a great example of how the Rust compiler can auto-vectorize code.

0. http://cliffle.com/p/dangerust/6/

neopallium··on FastCGI – The Forgotten Treasure (2002)
FastCGI doesn't set environment variables. Only CGI does that since the CGI binary is executed for each request. FastCGI binaries run as a server listening on a unix/tcp socket for connections from the frontend web server. It works the like a webapp listening for http requests on a local socket with reverse http proxy rules setup on the frontend web server.
neopallium··on Adam Neumann planned for his children and grandchildren to control WeWork
I wonder if a database of past videos of these people (after the truth is known) and machine learning would be able to make a bullshit/con detector? Maybe the same could be done with old PR news from companies that fail to deliver (pump and dump). Would ML be able to detect if the person truly believes what they are pushing?
neopallium··on Some notes about HTTP/3
The client doesn't have to use a "connected" UDP socket either. The client can bind to a random local port and use "sendto" to talk to many servers.
neopallium··on Bisected: The Unfortunate Reason Linux 4.20 Is Running Slower
Is there a comparison table of CPU vulnerabilities? I would like to know which CPU has the least vulnerabilities or the best performance after patching.

From what I have read so far is seems that AMD CPUs have had the fewest vulnerabilities/slowdowns? But I can't be sure since I haven't seen a complete comparison (including these new vulnerabilities).

neopallium··on This Is Why You Shouldn't Interrupt a Programmer (2013)
I have lost my train of thought from vague comments or questions like that.

The comment in the article was "…email about that 'thing'". I would immediately think "what thing?", then everything else goes down the drain.

neopallium··on Is serverless insecure? Let's find out
Also with: ps get

Found that accidentally because of autocomplete on my phone changed "-ef" to "get".

neopallium··on Darwin/macOS emulation layer for Linux
The readme has an example of unpacking Xcode and compiling a simple helloworld program using the CLI.
neopallium··on x64asm: C++ Library with in-memory assembler, parser, and linker
LLVM-lua was just an experiment. I wanted to see what kind of performance statically compiled Lua could achieve.

I didn't really need the JIT feature. But I did add support for it because Lua code can dynamically generate/load more Lua code.

I just found that the runtime overhead (memory and stack space) of LLVM was too high for my needs.

neopallium··on x64asm: C++ Library with in-memory assembler, parser, and linker
I don't think that answer was there when I had first come across that question (back in 2010-2011).

That answer is about typed languages. The issue is with dealing with dynamic types. There is a lot more code generated for the JIT to compile.

neopallium··on x64asm: C++ Library with in-memory assembler, parser, and linker
I hadn't heard about that. I have been working on other projects, so haven't had the time to keep up with the developments in LLVM.

Thanks for the heads up.

The main issues I had with LLVM was the partly do to the memory usage. It can product very good machine code, but the time needed to JIT a block of code was sometimes longer then just running the it in the interpreter.

One of the features I like about Lua the most is the low resource overhead. LLVM was just too big for what I was looking for.

Usage LLVM to JIT/AOT compile Lua code was really just an experiment, something I wanted to try out.

I re-wrote [0] the llvm-lua project with a simple C code-gen backend for compiling Lua code to native code. Which is a bit more useful, since it is easier to cross-compile pure C code, then LLVM bitcode. A lot of the people that tried using llvm-lua wanted to cross-compile Lua code and embed it into an iOS application.

0. https://github.com/Neopallium/slua

neopallium··on x64asm: C++ Library with in-memory assembler, parser, and linker
I have tried using LLVM as a JIT for Lua [0] (7 years ago), but found the codegen to be too slow [1] and use too much memory. It might have improved since then, but I really don't think it can compete with LuaJit's highly optimized tracing JIT.

Others [2] have also found LLVM to be unsuitable as a JIT.

0. https://github.com/Neopallium/llvm-lua

1. http://stackoverflow.com/questions/4077396/llvm-jit-speed-up...

2. http://stackoverflow.com/a/6833494/98107

neopallium··on How we're scammed into eating phony food
Companies using fake advertising need to be held accountable.

I have seen and heard about so much fake advertising, that I don't really even care what the label says anymore.

How hard would it be to create easy to use testing kits? (like home water tests for heavy metals)

If we could create test kits to help identify heavy metals, peanut oil, or other bad additives. Then that would help consumers to identify fake food.

I don't know how hard it would be to make kits like this, but it seems there is a market for it.

neopallium··on Understanding glibc malloc
ptmalloc[0] is developed by Wolfram Gloger and is a modified version of Doug Lea's malloc (aka dlmalloc [1]).

0. http://www.malloc.de/en/

1. http://gee.cs.oswego.edu/pub/misc/malloc.c

neopallium··on LuaRadio: a lightweight software-defined radio framework built on LuaJIT
Mike Pall has been looking for a new LuaJIT maintainer [0] for almost a year.

0. http://www.freelists.org/post/luajit/Looking-for-new-LuaJIT-...

neopallium··on Human Connectome Project Updates
Here is a link to the project's website:

https://www.humanconnectome.org/

neopallium··on Gag Order on Lavabit’s Levison Lifted
Either they had captured packets from Lavabit that they wanted to decrypt, or they wanted to be able to decrypt all traffic to/from Lavabit.

With how much traffic the NSA was collecting and archiving during that time, it is very likely that they had captured traffic that they thought might contain messages to/from Snowden.

neopallium··on P4: open-source programming language for high-performance packet switching
I am interested in Snabb and other methods of doing high performance packet processing to help with developing DDOS protection methods that could be implemented without expensive hardware.

If DDOS filtering modules can be written in P4, then they could be used on software switches and maybe even compiled into packet filters that run before packets are processed by the Linux kernel (or compiled into a kernel-bypass module for embedding in an application).

I think we really need to have better support for low-level high performance packet filtering (i.e. before the packet gets to higher levels in the kernel).

With multi-gigabit networking becoming cheaper, we really should have better support for dropping bad traffic.

How about a switch layer that runs in the kernel? Physical NICs and virtual NICs could be handled by the switch layer (programmed using P4 or eBPF). The kernel would only have to handle packets it receives from the virtual NICs. This would allow packet filtering and even some routing to happen without having to go through the kernels network-stack. The software switch layer would work like a data-plane with the kernel as the control-plane (like a lot of hardware routers/switches). If the computer has specialized hardware (smart NICs, co-processors), then the switch logic (P4 code) could be handled in that hardware instead of on the CPU.

neopallium··on A queue of page faults (2014)
The processor will try to do write-combining to fill entire cache lines, so it doesn't need to do a memory read. GCC already has a number of intrinsics to help with streaming data directly to RAM (helps to keep from evicting more important data from the cache).

The cache can be bypassed when doing a lot of writes. See section "Bypassing the cache" of [0].

That article says that memset() already uses cache bypassing instructions for large blocks.

0. https://lwn.net/Articles/255364/

neopallium··on P4: open-source programming language for high-performance packet switching
Someone has started working on P4 support [0] for Snabb Switch [1].

I think a language like P4 will be great for sharing switching logic. I hope we will see re-usable modules written in P4 that can be shared.

0. https://github.com/esbullington/snabbp4 1. https://github.com/snabbco/snabb

neopallium··on Cache Attacks Enable Bulk Key Recovery on the Cloud
The attack doesn't give them access to all information from another VPS on the same host. The are using timing analysis to detect what code the other VPS is executing and finding the private key from the noise caused by the victims VPS.

Crypto libraries can be fixed to prevent side-channel attacks like this. The article says that libgcrypt has already been patched.

neopallium··on Why is a Rust executable large?
Golang binaries are also very large [0].

  package main
  func main() {
    println("Hello, world")
  }
That compiles to 1.1M using go 1.5.

0. https://github.com/golang/go/issues/6853

Page 1 of 2Next →