HNHacker News
TopNewBestAskShowJobs

isaacfung

241 karma · joined January 13, 2023

submissionscomments
isaacfung··on Researchers have found a faster way to do integer linear programming
A lot of polynomial time algorithms for combinatorial optimization problems can be interpreted as primal dual algorithms for the corresponding LPs, e.g. mst, matching(bipartite or general graph), network flow, matroid intersection, submodular flow. The extreme point solutions of some LPs also have interesting properties that you can exploit to design approximation algorithms for NP-complete problems. For example, you can prove that there is always a variable with value at least half in an extreme point solution of the steiner forest problem, so you can just iteratively round a variable and resolve the LP to get a 2-approximation. When I was in grad school that was the only 2-approximation algorithm for this problem. Another interesting thing is that you can solve LPs with exponentially many constraints as long as you have a polynomial time separation oracle.
isaacfung··on Researchers have found a faster way to do integer linear programming
Yea, Daniel Spielamn and Shang-Hua Teng won the Gödel Prize for their work on smoothed analysis of simplex algorithms. They introduced a way to formally study the worst case complexity of algorithms when the inputs are randomly perturbed by a small amount.

https://www.di.ens.fr/~vergnaud/algo0910/Simplex.pdf

isaacfung··on Spotting LLMs with Binoculars: Zero-Shot Detection of Machine-Generated Text
First of all I dont think we should aim at 100% precision and 100% recall. Instead it is more realistic to use the result as a filter for more downstream testings and treat it as a mean to increase the cost of cheating.

Also we can use more than just the text output. A human writer doesn't generate a piece of text in one pass. Instead they go through the drafting and editing process. We can design devices to capture keypress or pen stroke(iirc ther were studies on fraud detection based on keypress patterns/mouse movment). One can attempt to train a new model to mimic themselves so we need to somehow make sure that the amount of training data required is too much to be worth the effort.

For downstream testing, the goal isn't mainly to verify whether a piece of text is AI genereted but to make sure that a student who can pass the test would essentially have to know the material sufficient well(so this defeats the purpose of cheating).

isaacfung··on Spotting LLMs with Binoculars: Zero-Shot Detection of Machine-Generated Text
Are You Normal? Hint: No

https://allendowney.github.io/ProbablyOverthinkingIt/gaussia...

isaacfung··on Ants recognise infected wounds and treat them with antibiotics
https://www.reddit.com/r/ChatGPT/comments/18qhcst/yooo_gpt4_...
isaacfung··on TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones
Something like this?

https://imgur.com/a/hPAaZUv

https://huggingface.co/spaces/Qwen/Qwen-VL-Plus

You can also ask it to give you bounding boxes of objects.

isaacfung··on Show HN: Emu2 – A Gemini-like open-source 37B Multimodal Model
Wow, the other link is so much faster. Almost the same speed as the much weaker llava demo on huggingface.
isaacfung··on Show HN: Emu2 – A Gemini-like open-source 37B Multimodal Model
The examples look very promising but the queue time of the huggingface demo is a bit long I actually got a timeout error :(
isaacfung··on ArXiv now offers papers in HTML format
This would never happen but in an ideal world, we should be able to click on a citation to jump to the part of the paper that is being referenced and each paper page should have a discussion board so we can easily communicate with the authors and group the discussion in one place instead of us having to google to see if there is relevant discussion on twitter/reddit. We can even put links to talks, tutorials, blogs, github repo, demo, paperswithcode/google scholar/open review, background material, a timeline of citations in tree form on the same page(actually I am seeing more machine learning papers that have a project page that does some of these) or even turn it into a mini wiki. I just think html has so much more potential(especially now with LLM we can do semantic search). I wonder if there would be interest in such a chrom extension overlay.

Related projects:

https://github.com/ahrm/sioyek

https://github.com/arxiv-vanity/engrafo

https://github.com/dginev/ar5iv

https://academ.us/article/2111.15588/ (powered by https://github.com/jgm/pandoc I believe)

isaacfung··on ArXiv now offers papers in HTML format
What device and app are you using to read the document?
isaacfung··on Linear Algebra Done Right – 4th Edition
This, betterexplained, ritvikmath, SeeingTheory will give you a very solid math background(I think they are better than 90% of the intro math classes in colleges).
isaacfung··on Using spaced repetition systems to see through a piece of mathematics (2019)
I think that is partly why LLMs are bad at math and often fail at counting subsequences. Play with the tokenizer and you see long numbers are split into groups of 2 or 3 numbers.

https://huggingface.co/spaces/Xenova/the-tokenizer-playgroun...

isaacfung··on Lego MRI Scanners donated to hospitals to help children cope with anxiety (2022)
Who else was hoping to see a real MRI scanner built with LEGO?
isaacfung··on AI hype is built on flawed test scores
Whisper, Stable Diffusion, VoiceBox, GPT4 vision, DALL.E3

Other breakthroughs in graph machine learning https://towardsdatascience.com/graph-ml-in-2023-the-state-of...

isaacfung··on RAG at scale: Synchronizing and ingesting billions of text embeddings
Depends on what you mean by search. Do you consider all Question Answering as search?

Some questions require multi-hop reasoning or have to be decomposed into simpler subproblems. When you google a question, often the answer is not trivially included in the retrieved text and you have to process(filter irrelevant information, resolve conflicting information, extrapolate to cases not covered, align the same entities referred to with two different names, etc), forumate an answer for the original question and maybe even predict your intent based on your history to personalize the result or customize the result in the format you like(markdown, json, csv, etc).

Researchers have developed many different techniques to solve the related problems. But as LLMs are getting hyped, many people try to tell you LLM+vector store is all you need.

isaacfung··on Chinese Netizen Fined over 1M Yuan for Using VPN
If they really want to catch these people, you wont be seeing only a few examples over half a decade. As mentioned in the article, even Hu Xijin (the former editor-in-chief and party secretary of the conservative popular media Global Times) has a youtube channel. Most top universities like tsinghua, SJTU and tech companies like alibaba have official github accounts.

There is no way to implement complete censorship without killing your own tech industry.

isaacfung··on Chinese Netizen Fined over 1M Yuan for Using VPN
You tried to prove these are not independent moves by providing a few incidents out of a 1.4 billion country over a span of several years.

One comment in the first article you linked said 99% of software engineers in china use github.

The CCP is 100% aware that people use vpn to access foreign websites.

isaacfung··on China is flooding Taiwan with disinformation
I know. I live in Hong Kong. Some of these are worrying but a few of them seem questionable. From what I can find, the previous asian games opening also wasn't attended by many world leaders. It's not Olympic. https://en.m.wikipedia.org/wiki/2018_Asian_Games_opening_cer...
isaacfung··on China is flooding Taiwan with disinformation
Can you provide some sources about the flood destroying military forces and Hangzhou government raiding temples?

About Huawei stores being empty, some seems pretty crowded at least on the first day. https://youtu.be/YFeE2xHQVRI?si=2-tqnqXj8aF8srN6

isaacfung··on Building a string art machine
Some people see innovative ways to combine art and technology.

Some people see money.

isaacfung··on Prophet: Automatic Forecasting Procedure
These may be related.

https://www.reddit.com/r/datascience/comments/11vzjhi/statis...

https://www.reddit.com/user/eamonnkeogh/submitted/?sort=top

isaacfung··on Hotel hackers redirect guests to fake Booking.com to steal cards
Dont send the cc info. Use private key to authenticate yourself. Digitally sign the transaction id and the date so people cant record the info and reuse it.

I guess the issue is that people may lose or forget their keys or people can gain access to the signing device. It is harder to understand than using password for verification so people like my parents do not trust it.

isaacfung··on LLM Agent Paper List
I am an idiot. I just read the comments without looking at the linked page.
isaacfung··on Two-Tower Embedding Model
OpenAI used this approach for CLIP before blip and llava. CLIP is used to encode the text prompt in stable diffusion. Not sure about Dalle.
isaacfung··on A hacker's guide to language models [video]
It was all over hackernews. Just google "hacker news gpt4 mixture of experts".
isaacfung··on Two-Tower Embedding Model
Two tower isn't a new term afaik. It's often used in the context of recommendation systems where the two modalities are users/query and items.

https://github.com/creyesp/Awesome-recsys

isaacfung··on Outperforming larger language models with less training data and smaller models
https://github.com/WooooDyy/LLM-Agent-Paper-List#114-reasoni...
isaacfung··on LLM Agent Paper List
https://github.com/WooooDyy/LLM-Agent-Paper-List
isaacfung··on LLM Agent Paper List
LLM has no long term memory(limited by context size or you have to do fine tuning which isnt efficient) and can't interact with the outside world.

You can augment it with retrieval systems(vector store, graph db, sql db + lexical/semantic/graph search) and give it access to tools like api, search engine and code interpreter. Then it becomes an agent.

isaacfung··on Outperforming larger language models with less training data and smaller models
LIMA: Less Is More for Alignment https://arxiv.org/abs/2305.11206

AlpaGasus: Training A Better Alpaca with Fewer Data https://arxiv.org/abs/2307.08701

Textbooks Are All You Need II: phi-1.5 technical report https://arxiv.org/abs/2309.05463

Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Models https://arxiv.org/abs/2307.14430

← PreviousPage 2 of 4Next →