D4Ha

its not always possible to flash your trust zone firmware, you get what the vendor forces on you.

I see your point, However the majority of Android devices have lots of closed source firmware/drivers, a large part of the Android OS doesn't run without them.

I'm refering to Bootloaders, TrustZone OS, Trusted Applications, then firmware for Bluetooth, Wifi, GPU, Sensors and power management. All of those are always closed source binary blobs running in the background.

Difficult to say at the moment, but there might be some speculation as to why.

One might be that they dont security patches reverse engineered and vulnerabilities to come out faster (some critical and high severity fixes in GPU drivers/bootloaders were often delayed), and this decision was done long before LLMs were considered powerful/useful for vulnerability research.

Second reason might be simply "Control of the android ecosystem", Google may just want to build a wall around android and make it frustrating for other vendors to compete.

Third, again, this is only speculation, is the current push for electronic ID in the EU and other countries, as well as DRM/copyright protections for media and locally ran LLMs (we've heard of how google pushed small LLMs with chrome updates), if they can do the same on some high end android devices, Google would be more invested into further locking down Android devices.

Unfortunately I checked, SLI doesnt work for this situation. Because the program loading the LLM uses CUDA library, which doesnt account for/takes advantage of SLI for this purpose at least.

Do you find it useful or worthwhile to split a large LLM across two GPUs on a desktop?

If you've tried it, what worked well and what didn't? I'm especially interested in mismatched VRAM setups, e.g. a 16 GB GPU + a 24 GB GPU.

How much overhead did you see from inter-GPU transfers, and did the extra usable VRAM outweigh the performance hit?

Correct me if I'm wrong, this is not the right way to ask this question.

LLMs are good at pattern recognition, so its less a type of math that they'll be good at, and more that when you provide documentation or text that can be easily parsed/compared to its training data/reasoning ability, the better answers you get from an LLM.

Also, you need to be knowledgeable at the same thing you are asking the LLM to do, to verify the answer it gives you (at least for the time being).

There is an unusual number of nvidia GPUs being sold on ebay from china that has higher than usual memory.

One can only assume its modified cards, using the same technique mentioned by other commentators here.

How does is anyone able to run this thing locally without paying too much? (I'm interested in specs or GPU that could handle it)

The current leap in research and its byproduct like ChatGPT are impressive (in the sense that it can pick up human text/speech and return back answers). But its not in any way a replacement for good Software Engineers, your job is safe (what is being posted on social media and news is extreme hype).

Here's the funny thing, if you ask people who interact with chat bots in any given service industry (banks, online orders...etc) they'll tell you how bad these things are, there is just no replacement for actual human beings with real life intuition (this kind of AI break through hasn't happened yet).

For the time being if you take time to specialize in a domain knowledge skill (physics, math, economy, or even agriculture) you'll be able to use the skills you learn from your CS degree in that domain and prosper as a result.

Here is the thing about pursuing a degree, it is a tedious awful long task. It doesnt actually look like the actual job you'll be doing when you finish this degree.

I'll go against the common advice and say, stick to it, finish the degree because it will give you good social mobility (when trying to travel for work to any country in the world, you'll either need to be a specialist in your field with few years experience or have at minimum a bachelor degree).

Hexrays used to be difficult to deal with if you want to purchase IDA Pro for the first time, due to their software getting leaked online.

They have eased the procedure to buy from them, but from time to time they'll ask you to fill out your info with national ID/passport (they say its because they don't want to sell their software to individuals under sanctions). This is despite them being based in Belgium (not the US).

For any serious work IDA Pro is highly suitable (the customization and scripting, loader examples and processor plugins...etc), on the other hand for side projects and basic security research Binary ninja and ghidra can go along way.

Shouldn't it be better described as a GRUB trojan. Unlike true bootkits, which persist through OS reinstalls by exploiting firmware.

This can be easily removed by reinstalling GRUB or the OS, making it less persistent and not a genuine bootkit.

Given that this is an open source project and sniffing bluetooth can be fun but expensive (only some specialized tools can offer the best results or features), I feel that this project is much appreciated, came close to help those doing any research of bluetooth devices.

Video of the project's author introducing the tool and pros/cons of sniffing bluetooth traffic.

https://www.youtube.com/watch?v=nClZzdvGlKg

happy you found it to be useful. I also wanted to write an implementation for sometime. The only things I could find were ready-made packages (python/Golang...etc).

The part I'd also like to know about is error correction, if you have anything useful related to QR codes for that.

I agree, the original link to the post, doesnt have a date as far as i can tell, and I can't edit it anymore.

That was lost on me, because the second quote felt like a prediction of the impact of technology on our social lives.

Interesting quotes to go with this Post

“So, your kids must love the iPad?” I asked Mr. [Steve] Jobs, trying to change the subject. The company’s first tablet was just hitting the shelves. “They haven’t used it,” he told me. “We limit how much technology our kids use at home.” (NYtimes article, Sept. 10, 2014)” ― Nick Bilton

“Technological progress has merely provided us with more efficient means for going backwards.” ― Aldous Huxley