leopoldj

Generated a Bangla (Bengali) podcast from a complicated property lease and price trend analysis document. I'm floored. Impossible to tell that the pod casters not real. I'm sure over a long term it will sound monotonic and disengaging. But what we have here is simply a breakthrough.

> there are eunuchs who have made themselves eunuchs

This appears to be a bad translation. Modern translation is "and there are those who choose to live like eunuchs for the sake of the kingdom of heaven". This is referring to celibacy. The issue is explained in more detail by Paul in 1 Corinthians 7 verse 8-9.

Google search is weirdly hallucinating saying "Theodore Scott Glenn is an American actor and Distinguished Professor at Rutgers University". As far as I can tell the actor and professor are two different people. Am I wrong? Can't tell what is true/false anymore.

For what LangChain does, most of the time I see no need for any framework. I would rather directly work with a vendor's official package. LangGraph is different. It is a legitimate piece of workflow software and not a wrapper framework. Now, when it comes to workflow there are many other well established engines out there that I will consider first.

My question was about running it unquantized. The author of the article didn't say how he ran it. If he quantized it then saying he ran it on a laptop is not a news.

>it can run on my laptop

Has anyone run it on a laptop (unquantized)? Disk size of the 32B model appears to be 80GB. Update: I'm using a 40GB A100 GPU. Loading the model took 30GB vRAM. I asked a simple question "How many r in raspberry". After 5 minutes nothing got generated beyond the prompt. I'm not sure how the author ran this on a laptop.

This is a faithful reproduction of the original Transformer paper [1]. Except, these days we use trainable parameters for positional embedding. The paper used a static calculation for positional embedding using sine and cosine.

Figure 1 in the paper can be seen implemented in the forward() method of the GPT class in model.py. Here are the rough steps:

1. Tokens are embedded using a nn.Embedding layer. 2. Tokens are positionally embedded using a nn.Embedding layer. 3. The two embedding values are added to make the input x. 4. A sequence of N number of transformer blocks are then executed. This is the grey box in the left of the Figure 1. This is where all the magic happens. Chiefly in the self attention calculation. You can see this in the forward() method of the CausalSelfAttention class. 5. A regular nn.Linear layer is executed. 6. Finally the output token probabilities are calculated using F.cross_entropy (shown as softmax in the figure).

I hope this helps a little. Please feel free to suggest improvements and additions.

[1] https://arxiv.org/pdf/1706.03762

You should be able to make a few small changes to support "mps".

In TrainingConfig set the device to "mps". The run training.

In sample.py modify parse_args() and add support for mps as a possible value for the --device argument.

The article goes into details of what the resize tool does:

> It uses generative AI to stretch the backgrounds of images to fit these required dimensions...

I have created a templating system in Java that uses the Vue syntax. Many of the common features are present. To minimize dependency, I chose to use Java's built-in XML parser to parse the template. Which means, the template has to be a valid XML. I've been using it mainly to generate email content.

https://github.com/bibhas2/Zippy

These "backdoors" are for court authorized wiretapping (The Communications Act of 1934). This is not some kind of a dark conspiracy. Our government argues that wiretapping is an essential tool for law enforcement. [1]

The NSA does have a program to intercept traffic [2]. But that's not what got hacked here.

I do blame CISA and NSA though for not protecting the system well enough.

[1] https://www.americanbar.org/content/dam/aba/publications/lit...

[2] https://en.wikipedia.org/wiki/PRISM

While I agree with you, this is the release of a research paper [1] and some accompanying demos on GitHub [2]. This is not a finished product fine tuned for high quality output.

[1] https://d1qx31qr3h6wln.cloudfront.net/publications/FUGATTO.p...

[2] https://fugatto.github.io/

This is the exact level of details I was looking for. I'm fairly experienced with deep learning and pytorch and don't want to see them built from scratch. I found Andrej's materials too low level and I tend to get lost in the weeds. This is not a criticism but just a comment for someone in a similar situation as I'm.

I wrote a series of blog post precisely for someone in your situation. I hope you find them useful.

Easiest way to run a local LLM these days is Ollama. You don't need PyTorch or even Python installed.

https://mobiarch.wordpress.com/2024/02/19/run-rag-locally-us...

Hugging Face can be confusing but in the end a very well designed framework.

https://mobiarch.wordpress.com/2024/03/02/start-using-mistra...

I agree with you. But what is the sentiment in the C++ community? I mean, other languages are profiting tremendously from package managers. In C++ things are going the other way - taking public libraries and putting in std (fmt, chrono, and all that).

Instead of giving you a list of books I'll give you a list of topics to learn well. They are listed in a proper learning sequence.

- Modern object initialization using {} and ().

- std::string_view

- std::map

- std::stack

- Emplace addition of objects to containers like vector and map.

- Smart pointers (std::unique_ptr, std::shared_ptr and their ilk).

- Ranges library.

- Concurrency support library (std::async, std::future, std::thread, locks and the whole deal).

C++ programmer and educator here. This (volume 3) is well organized good beginner level teaching material. You probably know most of it already.

I was looking for range-based for loop, std::array and std::span and happy to see that they are all there.

Because this book relates to HPC, I'd add a few things: Return Value Optimization, move semantics, and in the recursive function section a note about Tail Call Optimization.

As a beginner level material I can highly recommend it.

The use of ``this Self&& self`` in the ``at()`` method can appear to be an overkill. But this approach actually covers many scenarios and make the code much smaller. For details, you can read:

https://devblogs.microsoft.com/cppblog/cpp23-deducing-this/

You didn't say which std function you were using that was slow. But std::from_chars performs a lot faster than std::stoi et al due to many reasons.