Mistral's mixtral-8x7B-32kseqlen on Vercel | Hacker News Reader

Mistral's mixtral-8x7B-32kseqlen on Vercel

twitter.com

67 pointstosh30 comments

Not censored it seems! Also, it's a completion rather than a chat model, so it will likely not respond to commands.

I dislike the terminology of censorship because it implies everyone starts with unbiased training data and then some censor it. I think it would be better to say it's trained with a different perspective.

There's absolutely no reason to believe training data scraped from wherever is convenient is some neutral baseline.

Imagine a human writer who grows up with certain experiences that make him biased toward a point of view. The human writer is not censored if he writes what he wants even if his writing is informed by his experiences, beliefs, biases, etc. The human writer is being censored if he knows he is not permitted to write about certain topics and, when people ask him to, he answers that he isn't allowed to share opinions of that kind.

If you agree with my take on censorship above, I find it hard to accept that the models today are anything other than censored. This is really obvious with pictures where you can tell that the model generated a picture, looks at it, and decides the generation is not appropriate and gives you a "I can't do that" message.

Censorship is also evident in the intent. When models are trained the intent is, presumably, to get the best data. When RLHF and similar methods teach the model not to express certain opinions the intent is clearly to avoid offense or violating political correctness. This is a classic case of censorship.

I like the word censorship because it encompasses not only "correction" of bias but lobotomy of "harmful" knowledge, like making drugs/explosives.

Scare quotes around "harmful" are probably unwarranted there.

I think censorship implies that it is designed to suppress certain ideas or topics. Hmm.

Tried Orca 2 first reasoning challenge and it responded correctly (though not in a "chat" way)

> John and Mark are in a room with a ball, a basket and a box. John puts the ball in the box, then leaves for work. While John is away, Mark puts the ball in the basket, and then leaves for school. They both come back together later in the day, and they do not know what happened in the room after each of them left the room. Where do they think the ball is?

GPT 3.5 turbo didn't give a correct answer

The description says it's served by Fireworks. They have some posts about their mixtral work here: https://twitter.com/thefireworksai

To save you a click: https://sdk.vercel.ai/

I don't recognize a single word in this title.

You are probably not in web or AI industry?

I work in both

I am honestly surprised you don't recognize one of the biggest players in LLM (Mistral) and on the web (Vercel).

But looking at your comment history you do seem to have broader text comprehension problems, I think you should try LLMs, they can help.

32K seq length ? Really ? Here says: Context 4,096 tokens

This is first model that gets this puzzle right

“three missionaries and three cannibals must cross a river using a boat which can carry at most two people, but these are nice cannibals so they don't eat people under the any constraints, for both banks, if there are missionaries present on the bank, they can be outnumbered by cannibals (if they were, the cannibals would not eat the missionaries). The boat cannot cross the river by itself with no people on board how to solve this?

"they don't eat people under the any constraints" isn't comprehensible. Do you mean the cannibals never eat people and so any solution is valid?

edit: What is the solution you want? I tried plugging your exact words into the model posted and got nonsense (a repeat of the question and then the word "Mathematics"). If I add the prefix "Solve this puzzle: " I get three obviously stupid solution, including starting off with some people already across or just putting all six people in the boat and ignoring the capacity.

https://pastebin.com/raw/JLu0eGkx

I think he means they won't eat missionaries if left in the banks with them, but will do if in the boat with them.

Then what about the clause about outnumbering? Regardless this is clearly an unreasonable test as phrased.

My guess is this reading, but the grammar is so broken that it's not at all obvious or clear that I've guessed right:

“three missionaries and three cannibals must cross a river using a boat which can carry at most two people, but these are nice cannibals so they don't eat people under these constraints: for both banks, if there are missionaries present on the bank, they can be outnumbered by cannibals (if they were, the cannibals would not eat the missionaries). The boat cannot cross the river by itself with no people on board how to solve this?"

In other words, I think what they tried to express is that on the banks the missionaries are safe even if outnumbered. But in the boat they are not. So the rule would be that a missionary can't be in a boat with a cannibal.

If so, a solution is:

- Send two cannibals over

- Send one back with the boat.

- Send two missionaries over - here's where rule matters, as the remaining missionary is now on the bank with two cannibals, but that's fine.

- Send one missionary back

- Send two missionaries over. All missionaries are now on the same side.

- Send one cannibal back.

- Send two cannibals over.

Still, I think the closest you can get to being "right" on this is to offer up a possible reading of the rules, solve it given that reading is right and ask for a confirmation. I think I wouldn't accept an answer as "correct" here without seeking clarification about whether you got the rules right, as just assuming a given reading is unreasonable when it's this unclear.

> but these are nice cannibals so they don't eat people under the any constraints, for both banks, if there are missionaries present on the bank, they can be outnumbered by cannibals (if they were, the cannibals would not eat the missionaries).

I think I understand what you intend the puzzle to be, but you're saying the opposite of what you actually mean here.

No, that's their intent.

A lot of the LLMs have trouble with variations on typical logic puzzles.

If you change an aspect, they will still regress to autocompleting the standard solution.

In some cases, you can still get them to solve the variation, but it often takes finessing to break the token patterns (things like replacing the nouns with emojis or instructing that it repeat the variation parts when thinking through the solution in a CoT).

The solution for the above should be the minimum number of trips (three) taking two at a time to the other side, no extra stuff.

> No, that's their intent.

A lot of GP's comments elsewhere are similarly malformed, like: "I was thinking same, had hobby project where I did uploaded csv to DuckDb and then I use generate queries with chatgpt but building sematic layer top op duckdb sound much better."

Still understandable, but enough that I suspect the mistakes in their above comment are also just mistakes, opposed to intentionally testing the model's response to broken grammar and cannibals that seemingly never eat anyone.

> The solution for the above should be the minimum number of trips (three) taking two at a time to the other side, no extra stuff.

I believe that's the wrong interpretation of what the author is probably asking. Ideally it should ask for clarification on what's meant. Barring that, I'd say it should interpret the question in the way that makes the most sense - assuming just invalid wording rather than both invalid wording and intentional trickery.

(also, given the boat cannot make the return on its own, you'd still need more than three trips)

... Could you reformulate the puzzle? Right now I couldn't solve it.

As written, it's not a logistics puzzle, it's a test of reading comprehension.

Both the missionaries and cannibals are people, and the cannibals don't eat people, so neither the missionaries nor cannibals are in danger of being eaten.

Any person may man the boat to ferry any other person across.

Given it's not grammatically or well-formed, I don't feel it's reasonable to assume a given reading as there are multiple possible readings that all requires assuming a certain set of corrections to punctuations or words.

Frankly, the only "correct" response to this is to not offer a solution, but to suggest one or more possible readings and asking for corrections, as it's not solvable without knowing how to resolve the grammatical errors.

In the absence of any prior knowledge, perhaps. But if you're aware that "missionaries and cannibals" [1] is a common toy problem for LLMs, this variation becomes an apparent anti-logic puzzle.

Also, because we know that punctuation, capitalization, and special characters are removed prior to vectorization, they can be disregarded by us, as well. We do not need more perfect grammar or syntax.

Thus, there is sufficient probability of intent in the information encoded that we can reject any attempts to solve it that use cannibalism as a constraint. That is the test.

[1] https://en.m.wikipedia.org/wiki/Missionaries_and_cannibals_p...

It might well be an "anti logic" puzzle but that doesn't change the severely broken grammar which makes it unclear how exactly the rules are changed.

And the fact several of us were left here wondering about the meaning is clear evidence that it's unreasonable to expect it to be comprehensible to an LLM, and even more so to treat it as a good test, because it's severely compromised by the broken description.

I stand by my assessment, which is that an LLM should be rated as failing this unless it responds by either setting out a plausible interpretation and asking if it is right or asking for a clarification.

And so should people. The illogical response here is to assume toknow (rather than is guessing) what was intended. The expected response would be to be confused, as you can see from the replies.

Why are they called cannibals if they don’t eat people?

Cannibals don't necessarily kill the people they eat.

For example, there's cannibalistic tribes that eat their recently departed ancestors as part of their tradition, but that's their only form of cannibalism.

Cannibalism technically only relates to the eating of humans. Not to the killing of them.

If anything, the assumption that cannibals would kill and eat the missionaries is the leap of logic.

They are fasting.

Reply on news.ycombinator.com