skyde

We are talking about university teacher. They are definitely the one writing and grading the exams.

Right I guess I am not familiar how automated Benchmarks for LLM work. I assumed to decide if an LLM answer was good required Human Evaluation.

what do they mean by "evaluating the model on corpus." and "Evalutes the corpus on the model".

I know what a LLM is and I know very well what is Bayesian Optimization. But I don't understand what this library is trying to do.

I am guessing it's tryng to test the model's ability to generate correct and relevant responses to a given input.

But who is the judge ?

why just not make all robocalls illegal instead :-) I dont care if its a AI voice or a recorded message.

100% agree, that’s why MongoDB is so popular despite awful result of Jepsen

When you mention The Objective Information Criterion. I assume you mean something that takes into account the complexity of the model. A model with more parameters might fit the data better but could also be at risk of overfitting.

But isn’t there already a blackbox measure of overfiting one that focus on how well the model generalizes to new, unseen data, rather than on the model’s complexity. Like Cross-Validation, hold-out validation or bootstrap method.

Well some company try to implement Raft algorithm with good intentions but do a bad job a at so bugs make it so they fail Jensen test.

You can’t force people to test their software correctly unless it’s a regulated field like aviation.

They could at least introduce a new language construct like await synchronizedAsync(lock) { // some code }

C# introduced:

       await foreach (int item in RangeAsync(10, 3))

       Console.Write(item + " "); // Prints 10 11 12

So you dont have to type:

IAsyncEnumerator<int> e = RangeAsync(10, 3).GetAsyncEnumerator();

  try {

    while (await e.MoveNextAsync()) Console.Write(e.Current + " ");

  } finally { 

    if (e != null) await e.DisposeAsync(); 

  }

Why synchronized block are not preemptible? When compiling

public void syncMethod() {

        synchronized(lock) {
            // some code
        }

}

they could translate to

public void syncMethod() {

        await reentrantLockAsync.lockAsync();
        try {
            await somecodeAsync();
        } finally {
            await lock.unlockAsync()
        }
    }

how is this relevant?

I think those product can be sliced based on the typical use case.

Kafka and Azure EventHub and RedPanda are EventStream system use Retention policy. They can handle very high throughput efficiently but are not transactional and cannot be safely used when data consistency is the priority.

While system like ActiveMQ, Azure service bus, RabbitMQ are transactional message broker each subscriber to a topic has its own mailbox and message are removed from the mailbox when the client acknowledge that the message has been successfully processed. And deleted from the broker when no mailbox still contain the message. They can be used for workflow involving financial transaction (money) and are very safe. But they sometime dont maintain ordering, client might receive message in different order than the publisher put them inside the topic.

I didn't know that Thanks for your comment.

Except in the case of shared cache database connections with PRAGMA read_uncommitted turned on, all transactions in SQLite show "serializable" isolation

you still have to do

IF @@ROWCOUNT = 2 BEGIN COMMIT TRANSACTION; END ELSE BEGIN ROLLBACK TRANSACTION; END

to make sure both rows have been updated. so its a lot simpler to explicitly take a lock on the rows.

while true this is not what I mean.

if your goal is to transfer 10 dollars for account A to account B but only if account A balance is larger than 10 dollars.

You have to update account A to be 10 dollars less

and

you have to update account B to be 10 dollars more.

you can read both account a keep track of current version number for each.

then do

"UPDATE Accounts SET Balance = CASE AccountID WHEN 1 THEN Balance - 10 WHEN 2 THEN Balance + 10 END WHERE (AccountID, version) IN ((1,v1), (2,v2))"

But if using Read Uncommited isolation level I am not sure this UPDATE would actually lock both row until commit.

because row version does not make it possible to update more than 1 row atomically.

this is the current setup we use in my team but I am trying to move away for this to something more systematic.

yes if your test are using a mock of the db ex: https://github.com/microsoft/coyote/blob/main/Samples/Accoun... you can simulate the races.

The problem is your mock need to implement the same isolation level as your real DB and support transaction ...

You could use SQLITE in memory DB to run your test but that would make your test a lot slower I assume.

Even using tool like this it seem to be a combinatorial explosion issue.

If your app contain 10 distinct transactions.

you have to test what happen if 2 instances of tx #1 run concurrently.

But also if tx #1 and tx #2 run concurrently.

And also what if tx #1 and tx #3 run concurrently ....

I knew of this issue for some time (default DB isolation is less than SERIALIZABLE on most DB).

But assuming you do not want to change the default isolation for your transaction or for the whole DB. How do you write unit-test that check your code will work correctly because it's using "select balance from accounts FOR UPDATE"?

It seem to me the only way to really validate this is using tool like TLA+. Or something like https://github.com/microsoft/coyote

You mentioned group I am very curious what you think of the work Facebook did on memory pressure metric

Bayesian optimization is very good an optimizing black box Knob in shmoo plot usually called meta parameter. Assuming each probing is costly they allow you to find the optional combination of value for many knob (dimension) with a minimum of probing

first time earing about Gurobi, is it some kind of SMT solver? I tried looking at their website but could only find high level marketing description.

I have a question For people in the field!

Would GPT only with pretraining and no fine tuning be able to behave better when prompt is "let’s think step by step"?

Or this prompt only worked because a fine tuning dataset containing many "let’s think step by step" prompt was used?

Well when people say "Ada" they usually mean SPARK not old or feature limited version of Ada.

-Spark do verify contract using static analysis and proof. -While Ada is not as memory safe as Rust it's a lot more memory safe than popular languages. -Its one of the easiest languages to get compliance with safety standard DO-178C or DO-333 ...

C# used to have postconditions and verify them at compile time. But this has been deprecated :( I wish I knew why they removed it.

https://learn.microsoft.com/en-us/dotnet/framework/debug-tra...

this is awesome I was already trying to build the exact same thing but using Orange Pi 5 and Whisper speech to text model.

what kind of Hardware is considered to have "lower interrupt latency"? Is there some kind of Arduino board I could get that fit those lower interrupt latency required for real-time but still support things like Bluetooth?

in my opinion Citus is a lot better. its also available as a managed solution in azure under the marketting name of "Cosmosdb for Postgressql"

https://learn.microsoft.com/en-us/azure/cosmos-db/postgresql...

wow congrats to all people involved it feel good to finally see this feature land

What do you mean by "fresh grad programs require a certain age" this seem like an easy lawsuit for age discrimination.