knaik94

>For the 16 plaintiffs, the complaint indicates that they used ChatGPT, as well as other internet services like Reddit, and expected that their digital interactions would not be incorporated into an AI model.

I don't expect this lawsuit to lead anywhere. But if it does, I hope it leads to some clear laws regarding data privacy and how TOS is binding. The recent ruling regarding web scraping makes the case against OpenAI a lot weaker. [1] Data scraping publicly available data is legal. People didn't need consent to having their data be used, there was an implicit assumption the moment the data was published to the public, like on reddit or youtube.

I keep seeing this idea reoccur in the suit:

>Plaintiff ... is concerned that Defendants have taken her skills and expertise, as reflected in [their] online contributions, and incorporated it into Products that could someday result in [their] professional obsolescence ...

Anyone is able to file a suit, I wish people stopped assuming that a news report automatically means it has merit.

1. https://www.natlawreview.com/article/hiq-and-linkedin-reach-...

Just looking at the eval datasets used in the HF leaderboard, ARC is CC BY-SA, TruthfulQA is Apache, and HellaSwag and MMLU are both MIT. The first dataset many people will explore ml with, MNIST, is GPL-3.0 license. The audio courpus datasets I've used have all been similarly permissively licensed, for example LibriTTS is CC BY 4.0. LJ Speech, one of the main datasets used in Tacotron is public domain. Maybe I should have been more explicit and used the term permissively copyrighted instead to help avoid confusion.

https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...

1) 7B foundational model means this is a base model that has not been fine-tuned on the "prompt: response:" (instruct) structure, and it has 7 billion weights/biases. It's the working size of the model.

2) Currently every model that can run locally was trained with a 2K context size. It's a hard limit on prompt length. There have been recent advances with [A] position interpolation, but those methods explore fine-tuning/loras. This base model was trained with 8k sequences.

3. 1.5T tokens is the size of the total training corpus. Training cost and time increases with training size. [B]

A. https://arxiv.org/abs/2306.15595

B. https://www.semianalysis.com/p/the-ai-brick-wall-a-practical... (Jan 2023)

I don't mean evaluating a model at the end of training, I mean datasets used to quantify the relative performance across models and algorithms.

Copyrighted data would make it harder to share and use, to reproduce results. I don't remember coming across a benchmark dataset that was copyrighted.

The issue isn't that the results are necessarily bad from using a dataset with copyrighted exams. Using it for training data is "fine". But a copyrighted eval dataset makes it basically impossible to reproduce the results. One part of the original paper was validating the use of the MITQ dataset as a way to evaluate various models.

This is seperate from the issues related to the methodology not agreeing with conclusion that GPT4 could "get an MIT degree".

The second conclusion and methodology could have been fixed potentially.

>We want to emphasize that all the student authors in this paper worked really hard on what could have been a very interesting and valuable paper had the data been collected with consent. The many problems with the published work were not the fault of the students.

I appreciate the clear stance that MIT has taken regarding where the responsibility lies in this situation. I think some people are missing the context that many of the authors were undergraduate/early career. Research is an iterative process and every paper has to start somewhere. I don't agree that the paper should be withdrawn because Arxiv is not technically a publication, but I also wouldn't consider the paper properly peer reviewed. Teachers own the copyright the exam material. I was taught copyrighted material can't and shouldn't be used as part of an eval dataset.

The followup by three other MIT ('24) seniors is a great peer review.

https://flower-nutria-41d.notion.site/No-GPT4-can-t-ace-MIT-...

No, GPT4 Can’t Ace MIT - https://news.ycombinator.com/item?id=36370685 - June 2023 (120 comments)

I am confident my bottleneck is thermals and hardware related. I had the same speed when comparing koboldcpp and llamacpp a few versions ago. I am running it on a laptop on a full windows 10 install. It's a 10th gen i7 hex, 16g ddr4, and 2070 maxq mobile. In that context, I consider it remarkably fast.

Koboldcpp [1], which builds on llamacpp and adds a gui, is a great way to run these models. Most people aren't running these models at full weight, ggml quantization is recommended for cpu+gpu or gptq if you have the gpu vram.

GGML 13b models at 4bit (Q4_0) take somewhere around 9gb of ram and q5_K_M take about 11gb. Gpu offloading support has also been added, I've been using 22 layers on my laptop rtx 2070 max q 8gb vram and CLBlast. I get around ~2-3 tokens per second with 13b models. In my experience, running 13b models is worth the extra time it takes to generate a response compared to 7b models. GPTQ is faster, I think, but I can't fit a quantized 13b model so I don't use it.

TheBloke [2] has been quantizing models and uploading them to HF and will probably upload a quantized version of this online soon. His discord server also has good guides to help you get going, linked in the model card of most of his models.

https://github.com/LostRuins/koboldcpp

https://huggingface.co/TheBloke

Edit: There's a bug with the newest Nvidia drivers that causes speed slowdown with large context size. I downgraded and stayed on 531.61 . The theory is that newer drivers change how out of cuda memory management works when trying to avoid OOM errors.

https://www.reddit.com/r/LocalLLaMA/comments/1461d1c/major_p...

https://github.com/vladmandic/automatic/discussions/1285

The discord servers for a few of the projects are relatively popular. Most have a help channel you could post in if you have questions. The Discord for KoboldAI has some developers from koboldcpp ,which is the easiest and one of the most bleeding edge way of running these models locally. It builds on llamacpp and allows the use of different front ends among other things like using k quantized models. People also have had success with using something like runpods.

Native fine tuning is still out of consumer reach for the forseeable future, but there's people experimenting with QLORAs. The pipeline is still relatively new though and is a bit involved.

https://koboldai.org/discord

https://github.com/LostRuins/koboldcpp

When we were having a short in-person google cloud training session, we used Qwicklabs. From my understanding, we were trained by a Google contractor. I don't remember if the trainer described themselves as being from Qwiklab or Google.

In the same session, we were also introduced to AWS, and for that training we were trained by someone on the GovCloud team. The target for the training was in the field of proteomics and bioinformatics with machine learning. As a University there were on campus clusters we could use, but this was for using with smaller projects. Most people I knew ended up preferring AWS and sagemaker.

It was an open secret, and discussed off the record, that AWS will always give you better customer service than GCP, regardless of being a University scale client.

I don't think the team is going rogue, it's just another weird way in which Google refuses to provide direct customer support.

"A lot of developers are interested in getting access to ChatGPT plugins via the API but Sam said he didn’t think they’d be released any time soon. The usage of plugins, other than browsing, suggests that they don’t have PMF yet."

There might also be some concern related to the comments on the approach to plugins. Plugins not having a product market fit reflects slightly negatively, on the companies powering the plugin and OpenAI. It becomes harder to get companies to invest time in creating plugins. I am not sure if the there was an option to remove just the issue sections of the post.

Social exclusion is harmful to the development and mental health of teenagers too.

Their parents trust me to talk to their kids about things they didn't feel comfortable about. Their parents ask me to look out for their kids and implicitly understand that I'm told things that they aren't. I think it's a cultural difference in the role of education between Asian cultures and others. I was even occasionally asked by parents to take a bigger role than I was sometimes comfortable with, for certain things. I was not undermining the parent's roles. I understand your reservations, but they're misguided at best. The parents were who encouraged their kids to call me big brother and have shared their appreciation to me privately.

Social media is a method of communication at its core. Excluding kids from social media now also excludes them from social interactions. Being excluded is also damaging, and we no longer live in a world where not being on social media is practical.[1] It's better to teach them how to identify problematic content from a very young age and guide them. Blocking access will not accompish the goal you are hoping to accomplish. I think the demographics of who falls and shares misinformation and fake news demonstrates my perspective.

"It is possible that an entire cohort of Americans, now in their 60s and beyond, lacks the level of digital media literacy necessary to reliably determine the trustworthiness of news encountered online." [2]

https://www.unicef.org/press-releases/done-right-internet-us...

https://www.science.org/doi/10.1126/sciadv.aau4586

I am giving you perspective from someone that had social media limited. Not having a social media presence caused other kids to leave me out of things. I missed a lot of context on drama that happened online, and was always late in learning about things.

I don't have kids but I have been tutoring kids for the last 14 years, since I was basically a kid. I do individual tutoring for mostly high school, and occasionally middle school kids. Some of the have been with me from 7th through 12th grade, and a couple have even asked me to help them prep for finals even after starting college. They end up trusting me with things they wouldn't tell their parents, because I'm still someone they see as an authority figure. Because of cultural reasons, the kids I've tutor think of me and address me as an older brother. I've noticed covid catalyzed the shift to using digital platforms to keep in touch.

I don't have children, but empathize with the feeling of wanting to be protective. It took a lot of effort to make sure some of kids I tutored weren't taken advantage of by people like Andrew Tate. Telling them not to watch didn't help, pointing out the problems with Tate's message and taking time to dissect it did. Doing that also helped them learn how to identify similar manipulation tactics from other creators. It's important to separate the platform from the users.

I think we disagree on the importance of teaching kids how to use these tools. It feels sort similar to countries banning people from accessing AI tools like ChatGPT. The ban won't slow down AI development, those banned countries are simply left behind.

Social media is an important way for kids to communicate to a group of people outside of their immediate friends. Kids born before broadband did not have the option to communicate in the same way, and trying to live that way now will do more harm than good. It's similar to trying to using a dumb phone that is not capable of group messaging as your only form of communication. Mental health issues are very complex and social media is only a communication platform at its core.

YouTube is not social media because there's no real network of friends. You have creators making content for users to consume. There are comments under videos, but there's no friends feature or direct messaging. Anyone can't post anything. The rest of the world is going to continue to use social media, limiting access won't change that. My perspective mostly agrees with this Unicef report.

https://www.unicef.org/press-releases/done-right-internet-us...

You have the ability to help guide them through social media by helping them understand good and bad. I understand that you might have different beliefs, but social media plays a very critical role in young kids communicating with each other. Covid catalyzed them embracing social media to stay in touch. Youtube also shouldn't be considered social media.

I think we are on the tail end of a generation that grew up without internet trying to govern generations that did. The single question that helps me decide whether someone is equipped to address the underlying problem, is asking if they understand the differences between good and bad social media. The difference between good and bad social media is based on the target audience and how potential dark patterns implemented or addressed.

As someone that grew up with social media and the internet from the moment I could read, I struggle to understand how social connections worked before it. Broadband become mainstream in just the last ~20 years. Social media that quantifies someone's popularity absolutely affects teenagers' mental health. But social media also means support groups on reddit and places like HN.

The surgeon general wants to explore and understand the way different social media affects mental health. But the way this issue is presented by NYT, and in general, categorizes all social media as the same. This framing is going to make younger generations ignore these "warnings" as out of touch, because it is. Similar to how my parents call every game console or video game, "playing the Nintendo". These overgeneralized discussions make most people dig in to their already held beliefs. This also gives overbearing parents fuel to invade the privacy of their children, which harms children's mental health, and leads to trust issues, more than social media ever will.

I had a similar experience with an Android. Even though I never approved the use of my phone, Google account security showed it was registered already.

The prompts went away when I explicitly removed it as a sign in option.

I also had to explicitly disable "skip password when possible".

I didn't realize it misspelled the solution to level 8, and so I spent a few extra hours finding a second valid attack prompt. I am only annoyed with myself for not realizing. I enjoyed the difficulty increase in level 8. I tried using the final prompt, in a new session, for levels 1 through 7 and the response was occasionally filtered. I wonder what the temperature is and what kind of sampling is happening. It seems to be based on GPT3.5, it asserted that GPT4 is not public yet.

And from the c-span youtube channel, timestamp ~2hr 14m 50s.

https://youtu.be/_E-4jtTFsO4?t=8090

You can click the three dot menu next to the share button to pull up the autogenerated transcript or offical CC on c-span, on youtube. I don't think you can search in the yt app like you can on desktop/mobile. The comments section there speaks volumes about the divide between those congressman and younger generations.

It would require the camera to be on without notifying you, which isn't possible on modern OSes. Also, changes in pupil size are caused by many different things. That question is similar to asking whether heart rate can be used as a signal. They're not tracking eye dialation for funny face filters. That level of tracking requires cameras better than phone selfie ones.

"The responses can have a variety of causes, from an involuntary reflex reaction to exposure or inexposure to light—in low light conditions a dilated pupil lets more light into the eye—or it may indicate interest in the subject of attention or arousal, sexual stimulation,[3] uncertainty,[4] decision conflict,[5] errors,[6] physical activity[7] or increasing cognitive load[8] or demand."

https://en.wikipedia.org/wiki/Pupillary_response

https://www.microsoft.com/en-us/research/wp-content/uploads/...

I think I would use the word misinformed or misguided. I feel it's hard to get a clear understanding because there's a lot of performative politics. There isn't enough push back to obstructionism. There are people doing good work in legislature, but mainstream and social media tend to highlight the bad.

Regarding opinions about things like banning TikTok or protecting section 230, the divide is caused by bad technology literacy. I don't think it's fair to say it's been happening for a while, because the pace of tech development has been so much faster than legislature. The median age of the 118th congress is ~58 years old, broadband internet has only been around for the last ~20.

Mullvad has options to pay by cash, bitcoin, or even a physical gift card, in addition to the usual methods. There are no usernames, everything is tied to a randomly assigned account number. It's also moving many servers to a diskless infrastructure, running off ram and supports wireguard and openvpn.

https://mullvad.net/en/blog/2022/8/1/expanding-diskless-infr...

Part of the issue is that the government itself takes part in that marketplace as a seller and buyer. There's no resonable way to make laws for corporations while giving a pass to government agencies, it might be proposed but it would never pass. One of the biggest clients of Cambridge Analytica was the Trump campaign.

I became disillusioned about hoping laws pass that protect user data after watching the TikTok hearing. The congressman questioning whether TikTok tracks eye dialation as a signal for "the algorithm", and the CEO having to explain that it's used for funny face filters, gave me second hand embarrasment. It reminds me of the Facebook hearing where a congressman ask whether it was possible to ban "finsta" and the Head of Global Safety at FB reponding that it's not a product. It's people having a secondary personal account limited to specific friends. An explanation the congressman was having a hard time understanding.

https://www.youtube.com/watch?v=gPZA98whQGI

https://www.youtube.com/watch?v=TGt1Ukg7q4Y

This is only a single state ban, the majority of content creators are unaffected. There is zero chance of this ban having any impact on the network effect. If anything I predict the opposite, people will become frustrated with the walled-garden of iOS, I can see people choosing Android over it for the sake of being able to sideload TikTok. I remember when there was a sudden aftermarket of iOS and Android devices when flappy bird was delisted by the developer. TikTok is obscenely bigger.

"I apologize for any misunderstanding. To clarify, there is currently no GPT-4 model available publicly. As for "ChatGPT3Turbo," that is not a model that I am aware of. However, I can confirm that I am a language model developed by OpenAI, and I use a variant of the GPT architecture known as GPT-3 to generate responses. Is there anything else I can assist you with? "

In a similar tangent, I think Google and ClosedAI/OpenAI have intentionally made their public facing products perform worse over time as a way to assuage people's fears and risk of regulation. I'm sure some decisions are related to scaling, and it's well understood that the heavy handed filtering makes the output worse. But maybe this is seen as a good thing, not bad.

These foundational models have already been created. Government backing has already been used to create a fully open source model. The effort has been multi-national. Government licenses aren't currently required to run one.

"An award of computer time was provided by the INCITE program. This research also used resources of the Oak Ridge Leadership Computing Facility (OLCF), which is a DOE Office of Science User Facility supported under Contract DE-AC05-00OR22725."

Models:

https://www.together.xyz/blog/redpajama-models-v1

Training Data:

https://www.together.xyz/blog/redpajama

I am failing to see how some of the suggested guidelines would help create meaingful oversight without also hurting new startups in the field of AI.

I am appaled to see the suggestion that cloud providers should report when a large model is trained on their servers, citing the terrible draft of EU AI act. This does nothing for the bigger risk of state actors and infringes on the privacy of users of cloud compute.

On the issue of Model guidelines, I agree with the sentiment to create safeguards, but I don't have faith the government can decide acceptable risk that respects users rights. It has shown very little meaningful understanding of technology many times, most recently during the TikTok hearing.

Advocating for holding companies directly accountable for what these models generate makes sense only in theory and ignores how the math behind these models work. There's no way to enforce content authenticity standards without also attacking section 230 in some way.

I am not suprised to see this sort of letter from a company that supported "pausing" AI development.